A Byte of PhysicsLogo

Information in waves

Waves carry information when a sender deliberately varies a measurable property and a receiver knows how to interpret those variations. The property might be amplitude, frequency, phase, polarization, pulse position, or timing. The waveform alone is not the message: information depends on an agreed mapping from possible messages to physical signals and on a receiver that can distinguish them.

From symbols to physical states

A symbol is one choice from an agreed alphabet. A binary alphabet contains \(0\) and \(1\). A simple amplitude-shift scheme might map them to two carrier amplitudes:

\[s_b(t)=A_b\cos(2\pi f_c t+\phi_c),\qquad b\in\{0,1\}\]

where \(f_c\) and \(\phi_c\) are shared references and \(A_b\) is selected by the bit. Frequency-shift keying changes frequency; phase-shift keying changes phase.

Symbols last a finite interval \(T_s\), so the symbol rate is

\[R_s=\frac{1}{T_s}\]

measured in symbols per second or baud. If one symbol has \(M\) equally used states and \(M\) is a power of two, it can represent \(\log_2 M\) bits. The uncoded bit rate is then

\[R_b=R_s\log_2M\]

Bit rate and symbol rate are not generally identical.

Information depends on probabilities

For possible source symbols with probabilities \(p_i\), Shannon entropy is

\[H=-\sum_i p_i\log_2p_i\]

Its unit is bits per symbol when the logarithm has base two. An impossible outcome with \(p_i=0\) contributes zero by the continuous limit. A fair binary source has \(H=1\) bit per symbol. A highly predictable source has smaller entropy even if its symbols are stored in one binary digit.

Information in this sense is not physical wave energy. Sending a stronger pulse may make states easier to distinguish, but doubling energy does not automatically double message information. Conversely, one bit can be represented by many different physical energies, durations, or carrier frequencies.

Channel, noise, and receiver

A channel transforms the transmitted waveform. It may attenuate, delay, filter, disperse, reflect, add interference, or add random noise. A receiver must estimate timing and carrier references, filter the received signal, form a decision statistic, and map that statistic to a symbol.

For a tiny baseband example, encode \(0\) as amplitude \(0\) and \(1\) as amplitude \(1\). With additive received noise \(n\),

\[r=b+n\]

A threshold detector can decide

\[\widehat b=\begin{cases}1,&r\ge\tfrac12,\\0,&r<\tfrac12.\end{cases}\]

The threshold is optimal only under particular assumptions about equal priors, equal error costs, and symmetric noise. Bias, unequal symbol probabilities, or unequal consequences can move the appropriate threshold.

import numpy as np

rng = np.random.default_rng(42)
bits = rng.integers(0, 2, 10_000)
levels = 2 * bits - 1
received = levels + rng.normal(0, 0.8, len(bits))
decoded = (received >= 0).astype(int)
print("Uncoded bit error fraction:", np.mean(decoded != bits))

This model omits a carrier and samples once per symbol. It is useful for testing decision logic, not for predicting a complete radio or optical link.

Measuring errors

For \(N\) known test bits and \(N_e\) errors, empirical bit-error rate is

\[\widehat{\mathrm{BER}}=\frac{N_e}{N}\]

A short run with zero observed errors does not prove the true error probability is zero. Rare errors require long tests and an uncertainty interval. Count insertions, deletions, and lost synchronization separately when the received stream no longer aligns one-to-one with transmitted bits.

Repeat stochastic tests with recorded seeds and then with many independent seeds. A single fixed seed supports regression testing; an ensemble supports statistical estimation.

Bandwidth, noise, and ideal channel capacity

Bandwidth limits how rapidly distinguishable waveform details can change. Noise limits how finely amplitude or phase states can be distinguished. For an ideal additive white Gaussian noise channel of bandwidth \(B\), average received signal power \(S\), and noise power \(N\), Shannon capacity is

\[C=B\log_2\left(1+\frac{S}{N}\right)\]

Capacity has units of bits per second. It is an upper bound on rates for which arbitrarily small error probability is possible with sufficiently long and suitable codes under the model's assumptions. It is not the uncoded throughput of one threshold detector, and it does not say that any finite implementation reaches the bound.

Increasing signal-to-noise ratio has diminishing returns because it is inside a logarithm. Increasing bandwidth also collects more noise when noise spectral density is fixed, so \(N\) must be interpreted consistently.

Sampling rate is not channel capacity

The sampling theorem says a band-limited continuous waveform can be represented by samples when the sampling rate exceeds twice its highest frequency under ideal conditions. It does not say every sample carries one reliable bit, nor that capacity equals sample rate. Sample precision, noise, modulation alphabet, pulse shaping, coding, and channel bandwidth all matter.

Aliasing can make distinct transmitted frequencies produce the same samples. Anti-alias filtering must occur before digitization or rate reduction. Sampling at exactly the theoretical boundary leaves no transition band for a physical filter, so real systems use margin.

Coding has two different goals

Source coding removes predictable redundancy to represent messages more compactly. Channel coding intentionally adds structured redundancy so a receiver can detect or correct errors. These aims sound opposite but occur at different stages.

A parity bit can detect some errors but cannot correct an arbitrary corrupted bit. Repetition can enable majority decisions but spends extra time or bandwidth. Modern codes use carefully designed block or convolutional structure. When comparing schemes, report the net information rate after redundancy, not only the transmitted symbol rate.

# Threefold repetition spends three channel uses per source bit.
protected = np.repeat(bits, 3)
received = 2 * protected - 1 + rng.normal(0, 0.8, len(protected))
detected = (received >= 0).reshape(-1, 3)
recovered = (detected.sum(axis=1) >= 2).astype(int)
print("Repetition-code bit error fraction:", np.mean(recovered != bits))
print("This comparison holds energy per transmitted symbol fixed.")

Every stage can fail differently. Testing only whether one final message looks readable does not isolate synchronization, detection, decoding, or decompression errors.

Waves also carry energy and momentum

Mechanical and electromagnetic waves can carry energy, momentum, and information. These statements are related but not interchangeable. A constant unmodulated carrier transports energy but conveys no changing message once its known presence is established. Modulation adds distinguishable alternatives.

A receiver needs physical energy to make a measurement, and lower error rates generally require adequate received signal relative to noise. Yet information is counted from possible messages and their probabilities, while energy is calculated from fields and material properties. Keep the units separate: joules, watts, and joules per bit answer different questions from bits and bits per second.

End-to-end verification

Start with zero noise and a channel that introduces no delay or distortion. The recovered bits must exactly equal the input. Use a repeated pseudorandom but seeded sequence so the test covers both transitions and long runs. Then add a known fixed delay and verify that the synchronization stage, not hidden access to the source, restores alignment.

Sweep noise strength over many seeds and record bit count, error count, empirical BER, and confidence interval. Swap the labels of the two amplitudes and update only the shared mapping; a properly separated receiver should still work. Change sample timing and verify the predicted degradation rather than allowing the decoder to read a privileged ideal sample.

Real channels have colored noise, fading, interference, multipath propagation, dispersion, nonlinear amplifiers, finite clocks, quantization, security threats, and hardware constraints. The simple pipeline is valuable because it names those missing stages instead of disguising them inside one unexplained error rate.

A worked Python notebook

Transmit equally likely levels minus one and plus one through zero-mean Gaussian noise. The question for this calculation is precise: how does binary decision error probability change when we change noise standard deviation, while keeping the other choices fixed? The earlier sections explain the physical relationship. Here we turn one part of that relationship into a calculation that you can run, inspect, and change in Google Colab.

The receiver uses threshold zero and the noise is independent of the transmitted symbol. Keep that sentence beside the results. A formula can return a number outside its useful range, and a plotting library will usually draw that number without asking whether the physical assumptions still hold. The selected interval is from 0.2 to 2 signal amplitude units. It gives us a specific domain to discuss instead of leaving the graph's range to a software default.

Use the notebook link above the lesson, run the setup cell, and then run the cells in order. The code below is also included there. NumPy handles the arrays of numerical values, SymPy keeps the equation symbolic, and Matplotlib draws the results. If these libraries or notebook cells are unfamiliar, start with Your first physics notebook.

State the relationship before choosing a graph

In this small example, q stands for noise standard deviation, expressed in signal amplitude units. The output is binary decision error probability, expressed in dimensionless. The equation below uses the numerical values in those units. Any fixed coefficients therefore carry the units needed to make the relation consistent. A value supplied in another unit must be converted before it enters this expression.

\[f(q) = \frac{\operatorname{erfc}{\left(\frac{\sqrt{2}}{2 q} \right)}}{2}\]

This is a function of one changing input. It is not a statement that the physical system has only one variable. We deliberately hold the other inputs fixed so that one cause of variation is visible. Changing several inputs at once would make it harder to tell which change produced the output difference. Write down the held-fixed choices before trying a different experiment.

At 0.8 signal amplitude units, the expression gives 0.10565 dimensionless. This is our reference calculation. It is a prediction of the stated example, not a new measurement. Its displayed digits help us compare implementations. They should not be read as a claim that physical inputs are known to six significant figures.

import numpy as np
import sympy as sp
import matplotlib.pyplot as plt

q = sp.Symbol("q", real=True)
expression = sp.sympify("erfc(1/(sqrt(2)*q))/2", locals={"q": q})
reference_input = 0.8
reference_output = float(expression.subs(q, reference_input))
print("Input:", reference_input, "signal amplitude units")
print("Predicted binary decision error probability:", reference_output, "dimensionless")

sp.Symbol declares a mathematical symbol. It does not assign a numerical measurement to that symbol. subs then replaces the symbol with our chosen input, and float converts the resulting symbolic number into a standard Python numerical value. Keeping these steps separate lets us inspect the equation before numerical substitution. It also prevents a later plot from becoming the only surviving record of the model.

Compare a few cases you can explain

Before drawing hundreds of points, calculate three. The lower endpoint gives 2.86652e-07 dimensionless. The midpoint, at 1.1 signal amplitude units, gives 0.181651 dimensionless. The upper endpoint gives 0.308538 dimensionless. Read these together with the assumptions: they describe different choices of noise standard deviation within the same example, with the other inputs held fixed.

These three values are a useful first check, but they do not establish the shape between them. Two endpoints can be equal even when the curve bends or oscillates in between. Likewise, a large change between endpoints does not tell us whether the relationship is linear. We need the equation and intermediate samples to answer those questions.

scalar_evaluate = sp.lambdify(q, expression, modules="math")
evaluate = np.vectorize(scalar_evaluate, otypes=[float])
inputs = np.array([0.2, 1.1, 2.0])
outputs = np.asarray(evaluate(inputs), dtype=float)
outputs = np.broadcast_to(outputs, inputs.shape)
for input_value, output_value in zip(inputs, outputs):
    print(
        f"{input_value:.6g} signal amplitude units -> "
        f"{output_value:.6g} dimensionless"
    )
assert np.all(np.isfinite(outputs))

lambdify turns the symbolic expression into a numerical function. The complementary error function is available in Python’s math library. We use that backend and wrap the result with np.vectorize so the scalar function can accept an array. The same expression now has two useful forms: one for algebra and one for calculating many cases. The assertion checks that the chosen inputs produce finite outputs. It would catch an infinity or an undefined numerical result, but a finite value can still be physically inappropriate.

Make a graph that answers the question

The horizontal axis shows noise standard deviation; the vertical axis shows binary decision error probability. Every plotted point is a prediction of the same expression. We mark the reference case separately so that a reader can connect the earlier printed value to its place on the curve. The axis labels include units because a shape without a scale cannot tell us the size of a physical effect.

input_grid = np.linspace(0.2, 2.0, 301)
output_grid = np.asarray(evaluate(input_grid), dtype=float)
output_grid = np.broadcast_to(output_grid, input_grid.shape)
fig, ax = plt.subplots(figsize=(7, 4))
ax.plot(input_grid, output_grid, label="Stated model")
ax.scatter([reference_input], [reference_output], label="Reference case", zorder=3)
ax.set_xlabel("Noise standard deviation (signal amplitude units)")
ax.set_ylabel("Binary decision error probability (dimensionless)")
ax.legend()
ax.grid(alpha=0.25)
fig.tight_layout()
plt.show()

Look for a sign change, a turning point, or a region where the curve becomes steep. Connect each feature to the expression rather than to the colour of the line. If the curve has a minimum or maximum, changing the input in opposite directions may have similar effects on the output near that point. If it is nearly straight over a short interval, a local linear approximation may be useful even when the complete expression is nonlinear.

Increasing the number of samples makes the line smoother and may reveal features that a coarse grid missed. It does not improve the physical inputs or enlarge the model's validity range. For quantities that can take only discrete values, use the line as a guide to the relationship and interpret only the allowed input values. A plotting function does not enforce such restrictions for you.

Measure the local sensitivity

A derivative asks a more specific question than the whole graph: how quickly does binary decision error probability change near the reference input? Its units are the output unit divided by the input unit. For this calculation, the derivative at 0.8 signal amplitude units is 0.285389 in those ratio units. Locally, the output increases as the input increases. This statement is local; the sign or size of the slope can be different elsewhere on the curve.

\[f'(q) = \frac{\sqrt{2} e^{- \frac{1}{2 q^{2}}}}{2 \sqrt{\pi} q^{2}}\]

For a small input change, multiply the derivative by that change to estimate the output change. Then compare that estimate with a fresh evaluation of the full expression. The distinction between these two calculations matters: the derivative gives a tangent approximation, while substitution evaluates the chosen expression at the new point.

slope_expression = sp.diff(expression, q)
slope_at_reference = float(slope_expression.subs(q, reference_input))
changed_input = 0.809
linear_prediction = reference_output + slope_at_reference * (
    changed_input - reference_input
)
full_prediction = float(evaluate(changed_input))
print("Symbolic derivative:", slope_expression)
print("Tangent prediction:", linear_prediction, "dimensionless")
print("Full expression:", full_prediction, "dimensionless")
print("Approximation difference:", full_prediction - linear_prediction, "dimensionless")

Here the changed input is 0.809 signal amplitude units. The tangent estimate is 0.108218 dimensionless, while the full expression gives 0.108212 dimensionless. If the two values are close, it means the chosen input change is small enough for this local approximation at this point. It does not mean the entire model is linear. If the slope is zero, a quadratic or higher-order term may supply the leading change instead.

This is also the beginning of uncertainty propagation. If the input has a small stated standard uncertainty, the magnitude of the slope times that uncertainty estimates one contribution to output uncertainty. Do not automatically use the same rule for a wide interval, a discontinuity, a turning point with zero first derivative, or several correlated uncertain inputs. Those situations need more of the model than one slope.

Check the derivative with numerical differences

We can estimate the same slope from values on either side of the reference point. Subtract the lower output from the upper output and divide by twice the input step. Parentheses are essential: the entire output difference must be divided by the entire input interval. Without them, Python can execute a different mathematical expression while still producing a plausible number.

The calculation below uses three decreasing step sizes. Each estimate is compared with the symbolic derivative, so the reference is independent of the numerical difference formula. Near an endpoint, it is acceptable here to evaluate a small extension only because the displayed mathematical expression is smooth there; such a check does not assert that extrapolated values describe additional physical cases.

input_span = 1.8
for fraction in [1e-2, 1e-3, 1e-4]:
    step = input_span * fraction
    upper = float(evaluate(reference_input + step))
    lower = float(evaluate(reference_input - step))
    numerical_slope = (upper - lower) / (2 * step)
    difference = numerical_slope - slope_at_reference
    print(f"step={step:.6g}: slope={numerical_slope:.6g}, difference={difference:.6g}")

For a smooth function, reducing the step often reduces the approximation error at first. Very small steps can eventually make subtraction of nearly equal floating-point values less reliable. Therefore “smaller” is not an unlimited guarantee of “better.” Compare the observed behaviour over a useful range of steps instead of choosing the smallest number the computer can represent.

An absolute slope difference is useful when the expected slope is near zero. A relative difference would divide by a very small reference and could look enormous even when the absolute discrepancy is tiny. Away from zero, a relative difference can help compare quantities with very different scales. The tolerance should reflect the question and the numerical method, rather than a universal decimal threshold copied into every test.

Change the example and preserve the reasoning

Make one copy of the reference calculation before editing it. Choose a second value of noise standard deviation inside the stated interval, predict whether binary decision error probability will rise or fall, and then evaluate the expression. Place the prediction and the result in a text cell. If they disagree, identify whether the mistake was in the sign, the unit conversion, the algebra, or the assumption about how the curve behaves.

Next choose a shorter plotting interval around the reference point. Compare the local curve with its tangent approximation. Then widen the interval again. The difference shows why a useful local estimate need not describe the whole range. Keep the axis labels and numerical scale visible when comparing the plots; automatic axis limits can make two different effects appear equally large.

Finally, return to the physical limit stated at the start: The receiver uses threshold zero and the noise is independent of the transmitted symbol. Describe one change to the setup that would violate that limit. State what new input, equation, or observation would be needed before trusting an answer in the changed situation. This last step connects the Python experiment to the physics instead of treating successful execution as the only criterion for a useful result.

Before saving, restart the runtime and run all cells in order. A result that depends on an earlier, now-deleted input cell is not reproducible from the saved notebook. Keep unrounded values for later calculations, round only the reported output, and label any new data as measured, assumed, or generated. The graph and printed values should remain traceable to the same expression and the same choices of inputs.

Recover the output change by integrating the slope

There is another way to check that the expression and its derivative describe the same relationship. Add up the small output changes predicted by the slope across the chosen input interval. In the continuous limit, this integral equals the output at the upper endpoint minus the output at the lower endpoint. The result has units of dimensionless: the input unit in the integration cancels the input unit in the denominator of the slope.

This statement is about the mathematics of this particular relationship. It does not assign a new physical meaning to an arbitrary area under a graph. For example, an area under force versus distance represents work, while an area under a different pair of quantities may represent something else or have no standard name. Always carry the axis units through the calculation before naming an integral.

slope_function = sp.lambdify(q, slope_expression, modules="numpy")
endpoint_change = float(evaluate(2.0) - evaluate(0.2))
for points in [101, 201, 401]:
    grid = np.linspace(0.2, 2.0, points)
    slopes = np.asarray(slope_function(grid), dtype=float)
    slopes = np.broadcast_to(slopes, grid.shape)
    integrated_change = np.trapezoid(slopes, grid)
    print(
        points,
        "integrated change:",
        integrated_change,
        "endpoint change:",
        endpoint_change,
    )

The trapezoidal rule joins neighbouring slope samples with straight segments and adds their signed areas. A negative contribution remains negative; replacing every slope by its magnitude would calculate a different quantity. Increasing the sample count provides a practical refinement check. If the values do not settle toward the endpoint difference, first inspect the interval, singularities, and array ordering.

Do not conclude that two agreeing calculations prove the model is physically complete. Both use the same selected relationship, so an omitted effect would be omitted from both. Their agreement is evidence that the numerical tools implement that relationship consistently. Comparing with an independent experiment or a more appropriate physical model is a separate test.

Keep this comparison in the notebook when you change the expression. It is particularly useful after adding a coefficient, changing a sign convention, or converting an input unit. A code cell that produces a smooth graph can conceal these mistakes; a second calculation of the same output change gives you another place to notice them.

Sources

Share to: