Statistical fluctuations

Equilibrium does not make every measured quantity constant. A finite system keeps moving among microstates, so energy, particle count in a subregion, wall force, and other observables fluctuate around their expected values. A finite data set adds sampling uncertainty on top of those physical fluctuations.

The two effects are related but not identical. Physical variance describes the distribution of an observable. Standard error describes uncertainty in an estimated quantity, such as a mean, because only a finite amount of data was collected.

Mean and physical spread

For \(N\) observations \(x_i\), the sample mean is

\[\bar x=\frac{1}{N}\sum_{i=1}^{N}x_i\]

The Bessel-corrected sample variance is

\[s^2=\frac{1}{N-1} \sum_{i=1}^{N}(x_i-\bar x)^2\]

The denominator contains \(N-1\) because the same data were used to estimate the mean. Under independent sampling from a distribution with finite variance, this correction makes \(s^2\) unbiased for the population variance.

The standard deviation \(s\) has the same units as each observation and describes the typical spread of individual values. A variance routine needs at least two values. A constant data set has zero variance; a negative result indicates arithmetic or numerical failure.

Standard error is not standard deviation

For independent, representative observations, an estimated standard error of the mean is

\[\operatorname{SE}(\bar x)=\frac{s}{\sqrt{N}}\]

Individual measurements do not become less variable merely because more are collected. Instead, the estimate of their common mean becomes more precise. Doubling the independent sample count does not halve the standard error; reducing it by one half requires four times as many observations.

This formula addresses random sampling uncertainty. It says nothing about calibration bias, an omitted force, incorrect boundary conditions, or measurements drawn from the wrong population. Quoting a very small standard error from a very large but biased data set is false precision.

Counting fluctuations in a simple model

Suppose each of \(N\) independent particles has probability \(p\) of occupying region \(A\). If \(n\) is the count in that region, the binomial distribution gives

\[\langle n\rangle=Np, \qquad\operatorname{Var}(n)=Np(1-p)\]

The fractional standard deviation is

\[\frac{\sigma_n}{\langle n\rangle} =\sqrt{\frac{1-p}{Np}}\]

At fixed nonzero \(p\), the relative fluctuation decreases approximately with the inverse square root of particle count. This explains why macroscopic densities appear stable even though molecules cross an imaginary boundary continually.

The derivation assumes independent particle indicators. Excluded volume, interactions, conservation constraints, or proximity to a phase transition can correlate particles and change the variance.

Thermodynamic fluctuations contain physical information

In a canonical ensemble, a system exchanges energy with a reservoir at fixed absolute temperature \(T\). The system energy fluctuates, and its variance is related to the heat capacity at constant volume:

\[\operatorname{Var}(E)=k_BT^2C_V\]

Here \(E\) is the system energy, \(k_B\) is Boltzmann's constant, and \(C_V\) is the total, not molar, heat capacity of the modeled system.

Large heat capacity allows larger absolute energy fluctuations because more energy is required to change the temperature appreciably. For ordinary extensive systems, both mean energy and heat capacity grow with particle number. The energy standard deviation then grows approximately with the square root of particle number, while its fraction of the total energy shrinks.

This relation belongs to the canonical equilibrium ensemble. An isolated microcanonical system has fixed total energy, although one subsystem can still exchange energy with the rest and fluctuate. Applying the canonical equation to a driven or unequilibrated series without justification is an error.

Correlated samples reduce information

Observations recorded close together in time are often correlated. If a molecular energy changes only slightly between adjacent steps, storing every step produces a large file but not an equally large number of independent energy measurements.

For a stationary series with normalized lag correlation \(\rho_k\), a common approximation uses the integrated autocorrelation time

\[\tau_{\mathrm{int}} =\frac12+\sum_{k=1}^{\infty}\rho_k\]

The variance of the estimated mean is then approximately

\[\operatorname{Var}(\bar x) \approx\frac{2\tau_{\mathrm{int}}\,\sigma^2}{N}, \qquad N_{\mathrm{eff}}\approx\frac{N}{2\tau_{\mathrm{int}}}\]

Positive correlation makes the effective count

\(N_{\mathrm{eff}}\)

smaller than the stored count. The infinite sum is estimated only over the finite lags that the data support; extending a noisy estimate to arbitrary lags can be unstable.

Block averaging

Block averaging gives a practical correlation check. Divide a stationary series into non-overlapping blocks, calculate one mean per block, and examine the variation among those block means. Increase the block length until neighboring block means are approximately independent and the uncertainty estimate stops changing substantially.

import numpy as np

rng = np.random.default_rng(42)
series = np.zeros(16384)
for i in range(1, len(series)):
    series[i] = 0.8 * series[i - 1] + rng.normal()
for block_length in [16, 64, 256, 1024]:
    blocks = len(series) // block_length
    means = series[: blocks * block_length].reshape(blocks, block_length).mean(axis=1)
    error = means.std(ddof=1) / np.sqrt(blocks)
    correlation = np.corrcoef(means[:-1], means[1:])[0, 1]
    print(block_length, blocks, "standard error:", error, "correlation:", correlation)

Blocks that are too short remain correlated. Blocks that are too long leave too few block means for a stable variance estimate. The method requires a stationary analysis window; blocking cannot repair a trend caused by incomplete equilibration.

The central limit theorem and its limits

For many weakly dependent observations with finite variance, the distribution of a properly normalized sample mean approaches a normal distribution as the effective sample count grows. This central limit behavior supports familiar confidence intervals, but it is an asymptotic result rather than permission to assume a Gaussian shape for every data set.

Heavy-tailed distributions can converge slowly or lack finite variance. Rare transitions between metastable states can produce multimodal data. Near a critical point, long-range correlations enlarge fluctuations and reduce the effective number of independent regions. In those cases, a mean and one symmetric error bar can conceal the most important behavior.

Show the distribution, quantiles, or separate state-conditioned results when the shape is not approximately unimodal and symmetric.

Window choice changes the claim

A short window may capture a transient rather than equilibrium. A very long window can average away a genuine slow drift or alternating regime. The analysis window is part of the question, not a neutral formatting choice.

Compare early and late windows after the declared equilibration interval. Plot or inspect a running mean, but do not use visual flatness as the only criterion. Test whether means from non-overlapping windows agree within correlation-aware uncertainty and whether variance remains stable.

Changing the sampling interval can also create aliasing. If a periodic signal is sampled at an unfortunate cadence, the recorded series may appear constant or may show a false slow oscillation. Physical time stamps and characteristic time scales must accompany sample indices.

A trustworthy fluctuation report

Name the observable, units, system boundary, ensemble, and preparation. Report the raw sample count and the effective count or correlation method. Distinguish standard deviation from standard error. State the equilibration interval, sampling interval, window, block length, and number of independent runs.

Check that probabilities normalize, variance is nonnegative, a constant fixture gives zero spread, and independent synthetic data recover inverse-square-root standard-error scaling. Then use correlated test data with a known time scale to confirm that the method does not count every stored value as independent.

Finally, vary particle number, window length, time step, and independent preparation one at a time. A robust physical result should change in the direction predicted by the model and should remain larger than numerical and sampling uncertainties. Fluctuations are not clutter to remove; when measured with the right ensemble and uncertainty, they are part of the physics.

A worked Python notebook

Average independent observations whose standard deviation is five measurement units. The question for this calculation is precise: how does standard error of a sample mean change when we change independent sample count, while keeping the other choices fixed? The earlier sections explain the physical relationship. Here we turn one part of that relationship into a calculation that you can run, inspect, and change in Google Colab.

Correlated observations have a smaller effective sample count than the raw count. Keep that sentence beside the results. A formula can return a number outside its useful range, and a plotting library will usually draw that number without asking whether the physical assumptions still hold. The selected interval is from 10 to 1000 count. It gives us a specific domain to discuss instead of leaving the graph's range to a software default.

Use the notebook link above the lesson, run the setup cell, and then run the cells in order. The code below is also included there. NumPy handles the arrays of numerical values, SymPy keeps the equation symbolic, and Matplotlib draws the results. If these libraries or notebook cells are unfamiliar, start with Your first physics notebook.

State the relationship before choosing a graph

In this small example, q stands for independent sample count, expressed in count. The output is standard error of a sample mean, expressed in measurement units. The equation below uses the numerical values in those units. Any fixed coefficients therefore carry the units needed to make the relation consistent. A value supplied in another unit must be converted before it enters this expression.

\[f(q) = \frac{5}{\sqrt{q}}\]

This is a function of one changing input. It is not a statement that the physical system has only one variable. We deliberately hold the other inputs fixed so that one cause of variation is visible. Changing several inputs at once would make it harder to tell which change produced the output difference. Write down the held-fixed choices before trying a different experiment.

At 100 count, the expression gives 0.5 measurement units. This is our reference calculation. It is a prediction of the stated example, not a new measurement. Its displayed digits help us compare implementations. They should not be read as a claim that physical inputs are known to six significant figures.

import numpy as np
import sympy as sp
import matplotlib.pyplot as plt

q = sp.Symbol("q", real=True)
expression = sp.sympify("5/sqrt(q)", locals={"q": q})
reference_input = 100.0
reference_output = float(expression.subs(q, reference_input))
print("Input:", reference_input, "count")
print(
    "Predicted standard error of a sample mean:", reference_output, "measurement units"
)

sp.Symbol declares a mathematical symbol. It does not assign a numerical measurement to that symbol. subs then replaces the symbol with our chosen input, and float converts the resulting symbolic number into a standard Python numerical value. Keeping these steps separate lets us inspect the equation before numerical substitution. It also prevents a later plot from becoming the only surviving record of the model.

Compare a few cases you can explain

Before drawing hundreds of points, calculate three. The lower endpoint gives 1.58114 measurement units. The midpoint, at 505 count, gives 0.222497 measurement units. The upper endpoint gives 0.158114 measurement units. Read these together with the assumptions: they describe different choices of independent sample count within the same example, with the other inputs held fixed.

These three values are a useful first check, but they do not establish the shape between them. Two endpoints can be equal even when the curve bends or oscillates in between. Likewise, a large change between endpoints does not tell us whether the relationship is linear. We need the equation and intermediate samples to answer those questions.

evaluate = sp.lambdify(q, expression, modules="numpy")
inputs = np.array([10.0, 505.0, 1000.0])
outputs = np.asarray(evaluate(inputs), dtype=float)
outputs = np.broadcast_to(outputs, inputs.shape)
for input_value, output_value in zip(inputs, outputs):
    print(f"{input_value:.6g} count -> {output_value:.6g} measurement units")
assert np.all(np.isfinite(outputs))

lambdify turns the symbolic expression into a numerical function. We explicitly select NumPy as its numerical backend so it can accept an array. The same expression now has two useful forms: one for algebra and one for calculating many cases. The assertion checks that the chosen inputs produce finite outputs. It would catch an infinity or an undefined numerical result, but a finite value can still be physically inappropriate.

Make a graph that answers the question

The horizontal axis shows independent sample count; the vertical axis shows standard error of a sample mean. Every plotted point is a prediction of the same expression. We mark the reference case separately so that a reader can connect the earlier printed value to its place on the curve. The axis labels include units because a shape without a scale cannot tell us the size of a physical effect.

input_grid = np.linspace(10.0, 1000.0, 301)
output_grid = np.asarray(evaluate(input_grid), dtype=float)
output_grid = np.broadcast_to(output_grid, input_grid.shape)
fig, ax = plt.subplots(figsize=(7, 4))
ax.plot(input_grid, output_grid, label="Stated model")
ax.scatter([reference_input], [reference_output], label="Reference case", zorder=3)
ax.set_xlabel("Independent sample count (count)")
ax.set_ylabel("Standard error of a sample mean (measurement units)")
ax.legend()
ax.grid(alpha=0.25)
fig.tight_layout()
plt.show()

Look for a sign change, a turning point, or a region where the curve becomes steep. Connect each feature to the expression rather than to the colour of the line. If the curve has a minimum or maximum, changing the input in opposite directions may have similar effects on the output near that point. If it is nearly straight over a short interval, a local linear approximation may be useful even when the complete expression is nonlinear.

Increasing the number of samples makes the line smoother and may reveal features that a coarse grid missed. It does not improve the physical inputs or enlarge the model's validity range. For quantities that can take only discrete values, use the line as a guide to the relationship and interpret only the allowed input values. A plotting function does not enforce such restrictions for you.

Measure the local sensitivity

A derivative asks a more specific question than the whole graph: how quickly does standard error of a sample mean change near the reference input? Its units are the output unit divided by the input unit. For this calculation, the derivative at 100 count is -0.0025 in those ratio units. Locally, the output decreases as the input increases. This statement is local; the sign or size of the slope can be different elsewhere on the curve.

\[f'(q) = - \frac{5}{2 q^{\frac{3}{2}}}\]

For a small input change, multiply the derivative by that change to estimate the output change. Then compare that estimate with a fresh evaluation of the full expression. The distinction between these two calculations matters: the derivative gives a tangent approximation, while substitution evaluates the chosen expression at the new point.

slope_expression = sp.diff(expression, q)
slope_at_reference = float(slope_expression.subs(q, reference_input))
changed_input = 104.95
linear_prediction = reference_output + slope_at_reference * (
    changed_input - reference_input
)
full_prediction = float(evaluate(changed_input))
print("Symbolic derivative:", slope_expression)
print("Tangent prediction:", linear_prediction, "measurement units")
print("Full expression:", full_prediction, "measurement units")
print(
    "Approximation difference:",
    full_prediction - linear_prediction,
    "measurement units",
)

Here the changed input is 104.95 count. The tangent estimate is 0.487625 measurement units, while the full expression gives 0.488066 measurement units. If the two values are close, it means the chosen input change is small enough for this local approximation at this point. It does not mean the entire model is linear. If the slope is zero, a quadratic or higher-order term may supply the leading change instead.

This is also the beginning of uncertainty propagation. If the input has a small stated standard uncertainty, the magnitude of the slope times that uncertainty estimates one contribution to output uncertainty. Do not automatically use the same rule for a wide interval, a discontinuity, a turning point with zero first derivative, or several correlated uncertain inputs. Those situations need more of the model than one slope.

Check the derivative with numerical differences

We can estimate the same slope from values on either side of the reference point. Subtract the lower output from the upper output and divide by twice the input step. Parentheses are essential: the entire output difference must be divided by the entire input interval. Without them, Python can execute a different mathematical expression while still producing a plausible number.

The calculation below uses three decreasing step sizes. Each estimate is compared with the symbolic derivative, so the reference is independent of the numerical difference formula. Near an endpoint, it is acceptable here to evaluate a small extension only because the displayed mathematical expression is smooth there; such a check does not assert that extrapolated values describe additional physical cases.

input_span = 990.0
for fraction in [1e-2, 1e-3, 1e-4]:
    step = input_span * fraction
    upper = float(evaluate(reference_input + step))
    lower = float(evaluate(reference_input - step))
    numerical_slope = (upper - lower) / (2 * step)
    difference = numerical_slope - slope_at_reference
    print(f"step={step:.6g}: slope={numerical_slope:.6g}, difference={difference:.6g}")

For a smooth function, reducing the step often reduces the approximation error at first. Very small steps can eventually make subtraction of nearly equal floating-point values less reliable. Therefore “smaller” is not an unlimited guarantee of “better.” Compare the observed behaviour over a useful range of steps instead of choosing the smallest number the computer can represent.

An absolute slope difference is useful when the expected slope is near zero. A relative difference would divide by a very small reference and could look enormous even when the absolute discrepancy is tiny. Away from zero, a relative difference can help compare quantities with very different scales. The tolerance should reflect the question and the numerical method, rather than a universal decimal threshold copied into every test.

Change the example and preserve the reasoning

Make one copy of the reference calculation before editing it. Choose a second value of independent sample count inside the stated interval, predict whether standard error of a sample mean will rise or fall, and then evaluate the expression. Place the prediction and the result in a text cell. If they disagree, identify whether the mistake was in the sign, the unit conversion, the algebra, or the assumption about how the curve behaves.

Next choose a shorter plotting interval around the reference point. Compare the local curve with its tangent approximation. Then widen the interval again. The difference shows why a useful local estimate need not describe the whole range. Keep the axis labels and numerical scale visible when comparing the plots; automatic axis limits can make two different effects appear equally large.

Finally, return to the physical limit stated at the start: Correlated observations have a smaller effective sample count than the raw count. Describe one change to the setup that would violate that limit. State what new input, equation, or observation would be needed before trusting an answer in the changed situation. This last step connects the Python experiment to the physics instead of treating successful execution as the only criterion for a useful result.

Before saving, restart the runtime and run all cells in order. A result that depends on an earlier, now-deleted input cell is not reproducible from the saved notebook. Keep unrounded values for later calculations, round only the reported output, and label any new data as measured, assumed, or generated. The graph and printed values should remain traceable to the same expression and the same choices of inputs.

Recover the output change by integrating the slope

There is another way to check that the expression and its derivative describe the same relationship. Add up the small output changes predicted by the slope across the chosen input interval. In the continuous limit, this integral equals the output at the upper endpoint minus the output at the lower endpoint. The result has units of measurement units: the input unit in the integration cancels the input unit in the denominator of the slope.

This statement is about the mathematics of this particular relationship. It does not assign a new physical meaning to an arbitrary area under a graph. For example, an area under force versus distance represents work, while an area under a different pair of quantities may represent something else or have no standard name. Always carry the axis units through the calculation before naming an integral.

slope_function = sp.lambdify(q, slope_expression, modules="numpy")
endpoint_change = float(evaluate(1000.0) - evaluate(10.0))
for points in [101, 201, 401]:
    grid = np.linspace(10.0, 1000.0, points)
    slopes = np.asarray(slope_function(grid), dtype=float)
    slopes = np.broadcast_to(slopes, grid.shape)
    integrated_change = np.trapezoid(slopes, grid)
    print(
        points,
        "integrated change:",
        integrated_change,
        "endpoint change:",
        endpoint_change,
    )

The trapezoidal rule joins neighbouring slope samples with straight segments and adds their signed areas. A negative contribution remains negative; replacing every slope by its magnitude would calculate a different quantity. Increasing the sample count provides a practical refinement check. If the values do not settle toward the endpoint difference, first inspect the interval, singularities, and array ordering.

Do not conclude that two agreeing calculations prove the model is physically complete. Both use the same selected relationship, so an omitted effect would be omitted from both. Their agreement is evidence that the numerical tools implement that relationship consistently. Comparing with an independent experiment or a more appropriate physical model is a separate test.

Keep this comparison in the notebook when you change the expression. It is particularly useful after adding a coefficient, changing a sign convention, or converting an input unit. A code cell that produces a smooth graph can conceal these mistakes; a second calculation of the same output change gives you another place to notice them.

Sources

• OpenStax, University Physics Volume 2, Section 2.4, “Distribution of Molecular Speeds”.
• OpenStax, University Physics Volume 2, Section 4.7, “Entropy on a Microscopic Scale”.
• Benjamin Crowell, Light and Matter, Chapter 33, “Rules of Randomness”.
• Physics LibreTexts, “Thermodynamics and Statistical Mechanics”.