Probability and microstates
Statistical physics connects two descriptions of the same system. A microstate specifies all microscopic information required by the model. A macrostate keeps only a few large-scale properties, such as total energy, volume, particle number, pressure, or the number of particles in one region.
Many different microstates can produce the same macrostate. Probability becomes useful because ordinary measurements do not identify which of those microscopic possibilities the system occupies.
What counts as a microstate
For a classical point-particle model, a microstate at one instant consists of every particle position and momentum. This collection is a point in phase space. If particles have orientation, internal vibration, spin, or another modeled degree of freedom, those variables also belong to the microstate.
A different model can have a discrete state space. In a simple two-region model, each particle may be labeled left or right. The string LRRL is one microstate of four labeled particles. It contains much less detail than a real gas microstate, but it is enough to study occupancy probabilities.
The word "exact" is always relative to the model. A molecular model that omits electronic states cannot distinguish microstates that differ only electronically. State resolution, boundary conditions, particle labels, and conserved quantities must therefore be stated before a multiplicity is quoted.
A macrostate is a many-to-one description
A macrostate is produced by a reducer that discards microscopic detail. For the two-region model, one useful macrostate variable is the number of particles on the left. The microstates LRRL, RLLR, and RLRL are different arrangements, but each maps to the same macrostate because each has two particles on the left.
The number of microstates compatible with a macrostate is its multiplicity, written \(\Omega\). For \(N\) labeled particles and \(n\) particles on the left,
\[\Omega(n)=\binom{N}{n}=\frac{N!}{n!(N-n)!}\]The binomial coefficient counts which \(n\) labels are assigned to the left. It does not count the order in which those particles arrived.
For four labeled particles, the multiplicities for zero through four particles on the left are one, four, six, four, and one. The balanced macrostate is most numerous. It is not preferred by a new force; it is simply compatible with more microscopic arrangements.
Probability from equal weights
If every allowed microstate has the same probability, the probability of a macrostate is its multiplicity divided by the total number of allowed microstates:
\[P(M)=\frac{\Omega(M)}{\Omega_{\mathrm{total}}}\]In the two-region example, each of the \(N\) particles has two possible region labels, so the total number of microstates is \(2^N\). The probability of finding \(n\) particles on the left is
\[P(n)=\frac{\binom{N}{n}}{2^N}\]For four particles, the probability of a two-and-two split is six sixteenths, whereas the probability that all four are on one specified side is one sixteenth. Adding all five occupancy probabilities gives one, as every allowed microstate belongs to exactly one occupancy macrostate.
The equal-weight assumption needs justification. In the microcanonical description of an isolated equilibrium system, accessible microstates in a narrow energy range are assigned equal probability. If one side is energetically favored, particles are driven, or a sampling procedure is biased, raw multiplicity alone does not determine probability.
Weighted microstates
When microstates have probabilities \(p_i\), the probability of macrostate \(M\) is the sum of the probabilities of all microstates classified into it:
\[P(M)=\sum_{i\in M}p_i\]The probabilities must satisfy
\[p_i\ge 0,\qquad\sum_i p_i=1\]For a system in thermal contact with a large reservoir, equilibrium microstates are commonly weighted by the Boltzmann factor:
\[p_i=\frac{e^{-E_i/(k_BT)}}{Z}, \qquad Z=\sum_j e^{-E_j/(k_BT)}\]Here \(E_i\) is a microstate energy, \(T\) is the reservoir temperature, \(k_B\) is Boltzmann's constant, and \(Z\) normalizes the distribution. Lower-energy states receive greater individual weight, but a high-energy macrostate can still be important if it contains sufficiently many microstates.
Counting depends on distinguishability
The binomial example treats particles as labeled. Swapping particle one and particle two creates a different classical labeled microstate even if the occupancy count is unchanged. Identical quantum particles require different counting rules: exchanging two identical bosons or fermions does not create a new physical state in the same way.
This is not a small bookkeeping detail. Incorrectly treating identical particles as distinguishable overcounts states and changes entropy. At the introductory level, a safe practice is to state whether labels are physical, computational conveniences, or removed by the theory.
Continuous classical phase space also cannot be counted as isolated mathematical points; there are infinitely many. It is divided into finite cells before a multiplicity is formed. Quantum mechanics supplies a natural scale involving Planck's constant. Although an absolute classical count depends on resolution, physically defined entropy differences can remain meaningful when the same convention is used consistently.
Multiplicity and entropy
Boltzmann connected thermodynamic entropy to multiplicity:
\[S=k_B\ln\Omega\]The logarithm turns multiplication into addition. If independent systems have multiplicities \(\Omega_A\) and \(\Omega_B\), the combined multiplicity is their product, while their entropies add:
\[\Omega_{AB}=\Omega_A\Omega_B, \qquad S_{AB}=S_A+S_B\]The most probable equilibrium macrostate is generally the one with the greatest compatible multiplicity, subject to conserved energy, particle number, volume, and other constraints. For a macroscopic system, multiplicities differ by enormous factors, which is why equilibrium macrostates are reproducible even though individual microstates keep changing.
For unequal probabilities, the corresponding statistical entropy is
\[S=-k_B\sum_i p_i\ln p_i\]When \(\Omega\) states are equally likely, each probability is the reciprocal of \(\Omega\), and this expression reduces to Boltzmann's relation.
Exact enumeration and sampling
Small discrete models can enumerate every state. Large state spaces require sampling. Those methods should not be presented as if they provide the same evidence: enumeration gives an exact result for the encoded model, while sampling gives an estimate with uncertainty and possible correlation.
from itertools import product
from collections import Counter
microstates = list(product([0, 1], repeat=6))
counts = Counter(sum(state) for state in microstates)
probabilities = {
left: count / len(microstates) for left, count in sorted(counts.items())
}
assert sum(probabilities.values()) == 1
print("Number on the left -> probability:", probabilities)
For random sampling, use the same classifier but replace accumulated theoretical weight with observed frequency. Record the generator, seed, sample count, burn-in, and sampling interval. Samples from a Markov chain or time trajectory may be correlated, so the stored count can exceed the effective amount of independent information.
Checks that expose counting errors
Begin with a tiny state space that can be listed by hand. Confirm that every accepted microstate maps to exactly one macrostate, every impossible state is rejected, and the macrostate multiplicities add to the total accepted count. Then confirm that normalized probabilities add to one.
Test symmetry. With unbiased left and right regions, the probability for \(n\) particles on the left must equal the probability for \(N-n\) particles on the left. If not, either the physical weighting or the implementation has broken the symmetry.
Finally compare exact enumeration with a long, independently seeded sample for a small case. Agreement within sampling uncertainty checks the sampler, while the exact count checks the classifier. Neither check proves that the simplified state space is an adequate model of a real gas; that is a separate physical assumption.
A worked Python notebook
Group six independent binary particle locations into a macrostate with three on each side. The question for this calculation is precise: how does probability exactly three of six are left change when we change single-particle left probability, while keeping the other choices fixed? The earlier sections explain the physical relationship. Here we turn one part of that relationship into a calculation that you can run, inspect, and change in Google Colab.
Independence and a common single-particle probability are assumptions of the binomial model. Keep that sentence beside the results. A formula can return a number outside its useful range, and a plotting library will usually draw that number without asking whether the physical assumptions still hold. The selected interval is from 0 to 1 dimensionless. It gives us a specific domain to discuss instead of leaving the graph's range to a software default.
Use the notebook link above the lesson, run the setup cell, and then run the cells in order. The code below is also included there. NumPy handles the arrays of numerical values, SymPy keeps the equation symbolic, and Matplotlib draws the results. If these libraries or notebook cells are unfamiliar, start with Your first physics notebook.
State the relationship before choosing a graph
In this small example, q stands for single-particle left probability, expressed in dimensionless. The output is
probability exactly three of six are left, expressed in dimensionless. The equation below uses the numerical values in those
units. Any fixed coefficients therefore carry the units needed to make the relation
consistent. A value supplied in another unit must be converted before it enters this
expression.
This is a function of one changing input. It is not a statement that the physical system has only one variable. We deliberately hold the other inputs fixed so that one cause of variation is visible. Changing several inputs at once would make it harder to tell which change produced the output difference. Write down the held-fixed choices before trying a different experiment.
At 0.5 dimensionless, the expression gives 0.3125 dimensionless. This is our reference calculation. It is a prediction of the stated example, not a new measurement. Its displayed digits help us compare implementations. They should not be read as a claim that physical inputs are known to six significant figures.
import numpy as np
import sympy as sp
import matplotlib.pyplot as plt
q = sp.Symbol("q", real=True)
expression = sp.sympify("20*q**3*(1-q)**3", locals={"q": q})
reference_input = 0.5
reference_output = float(expression.subs(q, reference_input))
print("Input:", reference_input, "dimensionless")
print(
"Predicted probability exactly three of six are left:",
reference_output,
"dimensionless",
)
sp.Symbol declares a mathematical symbol. It does not assign a numerical measurement
to that symbol. subs then replaces the symbol with our chosen input, and float
converts the resulting symbolic number into a standard Python numerical value.
Keeping these steps separate lets us inspect the equation before numerical substitution.
It also prevents a later plot from becoming the only surviving record of the model.
Compare a few cases you can explain
Before drawing hundreds of points, calculate three. The lower endpoint gives 0 dimensionless. The midpoint, at 0.5 dimensionless, gives 0.3125 dimensionless. The upper endpoint gives 0 dimensionless. Read these together with the assumptions: they describe different choices of single-particle left probability within the same example, with the other inputs held fixed.
These three values are a useful first check, but they do not establish the shape between them. Two endpoints can be equal even when the curve bends or oscillates in between. Likewise, a large change between endpoints does not tell us whether the relationship is linear. We need the equation and intermediate samples to answer those questions.
evaluate = sp.lambdify(q, expression, modules="numpy")
inputs = np.array([0.0, 0.5, 1.0])
outputs = np.asarray(evaluate(inputs), dtype=float)
outputs = np.broadcast_to(outputs, inputs.shape)
for input_value, output_value in zip(inputs, outputs):
print(f"{input_value:.6g} dimensionless -> {output_value:.6g} dimensionless")
assert np.all(np.isfinite(outputs))
lambdify turns the symbolic expression into a numerical function. We explicitly select
NumPy as its numerical backend so it can accept an array. The same expression now has
two useful forms: one for algebra and one for calculating many cases. The assertion
checks that the chosen inputs produce finite outputs. It would catch an infinity or an
undefined numerical result, but a finite value can still be physically inappropriate.
Make a graph that answers the question
The horizontal axis shows single-particle left probability; the vertical axis shows probability exactly three of six are left. Every plotted point is a prediction of the same expression. We mark the reference case separately so that a reader can connect the earlier printed value to its place on the curve. The axis labels include units because a shape without a scale cannot tell us the size of a physical effect.
input_grid = np.linspace(0.0, 1.0, 301)
output_grid = np.asarray(evaluate(input_grid), dtype=float)
output_grid = np.broadcast_to(output_grid, input_grid.shape)
fig, ax = plt.subplots(figsize=(7, 4))
ax.plot(input_grid, output_grid, label="Stated model")
ax.scatter([reference_input], [reference_output], label="Reference case", zorder=3)
ax.set_xlabel("Single-particle left probability (dimensionless)")
ax.set_ylabel("Probability exactly three of six are left (dimensionless)")
ax.legend()
ax.grid(alpha=0.25)
fig.tight_layout()
plt.show()
Look for a sign change, a turning point, or a region where the curve becomes steep. Connect each feature to the expression rather than to the colour of the line. If the curve has a minimum or maximum, changing the input in opposite directions may have similar effects on the output near that point. If it is nearly straight over a short interval, a local linear approximation may be useful even when the complete expression is nonlinear.
Increasing the number of samples makes the line smoother and may reveal features that a coarse grid missed. It does not improve the physical inputs or enlarge the model's validity range. For quantities that can take only discrete values, use the line as a guide to the relationship and interpret only the allowed input values. A plotting function does not enforce such restrictions for you.
Measure the local sensitivity
A derivative asks a more specific question than the whole graph: how quickly does probability exactly three of six are left change near the reference input? Its units are the output unit divided by the input unit. For this calculation, the derivative at 0.5 dimensionless is 0 in those ratio units. Locally, the output has no first-order change as the input increases. This statement is local; the sign or size of the slope can be different elsewhere on the curve.
\[f'(q) = - 60 q^{3} \left(1 - q\right)^{2} + 60 q^{2} \left(1 - q\right)^{3}\]For a small input change, multiply the derivative by that change to estimate the output change. Then compare that estimate with a fresh evaluation of the full expression. The distinction between these two calculations matters: the derivative gives a tangent approximation, while substitution evaluates the chosen expression at the new point.
slope_expression = sp.diff(expression, q)
slope_at_reference = float(slope_expression.subs(q, reference_input))
changed_input = 0.505
linear_prediction = reference_output + slope_at_reference * (
changed_input - reference_input
)
full_prediction = float(evaluate(changed_input))
print("Symbolic derivative:", slope_expression)
print("Tangent prediction:", linear_prediction, "dimensionless")
print("Full expression:", full_prediction, "dimensionless")
print("Approximation difference:", full_prediction - linear_prediction, "dimensionless")
Here the changed input is 0.505 dimensionless. The tangent estimate is 0.3125 dimensionless, while the full expression gives 0.312406 dimensionless. If the two values are close, it means the chosen input change is small enough for this local approximation at this point. It does not mean the entire model is linear. If the slope is zero, a quadratic or higher-order term may supply the leading change instead.
This is also the beginning of uncertainty propagation. If the input has a small stated standard uncertainty, the magnitude of the slope times that uncertainty estimates one contribution to output uncertainty. Do not automatically use the same rule for a wide interval, a discontinuity, a turning point with zero first derivative, or several correlated uncertain inputs. Those situations need more of the model than one slope.
Check the derivative with numerical differences
We can estimate the same slope from values on either side of the reference point. Subtract the lower output from the upper output and divide by twice the input step. Parentheses are essential: the entire output difference must be divided by the entire input interval. Without them, Python can execute a different mathematical expression while still producing a plausible number.
The calculation below uses three decreasing step sizes. Each estimate is compared with the symbolic derivative, so the reference is independent of the numerical difference formula. Near an endpoint, it is acceptable here to evaluate a small extension only because the displayed mathematical expression is smooth there; such a check does not assert that extrapolated values describe additional physical cases.
input_span = 1.0
for fraction in [1e-2, 1e-3, 1e-4]:
step = input_span * fraction
upper = float(evaluate(reference_input + step))
lower = float(evaluate(reference_input - step))
numerical_slope = (upper - lower) / (2 * step)
difference = numerical_slope - slope_at_reference
print(f"step={step:.6g}: slope={numerical_slope:.6g}, difference={difference:.6g}")
For a smooth function, reducing the step often reduces the approximation error at first. Very small steps can eventually make subtraction of nearly equal floating-point values less reliable. Therefore “smaller” is not an unlimited guarantee of “better.” Compare the observed behaviour over a useful range of steps instead of choosing the smallest number the computer can represent.
An absolute slope difference is useful when the expected slope is near zero. A relative difference would divide by a very small reference and could look enormous even when the absolute discrepancy is tiny. Away from zero, a relative difference can help compare quantities with very different scales. The tolerance should reflect the question and the numerical method, rather than a universal decimal threshold copied into every test.
Change the example and preserve the reasoning
Make one copy of the reference calculation before editing it. Choose a second value of single-particle left probability inside the stated interval, predict whether probability exactly three of six are left will rise or fall, and then evaluate the expression. Place the prediction and the result in a text cell. If they disagree, identify whether the mistake was in the sign, the unit conversion, the algebra, or the assumption about how the curve behaves.
Next choose a shorter plotting interval around the reference point. Compare the local curve with its tangent approximation. Then widen the interval again. The difference shows why a useful local estimate need not describe the whole range. Keep the axis labels and numerical scale visible when comparing the plots; automatic axis limits can make two different effects appear equally large.
Finally, return to the physical limit stated at the start: Independence and a common single-particle probability are assumptions of the binomial model. Describe one change to the setup that would violate that limit. State what new input, equation, or observation would be needed before trusting an answer in the changed situation. This last step connects the Python experiment to the physics instead of treating successful execution as the only criterion for a useful result.
Before saving, restart the runtime and run all cells in order. A result that depends on an earlier, now-deleted input cell is not reproducible from the saved notebook. Keep unrounded values for later calculations, round only the reported output, and label any new data as measured, assumed, or generated. The graph and printed values should remain traceable to the same expression and the same choices of inputs.
Recover the output change by integrating the slope
There is another way to check that the expression and its derivative describe the same relationship. Add up the small output changes predicted by the slope across the chosen input interval. In the continuous limit, this integral equals the output at the upper endpoint minus the output at the lower endpoint. The result has units of dimensionless: the input unit in the integration cancels the input unit in the denominator of the slope.
This statement is about the mathematics of this particular relationship. It does not assign a new physical meaning to an arbitrary area under a graph. For example, an area under force versus distance represents work, while an area under a different pair of quantities may represent something else or have no standard name. Always carry the axis units through the calculation before naming an integral.
slope_function = sp.lambdify(q, slope_expression, modules="numpy")
endpoint_change = float(evaluate(1.0) - evaluate(0.0))
for points in [101, 201, 401]:
grid = np.linspace(0.0, 1.0, points)
slopes = np.asarray(slope_function(grid), dtype=float)
slopes = np.broadcast_to(slopes, grid.shape)
integrated_change = np.trapezoid(slopes, grid)
print(
points,
"integrated change:",
integrated_change,
"endpoint change:",
endpoint_change,
)
The trapezoidal rule joins neighbouring slope samples with straight segments and adds their signed areas. A negative contribution remains negative; replacing every slope by its magnitude would calculate a different quantity. Increasing the sample count provides a practical refinement check. If the values do not settle toward the endpoint difference, first inspect the interval, singularities, and array ordering.
Do not conclude that two agreeing calculations prove the model is physically complete. Both use the same selected relationship, so an omitted effect would be omitted from both. Their agreement is evidence that the numerical tools implement that relationship consistently. Comparing with an independent experiment or a more appropriate physical model is a separate test.
Keep this comparison in the notebook when you change the expression. It is particularly useful after adding a coefficient, changing a sign convention, or converting an input unit. A code cell that produces a smooth graph can conceal these mistakes; a second calculation of the same output change gives you another place to notice them.