A Byte of PhysicsLogo

Numerical instability and chaos

Numerical instability and physical chaos can both make computed trajectories separate, but they are not the same phenomenon. Instability can be created by a poor time step or algorithm even for a calm linear system. Deterministic chaos is sensitive dependence on initial conditions in the physical model after numerical error has been controlled.

Four sources of disagreement

When a calculated trajectory disagrees with another result, distinguish:

  1. Model difference: force laws, parameters, boundaries, or omitted interactions differ.
  2. Initial-data uncertainty: measured starting states are not exactly known.
  3. Numerical error: finite steps, finite precision, and incomplete solver convergence approximate the chosen equations.
  4. Chaotic sensitivity: the equations amplify tiny state differences over time.

A jagged or diverging plot alone identifies none of these. Record enough information to repeat the calculation and vary one cause at a time.

Numerical stability on a known oscillator

The ideal harmonic oscillator obeys

\[\ddot x=-\omega^2x\]

and has constant energy

\[E=\frac12mv^2+\frac12m\omega^2x^2\]

Forward Euler gives

\[x_{n+1}=x_n+hv_n,\qquad v_{n+1}=v_n-h\omega^2x_n\]

For this system,

\[E_{n+1}=(1+h^2\omega^2)E_n\]

so every nonzero step multiplies energy by a value above one. The outward spiral is a numerical artifact, not chaotic physics. Reducing the dimensionless step \(h\omega\) slows the growth, while a structure-preserving method changes the long-term behavior.

This linear oscillator is a control case: its exact solution is periodic and nearby exact trajectories do not exhibit sustained exponential sensitivity.

Convergence before interpretation

For a fixed physical time interval, solve with steps \(h\), \(h/2\), and \(h/4\). Compare states at identical physical times, not identical array indexes. Repeat with another appropriate integrator.

import numpy as np


def oscillator(steps):
    dt = 2.0 / steps
    x, v = 1.0, 0.0
    for _ in range(steps):
        x, v = x + dt * v, v - dt * x
    return np.array([x, v])


coarse, medium, fine = [oscillator(n) for n in [100, 200, 400]]
print("Coarse difference:", np.linalg.norm(coarse - medium))
print("Fine difference:", np.linalg.norm(medium - fine))

Convergence means the differences decrease consistently as resolution improves. It does not prove the physical model is realistic. Failure to converge can arise from an unstable step, discontinuity, incorrect event handling, stiffness, floating-point limits, or a programming error.

Deterministic chaos

A deterministic system assigns one future to an exact initial state. It can still be chaotic: nearby exact initial states separate rapidly, making long-term detailed prediction limited by finite knowledge of the initial state.

For an early interval before separation saturates, a small perturbation may behave approximately as

\[\delta(t)\approx\delta_0e^{\lambda t}\]

where \(\delta_0\) is small and \(\lambda\) is a finite-time or asymptotic growth rate depending on the measurement procedure. A positive maximal long-time Lyapunov exponent is a standard indicator of chaos.

Taking a logarithm gives

\[\ln\frac{\delta(t)}{\delta_0}\approx\lambda t\]

The slope is meaningful only during a growth regime where the perturbation remains small enough for local linearization and large enough to exceed roundoff noise.

A driven damped pendulum example

A nonlinear driven pendulum can be modeled by

\[\ddot\phi+2\beta\dot\phi+\omega_0^2\sin\phi=\gamma\omega_0^2\cos(\omega_Dt)\]

The sine makes the system nonlinear. Damping removes energy and periodic driving adds it. Depending on parameters, the long-time response may be periodic, period-multiplied, or chaotic. Nonlinearity is necessary for this common route to chaos but does not mean every nonlinear trajectory is chaotic.

Because the force depends explicitly on driving phase, the complete autonomous state can include

\((\phi,\dot\phi,\psi)\)

with \(\dot\psi=\omega_D\). Comparing only angle can be misleading because angles differing by a full turn describe the same orientation while the unwrapped rotation count may matter physically.

Measuring separation

A raw Euclidean norm of a state that mixes metres, metres per second, and radians has no clear physical meaning. Define characteristic scales and a dimensionless distance. For position and velocity,

\[d(t)=\sqrt{\left(\frac{\Delta x}{L_*}\right)^2+\left(\frac{\Delta v}{V_*}\right)^2}\]

The scales \(L_*\) and \(V_*\) must be reported. Different norms can change finite-time estimates, though asymptotic signs are robust under suitable equivalent norms.

def scaled_distance(first, second, scales):
    return np.linalg.norm((np.asarray(first) - second) / scales)


print(scaled_distance([1.001, 2.0], [1.0, 2.0], np.array([1.0, 3.0])))

For angular states, decide whether the distance should use the shortest wrapped angle or the difference in unwrapped rotations. The correct choice depends on the question.

Renormalizing perturbations

Two trajectories in a bounded chaotic region cannot separate forever. Once their distance reaches the size of the accessible region, it saturates and no longer measures local exponential growth. A common Lyapunov procedure periodically measures the grown perturbation, accumulates its logarithmic growth, and rescales the nearby state back to a small distance along the separation direction.

# A discrete logistic map illustrates tangent growth per iteration.
# It is a mathematical example, not an estimate for the oscillator above.
x = 0.123456
log_growth = []
for iteration in range(12_000):
    slope = abs(4 * (1 - 2 * x))
    if iteration >= 2_000:
        log_growth.append(np.log(max(slope, 1e-300)))
    x = 4 * x * (1 - x)
print("Lyapunov exponent per iteration:", np.mean(log_growth))
print("Analytic reference ln(2):", np.log(2.0))

The perturbation must be small enough for local behavior but not so small that roundoff dominates. The estimate should be repeated for different perturbation sizes, renormalization intervals, integration steps, initial transient lengths, and total durations.

Prediction horizon

If initial uncertainty is \(\delta_0\) and the largest acceptable forecast error is \(\delta_{\max}\), the exponential model gives an approximate horizon

\[t_{\mathrm{predict}}\approx\frac{1}{\lambda}\ln\left(\frac{\delta_{\max}}{\delta_0}\right)\]

Improving initial precision by a fixed factor adds time logarithmically rather than making the horizon unlimited. This does not mean all knowledge is lost afterward. Statistical properties, attractor structure, conserved quantities, or bounded ranges may remain predictable after a detailed trajectory is not.

Numerical error can imitate sensitivity

Two runs may diverge because:

First demonstrate that each unperturbed trajectory is converged over the measurement window. Then run neighboring states with identical algorithms and controlled randomness. For stochastic systems, divergence is not by itself evidence of deterministic chaos.

Energy drift is a diagnostic, not a universal chaos test

In a conservative model, unexplained energy drift can expose numerical error. The relative diagnostic

\[\epsilon_E(t)=\frac{E(t)-E(0)}{E_*}\]

uses a nonzero characteristic energy \(E_*\) so a near-zero initial total does not cause meaningless division.

A driven damped pendulum should not conserve mechanical energy, so nonzero energy change is expected. The correct check compares energy change with work by the drive and energy dissipated by damping. A chaotic conservative integrator can have bounded energy error, while a nonchaotic forward Euler oscillator can have exploding energy. Energy behavior alone cannot classify chaos.

Other evidence for chaos

A credible analysis often combines several diagnostics:

A complicated trace or broad spectrum can also come from noise, transients, or multiple incommensurate frequencies. Each diagnostic has limitations and should agree with the specified deterministic model.

Linear stability near an equilibrium

For a system

\(\dot{\mathbf y}=\mathbf f(\mathbf y)\)

near equilibrium \(\mathbf y_*\), small perturbations satisfy approximately

\[\dot{\boldsymbol\eta}=\mathbf J(\mathbf y_*)\boldsymbol\eta\]

where \(\mathbf J\) contains partial derivatives of the dynamics. Eigenvalues with positive real parts indicate local instability of that equilibrium. Local instability is not automatically global chaos; a trajectory can move away and settle into another simple attractor.

Reproducibility record

Record equations, parameter values with units, initial state and uncertainty, integration method, time-step or tolerance settings, floating-point precision, event rules, transient discarded, distance norm and scaling, perturbation vector, output cadence, and random seed when applicable. Without this record, another reader cannot separate physical sensitivity from implementation variation.

Required checks

Chaos is a property of a deterministic model's dynamics. Numerical stability is a property of how those dynamics are approximated. Trustworthy computational physics demonstrates numerical convergence before using trajectory divergence as physical evidence.

A worked Python notebook

A declared exponential growth rate of 0.2 per second illustrates how a small perturbation can amplify. The question for this calculation is precise: how does linear separation estimate change when we change elapsed time, while keeping the other choices fixed? The earlier sections explain the physical relationship. Here we turn one part of that relationship into a calculation that you can run, inspect, and change in Google Colab.

This prescribed growth curve alone does not establish chaos in any particular physical system. Keep that sentence beside the results. A formula can return a number outside its useful range, and a plotting library will usually draw that number without asking whether the physical assumptions still hold. The selected interval is from 0 to 10 s. It gives us a specific domain to discuss instead of leaving the graph's range to a software default.

Use the notebook link above the lesson, run the setup cell, and then run the cells in order. The code below is also included there. NumPy handles the arrays of numerical values, SymPy keeps the equation symbolic, and Matplotlib draws the results. If these libraries or notebook cells are unfamiliar, start with Your first physics notebook.

State the relationship before choosing a graph

In this small example, q stands for elapsed time, expressed in s. The output is linear separation estimate, expressed in dimensionless. The equation below uses the numerical values in those units. Any fixed coefficients therefore carry the units needed to make the relation consistent. A value supplied in another unit must be converted before it enters this expression.

\[f(q) = e^{0.2 q}\]

This is a function of one changing input. It is not a statement that the physical system has only one variable. We deliberately hold the other inputs fixed so that one cause of variation is visible. Changing several inputs at once would make it harder to tell which change produced the output difference. Write down the held-fixed choices before trying a different experiment.

At 5 s, the expression gives 2.71828 dimensionless. This is our reference calculation. It is a prediction of the stated example, not a new measurement. Its displayed digits help us compare implementations. They should not be read as a claim that physical inputs are known to six significant figures.

import numpy as np
import sympy as sp
import matplotlib.pyplot as plt

q = sp.Symbol("q", real=True)
expression = sp.sympify("exp(0.2*q)", locals={"q": q})
reference_input = 5.0
reference_output = float(expression.subs(q, reference_input))
print("Input:", reference_input, "s")
print("Predicted linear separation estimate:", reference_output, "dimensionless")

sp.Symbol declares a mathematical symbol. It does not assign a numerical measurement to that symbol. subs then replaces the symbol with our chosen input, and float converts the resulting symbolic number into a standard Python numerical value. Keeping these steps separate lets us inspect the equation before numerical substitution. It also prevents a later plot from becoming the only surviving record of the model.

Compare a few cases you can explain

Before drawing hundreds of points, calculate three. The lower endpoint gives 1 dimensionless. The midpoint, at 5 s, gives 2.71828 dimensionless. The upper endpoint gives 7.38906 dimensionless. Read these together with the assumptions: they describe different choices of elapsed time within the same example, with the other inputs held fixed.

These three values are a useful first check, but they do not establish the shape between them. Two endpoints can be equal even when the curve bends or oscillates in between. Likewise, a large change between endpoints does not tell us whether the relationship is linear. We need the equation and intermediate samples to answer those questions.

evaluate = sp.lambdify(q, expression, modules="numpy")
inputs = np.array([0.0, 5.0, 10.0])
outputs = np.asarray(evaluate(inputs), dtype=float)
outputs = np.broadcast_to(outputs, inputs.shape)
for input_value, output_value in zip(inputs, outputs):
    print(f"{input_value:.6g} s -> {output_value:.6g} dimensionless")
assert np.all(np.isfinite(outputs))

lambdify turns the symbolic expression into a numerical function. We explicitly select NumPy as its numerical backend so it can accept an array. The same expression now has two useful forms: one for algebra and one for calculating many cases. The assertion checks that the chosen inputs produce finite outputs. It would catch an infinity or an undefined numerical result, but a finite value can still be physically inappropriate.

Make a graph that answers the question

The horizontal axis shows elapsed time; the vertical axis shows linear separation estimate. Every plotted point is a prediction of the same expression. We mark the reference case separately so that a reader can connect the earlier printed value to its place on the curve. The axis labels include units because a shape without a scale cannot tell us the size of a physical effect.

input_grid = np.linspace(0.0, 10.0, 301)
output_grid = np.asarray(evaluate(input_grid), dtype=float)
output_grid = np.broadcast_to(output_grid, input_grid.shape)
fig, ax = plt.subplots(figsize=(7, 4))
ax.plot(input_grid, output_grid, label="Stated model")
ax.scatter([reference_input], [reference_output], label="Reference case", zorder=3)
ax.set_xlabel("Elapsed time (s)")
ax.set_ylabel("Linear separation estimate (dimensionless)")
ax.legend()
ax.grid(alpha=0.25)
fig.tight_layout()
plt.show()

Look for a sign change, a turning point, or a region where the curve becomes steep. Connect each feature to the expression rather than to the colour of the line. If the curve has a minimum or maximum, changing the input in opposite directions may have similar effects on the output near that point. If it is nearly straight over a short interval, a local linear approximation may be useful even when the complete expression is nonlinear.

Increasing the number of samples makes the line smoother and may reveal features that a coarse grid missed. It does not improve the physical inputs or enlarge the model's validity range. For quantities that can take only discrete values, use the line as a guide to the relationship and interpret only the allowed input values. A plotting function does not enforce such restrictions for you.

Measure the local sensitivity

A derivative asks a more specific question than the whole graph: how quickly does linear separation estimate change near the reference input? Its units are the output unit divided by the input unit. For this calculation, the derivative at 5 s is 0.543656 in those ratio units. Locally, the output increases as the input increases. This statement is local; the sign or size of the slope can be different elsewhere on the curve.

\[f'(q) = 0.2 e^{0.2 q}\]

For a small input change, multiply the derivative by that change to estimate the output change. Then compare that estimate with a fresh evaluation of the full expression. The distinction between these two calculations matters: the derivative gives a tangent approximation, while substitution evaluates the chosen expression at the new point.

slope_expression = sp.diff(expression, q)
slope_at_reference = float(slope_expression.subs(q, reference_input))
changed_input = 5.05
linear_prediction = reference_output + slope_at_reference * (
    changed_input - reference_input
)
full_prediction = float(evaluate(changed_input))
print("Symbolic derivative:", slope_expression)
print("Tangent prediction:", linear_prediction, "dimensionless")
print("Full expression:", full_prediction, "dimensionless")
print("Approximation difference:", full_prediction - linear_prediction, "dimensionless")

Here the changed input is 5.05 s. The tangent estimate is 2.74546 dimensionless, while the full expression gives 2.7456 dimensionless. If the two values are close, it means the chosen input change is small enough for this local approximation at this point. It does not mean the entire model is linear. If the slope is zero, a quadratic or higher-order term may supply the leading change instead.

This is also the beginning of uncertainty propagation. If the input has a small stated standard uncertainty, the magnitude of the slope times that uncertainty estimates one contribution to output uncertainty. Do not automatically use the same rule for a wide interval, a discontinuity, a turning point with zero first derivative, or several correlated uncertain inputs. Those situations need more of the model than one slope.

Check the derivative with numerical differences

We can estimate the same slope from values on either side of the reference point. Subtract the lower output from the upper output and divide by twice the input step. Parentheses are essential: the entire output difference must be divided by the entire input interval. Without them, Python can execute a different mathematical expression while still producing a plausible number.

The calculation below uses three decreasing step sizes. Each estimate is compared with the symbolic derivative, so the reference is independent of the numerical difference formula. Near an endpoint, it is acceptable here to evaluate a small extension only because the displayed mathematical expression is smooth there; such a check does not assert that extrapolated values describe additional physical cases.

input_span = 10.0
for fraction in [1e-2, 1e-3, 1e-4]:
    step = input_span * fraction
    upper = float(evaluate(reference_input + step))
    lower = float(evaluate(reference_input - step))
    numerical_slope = (upper - lower) / (2 * step)
    difference = numerical_slope - slope_at_reference
    print(f"step={step:.6g}: slope={numerical_slope:.6g}, difference={difference:.6g}")

For a smooth function, reducing the step often reduces the approximation error at first. Very small steps can eventually make subtraction of nearly equal floating-point values less reliable. Therefore “smaller” is not an unlimited guarantee of “better.” Compare the observed behaviour over a useful range of steps instead of choosing the smallest number the computer can represent.

An absolute slope difference is useful when the expected slope is near zero. A relative difference would divide by a very small reference and could look enormous even when the absolute discrepancy is tiny. Away from zero, a relative difference can help compare quantities with very different scales. The tolerance should reflect the question and the numerical method, rather than a universal decimal threshold copied into every test.

Change the example and preserve the reasoning

Make one copy of the reference calculation before editing it. Choose a second value of elapsed time inside the stated interval, predict whether linear separation estimate will rise or fall, and then evaluate the expression. Place the prediction and the result in a text cell. If they disagree, identify whether the mistake was in the sign, the unit conversion, the algebra, or the assumption about how the curve behaves.

Next choose a shorter plotting interval around the reference point. Compare the local curve with its tangent approximation. Then widen the interval again. The difference shows why a useful local estimate need not describe the whole range. Keep the axis labels and numerical scale visible when comparing the plots; automatic axis limits can make two different effects appear equally large.

Finally, return to the physical limit stated at the start: This prescribed growth curve alone does not establish chaos in any particular physical system. Describe one change to the setup that would violate that limit. State what new input, equation, or observation would be needed before trusting an answer in the changed situation. This last step connects the Python experiment to the physics instead of treating successful execution as the only criterion for a useful result.

Before saving, restart the runtime and run all cells in order. A result that depends on an earlier, now-deleted input cell is not reproducible from the saved notebook. Keep unrounded values for later calculations, round only the reported output, and label any new data as measured, assumed, or generated. The graph and printed values should remain traceable to the same expression and the same choices of inputs.

Recover the output change by integrating the slope

There is another way to check that the expression and its derivative describe the same relationship. Add up the small output changes predicted by the slope across the chosen input interval. In the continuous limit, this integral equals the output at the upper endpoint minus the output at the lower endpoint. The result has units of dimensionless: the input unit in the integration cancels the input unit in the denominator of the slope.

This statement is about the mathematics of this particular relationship. It does not assign a new physical meaning to an arbitrary area under a graph. For example, an area under force versus distance represents work, while an area under a different pair of quantities may represent something else or have no standard name. Always carry the axis units through the calculation before naming an integral.

slope_function = sp.lambdify(q, slope_expression, modules="numpy")
endpoint_change = float(evaluate(10.0) - evaluate(0.0))
for points in [101, 201, 401]:
    grid = np.linspace(0.0, 10.0, points)
    slopes = np.asarray(slope_function(grid), dtype=float)
    slopes = np.broadcast_to(slopes, grid.shape)
    integrated_change = np.trapezoid(slopes, grid)
    print(
        points,
        "integrated change:",
        integrated_change,
        "endpoint change:",
        endpoint_change,
    )

The trapezoidal rule joins neighbouring slope samples with straight segments and adds their signed areas. A negative contribution remains negative; replacing every slope by its magnitude would calculate a different quantity. Increasing the sample count provides a practical refinement check. If the values do not settle toward the endpoint difference, first inspect the interval, singularities, and array ordering.

Do not conclude that two agreeing calculations prove the model is physically complete. Both use the same selected relationship, so an omitted effect would be omitted from both. Their agreement is evidence that the numerical tools implement that relationship consistently. Comparing with an independent experiment or a more appropriate physical model is a separate test.

Keep this comparison in the notebook when you change the expression. It is particularly useful after adding a coefficient, changing a sign convention, or converting an input unit. A code cell that produces a smooth graph can conceal these mistakes; a second calculation of the same output change gives you another place to notice them.

Sources

Share to: