When models fail

A wrong result does not have one cause. It may come from an implementation bug, unstable numerical method, incorrect input data, or an approximation used outside its range. Good scientific software helps you tell those failure modes apart instead of silently turning them into a smooth-looking plot.

Think like a programmer

Debug in layers. First prove that code implements its stated rule. Then measure numerical error. Then validate inputs and units. Only after those checks should you decide whether the physical model's assumptions are inadequate.

Model checklist

Inputs: A declared model range, measured or chosen inputs with units, and an expected comparison case.
State: Current simulation state plus numerical-resolution settings and random seed when applicable.
Rule: The implemented update or relationship.
Output: Prediction with diagnostics, warnings, and validity status.
Check: At least one analytic, limiting, conservation, or reference-data comparison.

Classify before you patch

Symptom	First question	Likely next check
Result changes wildly when time step halves	Is the update method stable and converging?	Run a step-size sweep.
Units do not match	Was a value converted at the boundary?	Check quantity types and labels.
Test fixture passes but measurement disagrees	Does the fixture cover the operating range?	Inspect assumptions and input data.
Energy or probability drifts	Is the invariant actually preserved by this method?	Compare integrators and tolerances.

const coarse = simulate({ step: 0.1 });
const fine = simulate({ step: 0.05 });
if (Math.abs(coarse.output - fine.output) > tolerance) {
  warn("numerical result has not converged at this step size");
}

This does not prove the fine result is true. It tests a narrower claim: whether changing one numerical resolution parameter still changes the answer enough to matter. Record both values, the tolerance, and the measure being compared.

Do not hide useful failures

An error message can teach. “Mass must be positive” tells a reader which precondition was violated. “Explicit diffusion requires diffusivity × step ≤ 1/4” says why a visually exploding grid is not a surprising physical discovery. A chart should display missing data, invalid input, and uncertainty rather than interpolate over them without notice.

When a model fails physically, name the missing mechanism. A projectile model may need drag; a resistor model may need heating; a simple orbit may need another body. Replacing an equation without recording why loses the most useful part of the investigation.

Try this experiment

Prediction: If an error is numerical rather than physical, refining a controlled numerical setting should change the result systematically.

Choose a time-step or grid-size model. Predict what should happen as resolution improves, run at three settings, and decide whether the evidence points first to numerical error or to an invalid physical assumption.

Where this model breaks

Some disagreements cannot be resolved from code alone: data may be incomplete, measurement conditions may be unknown, or multiple physical models may fit the available evidence. Do not claim a failure classification with more confidence than the diagnostics support.

Summary

Failure is information when it is labelled. Separate code, numerical, data, and approximation errors; add diagnostics; and revise the smallest layer that the evidence actually challenges.

Glossary

Convergence: a numerical result becoming stable as resolution is refined.
Invariant: quantity expected to remain constant under a model's stated conditions.
Precondition: input requirement that must hold before a calculation is valid.

Self-check

What distinguishes a convergence test from physical validation?
Which layer should you inspect first when units disagree?
Why is hiding an invalid input scientifically harmful?

Sources

N. J. Higham, Accuracy and Stability of Numerical Algorithms, numerical failure modes and conditioning.
W. H. Press et al., Numerical Recipes, diagnostics and convergence checks.

Model contract

Treat the lesson as a small function before treating it as a fact to memorize. Give every value a unit, keep only the state needed for the next step, and make the output easy to inspect.

\[\text{observable output} = f(\text{inputs},\,\text{state})\]

Inputs: Quantities you set or measure, with units and useful bounds.
State: Values the program must retain to reproduce the next result.
Rule: The relationship or update that turns inputs and state into a result.
Check: A known limit, unit check, invariant, or measured result that can expose a bad model.

Implement the idea as a model

For When Models Fail, write down the quantities you can control, the values your program must retain, and the result a reader could inspect. In Thinking Like a Computational Physicist, the useful program is not the drawing: it is the smallest explicit model that makes a prediction you can test.

Guided experiment

Prediction: changing one declared input while holding the others fixed should change only the outputs that the model connects to that input. Choose one input, predict the direction of change, then check a limiting case such as zero, a symmetric arrangement, or a familiar low-speed or small-change approximation.

Where this model breaks

This lesson is a teaching model, not a complete simulator. Before using it outside the stated question, check which interactions, scales, uncertainties, boundary conditions, and measurement limits it leaves out.

Summary

Treat When Models Fail as a contract: named inputs and units enter a rule, the rule produces an observable result, and a known limit or invariant checks whether the implementation deserves trust.

Glossary

Input: a measured value or chosen parameter supplied to a model.
State: the smallest set of values needed to continue or reproduce a model.
Validation: comparing an output with a known result, limit, invariant, or measurement.

Self-check

Which values are inputs, and which values must remain state?
What observable result would tell you the model is behaving as expected?
Which assumption would you test first before applying the model to a real system?

Model review: turn When Models Fail into a test

Make validity ranges, residuals, counterexamples, and fallback explanations part of every model interface.

Name the inputs and units that the thinking like a computational physicist model needs.
Separate the state you must keep from values you can calculate when needed.
Write one rule that maps the current state and inputs to an observable result.
Choose a limiting case, unit check, invariant, or known result before trusting an output.
State one assumption you would change before using this simplified model for a real decision.

Share to: