Units, precision, and floating-point errors
A calculation can be logically correct and still give a misleading answer when its inputs use mixed units or its numbers are rounded too early.
Think like a programmer
Model checklist
- Inputs
- Measured values, units, and requested display precision.
- State
- The full-precision intermediate result.
- Rule
- Do arithmetic first; round only when presenting a result.
- Output
- A value with an honest number of digits.
- Check
- Compare early rounding with late rounding.
Rounding changes a program's state
- Input
- 0.3333333333333333
- Round first, then multiply by 3
- 0.99
- Multiply first, then round
- 1.00
const result = (1 / 3) * 3;
// result is close to 1, not guaranteed to be exactly 1 in binary floating point.
Try this experiment
Prediction: Keeping more digits makes the early-rounded result closer to one.
Move the digit slider. Identify the point where displayed precision stops being evidence that the measurement itself was precise.Where this model breaks
Summary
Check units, retain useful intermediate precision, and test sensitive calculations with known values. A long decimal is not automatically a trustworthy measurement.
Glossary
- Floating point: a finite computer format for approximate real numbers.
- Rounding: choosing a nearby representable or displayed value.
- Cancellation: loss of useful digits when nearly equal numbers are subtracted.
Self-check
- When should a program round a measurement?
- What is wrong with adding metres to seconds?
- Does an exact formula eliminate floating-point error?
Sources
- D. Goldberg, What Every Computer Scientist Should Know About Floating-Point Arithmetic.
Model contract
Treat the lesson as a small function before treating it as a fact to memorize. Give every value a unit, keep only the state needed for the next step, and make the output easy to inspect.
\[\text{observable output} = f(\text{inputs},\,\text{state})\]- Inputs
- Quantities you set or measure, with units and useful bounds.
- State
- Values the program must retain to reproduce the next result.
- Rule
- The relationship or update that turns inputs and state into a result.
- Check
- A known limit, unit check, invariant, or measured result that can expose a bad model.
Implement the idea as a model
For Units, Precision, and Floating-Point Errors, write down the quantities you can control, the values your program must retain, and the result a reader could inspect. In Physics as Computation, the useful program is not the drawing: it is the smallest explicit model that makes a prediction you can test.
Guided experiment
Prediction: changing one declared input while holding the others fixed should change only the outputs that the model connects to that input. Choose one input, predict the direction of change, then check a limiting case such as zero, a symmetric arrangement, or a familiar low-speed or small-change approximation.
Where this model breaks
This lesson is a teaching model, not a complete simulator. Before using it outside the stated question, check which interactions, scales, uncertainties, boundary conditions, and measurement limits it leaves out.
Summary
Treat Units, Precision, and Floating-Point Errors as a contract: named inputs and units enter a rule, the rule produces an observable result, and a known limit or invariant checks whether the implementation deserves trust.
Glossary
- Input: a measured value or chosen parameter supplied to a model.
- State: the smallest set of values needed to continue or reproduce a model.
- Validation: comparing an output with a known result, limit, invariant, or measurement.
Self-check
- Which values are inputs, and which values must remain state?
- What observable result would tell you the model is behaving as expected?
- Which assumption would you test first before applying the model to a real system?
Model review: turn Units, Precision, and Floating-Point Errors into a test
See how units, rounding, and finite binary numbers can change a physics calculation before the model itself is wrong.
- Name the inputs and units that the physics as computation model needs.
- Separate the state you must keep from values you can calculate when needed.
- Write one rule that maps the current state and inputs to an observable result.
- Choose a limiting case, unit check, invariant, or known result before trusting an output.
- State one assumption you would change before using this simplified model for a real decision.