9.5 Reproducibility and Verification

Computation is useful only when we can trust the result. If a calculation gives an answer, we should be able to ask two basic questions: can someone repeat the same computation, and can someone check that the result is correct?

Reproducibility means that the same method gives the same result when repeated. Verification means checking that the method and the result are correct.

Reproducibility

Suppose a student says:

“The answer is approximately $1.41421$.”

This may be correct, but it is incomplete. We need to know what was computed. Was the student approximating $\sqrt{2}$? Which method was used? How many steps were performed? How many decimal places were kept?

A reproducible computation gives enough information for another person to repeat it.

For example:

“To approximate $\sqrt{2}$, start with $x = 1$ and repeat the rule

After five steps, the result is approximately $1.41421$.”

This is better. Another reader can follow the same rule and check the result.

Why details matter

Small details can change a computation. Rounding too early can produce a different answer. Using a different formula can change the error. Stopping after three steps instead of five steps may give a less accurate result.

For simple arithmetic, these details may not matter much. For long computations, simulations, or large datasets, they matter a great deal.

A reproducible computation should state the input, the method, the stopping rule, and the final result.

Verification

Verification asks whether the result makes sense and whether the method was applied correctly.

Consider the equation

We solve it and get

To verify the answer, substitute it back into the original equation:

The statement is true, so the answer checks out.

This is a simple form of verification: put the result back into the problem and see whether it satisfies the original condition.

Checking by another method

Sometimes we can verify a result by using a second method.

For example, suppose we compute

We can check it by rewriting:

Then

and

Adding gives

The second method supports the first result.

Approximate results need error checks

When a computation is approximate, verification also means checking the error.

For example,

To check this, square the approximation:

This is very close to $2$, so the approximation is good.

For approximate answers, we usually cannot ask whether the result is exactly correct. Instead, we ask whether the error is small enough for the purpose.

Reproducibility in software

When computation is done by software, reproducibility requires more information. We may need to record the program, the input data, the version of the software, and the settings used.

For example, a graph drawn by a computer may depend on the plotting program. A numerical simulation may depend on the step size. A randomized experiment may depend on the random seed.

A random seed is a starting value used by a computer to generate random-looking numbers. If two people use the same seed, they can often reproduce the same random sequence.

Common mistakes

A computation becomes hard to trust when important details are missing. Examples include:

• Giving an answer without explaining the method
• Rounding too early
• Changing the input without saying so
• Using software without recording the settings
• Reporting an approximate value without saying how accurate it is

These mistakes do not always make the answer wrong, but they make the answer harder to check.

A practical habit

When presenting a computational result, write it so that another person can repeat it.

State the input. State the method. State when the method stops. State the result. If the result is approximate, state the accuracy or give a way to check the error.

This habit makes computation more reliable. It also connects computation back to proof: both require clear reasoning, visible steps, and results that can be checked.