Models, Assumptions & Sensitivity

Every model is a simplified representation of something more complicated.

That is not a defect.

Without simplification, a model would be as complicated as the world it is meant to help us understand.

The important question is whether the simplifications are appropriate for the claim being made.

Models turn assumptions into outputs

A model may specify:

• relationships between variables;
• parameter values;
• boundary conditions;
• distributions;
• causal structure;
• behavioral rules;
• measurement error;
• missing processes.

Some assumptions are strongly supported.

Others are pragmatic approximations.

The output is conditional on them.

A precise output can come from uncertain inputs

Computer models can produce many decimal places.

That numerical precision does not imply that the underlying assumptions are known equally precisely.

An estimate can be computationally exact under the model and still scientifically uncertain because:

• parameters are uncertain;
• structure is incomplete;
• data are sparse;
• key mechanisms are simplified.

This is why model output should be interpreted with uncertainty, not visual precision alone.

Sensitivity analysis asks what drives the result

A sensitivity analysis changes inputs or assumptions and observes how the output changes.

Useful questions include:

• Which parameters matter most?
• Does the conclusion survive reasonable alternatives?
• Is one uncertain assumption carrying the result?
• Are interactions between inputs important?
• Which uncertainties need better data?

A robust conclusion should not collapse under small, plausible changes unless the system itself is genuinely sensitive.

Sensitivity analysis can also be done badly

Research reviewing published sensitivity analyses has found substantial methodological weaknesses.

Changing one parameter at a time over an arbitrary range may miss interactions.

Testing only convenient assumptions can create false reassurance.

A sensitivity analysis should cover scientifically plausible uncertainty and match the decision or inference being evaluated.

Model validation is not one number

A model can be evaluated against:

• observed data;
• out-of-sample prediction;
• known physical constraints;
• historical cases;
• calibration targets;
• external datasets.

Success on one dimension does not guarantee validity for every purpose.

A weather model can be useful for one timescale and poor for another.

A biological model can explain mechanism without predicting clinical outcomes accurately.

All models have a domain

A model’s conclusions apply to the conditions represented.

Extrapolating outside that domain requires additional justification.

This is especially important when viral claims use outputs from simulations as though they were direct observations.

Models are evidence, but of a particular kind.

Assumptions should be inspectable

A trustworthy model-based claim should make clear:

• what is assumed;
• which parameters are estimated;
• where data come from;
• what uncertainty is propagated;
• what validation exists;
• how sensitive the conclusion is.

Hidden assumptions make disagreement difficult to diagnose.

The right question is conditional

Instead of asking:

Is the model true?

ask:

Given these assumptions and data, how well does the model answer this question, and does the conclusion survive reasonable alternatives?

That is a much stronger way to use models scientifically.