Burden of Proof & Evidential Thresholds

Evidence does not begin from the assumption that every imaginable explanation is equally likely.

The person making an empirical claim carries a practical burden of proof: there should be positive evidence capable of distinguishing that claim from relevant alternatives.

This is not a courtroom rule imported into science. It is a consequence of limited evidence and many possible explanations.

Possibility is cheap

For almost any unexplained observation, many possibilities can be invented.

A symptom could be caused by a new exposure. A light in the sky could be a new phenomenon. A statistical pattern could reflect a hidden mechanism.

The fact that an explanation is possible does not tell us how probable it is.

Evidence has to discriminate among alternatives.

Lack of disproof is not positive support

Suppose no study has ruled out a very small effect.

That does not establish the effect.

Likewise:

Nobody has proved this cannot happen.

is weaker than:

We have observed the predicted effect under conditions that distinguish this explanation from alternatives.

This is why the Reasoning Error Absence-of-Evidence Inversion matters.

HSW adds the methodological side: what kind of evidence would actually carry the claim?

Evidential thresholds depend on the conclusion

Different conclusions require different amounts and types of support.

Examples:

• “This mechanism is physically possible” requires less than “this mechanism produces a meaningful real-world effect.”
• “One study observed an association” requires less than “this exposure causes disease.”
• “A treatment may help” requires less than “this treatment is proven effective for most patients.”
• “We cannot exclude a small risk” is different from “the product is dangerous.”

The stronger, more general or more consequential the claim, the more evidence is needed to justify it.

There is no universal magic threshold

Science does not contain one number that turns uncertainty into truth.

A p-value below 0.05 is not a universal proof threshold.

One randomized trial is not always sufficient.

A meta-analysis is not automatically decisive.

The evidential threshold depends on design quality, prior knowledge, effect size, directness, consistency, potential bias and the costs of being wrong.

Frameworks such as GRADE make some of these judgments explicit instead of pretending they disappear.

Evidence and decisions are not identical

A policy or clinical decision may require action before scientific uncertainty disappears.

That means two questions should remain separate:

1. How certain are we about the evidence?
2. What should we do given the consequences, values and risks?

GRADE explicitly separates certainty of evidence from strength of recommendations for this reason.

Claim by Claim is primarily concerned with the first question when assigning an evidence status.

Extraordinary does not mean impossible

The phrase “extraordinary claims require extraordinary evidence” is often used loosely.

A better formulation is:

claims that conflict strongly with well-supported background knowledge, imply large effects, or require multiple established mechanisms to be wrong need evidence strong enough to overcome those competing explanations.

That is not institutional conservatism.

It is comparative inference.

How to set the threshold

Before looking at the result, ask:

• What exactly is being claimed?
• How large is the claimed effect?
• What alternative explanations exist?
• What kind of design could distinguish them?
• How much measurement error is plausible?
• How costly would a false positive or false negative be?
• Is the conclusion descriptive, causal, predictive or prescriptive?

These questions make the evidential standard visible.

The core rule

A claim should be supported because the evidence favors it over relevant alternatives — not because nobody has managed to disprove every conceivable version of it.