Systematic Reviews, Meta-analysis & Evidence Synthesis

A stack of papers is not yet an evidence synthesis.

Scientific literature can contain supportive studies, null studies, contradictory studies, duplicated datasets, selective publication and different populations.

A systematic review uses explicit methods to find, select, appraise and synthesize that evidence.

A meta-analysis is one possible statistical component of that process.

Systematic means the method is pre-specified and inspectable

A good systematic review states:

• the question;
• eligibility criteria;
• databases and search strategy;
• screening process;
• included and excluded studies;
• risk-of-bias assessment;
• synthesis method;
• handling of heterogeneity and missing evidence.

The goal is to reduce the opportunity to select studies because their results fit the desired conclusion.

A meta-analysis combines estimates, not truth

Meta-analysis can produce a pooled estimate from multiple studies.

That can improve precision and reveal patterns that individual studies cannot estimate well.

But the pooled number inherits the assumptions and limitations of the input studies.

If included studies are biased, indirect or poorly measured, averaging them does not remove those problems.

More studies do not automatically mean stronger evidence

Ten small studies with similar bias are not necessarily stronger than two large, rigorous studies.

Likewise, twenty publications may derive from only a few underlying datasets.

Evidence synthesis therefore evaluates:

• independence;
• study design;
• sample size;
• measurement quality;
• risk of bias;
• directness;
• consistency;
• publication bias.

Counting PDFs is not enough.

Heterogeneity is information

Studies rarely estimate exactly the same effect.

Differences can arise from:

• populations;
• doses;
• interventions;
• outcomes;
• follow-up;
• implementation;
• random variation.

A pooled estimate can be misleading if important heterogeneity is ignored.

Sometimes the central question becomes not “what is the average effect?” but why do results differ?

Fixed and random effects answer different models

Meta-analytic models make assumptions about how study effects relate.

A fixed-effect model treats included studies as estimating one common effect under its model.

Random-effects models allow true effects to vary across studies.

Neither model solves poor study quality or inappropriate pooling.

The choice should follow the scientific question and heterogeneity, not whichever produces a preferred p-value.

Missing studies can distort synthesis

Positive or striking results may be more likely to be published or easier to find.

Selective outcome reporting can also hide unfavorable results within published studies.

Systematic reviewers therefore look for:

• trial registries;
• protocols;
• unpublished results;
• publication-bias patterns;
• missing outcomes.

The absence of visible negative evidence may itself be part of the uncertainty.

PRISMA improves reporting, not validity by itself

PRISMA helps authors report systematic reviews transparently.

Following a reporting checklist does not guarantee that every decision was correct.

Likewise, the label “systematic review” should not be treated as an automatic quality badge.

The methods still need appraisal.

The Claim by Claim use

When several studies exist, we prefer high-quality evidence synthesis where it directly matches the claim.

But we still inspect:

1. what studies were included;
2. what outcomes they measured;
3. whether populations and exposures match;
4. risk of bias;
5. heterogeneity;
6. whether the pooled result is practically meaningful.

Synthesis is strongest when the question stays visible

A meta-analysis should answer a defined question.

It should not turn different populations, outcomes and interventions into one impressive number simply because they can be entered into the same software.