What Is a Meta-Analysis?

When reading research on a topic, you may notice that the studies disagree. One study might report a clear benefit from a treatment, while another finds only a small effect. A third may not observe much of a difference at all.

That can make the overall evidence difficult to judge.

A meta-analysis is one way researchers address the issue.It combines the results of a number of studies that have asked a similar question, and uses statistical techniques to work out what the results show when taken together.

Sounds simple enough, but it’s a bit more than just putting a bunch of numbers in a spreadsheet and averaging them out. Studies should be related to the question, and researchers should consider how they are alike and how they are different.

How does a meta-analysis work?

It normally starts with a clear research question. Researchers then look for studies that have investigated that question and decide which ones are suitable for inclusion.

Take a clinical treatment as an example. Perhaps five different trials have looked at whether the treatment helps people with the same condition. The trials may have different numbers of participants and may not have produced identical results.

Researchers compile the relevant results from each study into a comparable format. They can then use statistical methods to produce a combined result, often called a pooled estimate.

The studies do not necessarily have an equal influence on that final figure. A large study with more precise results may have a greater impact than a small study. The method used to calculate the final figure depends on the type of evidence and the assumptions made by the researchers.

Is a meta-analysis the same as a systematic review?

Not quite. The two terms often appear together, which is probably why they are easily confused.

A systematic review is a structured way of finding and assessing research that relates to a particular question. It looks at the available evidence as a whole.

The statistical component of a meta-analysis combines suitable numerical results from various studies.

A systematic review can include a meta-analysis, but it does not always do so. If the studies are too different to sensibly combine, researchers may simply describe and compare their findings instead.

Why do researchers combine studies?

There is a practical reason for doing so. An individual study may not tell us very much on its own.

For instance, a small trial might suggest that a treatment works, but studies with a larger number of participants are more likely to be published than those providing a very precise estimate of no effect, and studies where the effect was large enough to provide a very precise estimate. Other studies might show slightly different results. Looking at several studies together can provide researchers a better idea of the size of the effect.

That does not mean combining studies automatically produces a better answer. If the original studies have serious weaknesses, those weaknesses still exist. A meta-analysis cannot fix poor research simply by putting the results together.

What does heterogeneity mean?

One word you are likely to come across when reading a meta-analysis is heterogeneity.

In simple terms, it means that the studies are different from one another. They may involve different groups of people, use different treatments or doses, measure outcomes in different ways, or follow participants for different lengths of time.

Some variation is normal. The important question is whether the differences are large enough to affect the usefulness of the combined result.

Researchers can use statistical measures to examine this variation. I² is a common measure that indicates how much of the variation between results may be due to differences between studies.

If there is substantial heterogeneity, the pooled result needs a bit more thought. Researchers may look at why the studies differ rather than simply focusing on the final number.

Can a meta-analysis provide a misleading impression?

It can.

The quality of the included studies poses a significant issue. If several studies have problems with their design or a high risk of bias, combining their results does not make those problems disappear.

There is also publication bias to consider. Studies with positive or statistically significant results may be more likely to be published than studies where little or no effect was found. If some relevant research is missing, the meta-analysis may not provide a complete picture.

Another thing to watch is whether the studies actually belong together. Two studies may appear to investigate the same general topic but differ considerably in their participants, methods or outcomes. A statistical programme can still calculate a combined result, but that does not necessarily mean that combining the studies was sensible.

How can you read a meta-analysis?

Start with the question the researchers were trying to answer. Then look at what kinds of studies were included and who took part in them.

It is also useful to look beyond the final pooled result. Do the individual studies generally point in the same direction, or are their findings entirely unique?

The confidence interval is also worth checking. This provides an indication of the uncertainty around the estimated effect. A wide interval suggests that there is more uncertainty about the size of the effect, while a narrower one indicates a more precise estimate.

Finally, read the limitations. They are easy to skip, particularly when you are mainly interested in the headline finding, but they can tell you a lot about how much confidence you should place in the result.

The simple version

A meta-analysis is a statistical way of bringing together results from several studies that address a similar question.

It can be useful because one study usually provides us an incomplete picture. Looking at several studies together may provide a clearer estimate of an effect and show whether the findings are broadly consistent.

However, the final number does not provide a complete understanding. You still need to consider the quality of the research, differences between the studies, possible publication bias and the methods used to combine the results.

Therefore, rather than thinking of a meta-analysis as a shortcut to a definite answer, it is better to see it as a way of making sense of a body of research. The value of the result depends on the evidence that went into it and how carefully that evidence was analysed.