Previous - Historical Background Next - Definitions
3.2 Hierarchy of evidence:
For many years scientists have recognised two types of research:
In biomedical science there is general agreement over a hierarchy: the higher up a methodology is ranked, the more robust and closer to objective truth it is assumed to be. The orthodox hierarchy looks something like table 1:
|
Rank: |
Methodology |
Description |
|
1 |
Systematic reviews and meta-analyses |
Systematic review: review of a body of data that uses explicit methods to locate primary studies, and explicit criteria to assess their quality. Meta-analysis: A statistical analysis that combines or integrates the results of several independent clinical trials considered by the analyst to be "combinable" usually to the level of re-analysing the original data, also sometimes called: pooling, quantitative synthesis. Both are sometimes called "overviews." |
|
2 |
Randomised controlled trials (finer distinctions may be drawn within this group based on statistical parameters like the confidence intervals) |
Individuals are randomly allocated to a control group and a group who receive a specific intervention. Otherwise the two groups are identical for any significant variables. They are followed up for specific end points. |
|
3 |
Cohort studies |
Groups of people are selected on the basis of their exposure to a particular agent and followed up for specific outcomes. |
|
4 |
Case-control studies |
"Cases" with the condition are matched with "controls" without, and a retrospective analysis used to look for differences between the two groups. |
|
5 |
Cross sectional surveys |
Survey or interview of a sample of the population of interest at one point in time |
|
6 |
Case reports. |
A report based on a single patient or subject; sometimes collected together into a short series |
|
7 |
Expert opinion |
A consensus of experience from the good and the great. |
|
8 |
Anecdotal |
Something a bloke told you after a meeting or in the bar. |
You may note that this ranking has an evolutionary order, moving from simple observational methods at the bottom through to increasingly sophisticated and statistically refined methodologies. To some extent, the physical sciences are consistent: you can repeatedly measure the acceleration due to gravity, in different places and with different techniques, but it will always come out close to a constant value. But, biological systems are multivariate and less predictable: if you looked at the effect of dietary supplements on growth in different groups even from the same species, you would get very mixed results. Social phenomena are even more complex and difficult to isolate. Many biomedical discoveries have had to await the development of techniques capable of dealing with populations in a statistical manner. For instance:
James Lind hit lucky with his work on scurvy: he picked a disease that was nearly universal in sailors on long voyages, but where practically all cases could be cured by citrus fruit juice, or more accurately, the vitamin C it contained. Such discoveries were often empirical, and pre-dated the theories that would back them up. Another 150 years passed before vitamins were isolated and described, but that did not stop mariners protecting themselves from scurvy with lemon and lime juice!
Finally, you should remember two points in relation to this hierarchy:
1. Techniques which are lower down the ranking are not always superfluous. For instance, if you wanted to study risk factors for an illness, any ethics committee would quite rightly reject an RCT whereby you proposed exposing half your subjects to some noxious agent! Here, you need a cohort, where you find a group exposed to the agent by chance or their own choice and compare how they fare with another group who were not exposed. As we shall see later in the discussion of qualitative and quantitative methods, it is not that one method is better: for a given type of question there will be an appropriate method.
2. This hierarchy is not fixed in tablets of stone: the ranking may change, and there is debate over the relative positions of systematic reviews and large RCTs. The future may also see changes in the hierarchy of scientific evidence. Traditionally, the randomised controlled trial has been regarded as the most objective method of removing bias and producing comparable groups. But, the technique is often slow, expensive and produces results that are difficult to apply to messier, multivariate "real situations." In the future, expanding computer capacity may make observational studies with multiple variables possible.
In some ways, the wheel has come full circle. Printing presses with moveable type arrived in Europe in the fifteenth century, and provided the technology necessary to produce reading material quickly and cheaply. It was another two centuries before the rise of scientific societies and an interest in both performing and disseminating experimental work generated a need for technical periodicals. Now, three hundred years later, the rate of technological change, the arrival of the electronic database and the Internet, the volume of research and demands for services like healthcare to be justified by research evidence are driving the evolution of the scientific press into the twenty-first century.
|
Reflective Point: What has been your experience of using scientific literature so far?
Which aspects of the present system of journal publication, indexing and abstracting are good and which could be improved on?
|
Self Assessment Questionnaire
What options does a scientific editor have to safeguard the quality of his journal? How effective is each of the various options?
Next - Definitions