The use of Randomised Clinical Trials are generally heralded by medics as being the gold standard for clinical assessment of validity of use for medicines and are frequently held out by skeptics as the intellectual anvil on which to pound complementary and alternative medicine.
There has over the last few years been a dedicated strategy by individuals and groups opposed to natural medicine to raise questions concerning the validity of research conducted to identify efficacy of natural therapies and in part this has been undertaken by applying the same rules of engagement for those required or favoured by pharmaceutical objectives.
Clinical decisions should, as far as possible and I feel we would all agree, be evidence based so runs the current clinical dogma.[1] To confirm that evidence is being rightly placed we are urged to lump all the relevant randomised controlled trials into one giant metaanalysis and come out with a combined odds ratio for all decisions.
Yet we, as do all practitioners, deal with individuals in all their complexity and magnificence. This means as Sir Michael Rawlins, chair of NICE says – follow the NICE guidelines for each condition a patient presents with – there a very few that have only one! – and they will end up in more of a mess.
“Hierarchies attempt to replace judgement with an oversimplistic, pseudo-quantative, assessment of the quality of the available evidence.”[2]
But what proportion of current commonly used ‘medical treatments’, against which we are commonly compared are supported by good evidence and how big are the knowledge gaps in westernised medicine? According to the now defunct Clinical Evidence web site run by the BMJ of around 2,500 treatment covered just:
- 13% were rated as beneficial,
- 23% likely to be beneficial,
- 8% as a trade off between benefit and harm,
- 4% ineffective or harmful
- and a whopping 46% as having unknown effectiveness.
This as they say suggests that the research community has a large task ahead and that most decisions about treatments still rest on the individual judgments of clinicians and patients.
This number roughly equates to the number quoted by Dr Alan Roses world wide Vice President of genetics at GlaxoSmithKline when he stated in 2003 that more than 90% of drugs only work in 30-50% of patients identifying the reality of efficacy. [3]
We, on our side of this apparent fence should consider that we only have to do better than 50% in 90% of our treatments and we would match the results derived from billions of pounds of investment over many decades of development to be regarded as leaders in medicine.
What about Meta Analysis
Meta-analysis is a mechanism whereby all data from available studies are combined, regardless of the relative quality of the data. This is used to gain maximum statistical information from studies that may have small sample sizes or results of marginal significance. They are merged and analysed for similar or absent correlations. These are often used to bludgeon natural therapies as being ineffective.
This relatively new method of rehashing other people work and reinterpreting the results means that someone could get published without doing any real investigative work and if it features a popular dietary supplement he would most likely get some media exposure as well. An effect a bit like the bankers must have felt when they first discovered how to make money using other peoples!
Data manipulation like this is flawed, you cannot make bad data good by mixing it together – in fact the opposite is mostly what happens, in part because there is no unifying weighting system to the data weak studies are pooled with strong and regarded as the same in terms of data value.
Genes & Environment
The genetic and environmental backgrounds of the individuals being evaluated make a huge difference. These are not simply systematic errors to be averaged out, they are essential components of the way nutrients, or natural agents react with the body. The result may be that they average out the very differences that determine effect such as in the case of the antioxidant meta analysis that suggests increased risk of death.
Are Skeptics The New Manicheans?
Many commentators on this evidence are prone to falling into the dogma first alliterated by the prophet Mani and later proselytised by followers of his religious theology called Manichean’s in which life and the body for simplicity, are defined by good and bad (evil). This opposing and highly frictional dichotomy was defined as dualism. Some of this theological dogma is still found today in skeptical analysis, especially in their fundamentalist forms. This deterministic approach, seeing everything in black and white rather than shades of grey, is known by philosophers as the Manichean fallacy.
Medical Journal Adverts
Peer-reviewed medical journals are generally considered to be a source of unbiased and reliable information about drugs (albeit that this concept is under considerable scrutiny). But at the same time, most medical journals contain advertisements, almost all of which are for drugs, and which are, by their very nature, biased toward promoting sales of that drug.
Although physicians believe that they prescribe based on impartial evidence, advertising has been shown to increase prescriptions for targeted drugs in a dose-related manner even when other promotional efforts are held constant.[4]
A Dutch Journal published yet another article, just one in a long line of articles published in many journals showing how adverts in medical journals use dubious data.[5]
The researchers went through all the biggest medical journals in the world – the Journal of the American Medical Association, the Lancet, the New England Journal of Medicine, etc. between 2003 and 2005. All adverts were included, once each, if they made a claim about the effect of a drug. For all the claims in the ads, they checked the references, found the trials they referred to and then ran a comparison assessment.
The results were abysmal. Only half of the claims in the adverts were actually supported by the specific trials they referenced, and of all the trials, only 55% got a score of “high quality”: overall, only 39.2% of these adverts – in the leading medical journals in the world – referenced a high-quality trial which actually supported their claim.
Comment
Much is made about RCT being the gold standard in medical care and it certainly has its uses, as does Meta Analysis – but they like many other forms of research are not infallible and rarely provide the clinical direction for a patient presenting with multiple problems. Here the idea of a filter based in the head of the practitioner takes over, depending on his or her skills and training this may be unilateral or may be very diverse, the Functional Medicine matrix of assessment allows for a strategic gathering of data and translation into various categories of application that ultimately relies on a working and sharing relationship with the patient/client. Whilst it is very true that this model has considerable weaknesses in the emergency room presentation, it lends a fantastic opportunity for the morbidly chronic, or complex patient with non acute illness and loss of function.
References
[1] Keus F, Wetterslev J, Gluud C, van Laarhoven CJ. Evidence at a glance: error matrix approach for overviewing available evidence. BMC Med Res Methodol. 2010 Oct 1;10:90. View Paper
[2] Harevian Oration, Royal College of Physicians 2008 View Summary
[3] The Drugs don’t work Life Science Journal 2004
[4] Wang TJ, Ausiello JC, Stafford RS. Trends in antihypertensive drug advertising, 1985–1996. Circulation 99: (1999) 2055–2057. View Abstract
[5] Heimans L, van Hylckama Vlieg A, Dekker FW. Are claims of advertisements in medical journals supported by RCTs?. Neth J Med. 2010 Jan;68(1):46-9. View Full Paper
