Almost everyone who practices non western medicine will have an opinion about the role of drug companies in the proliferation of questionable medicines. One of the primary mechanisms used by pharma to deliver their message to the public and to physicians is through the media of advertising, on television in the states and in magazines, journals and on the web in the UK.
So, when looking at the data presented it needs to be seen n the context of reality, not in the context of simple prescriptive persuasion.
Here are the most common ruses used:
The mean methodological quality score of studies cited in journal advertisements is poor.
Primarily because the references used are from non-systematic narrative review articles and ‘other’ sources. Of 102 references in 31 advertisements in the BMJ, only 55 referenced articles in peer reviewed journals and of these, 13 were to sponsored symposiums published in the journal or a journal supplement.[1] They also refer to studies with methodological problems which diminish their validity in reference to the purported outcomes.[2]
Many adverts do not use appropriate confidence intervals.
In common usage, a claim to 95% confidence in something is normally taken as indicating virtual certainty. In statistics, a claim to 95% confidence simply means that the researcher has seen something occur that only happens one time in 20 or less.
If one were to roll two dice and get double six (which happens 1/36th of the time, or about 3%), few would claim this as proof that the dice were fixed, although statistically speaking one could have 97% confidence that they were. Similarly, the finding of a statistical link at 95% confidence is not proof, or even very good evidence, that there is any real connection between the things linked.
Graphs mislead
This type of visual provides a quick interpretation of the data and can be a compelling mechanism for efficacy assessment, but an analysis of 74 graphs that appeared in 63 advertisements in leading United States (US) medical journals found them seriously wanting. Of these, 64% did not contain sufficient information to interpret the results; 47% did not have a figure legend and 31% did not depict sample size. In addition, 36% had distorted the numerical data that lead to visual overestimation or underestimation of the outcome being shown in the graph. Of the 27 graphs that were reliant on a section of the data included in the advert the figures chosen were biased in favour of the drug’s effect 70% of the time and against its effect 0% of the time.[3]
Outcome selection is chosen for best presentation
The inclination to recommend a drug is found to be at its highest when the data is presented as RRR and least successful when presented as NNT, so look for the NNT in the adverts to see how many people need to take the medication to prevent 1 adverse effect.
- Relative Risk Reduction (RRR). RRR is the percentage reduction in the risk of targeted complications between two groups: a drop in mortality from 50% to 25% would be an RRR of 50%.
- Absolute Risk Reduction (ARR). ARR is the absolute percentage difference in the risk of targeted complications between two groups: a drop in mortality from 50% to 25% would be an ARR of 25%
- Number Needed to Treat (NNT). NNT is the number of patients that have to be treated to prevent one complication of their disease: a drop in mortality from 50% to 25% would be an NNT of 4 (100/ARR) Or in simple language:
“Out of every 100 people who take this drug, perhaps 4 heart attacks will be prevented, and the other 96 people will receive no benefit,”
For Example
Upon first glance, the Lipitor (Atorvastatin (INN) (Lipitor, Pfizer), is a member of the drug class known as statins) advert above boasts that the drug reduces heart attacks by 36%. But follow the asterisk, you find the following in much smaller type:
“That means in a large clinical study, 3% of patients taking a sugar pill or placebo had a heart attack compared to 2% of patients taking Lipitor.”
What this means is that for every 100 people who took the drug over 3.3 years, three people on placebos, and two people on Lipitor, had heart attacks. That means that taking Lipitor resulted in just one fewer heart attack per 100 people.
The NNT, in this case, is 100. 100 people have to take Lipitor for more than three years to prevent one heart attack. And the other 99 people, well, they increased their risk of side effects for nothing.
Comment
When reading adverts or looking at drug efficacy Vs risk (side effects) the following questions should be asked:
- Are results presented as either ARRs or, preferably, NNTs?
- Are confidence intervals used appropriately?
- Is the power of any statistics that are cited discussed?
- Does cited literature come from the main journal or from symposia and/or journal supplements?
- Do graphs properly present the data?
- Is hard numerical data given to support claims or are vague and/or emotive statements used?
Side effect linked to natural medicines, espescially food supplements are virtually unknown, and whilst they do not receive the same level of investigation that pharmaceuticals do – for many reasons their benefit to risk falls haevily on the benefit side.
References
[1] Cho MK, Bero LA. The quality of drug studies published in symposium proceedings. Ann Intern Med 1996; 124: 485–9. View Full Paper
[2] Mindell J, Kemp T. Only two fifiths of advertisements cited published, peer reviewed references. BMJ 1997; 315: 1622 View Full Paper
[3] Cooper RJ, Schriger DL, Wallace RC et al. The quantity and quality of scientific graphs in pharmaceutical advertisements. J Gen Intern Med 2003; 18: 294–7. View Full Paper
