August 08, 2008
When New Drugs Do More Harm Than Good
Princeton professor Donald Light released a new study at last week's annual meeting of the American Sciological Association that suggests new drugs are twice as likely to cause harm as provide added benefits.
"Systematic reviews indicate that one in seven new drugs is superior to existing drugs, but two in every seven new drugs result in side effects serious enough for action by the U.S. Food and Drug Administration (FDA), including black box warnings, adverse reaction warnings, or even withdrawal of the drug," Light's press release said.
Light also accused the drug industry of designing trials that minimize evidence of toxic side effects by enrolling healthier patients than those who actually take the drug once it's on the market. They systematically exclude patients who are older, poorer or who have multiple health problems, and don't run the trials long enough to detect long term side effects.
“Based on our current system, the designation of ‘safe and effective’ on today’s new drugs could be replaced with, ‘apparently safe based on incomplete information, and more effective than a placebo,’” Light said.
Before you dismiss Light because of his background in sociology, check out this study on Health Affairs' website from Shelby Reed and associates at Duke University's School of Medicine. A frequent consultant for the drug industry, Reed agrees that much larger pre-approval trials can eliminate drugs that wind up -- like Vioxx -- doing more harm than good when used in the general population.
We found that the potential to limit adverse events can be an important consideration in sample-size determinations for preapproval trials. Requiring larger preapproval databases could be a cost-effective means of reducing adverse events in postapproval populations.
But this isn't just about safety. Getting more information both pre- and post-approval is crucial to learning what works in medicine and what insurers, including Medicare, should pay for. That common sense approach is gradually being implemented through Medicare's "coverage with evidence development" program, which got an editorial boost from Stanford University's Alan Garber in the same issue of Health Affairs. A medical economist, Garber also chairs the Medicare Evidence Development and Coverage Advisory Committee. Here's what he had to say about gathering post-approval data to improve both safety and effectiveness of drugs, devices, screening tests and other costly medical interventions:
Under CED (coverage with evidence development), Medicare will, for example, pay for an implantable cardiac defibrillator only if the recipient is enrolled in a registry that collects basic clinical and health outcome data. In other instances, CED means a requirement to participate in a randomized controlled trial. It might mean simply being part of ongoing routine data collection, or a collection of EHRs (electronic health records). The FDA, at least in principle, could adopt analogous approaches to the postapproval evaluation of drugs. But payers may be in a more powerful position to ensure that the data collection actually occurs. One of the most compelling arguments for requiring more preapproval data is that the FDA has so little leverage to enforce agreements after a drug or device is already on the market. Insurers, in contrast, can deny payment for a procedure that was not performed under the terms of an agreement for CED.
Key, of course, is adopting easily transferrable electronic health records. "If EHRs can become a low-cost source of clinically detailed data, the costs of both preapproval data collection and postmarketing surveillance may decline, even for randomized trials. It may become cost-effective to collect much more data--whether pre- or postapproval--if the costs decline," he wrote.
The failure of the medical system, which consumes 16 percent of gross domestic product, to find the resources to rapidly move to electric health records can only be explained by physician and provider reluctance to have anyone peering over their collective shoulders. It's time for their footdragging to stop.
Posted by gooznews at August 8, 2008 08:05 AMIn a NEJM report some years ago, it pointed out that while newer chemotherapy regimens appear to be improving survival, when these same regimens are tested on a wider range of cancer patients, the results have been disappointing. Oncologists at a single institution may obtain a 40-50 percent response rate in a tightly controlled study, but when these same chemotherapy drugs are administered in a real world setting, the response rates decline to only 17-27 percent.
In clinical trials, many patients are excluded because they could not complete the rather arduous treatment. So randomized comparisons are of healthier treated patients against all the controls, rendering a lot of trials invalid. All the rigorous clinical trials that have been identified are the best treatments for the "average" patient. This has been referred to as the lowest common denominator theory of cancer treatment. But cancer is not an "average" disease.
And if a person in the untreated category dies at any time while he or she is being studied, this is recorded as a death in the control group and registered as a failure of the no treatment approach. However, if patients in the treated category die during the course of treatment (before the course is completed), their cases are rejected from the data since these patients do not then meet the criteria established by definition of the term treated. A patient dying on day 89 of a prescribed 90-day course of chemotherapy would be dropped from the list of treated patients.
The patients selected for trials are often healthier than others with the same disease, because it's safer to test new drugs in people without other medical problems. It's also easier to interpret the results if there aren't complicating factors that could influence the outcome. So when a drug comes to market after being tested on the healthiest subset of the intended population, it must then be dealt with by an individual physician and the individual for whom the drug treatment is intended (ergo trial-and-error medicine).
Posted by: Gregory D. Pawelski at August 9, 2008 12:03 AMI don't think that notion of "larger" clinical trials makes much sense. The history of chloramphenicol is a classic example -- one that is taught to every MPH student, every pharmacology graduate student, and almost every medical student. It goes like this: knowing what we now know about the risk of aplastic anemia from chloramphenicol(i.e., about 1 per 50,000), how many patients must be studied to be 95% likely that we will have observed this ADR? The answer is a stunning 150,000! Even for something much more familiar and common (e.g., anaphylaxis from penicillin, with a risk of about 1 per 10,000), the number that must be studied for the same 95% certainty is 30,000. I do not believe it is reasonable to ask drug companies to do studies of this magnitude without additional financial support.
Posted by: Jeffrey Lazar at August 9, 2008 09:27 AMDr Lazar's comment is correct -- but way UNDERSTATES the problem. You'd need 150,000 subjects to be 95% certain of seeing a case of aplastic anemia in the chloramphenicol-treated group, but you'd a number umpteen times greater to suspect (no less "prove") that it was due to the chloramphenicol -- rates of 1/150,000 vs 0/150,000 with placebo would never lead anyone to claim that it was due to the drug!
Furthermore, for many drugs that are intended to be used chronically (unlike chloramphenicol), it's impossible to know about LONG-TERM safety in a relatively short-term trial (no matter how large).
So while very large studies might help identify a few not-so-uncommon problems, the issue of safety simply cannot be adequately addressed without really careful post-approval surveillance (of the type we never mandate currently).
The issue of efficacy is altogether different -- it could be addressed a thousand times better than it is, by demanding "better than what we have" (rather than "better than nothing"), by making meaningful efforts to have the testing done in real-world relevant circumstances and on far more representative groups of subjects, and by demanding use of patient-oriented outcomes (rather than such things as "change in your 'good' cholesterol").
Even all of that wouldn't address the far more complex issue of BIAS in study design ... or the fundamental question of whether it makes sense to allow the entire research agenda to be set by those looking to find drugs that will make the best profit, rather than based on questions of greatest public health need.
Posted by: Jerome Hoffman at August 9, 2008 02:41 PMThanks for these comments. I should have made clear in my original post that Light's critique focused on blockbuster drugs, i.e., those destined for use in millions of people; not those used in relatively rare conditions like aplastic anemia. A part of Garber's editorial that I also should have highlighted suggested, as Dr. Hoffman did, that larger pre-approval trials should be reserved for widely used drugs, while the best approach for identifying rare side effects in sparingly used drugs would be to figure out ways to enforce "careful post-approval surveillance."
Posted by: Merrill at August 10, 2008 04:13 PMThanks again for alerting us, your readers, to issues like this. Love your blog.
Posted by: Zane Safrit at August 11, 2008 10:39 AMAs a general principle, larger trials should result in better screening of the safety and efficacy of new drugs. But recall that the trial that led to the approval of the antibiotic Ketek involved 25,000 people. That's large by conventional standards. So while size matters, so too does rigorous, arms-length monitoring to ensure that proper procedures are followed and that the results are not skewed by fraud.
Posted by: Mark Cohen at August 11, 2008 04:19 PM