In August 2010, a group of eminent statisticians and clinical trial specialists (Drs. Kent, Rothwell, Ioannidis, Altman, and Hayward) published online a proposal to deal with a major short-coming of clinical trials, the so-called heterogeneity of treatment effect (HTE). The authors offer a refinement in the way clinical trial results are analyzed and reported.
Perhaps sensing mounting dissatisfaction with the blunt tool of ‘evidence-based medicine,’ and perhaps in an attempt to dismiss any further rebuff, Kent et al. introduce their proposal by characterizing the critics of EBM as mere Luddites:
When the Scottish epidemiologist Archie Cochrane suggested that clinical practice should principally be guided by rigorously designed evaluations, in particular randomized clinical trials (RCTs), the reaction of the medical profession was largely negative. Critics suggested that relying on impersonal statistically-derived “evidence” based on averages to determine clinical decision-making was antithetical to the practice of medicine, which should rather be based on a physician’s expertise, acumen and clinical experience, and on knowing the individual patient and considering what is best for each person given their individual circumstances and needs [1-3].
One of the critics cited is no less than Alvan Feinstein, hardly a defender of the notion that “clinical decision-making…should rather be based on a physician’s expertise.” But the authors push their straw man argumentation further:
Although “evidence-based medicine” has become the dominant paradigm for shaping clinical recommendations and guidelines, recent work demonstrates that many clinicians’ initial concerns about “evidence-based medicine” come from the very real incongruence between the overall effects of a treatment in a study population (the summary result of a clinical trial) and deciding what treatment is best for an individual patient given their specific condition, needs and desires (the task of the good clinician)[4–7]. The answer, however, is not to accept clinician or expert opinion as a replacement for scientific evidence for estimating a treatment’s efficacy and safety, but to better understand how the effectiveness and safety of a treatment varies across the patient population (referred to as heterogeneity of treatment effect [HTE]) so as to make optimal decisions for each patient.
Of course, the criticism leveled against EBM is not that clinician opinion should be a “replacement for scientific evidence,” but rather that clinical opinion necessarily has primacy when it comes to applying medical knowledge to individual patients with unique circumstances. The authors may argue all they want about “scientific justification,” EBM recommendations can only apply to hypothetical, “average” patients.
*****************************
The rather imprecise term “evidence-based medicine” has clearly lost its glow. I remember the great enthusiasm when the User’s Guide series was published in JAMA in the 1990’s. Gordon Guyatt and his acolytes at McMaster were ostensibly introducing a”teaching method.” It wasn’t long before the teaching method evolved into its current role as adjudication tool for central planners. (Of note, and perhaps unsurprisingly, Guyatt was the founder in 1979 of the Canadian Medical Reform Group, which views the Canadian healthcare system as not universal enough. He has been running for office in Canada as a member of the New Democratic Party).
It was also not too long ago that, at the annual meetings of the American Heart Association, American College of Cardiology, or similar organizations, fervently awaited “Late-Breaking Clinical Trial” sessions were attended by huge crowds and generated excitement typically seen at rock concerts. In the last couple of years, the anemic turnouts have been subsidized by heavily discounted entrance fees.
Then, in 2005, maverick researcher John Ioannidis made a splash by publishing a paper in PLoS mathematically proving that most published research claims will be refuted by subsequent research! The paper is not for the neophyte, and was somewhat rebuked by Steven Goodman, but still, not auspicious for the brave new world EBM claimed to lead us to!
So what of this proposal to improve RCTs to which Ioannidis himself has added his name? We’ll broach the specifics in a separate post.