I mentioned in the last post a recent proposal by Kent et al. that would improve clinical trial reporting and—in the authors’ words—lead to “actionable” clinical trial results. The proposal addresses “heterogeneity of treatment effect (HTE)” and the relationship of that effect to the baseline risk of clinical trial enrollees.
HTE is the technical term used to reflect the fact that clinical trials (other than N-of-1 trials) are designed to reach a conclusion about an overall treatment effect, usually reported as relative or absolute risk reduction for the treated population when compared to the control group. Clinical trial results tell nothing specific about an individual subject’s response to treatment.
Clearly, not everyone enrolled in the treatment arm of a given clinical trial benefits from (or is harmed by) the intervention. Attempts to sort out who benefits most or least is usually done by dividing the trial population into subgroups, one characteristic at a time (eg. male vs female, old vs. young, hypertensive vs. normotensive, etc.), and by performing a statistical analysis on each subgroup. Traditional subgroup analyses, however, are fraught with many well known limitations and are not practically useful to help identify those likely to benefit.
In this proposal, Kent et al. begin by making explicit the intuitively evident fact that if one assumes an unchanging relative risk reduction across the board, patients at high baseline risk of developing the outcome will benefit more than those at low baseline risk in absolute terms (table 1 in the paper). The greater the baseline spread in risk, the greater the spread in benefit. If harm is included in the analysis, low baseline risk patients may have no demonstrable benefit and may even be harmed (table 2 in the paper).
The authors then make several recommendations. Chief among them is to tabulate the distribution of baseline risk in each treatment group using some pre-established baseline risk calculator that includes multiple baseline variables (Recommendation #1). The next recommendation, as anticipated, is to report on the absolute and relative risk reduction according to this baseline risk categorization (Recommendation #2).
The third recommendation is to perform single-variable subgroup analysis on pre-specified variables that have “strong a priori pathophysiological or empirical justification,” ie. not to go on a subgroup fishing expedition. The number of pre-specified single-variable subgroup analyses should be limited to avoid the “multiple comparison” trap. Cut-off threshold for baseline single variables should also be pre-specified before the analysis to avoid fishing.
The fourth recommendation is to designate any other single-variable subgroup analysis (non-pre-specified) as exploratory only.
The fifth recommendation is to subject all subgroup analyses to additional testing of heterogeneity (ie. did treatment effect vary between subgroups?) by using a proposed statistical tool (‘interaction test’), and to be mindful of a number of additional statistical caveats.
Will these refinements improve the clinical utility of clinical trials? It seems that it would, but 3 points come to mind:
- From a public health standpoint, dealing with heterogenity of treatment effect goes against Geoffrey Rose’s theory for preventive medicine, the dominant paradigm in public health for the last 30 years. Rose’s argued that more burden of disease is explained by many people with low risk than by small groups with large risk, ergo we must treat everyone. This “sick population” approach to prevention is still hailed as a major advancement in public health…
- Drug companies are unlikely to be keen on refining clinical trial results in a way that would reduce “the market size.”
- Most importantly, the proposal begs the question: if we are so good at characterizing baseline risk, why include low risk groups in the trial in the first place? (see 1. and 2. )
Affaire à suivre. All in all, this seems all right to me. But needless to say, the expected “actionable” outcome from this proposal is more secure employment for the statistician rather than better care for the patient at hand…