Risky decisions

I had a sort of epiphany about risk and medical decision-making last Thursday evening, even though I was particularly tired and in a state of mental fog.

I had been thinking about two recent articles dealing with risk prediction, but I also was preoccupied with risky career decisions I have made, and how these are affecting me and my family (I am pursuing a quixotic version of the “triple threat:” independent physician, apprentice schoolman, blogger).  So risk was very much on my mind.

One of the articles I had read was a recent viewpoint in JAMA by Allan Sniderman, Ralph d’Agostino, and Michael Pencina titled “The role of the physician in the era of predictive analytics.”  The other was a response to that article written by Bill Gardner at the Incidental Economist blog.

In their paper, Sniderman et al. discuss an important difficulty in the medical science of risk prediction.  The difficulty is apparent when one contrasts the notion of population risk with the notion of individual risk.

Population risk is a simple frequency of events and can be estimated empirically.  For example, 10% of people with a blood cholesterol level of x will die of a heart attack in 10 years.

Individual risk, on the other hand, is a problematic idea.  One is tempted to tell Mrs. Jones, whose blood cholesterol level is x, that she has a 10% risk of dying of a heart attack in the next 10 years.  But what does that mean?  As Sniderman et al. note:

Contrary to what is thought, this [10%] risk level is not that person’s personal risk because probability is not meaningful in an individual context (emphasis mine).

This is not a new idea.  Probability theorist Richard von Mises said as much in 1957:

We can say nothing about the probability of death of an individual even if we know his condition of life and health in detail. The phrase ‘probability of death’, when it refers to a single person, has no meaning at all for us.

Personal lives are not repeatable observations.  According to von Mises, no repeatable Mrs. Jones, no “individual risk” for Mrs. Jones.

And if the notion of individual risk “has no meaning at all,” then the whole enterprise of guideline medicine (including its pay-for-performance component) is also highly problematic, since it looks to group risk as proxy for individual risk.  Furthermore, individual risk is what we are supposed to discuss with patients when we obtain informed consent or engage in shared decision-making, the new examplar of patient-centered care.

Sniderman et al. recognize the insecure character of individual risk, but they are not so categorical as von Mises in denouncing it.  To salvage the concept—and with it, the clinical guidelines enterprise—they meet von Mises only halfway: individual risk is not meaningful, but that’s only because our knowledge is limited.

Thus they boldly propose that:

Hypothetically, if all past, present, and future predictors and processes that contribute to future events were known and quantifiable, algorithms could be constructed that produce perfect risk estimates for individuals—that is, they would predict with perfect accuracy whether an event would occur or not in every individual.

Although they don’t elaborate further on this hypothetical, Sniderman et al. seem to hark back to old school determinism (Laplace’s demon comes to mind), a position that is rather difficult to defend nowadays.  At the very least, this position is controversial enough that to slip it into an argument as casually as they do seems intellectually careless.

But the JAMA authors could have disagreed with von Mises without explicitly embracing nineteenth century determinism.  They could have said that a probability does not have to always represent a proportion of events from a uniform sequence of observation.   It can also represent a degree of belief, according to a “subjectivist” or Bayesian school of thought on probability theory.¹

That is precisely the position taken by Bill Gardner in his response to that JAMA article.  In the blog post, Gardner mostly agrees with Sniderman et al.’s defense of guideline medicine, but rebukes them for “badly going off the rails” in rejecting the notion of individual risk.  On the contrary, Gardner argues,

…it’s perfectly rational to use probabilities to express a consistent set of beliefs about the likelihoods of individual future events. Otherwise, saying “this patient will probably survive the operation” would be a logical absurdity.

Now, I have long taken the same position as von Mises regarding the notion of individual risk, i.e., that it is meaningless under any circumstances.  But I must admit that I have also struggled to reconcile my position with what I seem to be doing on a day-to-day basis when I talk to patients and advise them on the best course of action.

After all, I’m as likely as anyone else to say something like “well, Susan, the risk of x is not so high compared to doing y, and therefore I recommend…”  Am I engaging in a sort of mental reservation when I do this, speaking in generality about group risk, but knowing full well that Susan must be thinking instead about her “personal risk?” Am I being hypocritical?  Or am I a subjectivist in spite of myself?

Gardner’s statement must have hit a nerve, because I felt compelled to Tweet him a retort:

@IncidentalEcon@Bill_Gardner “saying ‘this patient will probably survive…would be a logical absurdity” or reflect limits of language…

— Michel Accad MD (@michelaccad) July 9, 2015

But retreating into a linguistic argument was not the most compelling point I could make, and I knew it.

So how did I resolve the mental tension I was experiencing about this tricky subject last Thursday?

The details are fuzzy but I remember thinking that the solution would not come from getting into the nitty-gritty details of the dispute between frequentistist and Bayesian theorists, as if the distinction between the two camps were clear cut (it’s not).  Anyone who dares to read about probability theory will soon enough realize how murky these abstract concepts can be.²

What eventually hit me, though, is the following realization:  doctors should not engage in risk prediction at all!

That was my ah ha! moment.  A seemingly outlandish one, for sure, so let me restate it more carefully:

When a doctor is engaged in the act of healing—which, after all, is what the medical profession is about—he or she should not first enter into a calculation of benefit, risk, cost, and the like before deciding on the proper course of action.

The key term is calculation.  In other words, in the course of helping patients, doctors should not explicitly add the pros and cons of the available options or engage in formal risk-benefit estimation, even if this estimation in only semi-quantitative (that is, not strictly-speaking arithmetic).

Of course,  I have a lot of explaining to do!  But I’m not coming to this conclusion without arguments to back me up.  In fact, I suspect this proposal of mine is not altogether original, but we’re already well past the 1000-word mark for this blog post, so I’ll leave the supportive evidence for my claim until next time.

For now, though, I’d be very interested in getting readers’ reaction to this idea.  What do you think?  Is a physician’s good counsel fundamentally a matter of doing a risk-benefit calculation for the patient?  Remember, your answers could put the future of pay-for-performance at risk!

[callout]This post is part of an on-going series on medical decision-making[/callout]

---

Notes:

1. The fact the Laplace himself was also the originator of Bayesian theory would suggest that this school of thought is bound to a deterministic worldview.  Nevertheless, I don’t imagine that all those who invoke Bayesian theory necessarily espouse that worldview.

2. Hacking, Ian.  The Emergence of Probability: A Philosophical Study of Early Ideas about Probability, Induction and Statistical Inference.

Additional references of interest:

Accad, M. Statistics and the rise of medical fortunetellers.  Tex Heart Inst J. (2009) 36(6):508-509 Free access

Accad, M and Fred HL.  Risk-factor medicine: an industry out of control?  Cardiology. (2010) 117:64-67 Paywall

Goodman, S.  Toward evidence-based medical statistics 1.: The P value fallacy. Ann Intern Med (1999) 130:995-1004 Paywall

Goodman, S.  Toward evidence-based medical statistics 2.: The Bayes factor. Ann Intern Med (1999) 130:1005-13 Paywall

[callout]If you enjoy what you read, don’t forget to share the content with friends and colleagues so they too might become Alert and Oriented! Also, sign-up at the upper right-hand corner of your browser (or at the bottom of the page on mobile devices) to receive a free monthly digest of all my posts. Thank you![/callout]