COI and empirical fundamentalism

Share with your friends










Submit

Over at the Incidental Economist, Austin Frakt has published a thoughtful commentary on Lisa Rosenbaum’s NEJM series on the obsession over conflict of interest. Frakt is supportive of Rosenbaum’s position but also touches on a dimension to the story which I did not address in my admittedly polemical piece yesterday.

Frakt’s most important statement is actually not in the post itself but in a Tweet linking to it.

Frakt is absolutely right and his statement points to a very fundamental assumption that underlies not only the COI concerns, but the legal practice of medicine in general.  Namely, the assumption is that in science and medicine, we should “let the data speak for itself.”

The notion that data can speak for itself is an old idea that has been repeatedly discredited since it was first championed by Francis Bacon in the sixteenth century (it may even have earlier antecedents among some pre-socratic philosophers, I’m not sure), but yet remains tempting enough to the human mind that it never seems to go away.

Data never speaks for itself.  Data needs to be interpreted, and the interpretation demands a framework of assumptions—biases—so that meaning can emerge.  Murray Rothbard dealt with Bacon’s idea of fundamental empiricism with his usual truculent style in a brief essay, part of his Austrian Perspective on Economic Thought, but the idea is alive and well in medicine and many other fields today.  Of course, Thomas Kuhn gave a more encompassing critique of the Baconian ideal.

The greatest manifestation of neo-Baconianism today is the so-called “evidence-based medicine” movement whose primary epistemic tool is the clinical trial.  For EBMers, true medical knowledge is essentially reduced to clinical trial results, which are viewed as sacrosanct messages of truth to be received without bias, preconception, or undue influence.  Because EBM ideology is a perfect fit for the promoters of population medicine, the COI disclosure hysteria has reached the proportion it now has.

Now, EBMers may well counter that their aim is simply to ensure that the data is not tampered with and that scientists interpreting the data are not personally skewed in their interpretation by financial incentives.  But both of these claims must be analyzed in proper context.

In the first instance, data tampering is dealt with most directly with systems of independent documentation and audits.  Disclosure of financial ties is irrelevant.  But it is noteworthy that we find ourselves in a situation where the data seems so “fragile” that it demands such draconian precautions.  Part of the reason for this fragility is again related to our love affair with population medicine.  As we try to identify minute clinical effects via mega clinical trials, misclassifying a couple of data points can turn a positive trial into a negative one or vice versa, and millions of dollars and lives are at stake.

In the second instance, financial ties are only one of myriads of mental influences that can affect a person’s interpretation of the data.  And, as Rosenbaum pointed out, our understanding regarding such influences—and whether they can or should be eliminated—is rather embryonic.

As for me, let me make a full disclosure:  My mind is hopelessly under the influence of ideas adopted from others or concocted in my own head.

[callout]If you enjoy what you read, don’t forget to share the content with your friends so they too can become Alert and Oriented! Also, sign-up at the upper right-hand corner of your browser (or at the bottom of the page on mobile devices) to receive a free monthly digest of all my posts . Thank you!  [/callout]

4 Comments

  1. Thanks for this essay! Do you have an essay that discusses what better alternatives you see to EBM?

    1. Thank you for the feedback, Bob. There is nothing wrong with conducting quantitative clinical studies per se. In my opinion, the problem comes when doctors are strongly influenced to use such studies essentially as the sole criterion for clinical-decision making. There will be more discussions of this issue (and you may browse prior posts on this as well), so stay tuned!

  2. Great commentary.
    I blogged more than once about related themes particularly in the context of meta-analysis.One of the better illustrations of the fact that data does not speak for itself is the situation in which we are faced with contradictory analysis from two meta-analyses. This occurred in the context of screening for breast cancer.I am fond of quoting from Dr. Steve Goodman’s comments in that regard. See http://mdredux.blogspot.com/2013/12/so-why-dont-we-really-know-about-breast.html
    for comments regarding his assessment of the dueling meta-analysis issue which is at least in part is telling us again that the data does not speak for itself.Another part is that someone has to make a value judgment as to what data to include.

    Michel,you are on a roll,keep going.

    1. Thanks, James. On the topic of meta-analysis, my favorite reference is the late Alvan Feinstein who referred to meta-analysis as “statistical alchemy.” I wrote about his classic paper here. He agrees with you: many more trade-offs than solutions…

Leave a Comment

Your email address will not be published.