COI and empirical fundamentalism

Over at the Incidental Economist, Austin Frakt has published a thoughtful commentary on Lisa Rosenbaum’s NEJM series on the obsession over conflict of interest. Frakt is supportive of Rosenbaum’s position but also touches on a dimension to the story which I did not address in my admittedly polemical piece yesterday.

Frakt’s most important statement is actually not in the post itself but in a Tweet linking to it.

I view COI disclosure as (meta) data. Like all data it doesn’t self-interpret. This is the source of all COI-debate: http://t.co/qIyds8T6Bu

— Austin Frakt (@afrakt) May 22, 2015

Frakt is absolutely right and his statement points to a very fundamental assumption that underlies not only the COI concerns, but the legal practice of medicine in general.  Namely, the assumption is that in science and medicine, we should “let the data speak for itself.”

The notion that data can speak for itself is an old idea that has been repeatedly discredited since it was first championed by Francis Bacon in the sixteenth century (it may even have earlier antecedents among some pre-socratic philosophers, I’m not sure), but yet remains tempting enough to the human mind that it never seems to go away.

Data never speaks for itself.  Data needs to be interpreted, and the interpretation demands a framework of assumptions—biases—so that meaning can emerge.  Murray Rothbard dealt with Bacon’s idea of fundamental empiricism with his usual truculent style in a brief essay, part of his Austrian Perspective on Economic Thought, but the idea is alive and well in medicine and many other fields today.  Of course, Thomas Kuhn gave a more encompassing critique of the Baconian ideal.

The greatest manifestation of neo-Baconianism today is the so-called “evidence-based medicine” movement whose primary epistemic tool is the clinical trial.  For EBMers, true medical knowledge is essentially reduced to clinical trial results, which are viewed as sacrosanct messages of truth to be received without bias, preconception, or undue influence.  Because EBM ideology is a perfect fit for the promoters of population medicine, the COI disclosure hysteria has reached the proportion it now has.

Now, EBMers may well counter that their aim is simply to ensure that the data is not tampered with and that scientists interpreting the data are not personally skewed in their interpretation by financial incentives.  But both of these claims must be analyzed in proper context.

In the first instance, data tampering is dealt with most directly with systems of independent documentation and audits.  Disclosure of financial ties is irrelevant.  But it is noteworthy that we find ourselves in a situation where the data seems so “fragile” that it demands such draconian precautions.  Part of the reason for this fragility is again related to our love affair with population medicine.  As we try to identify minute clinical effects via mega clinical trials, misclassifying a couple of data points can turn a positive trial into a negative one or vice versa, and millions of dollars and lives are at stake.

In the second instance, financial ties are only one of myriads of mental influences that can affect a person’s interpretation of the data.  And, as Rosenbaum pointed out, our understanding regarding such influences—and whether they can or should be eliminated—is rather embryonic.

As for me, let me make a full disclosure:  My mind is hopelessly under the influence of ideas adopted from others or concocted in my own head.

[callout]If you enjoy what you read, don’t forget to share the content with your friends so they too can become Alert and Oriented! Also, sign-up at the upper right-hand corner of your browser (or at the bottom of the page on mobile devices) to receive a free monthly digest of all my posts . Thank you!  [/callout]