This is a continuation on my previous post, “Evidence based vs anecdotal“.
I wrote an email to the main author of the chapter in “4th Paradigm”, Michael Gillam, and he graciously responded to my criticism by agreeing to everything I said and emphasizing that this is what they wanted to say in that chapter. He suggests that it may not have been clear enough in that regard, and I agree. Anyhow, it’s great to know that smart people are indeed having the right idea of how to handle the knowledge that appears in IT systems, often as a side benefit of having extensive amounts of data in them.
What Michael stresses instead is the benefit of having a real-time monitoring of the performance of treatments in the population. He points to the Vioxx debacle and how a lot less people would have been subjected to the increased risk of myocardial infarctions, had the systems been able to signal the pattern in large data sets. And in this he’s entirely correct, too.
So, in conclusion, we’re in total agreement. A problem remains, however, in that even I, the archetypal skeptic, was easily mislead to read the chapter in question as promoting the discovery of new treatment regimes from dynamic electronic health care data. And I think that is exactly what is happening when some new ill-conceived projects are started in health care. I’ve seen an increasing tendency to dream up projects based on just this, the idea that large sets of health care data will allow our electronic health care record systems to recommend for or against treatments based on the large accumulated set of experience data in the system. And I think the reason is that people like us that reason about how to handle that data and what to use it for and what not to use it for, don’t realize that snippets of our conversations taken out of context, may lead decision makers to take catastrophically wrong turns when investing in new projects. At least, that’s what seems to happen. Time for an anecdote from real life. (Note the strangely ironic twist, whereby I use an anecdote to illustrate why anecdotal knowledge is a bad thing.)
This is an entirely true story, happened to me about 30 years ago, while I was doing my residency in surgery. A trauma patient, comatose, with multiple rib fractures and abdominal trauma, in respiratory distress, was wheeled into the emergency room. I asked the male nurse to administer oxygen through a mask and bladder, while the blood gases got done. As the blood gas results came back, I stood a couple of meters away and quietly explained the blood gas results to an intern, saying something like “see how the oxygen saturation is way down, he’s shunting, while the carbon dioxide is lowered under the normal value, which you may find strange, but it’s because he’s compensating with hyperventilation”, etc. After a minute of explanations, I look up and I see that the nurse is holding the rubber mask over the patient as I ordered, but I see no oxygen line connected, so I tell him it fell off. He says “No, I took it off. You said he’s hyperventilating, so he should re-breathe without any oxygen.” OMG… this guy was actively suffocating the patient after overhearing one word of a half-whispered conversation and applying the only piece of knowledge he possessed that was associated with that word. Which was entirely wrong, as it turns out.
Admittedly, this particular nurse wasn’t the sharpest knife in the drawer; he did this kind of thing with frightening regularity. But still, this illustrates quite perfectly, in my opinion, what politicians and technicians are doing with health care related projects. They catch snippets of conversations, apply some wishful thinking, and formulate a thoroughly sexy project that in their opinion will revolutionize medicine. Except it’s all based on a fundamental misunderstanding. We have to become very much more clear in our discussions about exactly what we can use electronic health care records data for, and what we absolutely must not use it for. Yes, we can use it to provide warning signals to epidemiologists and pharmacologists, and ideas for future studies on new phenomena, but we can definitely not use it to make direct recommendations for or against treatments to doctors while they handle patients. The only recommendations that should be presented to them, are recommendations based on thoroughly and correctly performed studies, nothing else.
It’s up to us to see to it that the people in power get the entire conversation, and understand it, before we let them start projects that have the potential to destroy the advances in medical knowledge we have today. They’re entirely capable of suffocating this particular patient in the name of sexy IT health care projects.