Problem: lack of overview

Let’s expand on the first item in the list I made in the previous post. I called this item “Lack of overview of the patient”, and that’s actually a pretty serious understatement of the problem. What we get in most electronic healthcare records systems is an evenly thick layer of prose stretching from a variable point in the far past to some point in the near past, without any bumps or changes of scenery. It’s like a collection of badly written novels, intensely unreadable and intensely boring. And somewhere in there is that one fact you really need to know about, but since you don’t know what you’re looking for, you’ll never find it.

There is no overarching structure to it. You’d think that if a patient’s life, from a doctor’s viewpoint, is a sequence of medical problems that either are solved and thus in the past, or ongoing and still present in the future, that is what we’d see, but no, that is not what we see. Instead we see, for some reason only a government bureaucrat could fathom, a tree structure of how healthcare is organized in your area. That one administrative or political statement (what else can it be?) follows us around all through the medical record system, occupying prime user interface area, presumably to make sure you never once forget how many departments there actually are in your organization, and what they are called. It’s as if the most critical knowledge you need while treating uncle Bob’s diabetes is how many kinds of ear, nose and throat departments we have and what their names are and under what other larger department they reside. It could be useful for the postman, though.

(Click on image for full size.)

Just look at this screenshot from Cosmic. Most other EHR systems have the same layout, so Cosmic isn’t any worse than any others in this respect. As you can see, the workflow seems to be:

  1. Select a department
  2. Read everything that happened to the patient at that department in reverse chronological order
  3. Repeat from step 1 until you’ve read it all (unlikely) or you grow old and die (more likely)

Or, possibly, select “All physician notes” and then try to guess which departments were not included (yes, there are some, but they’re not going to tell you which ones unless you beg for mercy). This is obviously not the way we want to learn about a patient medical history. The only reason I can think of why EHR systems are built this way is because some bureaucrat who habitually views healthcare as a collection of departments, but has no idea of how a patient is actually diagnosed and treated, got the ultimate say in how the system was designed. Whatever the case may be, it’s a perfectly useless structure of an EHR.

What it should have been is a view of diseases, and then an outline of criteria, treatment options, and followup according to current science. We’ll get to that when discussing solutions later.

Getting organized

As the interest in iotaMed and the problems it is intended to solve clearly increases, we need to get our ducks in a row and make it simple to follow and to argue. Let’s do it the classic way:

  1. What is the problem?
  2. What is the solution?
  3. How do we get there?

Let’s do these three points, one by one.

What is the problem?

The problem we try to solve is actually a multitude of problems. I don’t think the below list is complete, but it’s a start.

  1. Lack of overview of the patient
  2. No connection to clinical guidelines
  3. No connection between diseases and prescriptions, except very circumstantial
  4. No ability to detect contraindications
  5. No archiving or demoting of minor or solved problems, things never go away
  6. Lack of current status display of the patient, there is only a series of historical observations
  7. In most systems, no searcheability of any kind
  8. An extreme excess of textual data that cannot possibly be read by every doctor at every encounter
  9. Rigid, proprietary, and technically inferior interfaces, making extensions with custom functionality very difficult

What is the solution?

The solution consists of several parts:

  1. The introduction of a structural high-level element called “issues”
  2. The connection of “issues” to clinical guidelines and worksheets
  3. The support of a modular structure across vendors
  4. The improvement of quality in specifications and interfaces
  5. The lessening of dependence on overly large standards
  6. Lessening of the rigidity of current data storage designs
  7. The opening of the market to smaller, best-of-breed entrepreneurs

How do we get there?

Getting there is a multiphase project. Things have to be done in a certain order:

  1. Raising awareness of the problems and locating interested parties (that is what this blog is all about right now)
  2. Creating a functioning market
  3. Developing the first minimal product conforming to this market and specs
  4. Evolve the first product, creating interconnections with existing systems
  5. Demonstrate the advantages of alternate data storage designs
  6. Invite and support other entrepreneurs to participate
  7. Invite dialig with established all-in-one vendors and buyer organizations
  8. Formalize cooperation, establish lean working groups and protocols

Conclusion

None of this is simple, but all of it is absolutely necessary. Current electronic health care systems are leading us on a path to disaster, which is increasingly clear to physicians and nurses working with these systems. They are, in short, accidents waiting to happen, due to the problems summed up in the first section above. We have no choice but to force a change to the design process, deployment process, and not least the purchasing process that has led us down this destructive path.

I’ll spend another few posts detailing the items in these lists. I may change the exact composition of the lists as I go along, but you’ll always find the current list on the iotaMed wiki.

If you want to work on the list yourself, register on the iotaMed wiki and just do it. That’s what wikis are for. Or discuss it on the VĂ¥rd IT Forum.

Design for updates

When designing new system architectures, you really must design for updating unless the system is totally trivial. This isn’t hard to do if you only do it systematically and from the ground up. You can tack it on afterwards, but it’s more work than it needs to be, but it’s still worth it.

I’ll describe how it’s done for a system based on a relational database. It does not matter what is above the database, even if it’s an object-relational layer, the method is still applicable.

I have a strong feeling that the problem is trivial on graph databases, since the nodes and relations themselves allow versions by their very nature. I haven’t designed using these graph databases yet, so I’m not 100% sure, though.

The reason I’m going into this now is that the iotaMed system must be designed with upgrades in mind, avoiding downtime and big bang upgrades. Just this weekend, the Cambio Cosmic system in our province (Uppsala, Sweden) is undergoing yet another traumatic upgrade involving taking the entire system offline for a couple of days. Very often it doesn’t come up on time or without serious problems after these upgrades, putting patients at risk entirely unnecessarily. The only reason these things need to happen is because of poor system design. A little forethought when building the system could have made a world of difference. The PMI communication system I built and which is used in Sweden has never (as far as I know) been taken down for upgrades, even though upgrading of both client and server systems is an easy and regular activity, and Cosmic could relatively easily have been build the same way. But it wasn’t, obviously. It’s not rocket science, exactly, just see for yourself in what follows.

The problem

The first step is to understand why the database structure is so tightly bound to the code layers, necessitating a simultaneous upgrade of the database structure and the code that accesses it. In the common and naive form, the data access layer is made up of application code outside the database itself which directly accesses tables for reads and writes. If there are several data access layer modules, all of them contain direct knowledge of table structures and all of them need to update simultaneously with the table structures. This means that if you change table structures you need to do all the following in one fell swoop, while all users are taken offline:

  1. Change table structures
  2. Run through all data and convert it with scripts
  3. Replace all application code that directly accesses table data
  4. And while you’re at it, compound the disaster by also replacing business layer code and user interface code to use new functionality added in the data access layer and database while removing old application code that won’t work anymore with the new data structures

This is what we call a “big bang” upgrade and it practically always goes wrong (that’s the “bang” part). It’s extremely hard to avoid disasters, since you have to do very detailed testing on secondary systems and you’ll never achieve full coverage of all the code or even all the little deviant data in the production database that will inevitably screw up the upgrade. And once you’re in the middle of the upgrade, you can’t back out, unless you have full downgrade scripts ready, that have been fully tested as well. The upgrade scripts, hard as they are to get right, are actually simpler than the emergency downgrade scripts, since the latter must take into consideration that the downgrade may be started from any point in the upgrade.

The result is that these upgrades are usually done without any emergency downgrade scripts. It’s like the high wire without a safety net. The only recourse is a total restore from backups, taking enormous amounts of time and leaving you shamefaced and nervous back at the spot you started after a long traumatic weekend. Since backing down is a public defeat, you’re under emotional pressure to press ahead at almost any cost, even in the face of evidence that things will probably go badly wrong, as long as there’s a chance of muddling through. Your ego is on the line.

This is no way to upgrade critical medical systems, but this is how they do it. Shudder.

The solution

The solution is to isolate the database structures from the application code. If different data access layers in application code can coexist, using each their own view of how the database is structured, while still accessing the same actual data within transactions, you can let two versions of application code stacks coexist while accessing the same data. If you can swing this, you’re free to change the database table structure without the application code in the data access layer even noticing. This implies that you can simply add a new view on the database that you need for a new version of the application, add in the new application code and start running it without removing the old application code or clients. New version clients, business layers, and data access layers run in parallel to old versions, so you can let the upgrade be slowly distributed out over the user population, allowing you to reverse the rollout at any point in time. Let it take weeks, if you wish. Or even leave some old clients there forever, if there’s a need for that.

To achieve the needed insulation, simply disallow any direct access to any tables whatsoever from application code. All accesses must be done through stored procedures or views.

SQL VIEWs where actually designed to achieve exactly this: different views on the table data, removing direct dependency of the application code on the tables, so the problem was clearly defined, known, and solved even before SQL hit the scene, so why are we even arguing this now? As an aside, I never use VIEWs, only stored procedures, since I can achieve the same effect with less constraints, but that does not detract anything from the argument that the problem was both recognized and solved ages ago.

Let’s assume you need to change the name of a table for some reason. (I never do that, I’m just taking a really radical example that ought to make your hair stand on end.) First, you edit the stored procedures that access the table to check if the old table name still exists, and if not, start using the new table name. Then you create the new table. Then you create trigger code on the old table that updates the new table with any changes. Then you use the core of that trigger code to run as a batch to transfer all the old table contents that aren’t accessed to the new table. You check a couple of times during actual use if the tables keep matching in contents. You drop the old table (or rename it if you’re chicken, and drop it later). Finally, you remove the check for the old table name in the stored procedures that access the table. Yes, this is somewhat “exciting”, but I’ve done this a number of times on critical systems and it works. And if you can do this, you can do anything.

A much simpler scenario is if you add columns to a table for a new version of your app. If the new column can safely remain invisible to the old version, just add it on the fly, using the right default values so any constraints don’t fire. Add a new stored procedure that is used for the new version of the application, implementing the parameters and functionality the new version needs. The old stored procedure won’t even know the column is there. If the new column must be set some particular way depending on old values, add a trigger for that and batch update the new column using the core of that trigger in a batch command. Again, there is absolutely no need to take down the database or even kick out users while doing all this. Once the new stored procedure is there, you can roll out new applications and have them come on line one by one, leaving old versions running undisturbed.

You can dream up almost any change in table structure, including new tables, splitting one table into two, combining and splitting columns, creating or removing dependencies between columns and tables, and I can catch all of that using a fairly simple combination of stored procedures, triggers, and the occasional user function. And all of it can be done on a live production database with very low risk (caveat: if you know what you’re doing).

To keep things easy and clean, I always use a set of stored procedures per application, where the application prefix is a prefix in the naming of the stored procedure. That lets you see at a glance which app is using which stored procedure. I never let two different apps use the same procedure (you can always let both stored procedures call a common third procedure to reduce code duplication). Versions of a procedure are named with a numeric postfix so they can coexist. Versions are only created if they have different behaviours as seen from the application code. So a procedure to find a patient, used by an iotaPad app, could be named “IP_FindPatient_2” if it was the third version (the first is without postfix, the second version with _1, etc).

Finally, since you only use stored procedures from application code, no application need have any read or write access at all to your database, only execute access to all the stored procedures with a prefix matching the app. This makes for a very easily verified set of GRANTs and a secure database.

Why this scheme isn’t used in products like Cambio Cosmic is a mystery to me. It can’t be laziness, since they’re working their butts off trying not to annihilate all the medical records we have, while stressing out both medical staff and their own programmers to the point of cardiac arrest everytime something goes wrong. A little forethought would have saved so much time and problems it’s almost unreal.

But on one point you can be certain: that little forethought is a definite requirement for the iotaMed project. It wouldn’t hurt if the other guys tried it as well, though.

iotaDB

iotaMed is intended as an organizing overlay to existing information. It adds the guideline layer, including knowledge transfer and checklist functionality, to the classic electronic health care record. The information needed cannot be stored in the existing EHR systems, simply because they lack the necessary concepts. In other words, we do need our own database to contain the iotaMed information, even though we may leave all the “old” information where it already resides in the legacy EHR. Even the new information added through iotaMed will result in new information to be added to legacy EHR systems, even though that information, by necessity, is as poor as the information currently stored there. There simply is no way to turn this particular toad (the legacy EHR) into a prince.

So, what do we choose for an iotaMed database, that is for the iotaDB? It’s painfully obvious to most that a relational database is very unsuited to this purpose and has always been. Using relational databases for medical information has never been a good idea due to the unstructured or semi-structured nature of medical data. Every EHR I’ve seen that uses relational databases has been a Frankenstein’s monster of inefficiency and inflexibility. It the wrong choice, pure and simple.

Lately, graph databases are becoming popular, and seem to be a much better fit for this purpose. Entities used in medical databases are practically always of variable structure, with a huge number of possible value types, but with only a small number of value types used in each instance. If implemented in a relational database, this results in sparse tables, or more often in blatant misuse of the database where one column is used to define the value type of another column, so that just about anything can be dumped into a single table. It’s horrible.

The “issue”, “issue worksheet”, “clinical guideline”, “item”, and “observation” objects clearly form a graph that would be difficult to squeeze into a relational DB, but would fit nicely into a graph database.

Currently, I’m looking at Neo4j, since it seems to be one of the more evolved open source graph databases.There’s a nice intro to Neo4j here. Neo4j does not seem to have a language binding for Objective-C yet, and it needs Java to run, but since I’m planning on deploying iPads logically tethered to an OSX machine, the Neo4j can run there. The interface between Objective-C and Java would then effectively be a server side proxy app. If somebody already did this, I’d appreciate hearing from you, though.

iotaMed domain analysis

As is simply too painfully obvious, current EHR systems aren’t fit for purpose. There is a serious defect in the domain analysis, so that fundamental concepts such as “disease” and “clinical guideline” are simply missing from the EHR domain analysis.

Let’s discuss the parts in this highly simplified diagram. The objects in green are unique to iotaMed and are missing in current EHR systems.

Patient

The patient object is the root of the tree. We’re assumed to already have created or chosen the patient here.

Issue

A patient has zero, or any number of “issues”. An “issue” is a disease, for instance “diabetes” or a symptom such as “headache” or “vertigo”.

Privacy

Access control is contained in the “privacy” object and the privacy object is connected to the issue. This is a major departure from how access control is normally done in EHR systems. Regular EHR systems tie the access control to the department or doctor that handles the patient, departing from the assumption that the information that the patient desires to protect is tied to the department or doctor that created or recorded the information. This is often the case, but not always, and when it is, it is by coincidence and not an essential attribute. For example: most of what a psychiatrist says belongs in the psychiatric domain so at first blush it seems only reasonable to limit access to records originating with the psychiatrist. But if the psychiatrist now treats the patient for a cold, which he can, then the cold becomes protected information, which is clearly absurd.

Another example is if a psychiatrist treats two different diseases, such as parkinsonism and depression, and the patient wishes for the depression to be protected but not parkinsonism. In that case, it becomes very difficult for the psychiatrist to keep the access control masks correct. We’re clearly using the wrong concepts and entities here.

Obviously, the privacy in real life is attached to the “issue”. If a patient wants to keep his depression a secret, it doesn’t matter which doctor is involved, it’s the depression itself that should be privacy protected. Hence the iotaMed design where the privacy record attaches to the issue.

Clinical Guideline

A “Clinical Guideline” is a template maintained by the medical authorities, which describes in outline form how a particular disease or symptom should be worked up and managed. It also contains differential diagnoses, treatments, new scientific discoveries, links to external documentation, reporting sheets, etc. Whenever an “Issue Worksheet” is created, one “Clinical Guideline” is attached and its contents copied into the “Issue Worksheet”. The relationship between the “Clinical Guideline” and the “Issue Worksheet” is maintained so that changes in the “Clinical Guideline” can be propagated to the already instantiated “Issue Worksheets” in use. An example would be if a new drug of choice has been selected for a particular disease, then the iotaMed system can alert the physician at next encounter that he should consider if that change should be done for the individual patient or not. This link-back allows effective distribution of new evidence-based knowledge into clinical practice without undue delay.

An issue that is not easily identifiable with a known condition, or if no clinical guideline exists that is suitable, the issue worksheet is connected to a dummy clinical guideline that does not contain any items.

Issue Worksheet

The “Issue Worksheet” is an interactive instance of the clinical guideline template. On this sheet, observations can be added or removed. Entire items can be added or deleted, such that any clinical guideline that is preloaded can be rearranged for the particular patient.

If the user creates an issue with an empty (dummy) clinical guideline, the issue worksheet starts out empty, so the user needs to add items to it before proceeding. This way, the user forms his ad hoc issue worksheet, which can later be replaced with a “real” issue worksheet based on a real clinical guideline as the narrowing down of the diagnosis proceeds.

Item

An issue worksheet contains a number of items. Each item is a clinical data point such as history, an allergy, a lab result, a blood pressure. The issue is a variable, not the value itself, which is an “observation”.

Observation

An observation is one data point in the clinical work. It is, for example, a blood pressure, a lab value, a description of the palpation of the abdomen, a description of the auscultation of heart sounds, or the patient subjective history for the encounter. For each item in an issue worksheet, there can be zero or any observations per encounter. In other words, for each encounter you can have none, one, or several blood pressures taken. Each observation allows free text comments as well.

Encounter

An encounter has a date and time of examination or conclusion. Each observation may belong to an encounter, but doesn’t have to. A lab result, for instance, is not encounter related. An observation never belongs to more than one encounter. Also, each encounter is related to one doctor, the responsible physician. In some cases, such as in training situations or operations, there may be multiple doctors.

You’ll find this page in the iota wiki as well: see here.

A Physician’s Bill of Rights

Jeff Atwood wrote “The Programmer’s Bill of Rights” a couple of years back, and it’s entirely applicable to physicians using electronic health care records, so I paraphrased it here for our domain. Here goes.

It’s unbelievable to me that a hospital or a healthcare organisation would pay a doctor more than $100,000 in salary, yet cripple him or her with terrible computing hardware and software. This makes no business or medical sense whatsoever. And yet I see it all the time. It’s shocking how many physicians aren’t provided with the essential things they need to succeed and to treat patients.

I propose we adopt a Physician’s Bill of Rights To Computing, protecting the rights of doctors by preventing their organisations from denying them the fundamentals they need to do their job.

  1. Every physician shall have two monitors
    With the low prices of LCDs and the ubiquity of dual-output video cards, you’d be crazy to limit your doctors to a single screen. The productivity benefits, and the improvement in digesting information, are obvious by now.
  2. Every physician shall have a fast PC
    Doctors are required to run a lot of software to get their jobs done: electronic medical records, pharmacy web interfaces, clinical guidelines, lab software, electronic dictation UI, and not least, Lotus Notes. Running all this software requires a fast PC with lots of memory. The faster a doctor’s PC is, the faster he can cycle through the EHR, prescription, messaging, and clinical guidelines. Time spent staring at a frozen desktop is wasted time.
  3. Every doctor shall have a comfortable chair
    Let’s face it. We make our lvings largely by sitting on our butts for 8 hours a day or more. Why not spend it in a comfortable, well-designed chair, instead of the broken down surplus junk most of us try to avoid falling out of. Sure, you hire doctors primarily for their giant brains and knowledge, but don’t forget their other assets.
  4. Every doctor shall have a fast network connection
    Good doctors use the network all the time. Whenever your doctor has to wait for servers and network responses, he’s losing concentration and time.
  5. Every doctor shall have software suitable to his work
    This is the most important of all these points and the most obvious. Without the right software, your doctors can’t do a good job and you’re wasting money, time, and the patient’s health this way.

These points aren’t extravagant demands. They’re fundamental to the quality of work life for a physician. If the organisation you work for isn’t getting this right, making it right is neither expensive nor difficult. Demand your rights as a physician! And remember: you can either change your organisation, or change your job.

The missing entities

There’s a reason why our medical records systems don’t work and that reason goes back quite a way. As I’ve mentioned before, it goes back to when IT people started implementing the paper records in computers, assuming the paper records contained a complete model of the medical management of the patient, which they didn’t. Automating that crippled model resultet in an automated crippled medical records system. All general medical records systems I’ve ever seen are based on a model somewhat like this:

As you can see, it’s a mess in general, but the problem I’m particularly focusing on is that the main entity, once the patient has been chosen, is the “Encounter”. Everything then dangles of either the encounter or the patient, making it totally impossible to find anything. If you search for information under the patient, you’ll find everything, but if you search under encounters, you’ll typically find nothing. For example, if I want to know if the patient ever said something in particular, or ever showed a particular clinical sign, I first have to know during which encounter that thing happened, if it ever happened. But if I knew that, I wouldn’t need to search for it, would I? Oh, by the way, did you see where they put the symptoms or diseases the patient has? No? Well, neither do I, because they didn’t. It’s not there. The ICD-10 codes don’t fill that function.

This entity relation diagram (ERD) is nuts. Not even a rank beginner would normally analyze a domain this poorly. But somehow it happened and keeps happening.

Let’s see how it should look:

Now we’re getting somewhere. A patient has issues, for which we, in turn, follow clinical guidelines, treat with medicines which in turn are refilled using prescriptions. Referrals are for issues, not for patients as in the first, bad, analysis. With this ERD as a basis, everything else falls into place. There are hundreds of entities missing in these two diagrams, but adding them to the correct, second, diagram is pretty easy and doesn’t break anything. Not so for the first ERD, the one we see in all current systems.

I actually made these two simplified diagrams for the iotaMed wiki. If you want to improve on them, please do that there. That’s what wikis are for.

The interconnection con

Every project and initiative in healthcare IT can be classified into one of two types: interconnection and the rest.

Interconnection projects, as the term indicates, all have in common that they involve improving just the exchange of data and nothing else, by actually interconnecting two or more systems, or by creating some standard that is intended to make interconnection easier. Almost without exeption, these projects are described as “improving healthcare” and almost without exception, no effort is expended on actually describing what this improvement is or means. Practically none of these projects are preceded by a reasonable cost/benefit analysis, or any other analysis of any kind.

Then you have all other projects, those that do not primarily speak of interconnection or standard structures or terms intended to make interconnections easier. Those projects are usually, but not always, meant to solve a defined and real problem, often preceded by cost/benefit analyses, and a decent technical analysis and design.

I’d say more than 90% of all projects we see in healthcare, at least those with public funding, are of the first, “interconnect”, variety. They are, as I said, usually without any decent motivation or ground work except the desire to “interconnect” for its own sake, and usually fail according to objective criteria. But note that they are usually declared successful anyway, and since the requirements were never clear, it’s just as unclear how to deem them successes or failures, so they could just as well call them a success, whatever happens.

The remaining 10% or less may have interconnection as a component, but are founded on some other functional principle, are usually scientifically and technically sound, have a real hard time getting funding, but are often successful and useful. They don’t get much publicity, though.

So, my advice to you is: if you want to contribute something real to healthcare, avoid any project exclusively having to do with “interconnections”. If you’re more interested in committee work without much risk of ever having to prove that you actually accomplished anything useful, jump on any chance you get to do “interconnection” work.

And if you’re in a budget committee and take that responsibility seriously, jump on any “interconnection” proposals and demand a detailed clarification of what exactly will be the benefit of the project, and don’t settle for “more information must be good”. That’s malarkey. If you find any other motivation that actually makes sense, please let me know.