Atul Gawande and the Challenge of Digital Checklists
Somehow, someway, Atul Gawande had time to take another job. He’s a surgeon at Brigham and Women’s Hospital in Boston, he’s Executive Director of Ariadne Labs, he’s a Professor of Surgery at Harvard and he writes constantly. Now he’s been tapped by Jeff Bezos, Jamie Dimon and Warren Buffett to run their new healthcare venture. Clearly, this is a man who knows how to manage tasks.
One of the things Gawande has become known for is his belief in the power of the checklist to reduce human error and improve outcomes in healthcare. In 2007, he began studying the implementation of a 19-item surgical checklist in eight hospitals around the world. To quote a recent New York Times article, “The results were startling: The mortality rate fell to 0.8 percent from 1.5 percent, and surgical complications declined to 7 percent from 11 percent. In the decade that ensued, surgical checklists proved effective in diverse settings. In South Carolina, in hospitals that implemented checklists, the 30-day mortality for certain surgical procedures fell to 2.8 percent from 3.4 percent.”
Checklists in surgical and emergency settings are statistically proven to be effective. Frequently they are still paper-based. As the industry continues to digitize, naturally there have been calls to replace these physical checklists with technology. There are obvious advantages. Digital checklists are flexible and can respond to changing conditions by presenting different action items. Information collected in digital formats is easier to use for post hoc analysis. Johns Hopkins began testing a home-brewed solution in 2014 and there are a number of technology companies developing stand-alone tools, not to mention the checklists built into all major EHR products. Some of these are highly sophisticated Clinical Decision Support (CDS) systems, but others are simply checks of standard operating procedure.
They’re proven to work in surgical theaters, but in other care settings, checklists—particularly digital ones—are more complicated than one might think. There’s a belief that modern medicine has advanced so far so fast that it’s impossible for clinicians to keep up with all the options available. The hope is that technology can be an interlocutor. It can be, but only if designed properly and applied in appropriate places.
In our field work, we’ve observed clinicians and other staff struggle against checklist workflows in a number of major EHR, but also other staff struggle with checklist creep that has them checking off items on paper, on white boards and on technology. It’s not surprising, because most people are really bad at using checklists or to-do lists, particularly digital ones. In a blog post for Lifehacker, Janet Choi of productivity tool iDoneThis shared that 41% of tasks added to the platform are never checked off.
In a seminal 2007 piece in the New Yorker that began burnishing Gawande’s public star, he said “Good medicine will not be able to dispense with expert audacity. Yet it should also be ready to accept the virtues of regimentation.” We agree wholeheartedly. As we wait anxiously to see what Gawande does with this exciting new opportunity, here are some things to consider when considering digitizing a checklist or decision-making tool.
Finding the Appropriate Level of Workflow Adaptability
Digital checklists allow for an endlessly forking logic tree. Building complicated logic trees in any system has the potential to increase effort and diminish returns. Yes, complicated logic can allow the system to show context-based options to users, but it can also spiral past the point of usability, manageability or maintainability. Each fork in a tree creates the potential for new forks, and there’s a limit to what a process designer can capture and what an end user can manipulate. Remember, checklists serve to reinforce desirable patterns of behavior. When workflows become too complicated and completely non-linear, they cease to create patterns at all. The less repeatable the task is, the more cognitive load required to complete it.
A few years ago, I sat down with a physician at a local health system in the process of implementing a new EHR. He showed me how tasks that he had previously completed quickly had been dissected into many irrelevant steps. The designer who created the workflow surely had the best intentions but had made the experience too granular to be useful. In response, the physician circumvented the process.
What made Gawande’s work in surgical theaters so impactful was its simplicity. In a podcast with the Harvard Business Review, Gawande said, “What I learned going to Boeing, to their checklist factory as I call it, their place where they generate, really more than a hundred checklists a year for situations that pilots have to handle, including crashing planes with enormous amounts of stress and chaos, is that handling those situations requires not trying to turn it into a cookbook... So the lesson that came to us in emulating what they did was we made an only two-minute checklist for the operating room.”
In short, even an adaptable checklist can’t account for every scenario.
Preventing Alert & Error Fatigue
One of the benefits of a digital checklist is that it can enforce a linear approach to certain tasks and account for contingencies. If you shouldn’t be doing “Y” without having done “X” first, a paper checklist allows you to skip ahead where a digital checklist can scream at you. One of the true benefits of a technological system is the ability to stop the workflow when it senses something has gone astray. This is a powerful tool for a product designer that can easily be abused and manifest itself in alert fatigue.
Consider the frustration you feel when you get to the end of a long web form only to have it not submit because you entered incomplete data. The best forms will show you exactly what you missed and make sure that everything you have entered already is saved. The most aggravating will throw another error when you didn’t enter the password you already entered two times more.
If designing digital tools, active alerts are preferable to passive ones because they force the designer to take ownership of decisions made in the product. It’s easy as a designer to not want to make any choices, to push all agency—and complexity—to the end user. Making an alert active forces a designer to confront how necessary it truly is. Active with specific explanation is better still. When designing digital products, we try to make each step active and have a definite outcome.
Stephen Claypool of Wolters Kluwer Health covers this thought in more detail in an excellent blog post called Effective Alerts: 10 Characteristics of an Effective CDS System.
Understanding the Limitations of Form Factor
In a study published as part of the International Conference on Pervasive Computing Technologies for Healthcare (Sarcevic, et al., 2016), researchers attempted to digitize an existing checklist in a pediatric trauma unit. The paper includes a thorough examination of the interface, including some novel solutions, but the researchers ran into two major problems.
- In pressure situations, users would not scroll, severely constraining the interface.
- Physicians frequently scribbled notes on paper checklists. Even when offered a stylus, the number of notes created dropped precipitously.
The first is solvable, but the second problem is mostly not given currently available technology. There’s an opportunity for continuing innovation in form factor or mode of data entry. Voice command of the interface is an interesting possibility. The lesson, though, is that the form factor of the solution must be appropriate for the challenge. If you are designing for an entirely linear process simply asking for a single piece of information per screen is a great option. This is the trend in consumer user interface—imagine how much harder it would be to customize a sub at Wawa by scrolling from option to option. But come complicated tasks call for more complicated interfaces, limiting human ability to use them quickly and effectively.
Making Adherence Matter
A few years ago, I was in a hospital snooping around a nurses station when I came across a folder full of papers tucked under the desk.
“What are these?” I asked
“Oh, checklists for preparing rooms for new patients,” a nurse answered
“Where do you put them when they’re done?” I asked.
“No one ever does them,” she replied, “No one cares.”
No tool, physical or digital can overcome a lack of institutional commitment to a task. That's a post for another day.