Epic-generated sepsis notifications increased during COVID-19, study shows

0
7
Epic-generated sepsis notifications increased during COVID-19, study shows

“Although the increase in the proportion of patients generating sepsis alerts in this study can be explained by the cancellation of elective surgeries and a higher average acuity among the remaining hospitalized patients, the 43% increase in total alerts illustrates the increased alerting burden imposed by COVID-19 on a sepsis model,” they wrote.  

WHY IT MATTERS  

“This increase in alerting could have resulted from dataset shift, a phenomenon in which model performance deteriorates as a result of changes in the case mix,” said researchers. “However, even accurate alerts can be disruptive in the presence of resource constraints.”

As noted by the researchers, concerns have been raised about the potential for sepsis models to cause alert fatigue, leading to the University of Michigan to pause Epic-generated alerts in April 2020.  

Story Highlights

  • Researchers from the University of Michigan Medical School and Washington University School of Medicine found that the total alerts per day increased by 43% in the three weeks before and after the first case of COVID-19 in each health system.  

  • Epic did not respond to requests for comment by press time.  

The team calculated Epic Sepsis Model scores from 24 hospitals across four different health systems between November 3, 2019 and April 25, 2020.     

In the three weeks before and after the first case of COVID-19 in each of the studied health systems, the proportion of patients generating sepsis alerts per day more than doubled from 9% to 21%.  

This took place even as the total hospital census declined by 35%.   “Larger hospitals generally experienced an increase in the proportion of patients generating sepsis alerts, whereas the change in the alerting proportion was more heterogeneous across smaller hospitals,” wrote the researchers in their letter.  

They noted that they did not evaluate the model’s accuracy for the study. “However, even if the alerts were accurate, many existing sepsis workflows are built around bacterial sepsis and thus may not be entirely appropriate in the context of COVID-19,” they observed.  

THE LARGER TREND Epic’s sepsis model has been the source of controversy this year, with a retrospective study – led by the same author as the research letter in JAMA Network Open – in JAMA Internal Medicine suggesting that it lacked predictive power.   

ON THE RECORD   “Being able to rapidly assess and disable AI-based alerts is a responsibility faced by health systems using AI to support clinical care,” wrote the Michigan researchers in JAMA Network Open.  

“Sepsis is a hard problem to solve, and it’s one the industry has been working for many years to address,” said Epic’s director of nursing, Emily Barey in an interview with Healthcare IT News about that study. “As a nurse at the bedside, sepsis can sneak up on you because it shows up in patients in different ways.”   A different study, however, found that the early warning system led to faster antibiotic administration and better patient outcomes without an increase in harmful clinical interventions.