An early warning score based on trend data from the electronic health record (EHR) may help predict clinical deterioration in inpatients in real time, according to a recent study in Australia.
Researchers used 43 variables and trends to develop a logistic model and deterioration index that helped predict patient deterioration one hour or more before an adverse event. Cases were patients at two Australian teaching hospitals who had had unexpected death, unplanned ICU transfer, urgent surgery, and/or a rapid-response alert between Aug. 1, 2016, and April 1, 2019. The model and index were tested on historical data and then used in another hospital's EHR for six months in a prospective “silent” validation period from October 2019 to April 2020. During the validation period, the logistic model was integrated with the EHR and alerts were triggered but not sent to clinical staff. The study results were published May 3 by Critical Care Medicine.
Data from 258,732 admissions were used to develop the model, and 8,002 adverse events occurred. When vital signs and laboratory trend values were added to the logistic model, the area under the curve increased from 0.84 to 0.89, and the sensitivity of the model to predict an adverse event one to 48 hours earlier increased from 0.35 to 0.41. In a 48-hour simulation, the logistic model had a higher area under the curve than the Modified Early Warning Score and the National Early Warning Score (0.87 vs. 0.74 and 0.71, respectively). During the “silent” validation period, 1,106 patients triggered an amber alert, defined as a deterioration index greater than 6, and 639 patients triggered a red alert, defined as a deterioration index greater than 8. The deterioration index had a sensitivity of 0.474 for predicting an adverse event any time before it happened, 0.369 for one hour before, and 0.327 for four hours before. The specificity in all of these cases was 0.972.
The authors noted that their data may not be generalizable to other settings and that the early warning score was tested in a private hospital, where event rates were lower, among other limitations. They concluded that their deterioration prediction model, converted to a deterioration index, had superior sensitivity and specificity versus traditional early warning scores when integrated into a hospital EHR. Inclusion of demographic data, laboratory values, and trends appeared to improve the predictive capability of the model, the authors said.
“Implementation practice is just as important as effective model generation. Collaboration with nursing staff to best understand how the output of the logistic model (a probability between 0 and 1) could be most effectively translated into a meaningful alert that would augment, rather than interfere with patient workflow, drove most of the implementation decisions in this article,” the authors wrote. “Feedback from clinical staff drove the decision to implement a two-tiered (amber and red) alert system.” Their deterioration index has now been launched in a live clinical trial, with alerts displayed in the EHR and sent to clinical staff, the authors said.