Real-time Inpatient Diagnostic Error Risk Prediction and Evaluation
Background
Compared to clinics, there is limited research on diagnostic errors (DEs) in hospitals despite autopsy studies indicating significant rates of preventable deaths (5% to 10%) among hospitalized patients.
Identifying and measuring DE in hospitals is challenging due to inconsistent confirmation methods and biases in retrospective reviews.
Study Goals
Our study aimed to create a reliable prediction model for diagnosing errors DEs in hospital care by analyzing patient data from electronic health records (EHR) to identify key risk factors. Using this data, we refined our DE risk prediction tool on the existing EHR-integrated Quality and Safety Dashboard.
Methods
Our team previously used validated tools like Safer Dx and the Diagnostic Error Evaluation and Research (DEER) Taxonomy to establish a structured process for reviewing EHRs to confirm or reject DEs. We applied advanced statistical methods to develop and validate a model for diagnosing errors during hospital stays and identified predictors of DE in a high-risk cohort.
Based on these analyses, we updated our DE risk prediction algorithm and configured the variables and weights in our Dashboard according to our algorithm. We assessed the algorithm by randomly sampling 175 new cases (low, medium, and high DE risk) prospectively and conducting independent chart reviews to determine the presence of DE.
Results
Our analysis suggests that multiple ambulatory encounters within the 14-days leading up to hospitalization may indicate risk of DE during the hospital encounter. We deployed initial findings from our prediction model into hospital-based clinical practice via our Quality and Safety Dashboard.
After completing chart reviews for 175 cases, we analyzed the performance of the algorithm at two cut-offs (high predicted risk versus medium or low; and high or medium versus low). The algorithm had moderate or high sensitivity at both cut-offs, but specificity was low.
Though our algorithm accuracy needs improvement, clinicians may be prompted to reconsider working diagnoses when they see DE risk flags.