Abstract: 

Emergency departments (EDs) aim to reduce length of stay and the number of patients leaving without being seen, a proxy for patient satisfaction. Clinicians often place clinical orders relevant for treatment late in a patient visit after the patient has been assigned a bed and a provider within the ED. Initiating orders at the time of triage, when the patient first enters the ED, would co-opt the waiting time for a bed and allow treatment to begin more quickly. We analyzed whether machine learning techniques using features derived from information available upon triage could be used to accurately predict orders placed during a patient’s visit.

Using data available from one emergency department and two urgent care centers within the VA Boston Healthcare System, we pulled information gathered upon triage and from the patient’s previous medical history. We extracted relevant features from structured fields as well as unstructured fields using a custom natural language processing pipeline. Formatting as a multi-label machine learning problem, we compared performance of binary relevance methods, the random k-labelsets (RAkEL) method, and multi-label algorithmic adaptation methods using the R statistical programming language and the caret package. We evaluated prediction over all orders and for each individual order. The relative importance of data features for the prediction of each order was also assessed and compared across orders. Finally, we developed a visualization dashboard to view historical and predicted results of ordering behavior at the three sites using the R Shiny package.

Multi-label learning approaches that accounted for correlations between orders performed better than binary relevance methods, which predict each order independently. The RAkEL ensemble classifier using a support vector machine exhibited the best performance with an F score of 0.76 when aggregated over all orders. Using a multi-label ensemble classifier greatly increased the performance as compared to binary relevance method using the support vector machine with an F score of 0.57. Prediction performance for each individual order varied greatly from 0.21 to 0.83, correlating with the number of instances of a particular order in the training data. Some variables, such as specific chief complaints or chronic diseases, were only important for prediction of particular orders whereas others, such as acuity level and age, were important for the majority of orders.

Overall, we have sufficient data at the time of patient triage to accurately predict common orders. The multi-label learning frameworks incorporating information from correlations between clinical orders provides added prediction performance relative to predicting all orders independently. This project demonstrates how the combination of machine learning, natural language processing, data visualization, and other data science techniques can support clinical decision making in healthcare.

Bio: 

Haley Hunter-Zinck is a health science specialist at the VA Boston Healthcare System. She has a Ph.D. in computational biology from Cornell University and transitioned to medical informatics during a postdoc in Porto Alegre, Brazil working with Brazilian public hospitals and a fellowship at VA Boston. She applies and develops machine learning techniques and visualization tools to improve hospital patient flow with a focus on the emergency department.