Getting Help from HAL: Applying Machine Learning to Predict Antibiotic Resistance

Highlighted Study: Lewin-Epstein O, Baruch S, Hadany L, Stein GY, Obolski U. Predicting antibiotic resistance in hospitalized patients by applying machine learning to electronic medical records. Clin Infect Dis. Published online October 18, 2020. doi:10.1093/cid/ciaa1576

Appropriate empirical antimicrobial therapy is paramount for ensuring the best outcomes for patients. The literature shows that inappropriate antimicrobial therapy for infections caused by resistant pathogens leads to worse outcomes.^1,2 Additionally, increased use of broad spectrum antibiotics in patients without resistant pathogens can lead to unintended consequences.^3-5 As technology advances, it may enable clinicians to better prescribe empiric antimicrobials. Lewin-Epstein et al studied the potential for machine learning to optimize the use of empiric antibiotics in patients who may be harboring resistant bacteria.

As machine learning and artificial intelligence technology improves, investigators are examining new ways to implement it in practice. Lewin-Epstein et al studied the potential for machine learning to predict antibiotic resistance in hospitalized patients.⁶ This study specifically targeted the use of empiric antibiotics, attempting to reduce their use in patients who may be harboring resistant bacteria.

The single-center retrospective study was conducted in Israel from May 2013 through December 2015 using electronic medical records of patients who had positive bacterial culture results and resistance profiles for the antibiotics of interest. The investigators studied 5 antibiotics from commonly prescribed antibiotic classes: ceftazidime, gentamicin, imipenem, ofloxacin, and sulfamethoxazole-trimethoprim. The data set included 16,198 samples for patients who had positive bacterial culture results and sensitivities for these 5 antibiotics. The most common bacterial species were Escherichia coli, Klebsiella pneumoniae, coagulase negative Staphylococcus, and Pseudomonas aeruginosa. The investigators also collected patient demographics, comorbidities, hospitalization records, and information on previous inpatient antibiotic use.

Employing a supervised machine learning approach, they created a model comprising 3 submodels to predict antibiotic resistance. The first 85% of data were used to train the model, whereas the remainder were used to test it. During training, the investigators identified the variable with the highest effect on prediction––the rate of previous antibiotic-resistant infections, regardless of whether the bacterial species was included in the analysis. Other important variables included previous hospitalizations, nosocomial infections, previous antibiotic usage, and patient functioning and independence levels. The model was trained in multiple ways to identify which manner of use would be the most accurate. In one analysis, the model was trained and evaluated on each antibiotic individually. In another, it was trained and evaluated on all 5 antibiotics. The model was also evaluated when the bacterial species was included and excluded. The model’s success was defined by the area under the receiver-operating characteristic (auROC) curve and balanced accuracy, which is the unweighted average of the sensitivity and specificity rates.

The ensemble model, which was made up of the 3 submodels, was effective at predicting bacterial resistance, especially when the bacteria causing the infection were identified for the model. When the bacterial species was identified, the auROC score ranged from 0.8 to 0.88 versus 0.73 to 0.79 when the species was not identified. These results are more promising than previous studies on the use of machine learning in identifying resistant infections, despite this study incorporating heterogenous data and multiple antibiotics. Previous studies that only included 1 species or 1 type of infection yielded auROC scores of 0.7 to 0.83. This shows that using the composite result of multiple models may be more successful at predicting antibiotic resistance.

One limitation of this study is that it did not compare the model with providers’ abilities to recognize potentially resistant organisms and adjust therapy accordingly. Although this study did not directly make a comparison, a previous study involving machine learning showed that a similar model performed better than physicians when predicting resistance. The model in this study performed better than the one in the previous study, which suggests that this model may perform better than providers when predicting resistance. Another limitation of this study is that it did not evaluate causal effects of antibiotic resistance. The authors believe that further research should be conducted in this area to evaluate whether machine learning could be employed to determine further causes of antimicrobial resistance. A third limitation is that this study only evaluated the 5 antibiotics included, which are the 5 antibiotics most commonly tested for resistance at that facility. Additional research and machine learning would likely need to be incorporated to apply this model to other antibiotics.

The authors concluded that the model used in this study could be used as a template for other health systems. Because resistance patterns vary by region, this seems to be an appropriate conclusion. A model would have to be trained at each facility that was interested in employing machine learning in antimicrobial stewardship, and additional training would have to occur periodically to keep up with evolving resistance patterns. Additionally, if a facility would like to incorporate this type of model, they might want to also incorporate rapid polymerase chain reaction testing to provide the model with a bacterial species for optimal predictions. Overall, the results of this study indicate that great potential exists for machine learning in antimicrobial stewardship programs.

References

1. ^{Paul M, Shani V, Muchtar E, Kariv G, Robenshtok E, Leibovici L. Systematic review and meta-analysis of the efficacy of appropriate empiric antibiotic therapy for sepsis. Antimicrob Agents Chemother. 2010;54(11):4851-4863. doi:10.1128/AAC.00627-10}
2. ^{Marquet K, Liesenborgs A, Bergs J, Vleugels A, Claes N. Incidence and outcome of inappropriate in-hospital empiric antibiotics for severe infection: a systematic review and meta-analysis. Crit Care. 2015;19(1):63. doi:10.1186/s13054-015-0795-y}
3. ^{Klompas M. Overuse of broad-spectrum antibiotics for pneumonia. JAMA Intern Med. 2020;180(4):485-486. doi:10.1001/jamainternmed.2019.7251}
4. ^{Jones BE, Ying J, Stevens V, et al. Empirical anti-MRSA vs standard antibiotic therapy and risk of 30-day mortality in patients hospitalized for pneumonia. JAMA Intern Med. 2020;180(4):552-560. doi:10.1001/jamainternmed.2019.7495}
5. ^{Webb BJ, Sorensen J, Jephson A, Mecham I, Dean NC. Broad-spectrum antibiotic use and poor outcomes in community-onset pneumonia: a cohort study. Eur Respir J. 2019;54(1):1900057. doi:10.1183/13993003.00057-2019}
6. ^{Lewin-Epstein O, Baruch S, Hadany L, Stein GY, Obolski U. Predicting antibiotic resistance in hospitalized patients by applying machine learning to electronic medical records. Clin Infect Dis. Published online October 18, 2020. doi:10.1093/cid/ciaa1576}