Public Health Watch: A Warning System for the COVID-19 Hospital “Hurricane”

At its peak, the COVID-19 pandemic wreaked havoc on healthcare systems worldwide. From El Paso, Texas to Brazil and, most recently, India and beyond, hospitals were so overtaxed at various points, as the virus spread, that patients were treated in temporary facilities—if they were able to access care at all.

As more and more people are vaccinated against SARS-CoV-2, these events may eventually fade into memory. However, no one working in healthcare, or public health, wants to see them repeated. Ever.

A new analysis published on June 17th by JAMA Network Open offers a preventive approach—using real-time syndromic surveillance of hospital employees to estimate subsequent COVID-19 hospitalizations, particularly in the absence of adequate testing capacity.

“To understand how to respond [to this public health emergency], our hospital needed a COVID-warning system, just as coastal towns need hurricane warning systems,” co-author Steven Horng, MD, clinical lead for Machine Learning, Center for Healthcare Delivery Science, at Beth Israel Deaconess Medical Center in Boston, and his colleagues wrote in an article in Harvard Business Review. “We knew it was hurricane season, but when would the storm arrive, and how hard would it hit? We were uncertain about what lay ahead.”

For their analysis, Horng and his colleagues conducted assessed whether a daily employee symptom reporting tool used at the Beth Israel Deaconess Medical Center in Boston could assist in forecasting subsequent hospital admissions for COVID-19 in the communities where employees lived. Among their study population of 6481 hospital employees, an increased frequency of employee-reported COVID-19 symptoms was associated with increased hospitalization rates at 10 network hospitals over the following week.

The analysis was conducted at the 10 hospitals in the Beth Israel Deaconess HealthCare Network between April 2 and November 4 of last year. Horng and his colleagues numbered the hospitals 1 through 10 based on the number of unique employees living in their local service area who filled out the symptom report during the study period. Thus, hospital 1, an academic teaching hospital in Boston, had the most employees living in its service area, while hospital 10 had the fewest.

Employees were included in the analysis if they worked at hospital 1, on-site and self-reported symptoms were recorded by the institution daily, using an automated text messaging system.

The study’s primary outcome was the mean absolute error (MAE) and weighted mean absolute percentage error (MAPE) of the 7-day forecasts of daily COVID-19 hospital census at all 10 hospitals. The syndromic surveillance model had a MAE of 6.9 patients with COVID-19 and a weighted MAPE of 1.5% for hospitalizations, according to the researchers. Individual hospitals had MAE that ranged from 0.9 to 4.5 patients, while weighted MAPE ranged from 2.1% to 16.1%. The mean network all-cause occupancy was 1,286 during this period; thus, an error of 6.9 is 0.5% of the network mean occupancy—a level of error that negligible operationally.

Indeed, the forecast model was reasonably accurate in predicting subsequent hospital demand; a doubling of the number of employees reporting symptoms (which equated to 4 additional employees reporting symptoms) at hospital 1 was associated with a 5% increase in COVID-19 hospitalizations at hospital 1 in 7 days, the researchers said.

And, it did so in a way that “required no knowledge or estimation of ongoing policies, was highly localized to a specific area, [and] was designed to estimate only into the immediate future (1 week),” they wrote.

“We used a series of methods derived from epidemiology, machine learning, and causal inference, to take a locally focused approach to predicting the timing and magnitude of COVID-19 clinical demands for our hospital,” Horng and his colleagues wrote in Harvard Business Review. “This forecasting serves as an example of a new opportunity in healthcare operations that is particularly useful in times of extreme uncertainty.”