Throughout the Covid-19 pandemic, the unprecedented use of infectious disease models provided insight about the spread of SARS-CoV-2 and helped authorities devise strategies for control. And although the pandemic appears to be waning in the United States, with more than half of the eligible US population having received at least one vaccination (the 7-day moving average of daily new cases has fallen below 15,000, the first time since late March 2020), we know that models will be needed again to predict the path of a future novel zoonotic disease, or an emerging infectious disease that has now become endemic, like Ebola, influenza, and new variants of Covid-19.
The Biden administration’s decision to fund a National Center for Epidemic Forecasting and Outbreak Analytics is an important and overdue step. However, an exclusive emphasis on forecasting would be misplaced.
Forecasting is important but difficult
I believe that infectious disease forecasting is possible, cost effective, and will save lives. Like weather forecasts, good infectious disease forecasts are probabilistic and will be wrong some of the time. Nonetheless, well-calibrated forecasts can be very useful for short-term planning and preparedness and efficiently allocating resources to containment. There is considerable scope for improving the science of infectious disease forecasting. However, as some of my research has shown, there are also fundamental limits to the accuracy of disease forecasts. There are several reasons for this.
One is that the spatial spread of an epidemic through a broadly distributed population is inherently noisy because it results from the individual long-distance movements of a small number of people. Another is that there is a very tight coupling between disease transmission and people’s behavior. Unlike a hurricane, whose physical force is independent of human activity, the magnitude of an epidemic is tightly tied up in the social interactions that enable transmission. For these reasons, forecasts should only be trusted over the very short term.
Six tasks for outbreak analytics
To be most effective for epidemic preparedness and response, forecasting should be complemented by a range of other analytic tasks, including:
Coherence. Some models, known in the jargon as mechanistic models, literally represent what we think is going on in an epidemic. These are different from statistical models and models based on artificial intelligence, which are merely concerned with patterns, i.e. how the epidemic appears. When mechanistic models disagree with the data, then there’s something important that we don’t understand. Such models can be used to check the holistic coherence of our understanding of an epidemic’s progression.
Decision support. Even perfect forecasts are unhelpful if you can’t do something to change the situation. Models for decision support define an action space and seek to characterize the possible, plausible, and likely outcomes of different interventions. Models for decision support focus on the value of different kinds of information and can guide data collection to those areas where reducing uncertainty is most useful.
Estimation. During an epidemic like Covid-19 there are lots of unknowns. Some of these unknowns are states of nature, such has how many people are infected and where they are located. Other unknowns relate to processes, like the rate of transmission, or even abstract concepts like the reproduction number. Even though such unknowns cannot be directly measured, they may still be estimated using statistical techniques.
Two additional tasks are closely related to estimation. These are situation awareness and inference.
- Situation awareness uses estimation to characterize current conditions. For instance, during the Covid-19 pandemic there was a lag between the time when a person became infected and when they were reported as a case. As a result, the tally of case counts that everyone was looking at reflected not the state of the epidemic, but rather transmission two or three weeks prior. To improve on this situation, several research groups developed algorithms for “nowcasting,” which provided much timelier information. Similarly, molecular surveillance is very useful for situation awareness about the evolution of genetic variants of a virus.
- Inference is a first cousin to estimation and concerns the statistical evidence for or against a hypothesis. In the Covid-19 pandemic, it would have been helpful to characterize certain questions in this fashion. For instance, the hypothesis that SARS-CoV-2 is spread in aerosols continues to be debated, even though the evidence supporting the hypothesis is strong.
Finally, scenario analysis is a bit like forecasting in that it projects forward to characterize possible future states of the epidemic. The difference is that whereas forecasting aims at well calibrated predictions of what the future will be like, scenario analysis asks counter-factual questions about how things would likely turn out under different scenarios for intervention.
Whereas forecasting aims at accurate prediction, scenario analysis aims to understand. It may seem like this is a rather nuanced difference, but the distinction is critical for effective engagement in epidemic containment.
The bottom line is that forecasting is just one among many analytic tools that should be used to understand, prevent, and contain outbreaks of infectious diseases. Because each kind of epidemic model has strengths and weaknesses, a pluralistic approach to epidemic modeling is essential. The US needs to use the whole toolbox.