Retrospective Influenza Hospitalization Pilot

Influenza-related hospitalizations are a major contributor to the overall burden of influenza in the United States. Accurate prediction of hospitalization rates could help ensure an appropriate public health response during an influenza season. To work towards this goal, CDC will coordinate a retrospective forecasting challenge for influenza-related hospitalizations that occurred during the 2016-2017 influenza season. For each week during the season, participants will be asked to provide national probabilistic forecasts for seasonal and short-term targets. The seasonal targets include the peak week and peak weekly rate of hospitalizations, while the short-term targets include weekly hospitalization rates one, two, three, and four weeks ahead from the date of forecast. Forecasts will be compared with values from the Influenza Hospitalization Surveillance Network (FluSurv-NET).

Forecasting Targets

Teams should submit forecasts for each of the six national targets defined below:

• The peak week will be defined as the MMWR surveillance week that the overall FluSurv-NET hospitalization rate is the highest for the 2016-2017 influenza season. • The peak rate will be defined as the highest numeric value that the weekly FluSurv-NET hospitalization rate reaches during the 2016-2017influenza season. • One- to four-week ahead forecasts will be defined as the weekly overall FluSurv-NET hospitalization rate for the target week.

Peak week, peak rate and weekly forecast rates will be forecast both at the overall level as well as for FluSurv-NET specific age groups (0-4, 5-17, 18-49, 50-64, 65+). FluSurv-NET rates will be rounded to one decimal point for determining the peak week, peak rate, and weekly forecast targets. In the case of multiple peak weeks (i.e. there is an identical peak FluSurv-NET value in two or more weeks), both weeks will be considered the peak week.

Data sources

Historical FluSurv-NET information dating to the 2010-2011 season with weekly updates from the 2013-2014 season onwards were provided to participating teams by CDC. Teams should note that during a season, FluSurv-NET surveillance data are typically not published until the total number of cases reaches approximately 300, and are then published weekly for the remainder of the season. Weekly updates early in the season that appear to be missing are likely a result of this reporting behavior. Teams are welcome to utilize additional data beyond FluSurv-NET - additional potential data sources include but are not limited to: Carnegie Mellon University’s Delphi group’s Epidata API
Health Tweets

Forecast Evaluation

All forecasts will be evaluated using the hospitalization rates pulled from the FluSurv-NET system during MMWR week 28, and the logarithmic scoring rule will be used to measure the accuracy of the probability distribution of a forecast. If p is the set of probabilities for a given forecast, and pi is the probability assigned to the observed outcome i, the logarithmic score is: $$S(\mathbf{p},i) = \text{ln}(pi)$$ For peak week, the probability assigned to that correct week plus the probability assigned to the preceding and proceeding weeks will be summed to determine the probability assigned to the observed outcome. In the case of multiple peak weeks, the probability assigned to each peak week and the corresponding preceding and proceeding bins will be summed.

Forecast submission

Participating teams can upload their submissions here. Please report any issues using the "Report Issues" button at the top of the page.