Verification and Validation Framework for COVID-19 Models

Lead Investigators

• Hamdi Kavak

Student and Senior Collaborators

• Maura Lapoff (MS student w/ CSS concentration - 2021-current)

Project Dates

2021-current

Summary

In this project, we present a verification and validation (V&V) framework to evaluate COVID-19 forecasting models on their report of eight V&V-related components: (1) Conceptual Model, (2) Code and Calculation Verification, (3) Data Validation, (4) Parameter Estimation, (5) Initialization, (6) Uncertainty Estimation, (7) Output Validation, and (8) Model-to-Model Comparison. The framework provides a structured method to qualitatively evaluate these models based on their reported V&V practices. We applied this framework as a checklist for nine models included in the COVID-19 Forecast Hub. One model scored the highest score by supporting seven components, while the lowest-ranked model scored only two. This framework can serve as part of a larger framework to qualitatively and quantitatively examine COVID-19 models’ V&V reported practices and provide credibility for those models that perform well and robust in model construction.

Publications & Presentations

• Towards a Verification and Validation Framework for COVID-19 Forecast Models.
  M. Lapoff and H. Kavak 2021 Annual Modeling and Simulation Conference (ANNSIM), Fairfax, VA, USA (Online), July 19-22, 2021
  [Paper] [BibTex]

Last updated on Jan 27, 2022.