Wk 12. Validation of Statistical Models

Lecture Date: November 6, 2023 - Monday
Lecturer: Dr. Hamdi Kavak

Statistical models are quite prevalent in many disciplines. Like any other modeling technique, statistical models require careful consideration of model assumptions, data quality, and interpretation of results. In this lecture, we will go over such considerations in the process of statistical modeling. Notably, we will cover the interpretation of p-values and Simpson’s paradox.

Slides:

Download PDF

Assigned Reading (choose one for your SWA 8):

• Iranitalab, A., & Khattak, A. (2017). Comparison of four statistical and machine learning methods for crash severity prediction. Accident Analysis & Prevention, 108, 27-36.
• Nateghi, R., Guikema, S. D., & Quiring, S. M. (2011). Comparison and validation of statistical methods for predicting power outage durations in the event of hurricanes. Risk Analysis: An International Journal, 31(12), 1897-1906.
• Muñoz, J., & Young, C. (2018). We ran 9 billion regressions: Eliminating false positives through computational model robustness. Sociological Methodology, 48(1), 1-33.

Recommended Reading:

• Atchadé, M. N., & Sokadjo, Y. M. (2021). Overview and cross-validation of COVID-19 forecasting univariate models. Alexandria Engineering Journal.
• Doornik, J. A., & Hendry, D. F. (2015). Statistical model selection with “Big Data”. Cogent Economics & Finance, 3(1), 1045216.
• Gelman, A., Skardhamar, T., & Aaltonen, M. (2018). Type M error can explain Weisburd’s paradox. Journal of Quantitative Criminology.
• Henley, S. S., Golden, R. M., & Kashner, T. M. (2020). Statistical modeling methods: challenges and strategies. Biostatistics & Epidemiology, 4(1), 105-139.
• Lee, G., Kim, W., Oh, H., Youn, B. D., & Kim, N. H. (2019). Review of statistical model calibration and validation—from the perspective of uncertainty structures. Structural and Multidisciplinary Optimization, 1-26.
• Lin, M., Lucas Jr, H. C., & Shmueli, G. (2013). Research commentary—too big to fail: large samples and the p-value problem. Information Systems Research, 24(4), 906-917.
• Pearl, J. (2014). Comment: understanding Simpson’s paradox. The American Statistician, 68(1), 8-13.
• Sen, K., Viswanathan, M., & Agha, G. (2005, September). Vesta: A statistical model-checker and analyzer for probabilistic systems. In Second International Conference on the Quantitative Evaluation of Systems (QEST’05) (pp. 251-252). IEEE.
• Spanos, A. (2011). Foundational issues in statistical modeling: Statistical model specification and validation. Rationality, Markets and Morals, 2(47).