An Independent Validation of Vulnerability Discovery Models

Viet Hung Nguyen
Fabio Massacci, University of Trento, Italy
Having a precise vulnerability discovery model (VDM) would provide a useful quantitative insight to assess software secu- rity. Thus far, several models have been proposed with some evidence supporting their goodness-of- t. In this work we describe an independent validation of the applicability of six existing VDMs in seventeen releases of the three popular browsers Firefox, Google Chrome and In- ternet Explorer. We have collected ve di erent kinds of data sets based on di erent de nitions of a vulnerability. We introduce two quantitative metrics, goodness-of- t en- tropy and goodness-of- t quality, to analyze the impact of vulnerability data sets to the stability as well as quality of VDMs in the software life cycles. The experiment result shows that the\con rmed-by-vendors' advisories" data sets apparently yields more stable and bet- ter results for VDMs. And the performance of the s-shape logistic model (AML) seems to be superior performance in overall. Meanwhile, Anderson thermodynamic model (AT) is indeed not suitable for modeling the vulnerability dis- covery process. This means that the discovery process of vulnerabilities and normal bugs are di erent because the in- terests of people in nding security vulnerabilities are more than nding normal programming bugs.