Linux still "benchmark of quality" in this year's Coverity Scan
Coverity has called Linux the "benchmark of quality" in its newly published 2012 Coverity Scan Open Source report. The company annually brings together millions of lines of code from open source and, using the same defect-scanning technology that it uses with its enterprise customers, scans that code for problems to produce data on defect densities.
The "accepted industry standard" for defects is 1.0; one in every thousand lines of code. Linux 3.8's 7.6 million lines of code has a defect density of .59, comparing well with 2012 (7.4 million and a .66 density) and 2011 (6.8 million and a .62 density). In general, the code quality of open source code is equivalent to that of proprietary code; Coverity uses an anonymous sample of its 300 customers to give it a defect density for non-open-source code. Proprietary code has an average defect density of 0.68 whilst open source code averages 0.69 – for two years now, the Scan report has shown densities below 1.0.
An interesting element to this years scan is the comparison of proprietary and open source code quality. The analysis looked at the lines of code in projects and how that related to defect density. For small proprietary projects between 500,000 and a million lines, the density was at .98, but as projects passed a million lines the defect rate dropped down to .66. Open source projects between 500,000 and million lines had a density down at 0.44, but, once past the million line mark, that rose to 0.75.
The suggestion is that open source projects start with a dedicated specialised core of developers who create better quality code, but that as the project expands, code management slips. This is compared to a proprietary project where, as the code base expands, more management effort and control would be applied to the code base. Oddly though, at less than a 100,000 lines, the defect density for open source and proprietary is 0.40 and 0.51 respectively, and between 100,000 and 499,999 this jumps up to 0.60 and 0.66 – still quite close.
The Coverity Scan began in 2006 as a US Department of Homeland Security research project to improve quality of code. The DHS ended the project, but Coverity has continued running the service. C, C++ and Java-based open source projects can apply to be scanned and get reports on defects by registering with Coverity. LibreOffice, MariaDB, NetBSD, NGINX, Git, zsh, Thunderbird and Firefox are among the well-known projects scanned by Coverity Scan. The full report is also available to those who register.