Google questions results of malicious site protection test

In a test conducted by NSS Labs, the beta version of Internet Explorer 9 warned testers about visiting malicious sites or downloading infected files in 99% of cases. Internet Explorer 8 achieved a respectable result of 90%. The good result for Internet Explorer 9 was reportedly due to the previously existing SmartFilter URL filtering and the newly added SmartFilter reputation-based filtering. The test by NSS Labs was financed by Microsoft.

Google's Chrome 6 browser reportedly only detected 3% of threats, although it had still warned users in 14% of cases in a previous test. Google has questioned the validity of the test results, criticising that there is no description of the testing methodology that would allow the tests to be independently verified. Although NSS Labs did describe the test set-up in its results publication, there are no details about which set of URLs was used for the test, and which criteria were used to determine potential threats. According to the description, the test only investigated URLs where a link directly lead to an infected file being downloaded – sites containing exploits for drive-by downloads were apparently omitted.

Google has criticised that the testers used Chrome 6 – the current version is Chrome 8. Relatively speaking, Safari 5 and Firefox 3.6.x still did reasonably well, successfully blocking 11% and 19% of cases. Opera 10 reportedly totally failed the test: 0%. A significant point concerning these results is that Chrome, Safari and Firefox were found to have different success rates although they all use Google's Safe Browsing to determine whether a site is malicious. However, The H's associates at heise Security have also observed in practical use that the various browsers don't always warn reliably, and that they rate threats differently: where Firefox already issues an alert, browsers such as Chrome still readily display a site. Additionally, the report makes no mention of false positives reported by any of the browsers that were tested.

(crve)