The prevalence data we give for clean files is just for informational purpose. We already allow a certain number of false alarms (currently 10) inside our clean set before we start penalizing scores, and in our opinion products which produce a higher number of false alarms are also more likely to produce false alarms with more prevalent files (or in other sets of clean files). While some AV vendors may play down the risk of false alarms and play up the risk of malware, we are not going to rate products based on what the supposed prevalence of false alarms is. In our opinion, anti-virus products should not have false alarms on any sort of clean files regardless of how many users are currently affected by them. Most false alarms will probably fall into the first two levels most of the time. Such cases are likely to be seen much less frequently in a false alarm test done at a specific time, as such files are usually either whitelisted or would be noticed and fixed very fast. Probably several hundreds of thousands or millions of users Probably several tens of thousands (or more) of users Initial distribution of such files was probably much higher, but current usage on actual systems is lower (despite its presence), that is why also well-known software may now affect / have only a prevalence of some hundreds or thousands of users. Individual cases, old or rarely used files, very low prevalence The prevalence is given in five categories and labeled with the following colors: Files which according to several telemetry sources had zero prevalence have been provided to the vendors in order to fix them, but have also been removed from the set and were not counted as false alarms. Due to that, a file with the lowest prevalence level (Level 1) and a valid digital signature is upgraded to the next level (e.g. Files which were digitally signed are considered more important. In order to give more information to the user about the false alarms, we try to rate the prevalence of the false alarms. It doesn’t mean the product with 5 FPs doesn’t have more than 5 FPs globally, but it is the relative number that is important. 30 FPs and another only 5, it is likely that the first product is more prone to FPs than the other. If, when using such a set, one product has e.g. What can be done, and is reasonable, is to create and use a set of clean files which is independently collected. There is no complete collection of all legitimate files that exist, and so no “ultimate” test of FPs can be done. distinguish clean files from malicious files, despite their context. False Positives Tests measure which programs do best in this respect, i.e. No product is immune from false positives (FPs), but some produce more than others. One aspect of reliability is the ability to recognize clean files as such, and not to produce false alarms (false positives). In AV testing, it is important to measure not only detection capabilities but also reliability. This report is an appendix to the Malware Protection Test March 2018 listing details about the discovered False Alarms.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |