matousec.com (site map)

Poll

How much system resources could your security products consume at most?

  → 0% - 1% (25.75%)

  → 2% - 5% (37.72%)

  → 6% - 10% (15.57%)

  → 11% - 15% (7.78%)

  → 16% - 20% (5.39%)

  → 21% - 25% (2.4%)

  → 25% - 30% (0.6%)

  → 31% and more (4.79%)

more

results

Windows Personal Firewall Analysis

How do we analyse and what do we offer?

This page is outdated! Current information is available on pages of Firewall Challenge project.

Our work is divided into several phases. We made a public survey before the first phase started. We were asking people which personal firewall families they used on their Windows computers. We say Personal Firewall family because one company usually produces more version of one product. For example Zone Labs offers ZoneAlarm product family which includes ZoneAlarm Internet Security Suite, ZoneAlarm Pro, ZoneAlarm Antivirus and other products.

The First Phase

Based on the results of our survey we have selected top five personal firewall product families. Top five product families from our survey were ZoneAlarm, Kerio, Norton, Outpost and Sygate. In the moment we started with the first phase of our analysis there were about 80% of Windows users using one of those products. In the first phase we want to test the best products from selected families. We know that Sygate, Inc., the former vendor of Sygate Personal Firewall, was bought by Symantec Corp. the vendor of Norton Personal Firewall. Because there will be no further development of Sygate Personal Firewall we have decided to analyse the following product from our survey. However the sixth place was taken by a native Windows XP Service Pack 2 firewall which we consider not to be a fully functional personal firewall. This firewall will be tested later in the one of next phases. So, for the first phase we have decided to take the product on the seventh place which was BlackICE. These families of personal firewalls will be tested in the first phase: ZoneAlarm, Kerio, Norton, Outpost and BlackICE. For the first phase a limited set of features and situations we want to analyse was already selected. These situations do not examine the most common personal firewall vulnerabilities, these are left for next phases. The first phase is more concentrated on the single product and problems specific to it. The testing environment for the first phase is a uniprocessor PC with 128 MB RAM and Microsoft Windows XP Service Pack 2 installed. The memory usage of the operating system after the boot is about 64 MB RAM. This means another 64 MB can be used by the tested product. If some product needs more memory the machine hardware is upgraded to 256 MB RAM.

Summary

The first phase of our analyses is over. We have published a final report called Comparison of top five personal firewalls that closes the first phase. This article compares all tested products and present our experience, we have gained during the first phase, with vendors of tested firewalls and end-users.

The Second Phase

The second phase was started with Leak-testing. We have tested more than twenty personal firewalls and published leak-testing results on a separate page. The second phase will continue with deeper analyses of more personal firewalls. These analyses will have the same methodology as the first phase analyses. We have not limited the set of firewalls we will test in this phase. We are more interested in products that have a potential to succeed in our tests. This is why we will test only firewalls with high leak-testing score in this phase. We have already decided to test Comodo Personal Firewall, Jetico Personal Firewall and Kaspersky Internet Security. We are not interested in firewalls that uses the engine of already tested product. This is why we are not going to test Lavasoft Personal Firewall, which is based on the Outpost Firewall PRO engine.

The Methodology

The Methodology reference page contains a complete list of all analysed features. There are also definitions and descriptions of measurement procedures and their classification for every interesting feature. We describe how we work and judge found bugs too.

Public and Private Results

The result page contains the results of our analyses that are public. It offers a brief comparison of already tested products. There is also a short public overview for every tested product. This overview contains a list of bugs and vulnerabilities that were found during the product analysis. Every bug is described using properly defined attributes from the Methodology reference page. The are a few important properties missing in a public view of this bug list, namely bug name, bug description and Testing program or Testing method.