Poll
How much system resources could your security products consume at most?
News
New tests added to SSTS and FWC (2008/05/06 12:42)
We have implemented three new tests and added them to Security Software Testing Suite (SSTS) and also to Firewall Challenge (FWC). There are two new performance tests, PerfTCP and PerfUDP, that measure the impacts of using personal firewall on the network performance. The last new test is called SockSnif and it tests the protection against unwanted packet sniffing. Firewall Challenge now contains 73 tests.
In past few weeks, we have focused on implementation of new tests. In upcoming weeks, we will focus more on testing again.
Online Armor recommendation (2008/05/02 18:09)
We have joined Tall Emu's affiliate program and so you can now buy commercial versions of Online Armor Personal Firewall through our pages and thus support our Firewall Challenge project.
ShadowHook and keylogger tests to Firewall Challenge (2008/04/24 19:40)
We have implemented seven new keylogger tests to Security Software Testing Suite and together with ShadowHook we have added them to Firewall Challenge. We have 70 tests in the system of Firewall Challenge now. Newly tested products as well as new versions of already tested products are always tested against all the tests in the system on levels that they reach.
Firewall Challenge recommendations (2008/04/20 15:16)
Firewall Challenge revealed several notable security products. We have decided to recommend the best products to you via affiliate programs of their vendors. Every product that scores at least 80% in the challenge and thus receives our mark of Very good or Excellent Protection level may become a recommended product. We contact the vendors of such products in order to join their affiliate programs. The vendors that agree and provide us the necessary technical instruments will be linked from the result page of Firewall Challenge.
What does it mean for end users? If you like Firewall Challenge, you can help us and support this project by buying one of the recommended products through the links in Firewalls' ratings table. To get more information about how does it work, please read How you can help us paragraph on the index page of Firewall Challenge.
We start our recommendations with affiliate programs of Agnitum Ltd. and Kaspersky Lab. If you are going to buy any of their products or prolong your license, you can help us by doing so through our links. In the next weeks we will try to join affiliate programs of vendors of other great products tested in Firewall Challenge.
ShadowHook (2008/04/14 12:26)
We are happy to publish the second version of BSODhook utility codenamed ShadowHook. The original version supported the native SSDT calls only, the new version adds support for the GDI SSDT calls. We have also made BSOShook a separate project with its own project web page.
FWC update (2008/04/06 13:09)
We have tested eight more products in Firewall Challenge. System Safety Monitor confirmed its qualities, Norton Internet Security 2008 was significantly worse, but still better than AVG Internet Security, FortKnox Personal Firewall 2008, iolo Personal Firewall, Look 'n' Stop 2.06, PC Tools Firewall Plus and Rising Personal Firewall that failed completely.
Rules update and FAQ in FWC (2008/04/01 11:58)
We have added two new rules to Firewall Challenge to improve its quality and we have also created a FAQ page. If you have any questions related to Firewall Challenge, please consult its FAQ first.
Firewall Challenge (2008/03/18 23:15)
We have not updated our website for a long time, we have received many of emails asking for new tests, results and updates. Today, we present new projects called Firewall Challenge and Security Software Testing Suite to you and hope that it was worth waiting.
Firewall Challenge replaces our former project Windows Personal Firewall Analysis and its subproject Leak-testing. Firewall Challenge combines the depth of our analyses with the simplicity of leak-testing. The whole system of the new project is very extensible, we can and we will add new tests to it to get even more information about the protection of the tested products. If you are interested in the initial results of the challenge or in its rules, scoring system and methodology, just visit the Firewall Challange's pages!
Firewall Challenge testing highly relies on Security Software Testing Suite. This suite is a collection of simple tests with common interface. To make the testing as much transparent as possible we publish the suite with full source codes.
Leak-tests winners, new tests and the future (2007/11/02 19:46)
Today, we have closed the testing with the old set of leak-tests. There were many reasons that led us to think about recoding all leak-tests from the scratch, and another good reason appeared today. What is the future of leak-testing and other kinds of testing and of Matousec group?
Two vendors of personal firewalls asked us to perform extra testing of the latest versions of their products. We have satisfied these requests and as a result we have two new champions in anti-leak protection. Online Armor Personal Firewall 2.1.0.19 Free and Outpost Firewall Pro 2008 6.0.2162.205.402.266 are perfect against the current set of leak-testing programs and pass them all on their default settings. We believe that it is also a result of our work that these products were improved to their today's quality. The results of the tests are available on the Leak-tests results page as usual.
Having two products that are perfect against leak-tests is quite a good reason to come with new tests. But there are much stronger reasons to do so. Firstly, many of the tests do not work properly on their own. When we want to test a personal firewall properly against such tests, we have to hack them to get correct results. For example, some of the tests rely on Internet servers that do not exist anymore and so we have to simulate these servers when we perform these tests. Other tests are not able to clearly show the testing results. There are many more problems with the whole set. So, to make the leak-testing easier, faster and more reliable, we have decided to recode the tests from scratch, to remove duplicate methods, and possibly to add some new methods.
However, leak-tests are, and will always be, tests that do not fully reflect the quality of personal firewalls. Our group provided deeper tests and reviews of some of the most popular personal firewalls and similar solutions. Based on our results we have established partnerships with several vendors of personal firewalls and other security software. In the last 12 months we have provided commercial and non-commercial research, consultations and independent testing to top companies that develop desktop security products for Windows platforms. This is how we have silently contributed to the security of products used by millions, maybe even you. Currently, we have more commercial orders than we can handle and this affects our non-commercial activities such as public analyses of more firewall products. Although we are out of human resources, this does not mean we will not continue in our non-commercial activities, it just means that a progress in such projects will not be as fast as we want.
BSODhook revealed 67 bugs in 14 software products (2007/09/18 20:07)
In the past few weeks, we have performed a research of SSDT hooks implementations in various software products, mostly personal firewalls. The results are shocking! Almost no software vendors that implement SSDT hooking drivers are able to write them correctly and/or test them properly. Today, we have published an article that describes one specific bug we focused on. We have also introduced the BSODhook utility, which may help vendors of such drivers to find hooks that are implemented incorrectly. With BSODhook, we have revealed 67 bugs in 14 software products, for which we also published an advisory. More details can be found in the article.