On Windows 7 (or Vista) I use
Do not use GRC's LeakTest (2010/02/17 13:37)
During the last few weeks, we have received a couple of emails concerning the security of PC Tools Firewall Plus. Our visitors ask us, how is it possible that PC Tools Firewall Plus is rated highly in Proactive Security Challenge when it is not able to block the very simple GRC's LeakTest, a tiny testing program that was written many years ago. Regardless the configuration of PC Tools Firewall Plus, clicking Test For Leaks button in GRC's LeakTest leads to the big red
Firewall Penetrated! alert.
Being that a repetitive question, we have decided to analyze the situation. We found out that GRC's LeakTest is just a poorly written program that suffers from reporting false results in some cases, especially in case of PC Tools Firewall Plus. Why is PC Tools Firewall Plus so special compared to others in a way it does not pass GRC's LeakTest even if the user uses the block button in PC Tools Firewall Plus's alert? In case of most products on the market, when the action of outbound connection is blocked, the product cuts the connection completely and report an error message to the offending application. For example if a web browser is blocked, it reports some kind of connection failure message to the user. PC Tools Firewall Plus, however, does not do it that way. Its developers implemented it in a way that might be considered as more polite to the end user. If the connection is blocked on the machine via PC Tools Firewall Plus, it seems to the application as if the connection was successful and then any attempt to read the data from the server leads to reception of an informative message that explains that the connection was blocked by PC Tools Firewall Plus and also explains what to do to allow the blocked application to connect in case it was not the real user's intention to block it. So, if the user accidentally blocked the legitimate browser application, they will see the informative message and will have no problem to fix the situation. This may be considered as a better approach compared to the situation when the default error message is shown to the user, which is also displayed in case of many other error situations including the target server failure, network failure etc.
The problem with GRC's LeakTest is that it does not verify that it connected to the target server. No proper verification is done and since it is able to read some data it suppose the firewall was penetrated while in fact it is just a message from PC Tools Firewall Plus.
Testing programs are important tools for developers, testers and users, but they should never be blindly trusted. Unlike GRC's LeakTest, our tests in Security Software Testing Suite are designed to always verify and provide proves of the reported results if possible and even then our testers never blindly rely on the test's output.