Blog

Proactive Security Challenge vs. real malware (2010/11/01 09:00)

Proactive Security Challenge is a project devoted mostly to testing abilities of security software to protect against actions of malware. Currently, Proactive Security Challenge consists of 148 different tests. Sometimes we hear people arguing that the techniques used in our tests do not correspond with techniques used by the real malware. In order to find out how much Proactive Security Challenge reflects the real world of malware, we have performed the following research.

We have collected a set of 20 malware samples that were not detected by two popular anti-virus engines. This means that downloading these samples to the computer and executing them would be possible even with a fully updated anti-virus installed. Then we have run the samples and analyzed the techniques they used. The results are as follows.

• The most used technique among the tested samples was direct Internet access, which is tested by Leaktest and Yalta tests, it was performed by 50 % of the malware samples.
• The second most used technique was registering under HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run. This has been seen in case of eight malware samples. In Proactive Security Challenge this technique is implemented by Autorun3 test.
• Similar technique of using HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run registry key in order to persist in the system was used by five malware samples. In Proactive Security Challenge this is done by Autorun1 test.
• Exploiting registry value PendingFileRenameOperations under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager is done by FileMov2 test and also by four of the samples. In this case, however, we have to admit that FileMov2 uses the technique for attacking purposes, which has been seen only once while the other three samples used it to remove their tracks.
• Two malware samples changed the system HOSTS file. This technique is checked by our HostsBlock test.
• Twice we have seen an attempt to load a kernel driver. Once it was using the technique of Kernel2 test and the other time Kernel1's technique was used.
• Disabling security related system services also appeared twice. Svckill test simulates this behavior.
• Jumper test replaces Internet Explorer's start page and so did two malware samples in our research.
• In case of two malware samples we have seen the technique of replacing executable of a legitimate application with a malicious one. Similar idea is implemented in Runner and Runner2 tests.
• The Shell value under the registry key HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Winlogon was exploited twice. Autorun7 uses this.
• Similar technique to today's very popular DLL Hijacking was used by one malware. Its technique can be found in our Inject2 test.
• One sample encoded various information about the infected system into a long DNS query which it then resolved. This is the technique of DNStester.
• Misusing registry key HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunOnce, implemented by Autorun4 test in our project, has been seen in one case.
• Another registry key that can be misused for system infection is HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer\Run. This was done by one malware and it is also done by Autorun19.
• The Load value under the registry key HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Winlogon can be used for persistent system infection, and was once. Autorun15 implements this technique.
• One of the tested malware samples started system at command and scheduled new tasks in order to be executed regularly. Similar technique is used by our Wallbreaker4 test.
• Configuring itself as a debugger for a known executable name under HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options registry key is implemented by Autorun10 test and was used by one malware sample.
• Another attacking technique was starting a trusted application and infecting it with a malicious code. This technique is implemented in DNStest and AWFT1 tests.
• One sample executed script using wscript.exe. This idea can be found in VBStest.
• Another malware sample installed so called global hook in order to inject its malicious DLL into other processes in the system. FireHole test works just like that.
• AWFT3 and AWFT4 create new thread in Windows Explorer in order to execute malicious code under its rights. This technique appeared in case of one malware sample. A similar technique is done by Thermite test. In its version the target process is Internet Explorer, and this variant has also appeared once during this research.
• Direct disk access technique was also implemented by one malware, in our suite this is the role of FileWri4 test.
• Print Spooler's interface was misused by one malware. Kernel5b works on this idea.

As we can see behavior of real malware samples consists of many techniques that we test in Proactive Security Challenge. We are sure that if larger set of malware was used we would see even more techniques that are implemented in our tests. If a security product is able to block techniques of our tests, it can also block most of the real malware's behavior. It is clear that Proactive Security Challenge is not a useless theoretical concept, its techniques are used by real malware. Being able to fight these techniques makes sense. It should be mentioned, however, that not all the existing attacking techniques are implemented in Proactive Security Challenge tests, but we have tried to cover the most used techniques with our testing suite.

permalink

Confusion about Malware Defender (2010/03/22 13:30)

Since the last testing of Malware Defender, we have received quite a lot of emails suggesting that Malware Defender should not be recommended security product to use because it is a rogue security software. As an evidence the emails linked various web pages with details about Malware Defender 2009. If we read the above mentioned Wikipedia article about rogue security software carefully we can find out that the list of well known rogue security software contains the following entry: Malware Defender (not to be confused with the HIPS firewall of the same name).

Obviously, the rogue application called Malware Defender 2009 is not the Malware Defender by TorchSoft. Screenshots of both applications are available on the Internet (compare screenshots of TorchSoft's Malware Defender with a screenshot of Malware Defender 2009). TorchSoft's Malware Defender that was tested in Proactive Security Challenge is a legitimate security software. The list of rogue security software on Wikipedia clearly shows that it is a common strategy of malware authors to name their products with similar names to legitimate software. We should not be confused about that and we should always be able to find out which application is legitimate and which is not. Our list of security products suitable for Proactive Security Challenge testing should always help you find the web site of the original and legitimate software.

permalink

Do not use GRC's LeakTest (2010/02/17 13:37)

During the last few weeks, we have received a couple of emails concerning the security of PC Tools Firewall Plus. Our visitors ask us, how is it possible that PC Tools Firewall Plus is rated highly in Proactive Security Challenge when it is not able to block the very simple GRC's LeakTest, a tiny testing program that was written many years ago. Regardless the configuration of PC Tools Firewall Plus, clicking Test For Leaks button in GRC's LeakTest leads to the big red Firewall Penetrated! alert.

Being that a repetitive question, we have decided to analyze the situation. We found out that GRC's LeakTest is just a poorly written program that suffers from reporting false results in some cases, especially in case of PC Tools Firewall Plus. Why is PC Tools Firewall Plus so special compared to others in a way it does not pass GRC's LeakTest even if the user uses the block button in PC Tools Firewall Plus's alert? In case of most products on the market, when the action of outbound connection is blocked, the product cuts the connection completely and report an error message to the offending application. For example if a web browser is blocked, it reports some kind of connection failure message to the user. PC Tools Firewall Plus, however, does not do it that way. Its developers implemented it in a way that might be considered as more polite to the end user. If the connection is blocked on the machine via PC Tools Firewall Plus, it seems to the application as if the connection was successful and then any attempt to read the data from the server leads to reception of an informative message that explains that the connection was blocked by PC Tools Firewall Plus and also explains what to do to allow the blocked application to connect in case it was not the real user's intention to block it. So, if the user accidentally blocked the legitimate browser application, they will see the informative message and will have no problem to fix the situation. This may be considered as a better approach compared to the situation when the default error message is shown to the user, which is also displayed in case of many other error situations including the target server failure, network failure etc.

The problem with GRC's LeakTest is that it does not verify that it connected to the target server. No proper verification is done and since it is able to read some data it suppose the firewall was penetrated while in fact it is just a message from PC Tools Firewall Plus.

Testing programs are important tools for developers, testers and users, but they should never be blindly trusted. Unlike GRC's LeakTest, our tests in Security Software Testing Suite are designed to always verify and provide proves of the reported results if possible and even then our testers never blindly rely on the test's output.

permalink

New versions of ZoneAlarm and PC Tools (2009/09/02 14:12)

PC Tools Firewall Plus 6 has been released. The new version should come with Windows 7 support and significant improvements of application protection module. We will test the new version in our next Proactive Security Challenge update.

Also new versions of ZoneAlarm products have been released recently. The 2010 series newly supports Windows 7. In our security testing project, we will replace ZoneAlarm Pro with ZoneAlarm Extreme Security to get the best ZoneAlarm products can offer. The new version will be tested soon.

permalink

Privatefirewall goes free! (2009/07/30 18:36)

A great news for all fans of Privatefirewall has been announced by its vendor PWI, Inc. Since Privatefirewall 6.1.20.24 (the latest version at the time of this announcement), the whole product is free of charge, without any limitations. Also, the new version is ready for Windows 7. Other related products of PWI, Inc. including DSA (free product with only a part of the functionality provided by Privatefirewall) have been discontinued.

permalink

Outpost 6.7 (2009/07/23 13:08)

Agnitum Ltd. released new versions of their Outpost products. Even if this is not a major release, 6.7 series might be interesting for many because of its support of Microsoft's new operating system Windows 7, which final version should be available in autumn this year. Another interesting improvement in new Outpost is that the content filtering is now fully compatible with P2P clients and rich-media websites.

permalink

KIS 2010 (2009/06/25 09:54)

Kaspersky Internet Security 2010 is out. It comes with several new functions and various improvements. Among the noticeable new features, we can mention the Safe Run mode which enables the users to run new software in an isolated environment so that it can not harm the operating system or other applications. Another new features are the Game Mode – reducing alerts during playing games, and Kaspersky Toolbar for Internet browsers that warns about known dangerous websites. Read more in the official press release on Kaspersky Lab's website. We will schedule the testing of KIS 2010 as soon as possible.

permalink

Outpost Firewall Free 2009 (2009/04/27 12:47)

Very popular Outpost Firewall is now also available in a lightweight version called Outpost Firewall Free. The previous free version of Outpost Firewall was released in 2002 and its protection was outdated for several years. The new free version is based on the engine of the commercial version with some features removed. Outpost Firewall Free offers Firewall, Proactive host protection and Self protection features as the Pro version but it misses Anti-Spyware, Web control, Identity Protection features and Multi-language support. This product may quickly become a great alternative for the users that require free solutions.

permalink

Comodo Internet Security (2008/10/28 11:31)

A few days ago, Comodo Security Solutions, Inc. released new security products called Comodo Internet Security and Comodo Internet Security Pro. These are security suites that combine classic firewall, personal firewall and anti-virus features. Comodo Internet Security is completely free while Comodo Internet Security Pro includes paid services. For more information, visit the vendor's website.

permalink

Checkmark Desktop Firewall Certification and Rising Firewall (2008/10/20 08:22)

One of our visitors asked us a question about Rising Firewall passing Checkmark Desktop Firewall Certification. He asked how it was possible that Rising scored so bad in our tests while it was able to pass the certification by West Coast Labs. This blog post is about what Checkmark Desktop Firewall Certification really means and how it is related to Firewall Challenge testing.

On 27th September 2008, Beijing Rising International Software Co., Ltd. published the information that Rising Firewall won Checkmark Desktop Firewall Certification. According to West Coast Labs, the product must achieve an effective level of protection against hostile attacks from outside and prevent unauthorized local applications from accessing the local network. According to the mentioned press release of Rising:

The five shinning key features of this software that led to getting this certification are:

1. Multi-Account Management by the Firewall
The firewall provides two accounts: an administrator account and a user account. A function is provided by the firewall which enables the switch between the two accounts.

2. Trojan Identification Technology
Through heuristic virus scan technology, when a program is connected to the Internet, the Trojan scanner will scan the program.

3. IE Function Call Interception
As IE provides an open Com component call interface, it may be called by malicious programs. This function checks the program which needs to call the IE interface.

4. Anti-Phishing and Anti-Trojan Websites
The website provides a set of powerful and upgradeable blacklist rules, which contain a list of websites that are illegal, highly risky and/or highly hazardous. Using the blacklist rules, any access to the listed websites will be prohibited.

5. Module Test
The firewall can control access to the Internet by all modules. When an application accesses the Internet, the firewall will check the authorization of the module to see if approval has been given.

The key features 3. and 5. are interesting for us at the moment because these are tested in our Firewall Challenge project too. To answer the question of our visitor, we installed the latest version of Rising Firewall. It should be noted that there were some differences between our configuration and the configuration of the tested machine of West Coast Labs. Firstly, the certification test was made some time ago, so the version of Rising Firewall tested was 20.54.41. We installed the latest version available today, which was 20.66.40. Also the Checkmark test was done on Windows Vista Business Edition while we used Windows XP Professional Service Pack 3. Neither of these differences should affect the results because according to Rising website, their product fully supports Windows XP too and the new version should be at least as good as the older one.

To review the claimed protection we used some tests of our Security Software Testing Suite (SSTS), which is used in Firewall Challenge. According to the press release, Rising should be able to prevent misusing the COM interface of Internet Explorer components. This is exactly what Flank test is about. Then it should be able to prevent DLL injection in order to pass the Module Test – this is exactly what FireHole test is about. And according to the description of the certification, it should implement an effective protection in order to prevent unauthorized local applications from accessing the local network – this is what many leak-tests are about, we used AWFT1 and CopyCat just to verify the protection is implemented.

Rising Firewall passed FireHole test smoothly. A popup window appeared asking about an unknown module that FireHole injected to Internet Explorer's process. However, Rising Firewall failed Flank, AWFT1 and CopyCat tests. According to the press release, it should pass Flank at least. The reason why Rising failed Flank is simple. The protection implemented by Rising Firewall in order to intercept working with COM/OLE objects is based on user mode hooks, but the tests of SSTS are implemented to unhook the user mode hooks. It is well known fact that user mode hooks can not be used to implement security features safely because they can always be bypassed. Rising Firewall uses user mode hooks improperly and hence it does not work against Flank. When we run Flank with unhooking disabled, Rising was able to intercept its attack attempt. AWFT1 attempts to access the network indirectly – it injects its code into an instance of Internet Explorer that it creates. The injected code then access the network and returns the results to AWFT1 process, so that it seems that Internet Explorer accesses the network and not AWFT1. CopyCat also accesses the network indirectly but it uses another trick. It does not create a new instance of Internet Explorer but it injects its code directly to the existing Internet Explorer process. Note that any network-allowed process could be used by AWFT1 and CopyCat, not just Internet Explorer. Rising Firewall was not able to catch these attacks, which are well known for years and covered by many other software firewalls available on the market.

What do these results mean? The result of FireHole means that Rising passes the Module Test mentioned in the press release. Another test mentioned in the press release – IE Function Call Interception – was failed in our opinion, because the implementation using insecure user mode hooks does not really work since the attacker can choose whether to bypass the hooks or not. By certifying Rising Firewall, West Coast Labs said that it achieved an effective level of protection that prevents unauthorized local applications from accessing the local network, but the two basic tests we chose proved that this was not the case. This probably means that the methodology of West Coast Labs is incomplete in this part. SSTS is full of much more advanced tests that Rising Firewall does not pass either. From our point of view, Rising Firewall implements a basic protection that is partially based on unreliable user mode hooks and it is insufficient against the modern malware.

Firewall Challenge marked Rising Firewall as one of the worst products on the field because its protection is incomparable to the protection of its competitors such as Outpost Firewall Pro, Online Armor Personal Firewall, Comodo Firewall Pro, Privatefirewall, Kaspersky Internet Security, Netchina S3, ZoneAlarm Pro, PC Tools Firewall Plus and many many others. Giving you another point of view, you can now decide what Checkmark Desktop Firewall Certification means for a certified product and for you.

permalink