matousec.com (site map)

Poll

On Windows 7 (or Vista) I use

  unlimited administrator's account (57.95%)

  limited administrator's account (16.51%)

  common user's account (13.64%)

  nothing (I do not use Win 7/Vista) (14.16%)

more

results

Articles

Features of Modern Security Suites – Part 2

Published: 2012/09/14

The second part of the Features of Modern Security Suites series is devoted to Behavior Control features. Application behavior and its monitoring and controlling is a topic that is very close to us as it is also the primary focus of our public research as well as our commercial work for security software vendors. In this article we describe basic functionality of Behavior Control features and we enumerate common categories of application behavior that are being controlled by modern security suites.

Contents:


Back to contents

Behavior Control

Also called: Proactive Protection, Active Virus Control, Intrusion Guard, Proactive Defense, Host-based Intrusion Prevention System (HIPS), Behavioral Shield

The Behavior Control component monitors actions of all applications in the system and blocks actions that threaten the security of the system or its users. It maintains a database of rules that determine which actions should be allowed or blocked for each application. The protection system takes control and stops the execution of the application when it is about to execute a potentially dangerous action. If there is a rule that determines the particular situation, it is used and the action is either blocked, in which case the execution flow is modified so that the action is not performed at all or its parameters is altered to ensure it is safe to execute it; or allowed, which means that the action is executed as if there was no protection. It may happen that there is no appropriate rule in the database of rules. In such cases, depending on the configuration of the behavior control component, either a user can be asked to make a decision, or the action can be automatically allowed or blocked, or a heuristic-based algorithm can be used to decide whether it is safe to allow the execution of the action in question.

Behavior Control does not scan the application files before they are executed and hence it is unable to detect malicious programs before they are stared. However, it can effectively block dangerous behavior and thus prevent damage and protect the system from both known and unknown malware.

It is the Behavior Control feature that our Proactive Security Challenge 64 (PSC64) focuses on. In this project we try to examine the quality and the level of security of the product's Behavior Control component. Other components such as Anti-virus Engine may be strongly connected to Behavior Control and may be necessary for its correct behavior, but the primary focus of our project is on the behavior blocking – the ability of the product to recognize and block malicious behavior even if the offending program is not known and can not be recognized prior its execution. We have created a set of tests called Security Software Testing Suite 64 (SSTS64). Each test within this suite implements different technique that can be misused by malware. The goal of the security product tested in PSC64 is to block the attacking techniques of the SSTS64 tests.

Operation Modes

Similarly to Firewall Policy which determines how decisions are made by the Program Control component, the Behavior Control feature can operate in several different modes. Most products do not provide separated configuration here and use Firewall Policy to define how the rules are applied by both the Firewall component and the Behavior Control module. Some products do allow the users to configure this feature separately but in general, the available operation modes work on exactly same principles as in case of Firewall Policy – in the Learning mode, the Behavior Control component will automatically allow the actions and create new rules for them; in the Interactive mode, the situations that can not be decided using the current rule set will alert the users and ask for their decisions; in the Silent mode, everything will be decided automatically etc.

Some products allow to set the protection level of Behavior Control. This setting defines which actions of the applications are considered as potentially dangerous. On lower protection levels applications are allowed to freely perform almost everything they want except for the most dangerous actions. On high protection levels the protection is very tight and the applications are monitored so closely that it may happen that even some of their legitimate actions are sometimes blocked.

Sometimes it is possible to set particular reactions to each of the potentially dangerous actions that can occur. Allow, Block or Prompt are the common options here meaning the action will be automatically allowed or blocked or that the user is to be asked when the corresponding event occurs.

The default configuration of many products prefers more automation, less user interaction, while the security is said to be optimal. This usually means that more advanced methods of behavior control are disabled and so only the most common malicious techniques are being watched. If this is the case and the user wants to be fully protected against the most sophisticated types of attacks it is necessary to enable the appropriate setting. Its name varies, for example, it can be called Advanced events monitoring, Anti-leak, but often a unique trademarked name is used for the technology.

Sandbox

When running under one of the automatic modes the security product may allow a malicious action to be executed (in case of Allow All/Allow Most modes), or block some of actions of legitimate programs (in case of Block All/Block Most modes). This situation is not ideal but using the Interactive mode requires users to make various decisions and so this mode assumes that the users are qualified to do so. Also the number of questions in the Interactive mode might be annoying for the users. An alternative approach that helps solving these problems is using a sandbox.

A product that implements a sandbox treats all unknown or suspicious programs in a special way that ensures they can not cause any damage to the system. It creates a special environment, the sandbox, that looks like the real system to the application that runs inside. The application can freely manipulate objects that it could not in the real system. For example, it can modify important registry entries. The sandbox makes sure that no such modification actually happens to the real system but it tries to simulate a behavior of the system so that the sandboxed application can not recognize it. In the example with the changed registry entry, when the application tries to read its value after the change the sandbox can return the modified value instead the one from the real registry that was not changed. There are several reasons why it is not possible to create perfectly authentic sandbox and this is why some of the most critical operations are always blocked in the sandboxed environment. The better the sandbox is the harder it is for the application to recognize that it is sandboxed.

Trusted programs always run outside the sandbox so that they are fully able to do their job without any limitations. When a new or an unknown application is installed by the user and the sandbox environment prohibits it to operate correctly, the user can exclude the particular application from the sandbox.

Some security suites implement the sandbox as a separate feature of the Behavior Control component. These products let users switch the sandbox off and still control the behavior of the applications. Others have it as an integral part of the Behavior Control component. There are products that allow to configure which actions are blocked automatically in the sandbox and which should be decided using the current policy and rules.


Back to contents

Potentially Dangerous Actions and Techniques

Potentially dangerous actions that security suites monitor can be divided into several categories or groups. We will describe the most common types of techniques that security suites control.


Back to contents

Component Control

Also called: Known Components

Every application uses one or more executable modules, sometimes called components. The main module is usually a .EXE file that requires several dynamic-linked libraries (DLLs) to be loaded into the same address space. Common applications rely on core system modules, such as Kernel32.dll, KernelBase.dll, ntdll.dll, Advapi32.dll, user32.dll etc. Many applications also rely on their custom libraries that are installed to the system with the main application module. One of the basic characteristics of DLLs is that they can be loaded during the initialization of the application's process as well as to be loaded later, just before their functionality is required.

Every application thus has more or less fixed set of DLLs that it loads and depends on. The Component Control feature recognizes these dependencies and controls loading of modules into applications' processes. When a malicious program wants to inject its DLL into another process the component control can recognize a new module being loaded to the target process and it can deny the action.

The component control is also responsible to guarantee the integrity of the trusted modules. Any attempts to modify the on-disk files of the known modules can be recognized and denied. This applies for both the main executable modules as well as for DLLs.

System Guard

Some security products implement different level of protection for the third party applications and the operating system and its components. System Guard is the part of the behavior control protection that is responsible for protecting the operating system itself from being infected by malware, for preserving the integrity of the operating system's parts. Important system files, critical registry keys (including autorun locations), and other system resources, such as COM interfaces, that can be attacked or misused are watched by the system guard. Any attempt to change the system or infect it is denied.

Removable Media Protection

The basic functionality of a security suite when dealing with removable media is to disable AutoRun and AutoPlay features. When a removable medium is inserted and its root directory contains Autorun.inf file, it is possible that a custom program is automatically started by the system. This can lead to a silent infection of the computer.

Many security suites also define a special set of rules for all programs stored on removable media. It is assumed that files stored on removable media may come from other computers that are not protected sufficiently and may be infected. This is why removable media programs are untrusted by default and their actions are usually highly restricted. Some suites can recognize programs that are digitally signed by trusted certificates and do not limit such applications even if they are stored on removable media.

Self-protection

One of the most critical features that the behavior control engine is responsible for is the product's self-protection. Any security protection the product implements becomes useless if malware can disable it. Modern security suites protect all their components so that they can not be switched off or damaged by malware. Self-protection requires protection of the product's processes and threads, files and directories, registry keys and values, installed system services and drivers, COM interfaces, and other resources created by the product that could be accessed by other processes running in the system.

It is vital for the security suite to prevent its own most trusted processes to be infected with malicious code or terminated. Many security suites also rely on regular updates of their databases. The update mechanisms must be designed securely so that malware can not subvert fake update files or block downloading or installing the updates.

Self-protection can be implemented naturally using common behavior control rules that prohibit manipulation of the product's resources; or it can be implemented separately from the security model provided to other applications, in which case the components of the security product are protected better than any other application and resource in the system. Both approaches are common in today's security suites.


Back to contents

Continue to Features of Modern Security Suites – Part 3