Statistical Anomaly Detectors

Statistical anomaly detectors establish a baseline for a number of variables for each subject of interest. The usual example given is tracking the number of times a given user runs particular commands. A threshold is set by the tool administrator. When the threshold is exceeded, an event is generated.

IDSs based on statistical anomaly have a number of proponents. CMDS is the often-quoted example that has a good-sized install base. A number of hacks and misuses can be caught by threshold monitoring. However, difficulties are encountered with this approach. Setting the baseline can be a problem. For experienced users, it’s not uncommon to find noisy data and, thus, run the risk of too many false positives. The soundness of the underlying statistical assumptions also has been questioned. The difference between a legitimate and an illegal privilege escalation is not something that is detected by counting commands. CMDS contains a pattern-matching component to offset these shortcomings with the anomaly detector piece of the tool.

NT System Level Tools

Centrax is developing some very impressive NT system-level IDS tools. The team there is very experienced at building commercial intrusion detection systems. Keep an eye on them for what is likely to be a market leader. Competition is expected from other vendors who also now offer system-level IDS for NT. The Kane Security Monitor sold by Security Dynamics is another player is this space.

Over time, NT IDS offerings will increase in the marketplace. Understanding heterogeneous UNIX audit logs is a complex process that was mastered by only a small number of vendors. NT event logs are more easily available to developers and are more easily understood. Thus, more players are sure to emerge in the NT system-level IDS space.

Network Sniffers

Network IDSs are a critical component of your perimeter defense because they catch attacks that system-level IDSs cannot. IDSs that detect attacks by sniffing network traffic in real time watch for protocol attacks, for attempts to run well-known hacked programs, and for strings that may indicate policy violations. Most sniffers support a client-server model in which a central engine receives notifications from multiple sensors. Like most other real-time IDSs, network sniffers provide real-time alerts and options to terminate offending connections. Some risk exists in automating the kill connection response because often hackers forge IP addresses or launch attacks from compromised systems at legitimate sites (such as universities).

To catch attacks, the sniffer must reside where all of the packets can be seen. As shown in Chapter 9, “Sniffing For Intruders,” in some configurations network packets will not be seen by the sniffer. The best placements for a network IDS sniffer are just inside the perimeter network, in the secure network immediately after the firewall, and after the router or gateway on other subnets. In these positions, the inbound and outbound network packets will be visible. However, anytime two nodes within a subnet exchange packets, and the traffic does not flow past the network IDS, there is a potential for missed attacks.

Sniffers are hampered when network traffic is encrypted between two arbitrary nodes. If the sniffer sits in the network after the firewall, and the firewall is the system that decrypts packets, then the sniffer will see cleartext packets. However, if two nodes have their own IP tunnel, only those nodes will decrypt the packets. In order to be effective in this scenario, the sniffer code must be running in the IP stack just after the packets are decrypted by the IP layer. To complicate the effectiveness of sniffers further, application-level encryption limits what a sniffer can detect to only those attacks in the network protocol itself—address spoofing, session hijacking, SYN Flood, and others. An argument put forward in this book is to allow applications to call IDS routines after the packets have been decrypted.

Some overlap exists between firewalls and network IDSs in that firewalls also look for attacks, such as SYN Flood, Ping of Death, and other protocol exploits. Because both firewalls and sniffers examine network traffic, it’s likely that there could be convergence of these two functions. A recent Infoworld report rated the IBM Emergency Response Center, ISS RealSecure, and Network Flight Recorder highly in their comparison tests. However, only Network Flight Recorder caught all of the attacks in the test suite. In some respects, this result is not surprising because the evaluation team wrote custom signatures specifically to catch the attacks in the test suite.

Finally, some recent papers identify shortcomings in network IDSs. The weaknesses stem from attacks that can cause the IDS and the actual destination of the IP packet to process different datastreams. In some cases, the IDS processes packets that are ignored by the destination node. The converse also is possible. Both of these cases can lead to network-based attacks that are not detected by some sniffers. The products’ owners are undoubtedly addressing these issues in upcoming releases.

Improving upon IDSs

Because IDSs are not the last word in security, opportunities exist for improvement. Here are some important areas in which advances are needed.