The Network IDS Is Not the Destination Node

Recently, Ptacek and Newsham (1998) identified several weaknesses in intelligent network sniffers. Similar concerns about network IDSs were simultaneously identified by Paxson (1998). At the heart of the discussion is the fact that a network IDS cannot know for sure what is happening on the network nodes themselves. An inbound packet has a destination IP address for the intended recipient. The network IDS does not know for sure whether the destination node will accept a packet or discard it. To really know the behavior of the remote node, the network IDS must contain a good deal more knowledge than it does today.

For example, operating systems handle fragmented packets differently. Some OSs will discard a packet if it contains overlapping sequence numbers with previously received packets. Other OSs will accept the packet and process it; thereby overlaying existing data received earlier.

One example given by Ptacek and Newsham is the checksum on a packet. Most systems will discard a packet with a bad checksum, but some network IDSs do not currently check this as part of packet inspection. Now that the issues have been publicly indicated, you can expect future IDS releases to include this feature. Because the IDS is not the real recipient of the packet itself, the IDS cannot know for sure how the destination node will really handle the packet. Many factors, such as memory limitations, determine whether a packet is dropped or processed by the intended receiver. The IDS cannot possibly know all of these variables for each node on the subnet it monitors.

This subtle point—not knowing for sure how the destination node will respond—can lead to some interesting attacks. In an insertion attack, an adversary will inject a packet that the IDS will accept but which the destination node will reject or drop. The IDS and the destination node are thus in different states because they are processing different data. A few clever hacks show how to sneak a phf (or similar) attack past current network IDSs even though they will be executed successfully on the destination machine. As a teaser to completely read the referenced papers, think about how you could send a string such as “ppppphhhhhhhffffffff?rrrrmmmm *”. You could send individual one-byte packets formed in a such a way that the destination node would drop all packets except those that combine into “phf?rm*”.

You also can inject SYN packets that can trick the IDS into resetting its state, even though the target node ignores the packet. When the IDS resets its state, pattern matching that was in progress is restarted; thus, losing any attacks in progress for that TCP session.

In an evasion attack the destination node accepts a packet that is ignored by the IDS. For example, the attacker could send extra packets with the same sequence number as a previous packet but with different data. The IDS might drop the packet because the sequence number was already used. The destination node (depending on the OS) would accept and process the packet, replacing a previous substring with one that turns the entire message into a hack. Many other evasion attacks are possible because of protocol problems with IP, TCP, or UDP.

Another problem to watch for is denial-of-service launched against the IDS itself. While sniffing network traffic, the IDS maintains a queue for incoming packets. The amount of memory that is allocated for these queues often can be configured, but it is eventually bounded by some constraint—whether physical memory or virtual memory. If a hacker knows what the IDS is looking for, it’s not difficult to inject a number of packets that must be processed. The IDS can run out of resources.

Recall that network IDSs operate in passive mode and sniff the network. They do not block packets, which is one thing that firewalls do for you. When a firewall fails, it (generally) does not let any packets through, and thus your security policy is not violated. If a network IDS fails due to resource exhaustion, your policy can be violated because the IDS does not block packets; it works by alerting you when there is a problem.

In this section, you’ve seen a number of problems with network IDSs. Don’t let this discussion give you the impression that these IDSs are weak products. No security tool is perfect. Network IDSs are critical for improving security at your site. These IDSs also are very useful for catching a number of attacks against your network. Vendors have been responsive to concerns such as those mentioned previously and have speedily added fixes in new releases.

Getting around the Encryption Problem

The encryption issue is a particularly sticky one for network IDSs. It is highly likely that over time, you will see more encryption of application-level data. You need encryption for secure communications. There isn’t much you can do to change the limitations on network IDSs when encryption hides the network packet content. IDS vendors do have a few alternatives, though.

Instead of spelling the demise of network IDSs, encryption argues for a repositioning of the technology. If a large number of applications rely on SSL, for example, the SSL libraries could be enhanced to invoke network IDS routines after the packets are decrypted. The added advantage is that the IDS routines also would be running on every node that uses SSL, thus providing previously unavailable opportunities for misuse detection.

A similar approach would be to embed the network IDS routines directly in the network stack of the operating systems. As data from a socket is read by the application, the IDS routines optionally could be called to look for attacks. A socket option that each application could set would provide granular control over when to invoke the IDS routines. A system-wide switch to enable the IDS for all socket reads is another configuration possibility. The IDS routines would be running on each node with the advantage for misuse detection again.

Repackaging network IDSs as a set of library APIs is another possibility. As applications read data from sockets, they optionally could call the IDS routines to check for problems. This is a slight variation on the socket option suggested above. One difference is that it would be easier for a vendor to market network IDS libraries than it would be to convince a number of OS providers to embed the code. Unfortunately, if the APIs are optional, it’s difficult to encourage programmers to modify existing software to take advantage of the libraries. When the IDS routines are a default option in the network stack of the OS, the solution is more transparent and easier for application programmers to use.