Intrusion Detection: Network Security beyond the Firewall
(Publisher: John Wiley & Sons, Inc.)
Author(s): Terry Escamilla
ISBN: 0471290009
Publication Date: 11/01/98

Previous Table of Contents Next


Finally, attacks that operate on data at the application level can reasonably be detected only by the application. True, some application-level attacks will leave a residue in the operating system logs, and an OS level IDS could detect them. An example is someone who uses the test.cgi attack to trick the Web server into deleting some of the GIF images on the system. A system-level monitor would detect this activity because the subject and object (the Web server daemon and the file) have meaning within the OS level.

An application with its own notions of subjects and objects will not leave any activity residue that the OS can understand. Databases are the best example of this. The database files are divided into logical areas for records, fields, and metadata. Only the DBMS knows about access control to these various subdivisions of the file. When several database users read and write database content, the OS sees only that the database daemon was repeatedly accessing the file in which the database is stored. The OS does not see into the file in the way that the database manager does. If a database application program can be hacked, the OS will still not know what is going on. Only the DBMS will be able to detect the intrusion because it understands the actions performed by the hack.

In the same way that various security tools were needed to solve different problems, IDSs are also needed at different levels of the stack—network, system, and application. Until a single IDS can combine all of these capabilities, you will need to pick up a different tool for each layer.

At the network layer, the interesting data to look for includes any information that might appear in the headers of network packets:

  IP addresses
  Port numbers
  Sequence numbers
  Protocol type
  Names of network entities, such as server names in SMB
  Checksums
  Option settings, like source routing for IP

These fields are the ones that network monitors inspect when looking for attacks such as address spoofing and session hijacking. At this level, a number of events cannot be easily detected such as failed logins or file deletions. Assuming that the network IDS relies on network traffic only, the only way to know that a telnet login failed is by knowing as much about the packets as telnet does. In other words, just to detect a telnet failure, one would need to know the telnet protocol and watch it through the packets. The same would be needed for FTP failed logins. Detecting failed authentication in third party products such as SecurID would be next to impossible, particularly if the network traffic is encrypted.

You can see that the more types of attacks an IDS wants to glean from data, the more details it must know about the data. At the network data, the most easily extracted information comes from the list shown here, such as IP addresses. Inspecting the packets can be trivial for some types of attacks, but general-purpose packet data scanning requires code that approaches the complexity of the application itself, which processes the packets. This is really not practical.

Types of network threats that can be detected are those which are arguably network or system attacks. One example is the AIX vulnerability known as rlogin -froot, which exploited a programming bug in rlogind. The bug permitted a remote user to gain root shell access (ouch!) without specifying a password. Because rlogin is really a network application for TCP/IP, you might call this a network attack. Given that the result is a root login shell, it is also a system attack. So much for clear definitions in intrusion detection.

How Does One Get the Data?

In order to monitor a system, the data must be obtained somehow. To get the most realistic view of events occurring in real time, the best approach is to go directly to the source. Replacing or intercepting system calls in the bowels of the software, the kernel of the OS for example, is the most reliable way to gather data for a single event.

If you are an IDS vendor and you choose to intercept an OS routine, performance consequences will occur. You cannot avoid these consequences if the original routine has M instructions, and you intercept and require N additional instructions, the total path length will be at least M+N. Because the OS incurs penalties when loading and unloading your subroutines, the price is more than just M+N instructions. This simple principle can be stated as follows:

Interception of routines will tradeoff performance for accessibility to the data.

Sometimes this tradeoff is unavoidable. Firewalls provide a good example. Without the additional packet filtering code in the IP stack, the network packets would travel faster. However, to make intelligent security policy decisions, the packets are intercepted and fed through additional code paths that implement the firewall software. As you saw in Chapter 3, “The Role of Access Control in Your Environment,” Memco’s SeOS is another product that intercepts system calls to expand the security policy capabilities of the base OS and applications running on top of it. Recall, however, that SeOS and firewalls are designed to be preventative products, although both product types provide ample logging from which one can deduce intrusive behaviors.

In the previous chapter you saw that network monitors can intercept data without affecting performance. As long as the network sniffer is running on an independent node and merely passing packets through, no performance penalty should occur. If the sniffer is also making decisions about whether to route the packet, because it is being inspected for decisions, performance implications will occur. Network IDSs that sniff packets and report events after the fact are nonintrusive.

Intrusion Detection Inside a Firewall

Is a firewall an IDS? Today, the answer is “No.” Commercial firewalls are not marketed as IDSs, although many probably will include scanners if not already doing so. The purpose of the scanner is to check the firewall for problems, but add-ons make the scanner useful for checking other systems.

Could a firewall be an IDS? Sure. Checkpoint’s firewall provides for abitrary user-exits on received packets. The language the firewall demands is INSPECT, a proprietary language that people can use to add other filtering capabilities to received packets. If INSPECT is general enough to support stored data values and the main programming statements (sequence, selection, and iteration), then it can be used to write intrusion detection code. Axent-Raptor Eagle also checks for some application level attacks in its proxy servers.


Previous Table of Contents Next