Chapter 9
Sniffing for Intruders

This chapter describes how you can catch intruders by watching the network traffic at your site. From the title, you probably thought you were going to learn how to really put your nose to the trail and smell for hackers. On the other hand, perhaps you envisioned that you might pick up a good bloodhound and track down perpetrators through the electronic jungle. Luckily, no vendor is pushing either of these sniffer approaches. The IDSs described in this chapter work by monitoring network traffic as it flows across subnets at your site.

The material in this chapter is broken into two main sections. First, you’ll look at how network IDSs work and examine their pros and cons. Next, some popular network IDSs are described to give you a glimpse of what to expect from leading vendors in this field. Like many of the chapters so far, as you get near the end of this chapter, you will find yourself wondering whether the use of scanners, system IDSs, and network IDSs is finally enough to solve your intrusion detection problems. You probably can guess the outcome.

How Network IDSs Work

What makes a network IDS different from other tools about which you’ve read? The next few sections describe the basic approach to network intrusion detection. After this, you’ll see some example attacks that can be discovered by looking at network packets.

Networks and Subnets

Networked environments usually are divided into multiple subnets for various reasons. By separating computers into physically distinct groups, a network administrator can regulate the flow of traffic into and out of subnets. Gateways and routers control the boundaries of subnets and decide how packets are delivered to each subnet. The primary advantage of subnetting is performance.

For example, if all of the accounting computers are on the sixth floor of a building, physically configuring a separate accounting subnet will limit accounting network traffic to that floor. If other floors do not need to see these packets, why bother routing the network traffic to them? The degenerate case in which all computers are on the same subnet is undesirable because all network traffic flows on the same network. Subnets usually are set up according to departmental organizations, physical buildings, or security boundaries. Therefore, another advantage of subnetting is security.

A router can be used to block the flow of packets from one network to another. The simplest type of packet filtering firewall is a router. Other uses of screening routers were identified in Chapter 4, “Traditional Network Security Approaches,” such as blocking IP address impersonation attempts into your perimeter network.

Naturally, some packets will cross subnets as computers communicate with each other or try to reach computers outside the company’s firewall boundary. Sometimes, a subnet is completely disjointed and communicates with no other subnets. Examples include test environments in many companies. Test coordinators prefer to limit the damage of programs that are not ready for the production environment by restricting the network test environment.

As you can see, in a typical enterprise network, many different subnets are self contained. Network packets may uniquely flow within a subnet and not cross into any other subnet. If you wanted to monitor network traffic in subnets for performance, each subnet would need a separate monitor to watch its unique packets. Why? Simply put, not all subnet traffic is available from a single point on the network. If you really want to watch what everyone on the network is doing—including both insiders and outsiders—then your monitors need to pick up all of the packets. If all of your computers are on a single network, a single monitor is enough. Logically, if you have a network divided into subnets, then you need a monitor on each subnet.

Network IDSs Sniff Network Traffic

Network adapters can be configured to run in promiscuous mode, which allows the adapter to grab all of the packets that it sees on the network. The default behavior of a network adapter for a computer is to grab only those packets destined for that computer (sounds like a reasonable performance enhancing choice). In promiscuous mode, the adapter grabs all packets that it senses on the subnet and passes them up to the device driver. The captured packets are passed from the device driver up to the IDS for analysis.

NetRanger developed by the WheelGroup and now owned by Cisco, has a slightly different setup. In its original form, NetRanger works with routers provided by Network Systems Corporation. The router provides network traffic information to NetRanger software running on a designated computer. Communications between the computer and the router are cryptographically secured for privacy and integrity across a socket connection. One advantage of this architecture is that the monitoring node running NetRanger and the router do not need to be on the same network. The two components of the solution need to be reachable only via some type of network, such as via a private dial-up line. A standalone version of NetRanger that uses a computer with a promiscuous mode adapter is also available now. As you might expect, NetRanger also works quite well with Cisco routers, too.

To be effective at catching intruders, network IDSs need to be positioned properly in the network. In order to catch inbound packets for a subnet, the network IDS must be positioned as the first node after the router in the subnet. Alternatively, the IDS may be placed on a gateway between two subnets to watch for attacks across the subnets. A common placement for network IDSs is immediately after the firewall in an enterprise. Because all inbound and outbound traffic must pass through the firewall, the network IDS easily can reside directly after the firewall and as the first node inside the secure network as shown in Figure 9.1.

Figure 9.1  A properly positioned IDS will see all of the inbound and outbound packets.

You should be sure to understand that the network IDS does not intercept and hold packets before forwarding them to the intended node. Because the IDS’s network adapter is capturing packets in promiscuous mode, it is merely reading the packets as they appear, not grabbing them and holding them until they are analyzed. A network IDS should not interfere with the performance of your network.

Even though the network IDS does not alter your network performance, you should have some performance concerns about the IDS itself. Because it is potentially processing packets for a large number of nodes in your network, the IDS must scale well or else it will drop packets and miss attacks. Before you plop down large amounts of money for a large-scale network IDS solution, insist on performance testing to ensure that the solution meets your needs. You may be able to get around scalability problems by deploying multiple IDSs in a chain and configuring each one to look for a disjointed set of attacks.