In general what happens is that layer N asks layer N-1 to perform some service by calling interfaces provided at the boundary between the layers. Generally, layer N is asking layer N-1 to help transmit some data. Other interfaces might exist for checking the status of requests or querying the state of a layer. When a transmission request is sent from layer N to layer N-1, layer N-1 takes the information packet(s) sent by layer N and wraps them with a packet header to construct a meaningful packet for layer N-1.

Each layer of the network model has a data structure or packet that it understands. The data structure includes the actual data value to transmit and information describing the packet being transmitted. If you want to send your mailing address, the data value would be the character string for your postal address. Other information in the packet might include the length of the data, a sequence number, a protocol identifier, a timestamp, and perhaps some I&A information. The peer layers on communicating systems need this header data in order to be able to assemble, disassemble, and interpret packets. Another way to think about this is that the packet created at layer N and sent to layer N-1 becomes the data in the packet created at layer N-1.

When a message needs to be sent between systems, layering is employed to wrap the message as it passes down through the layers so that peer layers on the receiving system will have the necessary information for handling the packets. When Layer 1 on node B receives the packet from Layer 1 on node A, it performs various checks to ensure that the transmission was successful by inspecting the Layer 1 packet. If no transmission errors have occurred, Layer 1 on node B unwraps the packet and captures the data portion that it then passes up to Layer 2 (on node B). Layer 2 receives the data sent from Layer 1, which is of course a full packet meaningful to Layer 2, and processes the packet. If the packet was fragmented, the data from the Layer 2 packet may need to be cached and concatenated with other incoming packets from node A. At a given layer of the network model, a number of complex processing steps may occur including retransmission of lost packets, special messages for synchronization with the peer layer on the other node, or coalescing of fragmented packets.

Security between Layers on a System

How do you know that layer N and layer N-1 are communicating securely? Because it is assumed that layer N and layer N-1 run on the same node, whatever access control mechanisms ensure communication integrity as part of the standard operating system are assumed to be sufficient for implementation of the network layers on a given node. Network security is not generally concerned with how layers on the same system communicate with each other. Instead, access control facilities supported for communicating processes on the same system are used between layers.

Access control for IPC such as semaphores, message queues, and shared memory work much like file system permissions. Different users or groups are allowed to read, write, create, or perform other operations on message queues depending on how the access control rules are defined for the queue. The layers could communicate just by calling subroutines or library routines. In this case, security of the message exchanges between layers also is handled by the operating system’s security kernel.

A subtle issue about network security within node layers is worth mentioning. The design of most network protocols allows one process on the system to communicate with another process on the same system as if they were talking across a network. You probably are familiar with how this works because the same network node can be running both the client and server programs for a database. On that node, you can access the database server by invoking the database client. As far as the client and server are concerned, they think communications are occurring over a network.

Security between Peer Layers across Systems

The interesting aspect of network security is secure communications between peer layers on different systems. Most network security threats occur during communications between layer N on node A and layer N on node B.

You should know that each layer of the network model is not required to implement the same security architecture as all other layers. Figure 4.2 shows an example in which each layer is using a different technique for network security. Each layer might require an independent security model from the other layers. This practice is actually common. A company with two sites might connect the sites using an encrypted physical connection (the lowest layer). Applications running on the system, such as e-mail, additionally could provide encryption to protect the privacy of messages exchanged between employees. The Application layer and the Physical layer are likely to be using two very different cryptographic implementations, but both are part of the network security solution. Of course, peer layers are required to implement the same security model.

Figure 4.2  Different layers with alternative security solutions.

Recall that packets are wrapped and unwrapped as they are passed between layer N and layer N-1. The security information needed in the packets of layer N will be encapsulated along with the rest of the header when processed by layer N-1. Layer N-1 is unconcerned with the peer security needs of layer N. This notion is a logical extension of the layered model, which is based on the theme that a layer provides services for adjacent layers without understanding the intricacies of packets from those other layers.

From a site security officer’s point of view, this practice can lead to some very complex implementations. Network software layers might each be protected using a different security solution. One reason this is not unusual is because of the way in which systems evolve. The person responsible for secure network communications between two different endpoints, perhaps representing two different sites for the same company, is probably not the same person responsible for all of the applications that run between the sites. If a database transaction system is enabled between the sites, the project administrator for the database probably decides whether the nature of the transactions requires secrecy. The network administrator might want to have some input in the decision process, but most of the time various islands of authority exist in a company responsible for the security of individual application subsystems.