Security Mechanisms

Although a robust architecture is a good start, real security requires that you have security mechanisms in place to control processes and applications. Some good security mechanisms are described in the following sections.

Process Isolation

Process isolation is required to maintain a high level of system trust. To be certified as a multilevel security system, process isolation must be supported. Without process isolation, there would be no way to prevent one process from spilling over into another process's memory space, corrupting data or possibly making the whole system unstable. Process isolation is performed by the operating system; its job is to enforce memory boundaries.

For a system to be secure, the operating system must prevent unauthorized users from accessing areas of the system to which they should not have access. Sometimes this is done by means of a virtual machine. A virtual machine allows the user to believe that they have the use of the entire system, but in reality, processes are completely isolated. To take this concept a step further, some systems that require truly robust security also implement hardware isolation. This means that the processes are segmented not only logically, but also physically.

Operation States

When systems are used to process and store sensitive information, there must be some agreed-upon methods for how this will work. Generally, these concepts were developed to meet the requirements of handling sensitive government information with categories such as sensitive, secret, and top secret. The burden of handling this task can be placed upon either administration or the system itself.

Single-state systems are designed and implemented to handle one category of information. The burden of management falls upon the administrator, who must develop the policy and procedures to manage this system. The administrator must also determine who has access and what type of access the users have. These systems are dedicated to one mode of operation, so they are sometimes referred to as dedicated systems.

Multistate systems depend not on the administrator, but on the system itself. They are capable of having more than one person log in to the system and access various types of data, depending upon the level of clearance. As you would probably expect, these systems are not inexpensive. Multistate systems can operate as a compartmentalized system. This means that Mike can log into the system with a secret clearance and access secret-level data, while Carl can log in with top-secret level access and access a different level of data. These systems are compartmentalized and can segment data on a need-to-know basis.

Unfortunately, things don't always operate normally; they sometimes go wrong, and a system failure can occur. A system failure could potentially compromise the system. Efficient designs have built-in recovery procedures to recover from potential problems:

• Fail-safe If a failure is detected, the system is protected from compromise by termination of services.
• Fail-soft A detected failure terminates the noncritical process, and the system continues to function.

Protection Rings

So how does the operating system know who and what to trust? It relies on rings of protection. Rings of protection work much like your network of family, friends, coworkers, and acquaintances. The people who are closest to you, such as your spouse and family, have the highest level of trust. Those who are distant acquaintances or are unknown to you probably have a lower level of trust. It's much like the guy I met in Times Square trying to sell me a new Rolex for $100I had little trust in him and the supposed Rolex!

In reality, the protection rings are conceptual. Figure 5.1 shows an illustration of the protection ring schema.

Figure 5.1. Rings of protection.

The protection ring model provides the operating system with various levels at which to execute code or restrict its access. It provides much greater granularity than a system that just operates in user and privileged mode. As you move toward the outer bounds of the model, the numbers increase and the level of trust decreases.

• Layer 0 is the most trusted level. The operating system kernel resides at this level. Any process running at layer 0 is said to be operating in privileged mode.
• Layer 1 contains nonprivileged portions of the operating system.
• Layer 2 is where I/O drivers, low level operations, and utilities reside.
• Layer 3 is where applications and processes operate. Items such as FTP, DNS, Telnet, and HTTP all operate at this level. This is the level at which individuals usually interact with the operating system. Applications operating here are said to be working in user mode.

Trusted Computer Base

The trusted computer base (TCB) is the sum of all the protection mechanisms within a computer and is responsible for enforcing the security policy. This includes hardware, software, controls, and processes. The TCB is responsible for confidentiality and integrity. It monitors four basic functions:

• Input/output operations I/O operations are a security concern because operations from the outermost rings might need to interface with rings of greater protection. These cross-domain communications must be monitored.
• Execution domain switching Applications running in one domain or level of protection often invoke applications or services in other domains. If these requests are to obtain more sensitive data or service, their activity must be controlled.
• Memory protection To truly be secure, the TCB must monitor memory references to verify confidentiality and integrity in storage.
• Process activation Registers, process status information, and file access lists are vulnerable to loss of confidentiality in a multiprogramming environment. This type of potentially sensitive information must be protected.

An important component of the TCB is the reference monitor, an abstract machine that is used to implement security. The reference monitor's job is to validate access to objects by authorized subjects. The reference monitor is implemented by the security kernel, which is at the heart of the system and handles all user/application requests for access to system resources. A small security kernel is easy to verify, test, and validate as secure. However, in real life, the security kernel is usually not that small because processes located inside can function faster and have privileged access. To avoid these performance costs, Linux and Windows have fairly large security kernels and have opted to sacrifice size in return for performance gains. No matter what the size is, the security kernel must

• Control all access
• Be protected from modification or change
• Be verified and tested to be correct