Solaris Internals: Solaris 10 and OpenSolaris Kernel Architecture (2nd Edition)

2017-07-07 02:10:07

6.8. Devices

All applications make use of devices; however, the great majority of applications interact directly only with pseudo-devices, which makes the task of providing a zoned device environment feasible. These are the key goals for providing devices in a zone:

Security. Users interacting with devices appearing in a zone must not be able to use those devices to compromise another zone or the system as a whole.

Virtualization. Some devices must be modified to provide namespace or resource isolation for operation in a zone.

Administration. It must be easy to place a default, safe collection of devices in a zone; warnings must occur when an administrator attempts to place unsafe devices into a zone; it must be possible for a knowledgeable administrator to assign physical devices to zones when needed.

Automatic Operation. With the advent of devfsadm(1M), device file system management in Solaris became largely automated. Zones should not require administrator intervention to create dynamically managed device nodes.

This section begins by classifying devices according to their virtualization and security characteristics. Discussions of the /devices namespace, device privilege and permission, device administration tools, and the special handling required to support pseudo-terminals follow.

6.8.1. Device Categories

Treatment of devices with respect to zones must of necessity vary depending on the type of device. For this discussion, we divide devices into the following categories:

Unsafe. Devices that cannot be safely used within a zone.

Fully virtual. Devices that reference no global state and may safely appear in any zone.

Sharable-virtual. Devices that reference global state, but may safely be shared across zones, possibly as the result of modification made by the zones project.

Exclusive. Devices that can be safely assigned to and used exclusively by a single zone.

6.8.1.1. Unsafe Devices

Examples of unsafe devices include those devices that expose global system state, such as:

/dev/kmem

/dev/cpc (cpc(3CPC))

/dev/trapstat (trapstat(1M))

/dev/lockstat (lockstat(7D))

There is no way to allow use of such devices from a zone without violating the security principles of zones. Note that this is not restricted to control operations; for example, read access to /dev/kmem will allow a zone to snoop on activity within other zones (by looking at data stored in kernel memory).

This category includes most physical device instances present on the platform, including bus nexus devices, platform support drivers, and devices in support of the device administration infrastructure. All these devices are central to the operation of the platform as a whole and are not appropriate to expose for monitoring or control by the lower-privilege environment inside a nonglobal zone.

6.8.1.2. Fully Virtual Devices

A few of the system's pseudo-devices are "fully virtualized"; these are device instances that reference no global system state and may safely appear in any zone. An excellent example is /dev/tty (tty(7D)), which references only the controlling terminal of the process in whose context it executes.

Other fully virtual devices include

/dev/null (null(7D)) and /dev/zero (zero(7D)).

/dev/poll (poll(7D))

/dev/logindmux, used to link two streams in support of applications including telnet(1).

6.8.1.3. Sharable Virtual Devices

Those device instances that reference some sort of global state but may be modified to be zone-compatible are said to be sharable virtual devices. Some examples:

/dev/kstat (kstat(7D))

/dev/ptmx (ptm(7D)), the pty master device. See the "Pseudo-Terminals" discussion in Section 6.8.5.1.

A principal example of such a device is the random(7D) driver, which exports the /dev/random and /dev/urandom minor nodes. In this case, the global state is the kernel's entropy pool, from which it provides a stream of cryptographic-quality random bytes.

6.8.2. `/dev and /devices` Namespace

The devfs(7FS) file system is used by Solaris to manage /devices. Each element in this namespace represents the physical path to a hardware device, pseudo-device, or nexus device; it is a reflection of the device tree. As such, the file system is populated by a hierarchy of directories and device special files.

The /dev file hierarchy, which is today part of the / (root) file system, consists of symbolic links (logical paths) to the physical paths present in /devices. /dev is managed by a complex system comprising devfsadm(1M), syseventd(1M), the devinfo(7D) driver, libdevinfo(3lib), and RCM.

Few end-user applications reference /devices. Instead, applications reference the logical path to a device, presented in /dev. So, while the system's /devices file system is important to a few system administration applications, it was decided that /devices would not appear within a zone.

6.8.3. Device Management: Zone Configuration

An interface indicates which devices should appear in a particular zone; this functionality is provided by the zonecfg(1M) infrastructure according to a flexible rule-matching system. Devices matching one of the rules are included in the zone's /dev file system.

To include an additional device in the zone, the administrator can specify it by adding an additional rule:

zonecfg:demozone> add device zonecfg:demozone:device> set match=/dev/scsi/scanner/c3t4* zonecfg:demozone:device> end zonecfg:demozone> info device device match: /dev/scsi/scanner/c3t4*

With this syntax, a device can be specified in one of three ways.

6.8.4. Device Management: Zone Runtime

Zones adds new interfaces and capabilities to the devfsadm daemon; the system's single instance of devfsadmd is part of the virtual platform layer for each zone. The changes are somewhat obscure and are enumerated below.

When the daemon starts up, it discovers which zones on the system are ready or running; it is assumed that such zones have a valid /dev file hierarchy. devfsadmd retains a list of said zones; it also loads the zone's configuration database in order to know the <device> matching rules.

When the virtual platform is set up, the zoneadmd(1M) invokes the devfsadm command (not the daemon) and passes arguments to it indicating which zone's /dev directory should be populated. Additionally, the zone is registered with the devfsadmd daemon by a door call to a new door, /dev/ .zone_reg_door.

When a zone is discovered by (1) or (2) above, devfsadmd creates a file under the zone's root directory at /dev/.devfsadm_synch_door and attaches the appropriate door to it. It also loads the zone's configuration database in order to know applicable <device> matching rules. If the zone being registered is already registered, its configuration is reread, and its door file is recreated.

When the virtual platform is destroyed, the zone is unregistered from devfsadmd, which then frees associated resources and ceases to manage the zone.

When a device-related event occurs (for example, in response to a hotplug event), a device entry in /dev may need to be created. When this occurs, the devfsadmd link-generator module calls the devfsadm_mklink() routine.
This routine first services the global zone, establishing the device symbolic link requested by the link-generator. Next, the routine iterates across all registered zones. Instead of creating links, however, the routine creates device nodes with mknod(2); node creation is subject to the filtering rules for the zone's configuration.

6.8.4.1. Read-Only Mount of `/dev`

A significant problem with this approach is that devfsadmd must "reach into" zones in order to execute mknod, create, delete and symlink on files. This violates zone file system principles outlined in Section 6.6, and there is a distinct danger of buffer-overrun and symlink attacks.

To solve this problem, the /dev file system is loopback-mounted into the zone by means of a read-only mount. Device nodes can be freely accessed (opened O_RDWR, for example) by zone processes, but other file operations (creation, unlink, sym-link, link, etc.) are prevented.

A risk is associated with this change; some applications may expect to be able to manipulate /dev enTRies. However, such applications are generally not zone-appropriate, so we believe this risk is minimal.

6.8.4.2. Device Privilege Enforcement

In Solaris, privileges held by applications interact with device drivers in a complex way. File system permissions, device policy (see getdevpolicy(1M)), and driver-based privilege checks (drv_priv(9F) and priv_policy(9F)) may all come into effect during a call to open(2). This section discusses issues related to driver privileges.

6.8.5. Zone Console Design

Figure 6.1 demonstrates that zones export a virtualized console. More generally, the system's console is an important and widely referenced notion; as seen in previous examples, the zone console is a natural and familiar extension of the system for administrators.

While zone consoles are similar to the traditional system console, they are not identical. In general, the notion of a system console has the following properties:

Applications may open and write data to the console device.

The console remains accessible when other methods of login (such as telnet(1)) fail.

The system administrator uses the system console to interact with the system when no other system services are running.

The console captures messages issued at boot-up.

The console remains available even when the operating system has failed or shut down; that is, you can remain "on console" of systems that are powered down or rebooted, even though no I/O may be possible.

The console need not have a process consuming data written to it in order to drain its contents.

Windowing systems make use of the SRIOCSREDIR ioctl and the redirmod STREAMS module to redirect console output to a designated terminal in the window system.

The console can be configured such that console messages are copied to auxiliary hardware devices like terminals or serial lines by the consadm(1M) command.

The zone console design implements the most crucial subset of these; future projects could enable additional functionality as customer needs demand. The zone console is implemented by the zcons(7D) driver. As in a normal Solaris instance, a non-global zone's console I/O (including zone boot messages) are directed to this device.

Within a non-global zone, /dev/console, /dev/msglog, /dev/syscon, /dev/ sysmsg, and /dev/systty are all symbolic links to the /dev/zconsole device.

The auxiliary console facilities provided by consadm(1M) are not supported for zone consoles; additionally, the SRIOCSREDIR ioctl is not supported.

A zone's console is available for login once the zone has reached the ready state and can last across halt/boot cycles.

6.8.5.1. Pseudo-Terminals

Solaris's pseudo-terminal support consists of a pair of drivers, ptm(7D), and pts(7D); several STREAMS modules, including ptem(7M) (terminal emulator), ldterm(7D) (line discipline), and ttcompat(7M) (V7, 4BSD and XENIX compatibility); and userland support code in libc and /usr/lib/pt_chmod.

The ptm and pts drivers are enhanced such that an open of a pts device can occur only in the zone that opened the master side for the corresponding instance (the ptm driver is self-cloning).

In the case of the zlogin(1) command, it is necessary to allocate a pty in the global zone and to then "push" that pty into a particular zone. To accomplish this, a private zonept(3C) library call is introduced. zonept issues the ZONEPT ioctl to the master device requesting that the current terminal be assigned the supplied zone owner.

The number of pts devices may grow without bound.^[6] This dynamic growth is triggered when the number of ptys must grow beyond the current limit (for example, when allocating the 17th pty). This must function even when the application opening /dev/ptmx is unprivileged. This functionality (provided in libc) relies on a door call to devfsadmd, which is wrapped by the di_devlink_init() inter-face. The code can also start devfsadmd as needed.

^[6] In practice, the number of pts devices on the system is limited by the size of the minor number space in the operating system.

To solve this problem in the zone, we place the appropriate door to the global zone's devfsadmd into the non-global zone. This allows the zone to demand that the global zone install the appropriate /dev/pts/ device nodes as needed. There is some risk of denial-of-service attack against devfsadmd here. A future enhancement may be to allow the zoneadmd(1M) process (see Section 6.2.3.1) to act as a "proxy server" for such door calls; zoneadmd would simply turn the zone's request for devlink creation into a call to di_devlink_init(), which would in turn start devfsadmd if it was not running. zoneadmd could throttle requests as needed.

A final issue is that the global administrator should be able to limit the number of pseudo-terminal devices available to each zone. One possibility is to implement a zone-scoped resource control (See Section 6.10) for pty creation.

6.8.6. ftpd

Solaris provides the ftpconfig(1M) command to set up anonymous FTP environments. Anonymous FTP allows users to remotely log on to the FTP server by specifying the user name "ftp" or "anonymous" and the user's email address as password; anonymous FTP environments are run in a chroot'd environment, and ftpconfig uses cpio(1) to propagate device special files from /dev to the chroot area as follows:

cpio -pduL "$home_dir" >/dev/null 2>&1 <<-EOF /dev/conslog /dev/null /dev/udp /dev/udp6 /dev/zero EOF

The zone application environment lacks the SYS_DEVICES privilege, so this will fail. Chroot environments that require device special files cannot be created from within a zone. The global zone administrator can, however, cooperate with the zone administrator to set up such an environment.