Developing Drivers with the Windows Driver Foundation (Pro Developer)

Synchronization Scope and I/O Callback Serialization

Concurrently active callbacks on the same object might require access to shared object-specific data or might share such data with helper functions. For example, a driver's cleanup and cancellation callbacks often use the same data as its read, write, and device I/O control callbacks. A driver can set synchronization scope to manage concurrency among these functions.

Synchronization scope determines the level in the object hierarchy at which the framework acquires an object presentation lock. This, in turn, controls whether the framework calls certain I/O event callback functions concurrently or serially. The I/O event callbacks that are serialized depend on the synchronization scope and the framework (that is, UMDF or KMDF).

WDF defines the following three synchronization scopes:

• No scope

  No scope means that WDF does not acquire an object presentation lock and therefore can call any I/O event callback concurrently with any other event callback. No scope is the default for KMDF. A KMDF driver can "opt in" to serialization for a device or queue object by setting device scope or queue scope when it creates the object.

• Device scope

  Device scope means that WDF acquires the device object presentation lock before calling certain I/O event callbacks. Therefore, device scope results in serialized calls to the I/O event callbacks for an individual device object or for any file objects or queue objects that are its children. UMDF uses device scope by default.

• Queue scope

  Queue scope means that WDF acquires a queue object presentation lock, and thus the I/O event callbacks are serialized on a per-queue basis. A KMDF driver can specify queue scope for a device object so that the I/O event callbacks for each queue are serialized but the callbacks for different queues can run concurrently.

  The initial UMDF release does not support queue scope.

Device Scope and Queue Dispatch Methods

The combination of synchronization scope and queue dispatch method provides great flexibility for the control of I/O requests through your driver.

Figure 10-4 shows the objects that are affected if the driver sets device scope for a device object.

Figure 10-4: Device synchronization scope

In the figure, a driver object has a single device object and the device object has two child queues and a child file. The read/write queue has a child request object. If the driver sets device synchronization scope on the device object, by default certain callbacks of the device object, the queue objects, the request, and the file would be serialized.

To examine how the synchronization scope and queue dispatch method interact, consider a hypothetical I/O sequence. Assume that the driver in Figure 10-4 configures the read/write queue for parallel dispatching of read and write requests and device I/O control queue for sequential dispatching of device I/O control requests. The following list shows the framework's actions as I/O requests arrive for the driver.

When the first read request arrives, the framework acquires the device presentation lock and dispatches the request to the queue's read callback function. Meanwhile, two device I/O control requests arrive. The framework adds them to the device I/O control queue but does not invoke a callback-even though no other requests from this queue are active-because the device object presentation lock prevents additional callbacks until the read callback returns.

When the read callback returns, the framework calls the device I/O control callback with the first device I/O control request. The device I/O control callback cannot complete the request, but instead sends it to the default I/O target and then returns. However, the framework does not dispatch the next device I/O control request, because the queue is sequential and the driver has neither completed nor requeued the first device I/O control request.

While the driver handles the device I/O control request, another read request and a write request arrive. The framework dispatches the first of those requests to an I/O event callback on the read/write queue. When that callback returns-assuming that the device I/O control request on the other queue still has not completed-the framework dispatches the next request from the read/write queue. Even though the read/write queue is configured for parallel processing, the device-level lock prevents the framework from calling the queue's read and write event callbacks concurrently.

Although using synchronization scope can greatly simplify a driver, it can also introduce deadlocks. For example, assume that the I/O target in this example cannot complete the device I/O control request without getting additional information or requesting an action from the driver's device stack. If the I/O target sends a request to the device stack while the original request is still pending, the driver deadlocks. The framework cannot dispatch the new request to the driver until the original request completes.

Never call outside your driver when holding a lock.

The bottom line is that synchronization scope is in general useful and safe in monolithic drivers. If your driver is going to be in a stack of drivers, unless you know and can control the behavior of the other drivers in the stack, it's problematic to opt for synchronization scope.-Eliyas Yakub, Windows Driver Foundation Team, Microsoft

Synchronization Scope in UMDF Drivers

UMDF drivers can configure synchronization scope for its I/O queue and file callbacks. To set synchronization scope, a driver calls the SetLockingConstraint method of the IWDFDeviceInitialize interface before it creates the device object. Possible values are the following constants of the WDF_CALLBACK_CONSTRAINT type:

• WdfDeviceLevel

  Sets device scope. This is the UMDF default.

• None

  Sets no synchronization.

Table 10-1 lists the UMDF callback interfaces and methods that a driver can implement on device, queue, and file callback objects and shows whether the framework acquires the device presentation lock before it calls each method.

Callbacks for different device objects are not serialized. Thus, if a UMDF driver manages two devices and creates a device object and one or more I/O queues for each device object, the framework can concurrently invoke the callbacks for the two device objects. Each device object is part of a different device stack and thus in a different host process that has its own runtime environment and thus no risk of a race condition.

If the driver sets the synchronization scope to None, UMDF can concurrently call any event callback with any other event callback and the driver must create and acquire all its own locks. A critical section is often a good choice for such a lock because it prevents thread preemption. However, a multithreaded driver might instead require a mutex, which can be acquired recursively.

The USB Filter sample driver sets synchronization scope to None in the Device.cpp file, as follows:

FxDeviceInit->SetLockingConstraint(None);

The USB filter driver uses no synchronization scope because it does not maintain any state information from one request to the next and does not require locks.

Synchronization Scope in KMDF Drivers

A KMDF driver can configure synchronization scope for I/O event, file, and queue callbacks by setting the SynchronizationScope field in the object attributes structure when it creates a device object, I/O queue object, or file object. The framework can also apply synchronization scope to DPC, timer, and work item callbacks, as described in "Automatic Serialization for KMDF DPC, Timer, and Work Item Callbacks" later in this chapter.

Possible values for SynchronizationScope are the following WDF_SYNCHRONIZATION_ SCOPE constants:

• WdfSynchronizationScopeDevice

  Sets device scope.

• WdfSynchronizationScopeQueue

  Sets queue scope.

• WdfSynchronizationScopeNone

  Sets no synchronization scope.

• WdfSynchronizationScopeInheritFromParent

  Sets the scope for an object to the same value as that of its parent object.

If the driver sets synchronization scope for a device or queue object, the framework serializes callbacks to a driver's I/O request cancellation callbacks but not to the completion callbacks. A driver cannot set synchronization scope for an I/O request object and should accept the default setting for this field.

Table 10-2 lists the KMDF callbacks that are subject to synchronization scope. The table indicates which callbacks are serialized when the driver sets device scope and which are serialized when the driver sets queue scope on the device object, along with the execution levels at which each callback can run.

If the driver accepts the defaults for synchronization scope and automatic serialization, the framework does not serialize any of the callbacks in the table.

Synchronization Scope Defaults

The default scope for the driver object is WdfSynchronizationScopeNone, and the default scope for all other objects-which are children of the driver object-is WdfSynchronizationScopeInheritFromParent. Thus, by default, the framework does not acquire any locks or serialize any of the I/O event, file, and queue callbacks. To use framework synchronization, a KMDF driver must explicitly set the synchronization scope on the device or queue objects.

 Chapter 5 covers the object hierchy  Because scope is inherited by default, a driver can easily set synchronization scope for its I/O queues by setting the scope for the device object, which is the parent of the I/O queues.

Device Scope

If the driver sets device scope, the framework acquires the device object presentation lock before it invokes the callbacks for an individual device object or for any file objects or queues that are children of the device object. However, callbacks for different device objects are not serialized and therefore can run concurrently.

To use device scope, the driver sets WdfSynchronizationScopeDevice for the device object.

Queue Scope

 KMDF  If a KMDF driver specifies queue scope for a device object, the framework serializes calls to the I/O event callbacks for the queue object but not for the file object. Consequently, the EvtDeviceFileXxx and EvtFileXxx callbacks for the device object and its queues can run concurrently.

To use queue scope, the driver sets WdfSynchronizationScopeQueue for the device object and WdfSynchronizationScopeInheritFromParent for the queue object. Queue scope means that only one of the listed callback functions can be active for each queue at any one time. A driver cannot set synchronization scope separately for each queue.

Figure 10-5 shows the objects that are affected if a KMDF driver sets queue scope for the device object that was shown in Figure 10-4 earlier in the chapter.

Figure 10-5: Queue synchronization scope for a KMDF driver

If the driver sets queue scope, the framework acquires the presentation lock at queue level. Consequently, certain callbacks for the read/write queue and the request object are serialized with respect to each other. Callbacks for the device I/O control queue are similarly serialized. However, callbacks for the read/write queue and the request object can run concurrently with callbacks for device I/O control queue.

 Queue scope and file objects  Queue scope does not apply to file objects. However, by default, a file object inherits its synchronization scope from its parent. Therefore, if the driver sets queue scope for the parent device object and registers one or more file object callbacks, the driver must explicitly set either device scope or no scope for the file object. If the driver fails to do so, the framework generates an error.

If a driver sets device scope for a file object, it must also set the execution level for the object to WdfExecutionLevelPassive, as explained in "Execution Level in KMDF Drivers" later in this chapter.

The best practice for file objects is to use no scope and to acquire the appropriate locks in the event callback functions when synchronization is required.

KMDF Example: Synchronization Scope

Listing 10-1 shows how the Serial sample driver-which manages serial ports-sets WdfSynchronizationScopeNone for its file objects. This code appears in the Serial\Pnp.c file.

Listing 10-1: Setting synchronization scope in a KMDF driver

WDF_OBJECT_ATTRIBUTES_INIT(&attributes); attributes.SynchronizationScope = WdfSynchronizationScopeNone; WDF_FILEOBJECT_CONFIG_INIT( &fileobjectConfig, SerialEvtDeviceFileCreate, SerialEvtFileClose, WDF_NO_EVENT_CALLBACK // Cleanup ); WdfDeviceInitSetFileObjectConfig( DeviceInit, &fileobjectConfig, &attributes );

The code in the listing appears in the driver's EvtDriverDeviceAdd callback before the driver creates the device object. To set no synchronization scope for the file objects, the driver initializes an object attributes structure and sets the SynchronizationScope field to WdfSynchronizationScopeNone. It next initializes a WDF_FILEOBJECT_CONFIG structure with pointers to the driver's file object event callback functions. Finally, the driver calls WdfDeviceInitSetFileObjectConfig to record the file object configuration information and the object attributes in the WDFDEVICE_INIT structure that it will eventually pass when it creates the device object.

In the Serial driver, the SerialEvtDeviceFileCreate and SerialEvtFileClose file object callback functions do not share data with other device object callback functions and therefore do not require locks. "Serial" in the function names indicates that the functions are part of the Serial sample driver; it does not mean that the callbacks are serialized.

Automatic Serialization for KMDF DPC, Timer, and Work Item Callbacks

Every DPC, timer, or work item object is the child of a device object or of a queue object. To simplify a driver's implementation of callbacks for DPCs, timers, and work items, KMDF by default automatically serializes the execution of those callbacks with the execution of the callbacks for the parent object. A driver can override the default by setting AutomaticSerialization to FALSE in the configuration structure during creation of the DPC, timer, or work item object.

When automatic serialization is enabled, the framework applies the SynchronizationScope setting for the device or queue that is the parent of the object. Thus, if the parent of a timer is a device object for which the driver set device scope, the framework acquires the device object presentation lock before it invokes the EvtTimerFunc callback.

You should use automatic serialization only for callback functions that can run at the same IRQL. For example, if a single device object has a child DPC object and a child work item object, you should not enable automatic serialization for both of the children because a DPC callback always runs at DISPATCH_LEVEL and a work item callback always runs at PASSIVE_LEVEL. The framework cannot use the same lock to serialize both of these functions. The following topic, "Execution Level in KMDF Drivers," provides more information about how IRQL affects serialization.

The Serial sample creates DPCs that run when its timers expire and when it completes a read or write operation. The driver enables automatic synchronization for all of these DPCs so that they are serialized with the other callbacks for the I/O queue or device object. Consequently, the DPCs can access the device and queue context areas without using locks. Listing 10-2 shows how the Serial driver sets automatic serialization for one of these DPCs. This code appears in the Kmdf\Serial\Utils.c source file.

Listing 10-2: Setting automatic serialization for a DPC object

WDF_DPC_CONFIG_INIT(&dpcConfig, SerialCompleteWrite); dpcConfig.AutomaticSerialization = TRUE; WDF_OBJECT_ATTRIBUTES_INIT(&dpcAttributes); dpcAttributes.ParentObject = pDevExt->WdfDevice; status = WdfDpcCreate(&dpcConfig, &dpcAttributes, &pDevExt->CompleteWriteDpc);

In the listing, the sample driver initializes a WDF_DPC_CONFIG structure with the name of the DPC function and then sets the AutomaticSerialization field of the structure to TRUE.

This DPC is used only with a particular device object, so the driver initializes an object attributes structure and assigns the device object as the parent. The driver then calls WdfDpcCreate to create the DPC, passing a pointer to the DPC configuration structure, the attributes structure, and the DPC function.

Execution Level in KMDF Drivers

Every locking and synchronization primitive has an associated IRQL. In some situations the framework uses a PASSIVE_LEVEL synchronization primitive to serialize callbacks, and in other situations it uses a primitive that runs at DISPATCH_LEVEL or-for interrupts-at DIRQL. For a KMDF driver, you can force the framework to use a PASSIVE_LEVEL lock when it serializes certain callbacks.

 Chapter 15 describes IRQL guidelines  By setting the execution level, KMDF drivers can specify the maximum IRQL at which the callbacks for driver, device, file, and general objects are invoked. As with synchronization scope, execution level is an object attribute. The driver sets the ExecutionLevel field of the attributes structure when it creates a driver, device, file, or general object.

KMDF supports the following execution levels:

• WdfExecutionLevelPassive

  Passive execution level means that the framework calls all event callbacks for the object at PASSIVE_LEVEL. If necessary, KMDF invokes the callback from a system worker thread.

  Drivers can set this level only for device and file objects. Typically, a driver should set passive execution level only if the callbacks access pageable code or data or call other functions that must be called at PASSIVE_LEVEL.

  Callbacks for events on file objects are always called at PASSIVE_LEVEL because these functions must be able to access pageable code and data. The EvtDeviceFileCreate and EvtFileCleanup callbacks run in the thread context of the application that opened the file handle. EvtFileClose can be called in an arbitrary thread context.

• WdfExecutionLevelDispatch

  Dispatch execution level means that the framework can invoke the callbacks from any IRQL up to and including DISPATCH_LEVEL.

  This setting does not force all callbacks to occur at DISPATCH_LEVEL. However, if a callback requires synchronization, KMDF uses a spin lock, which raises IRQL to DISPATCH_LEVEL. You should therefore design the callback to run at DISPATCH_LEVEL and to use a work item to perform tasks that require execution at PASSIVE_LEVEL. If the work item shares data with the other callbacks, the work item can use the device or queue presentation lock to synchronize access. The driver acquires this lock by using the WdfObjectAcquireLock method.

• WdfExecutionLevelInheritFromParent

  Inherited execution level means that the framework uses the same execution level value that is set for the parent of the object.

  This setting is the default for all objects except the driver object. The default execution level for the driver object is WdfExecutionLevelDispatch.

The execution level determines the locking mechanism behind the framework's object presentation locks, as follows:

• If the driver sets passive execution level, the framework serializes event callbacks by using a fast mutex, which is a PASSIVE_LEVEL lock.

  Therefore the framework calls the serialized callback functions at PASSIVE_LEVEL regardless of the synchronization scope.

• If the driver sets dispatch execution level and either device or queue synchronization scope, the framework serializes event callbacks by using a spin lock, which raises IRQL to DISPATCH_LEVEL.

  If the synchronization scope is set to none, the framework does not acquire a presentation lock and can invoke the callbacks at either DISPATCH_LEVEL or at PASSIVE_LEVEL.

 Exception for file callbacks  The EvtDeviceFileCreate, EvtFileCleanup, and EvtFileClose callbacks run in the caller's thread context and use pageable data, so they must be called at PASSIVE_LEVEL. If a driver sets device synchronization scope for its file objects, it must set the execution level for the file objects to WdfExecutionLevelPassive to force the framework to use a PASSIVE_LEVEL lock.

The following example, from the Toaster\Func\Featured\Toaster.c file, shows how the Featured Toaster sample sets synchronization scope and execution level:

WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&fdoAttributes, FDO_DATA); fdoAttributes.SynchronizationScope = WdfSynchronizationScopeDevice; fdoAttributes.ExecutionLevel = WdfExecutionLevelPassive; . . . //Code omitted for brevity status = WdfDeviceCreate(&DeviceInit, &fdoAttributes, &device);

As the example shows, a driver sets execution level in the object attributes structure by assigning a value to the ExecutionLevel field. The Featured Toaster sample sets device synchronization scope and PASSIVE_LEVEL execution for the device object, so that the framework serializes calls to I/O event and file object callbacks and invokes all such callbacks at IRQL PASSIVE_LEVEL.

Although forcing all callbacks to run at PASSIVE_LEVEL initially seems to make driver programming much easier, this setting is not appropriate for most drivers. In a busy driver, I/O processing might be delayed because the driver's I/O callbacks cannot run until all other DISPATCH_LEVEL processing is complete.