Win32 API Programming with Visual Basic

Page 209
  13. Windows Memory Architecture  
   
  In this chapter, we take a look at how Windows NT and Windows 9x use memory. The primary purpose of this chapter is to give you some background on virtual memory and how it is used. As a rule, Visual Basic programmers don't generally need to manipulate memory directly, but we will see one important reason to do so when we discuss accessing foreign processes. In any case, the subject is both interesting and instructive, so I have decided to cover the highlights in this book.  
 
  Types of Memory  
   
  We should begin by defining our terms. Figure 13-1 shows some of the concepts involved.  
   
  Physical Memory  
   
  Physical memory is the actual RAM that is installed in a PC. Each byte of physical memory has a physical address, which is a number from 0 to one less than the number of bytes of physical memory. For instance, a PC with 64 MB of RAM has physical address &H0000 0000-&H0400 0000 in hex, which is 0 67,108,863 in decimal.  
   
  Physical memory (unlike pagefile and virtual memory discussed later) is executable memory, that is, memory that can be read or written to by the CPU through its instruction set. Thus, for instance, in the assembly language instruction:  
 
  mov [si], ax  
   
  the address in the si register must be a physical address in order for this instruction code to actually execute.  
Page 210
   
   
   
  Figure 13-1.

Types of memory

  
   
  Virtual Memory  
   
  Virtual memory is simply a range of numbers, referred to as virtual addresses. It is nothing more. Thus, while a programmer can refer to a virtual address, Windows cannot access data at that address, since the address does not represent actual physical storage, as do physical and pagefile addresses. In order for code that refers to a virtual address to be executed, the virtual address must be mapped to a physical address, where the code or data can actually exist. This is done by the virtual memory manager (or VMM). We will discuss the operation of the VMM a bit later. The Windows operating system designates certain areas of virtual memory as accessible, that is, accessible to user mode code. All other areas are designated as reserved. Which areas are accessible and which are reserved varies with the operating system version (Windows 9x or Windows NT), as we will see in a moment.  
   
  Memory Page Frames  
   
  As you probably know, the smallest unit of memory that can be addressed is a byte. However, the smallest unit of memory that the Windows VMM deals with is a  
Page 211
   
  memory page, also called a memory page frame. On Intel-style computers, a page frame has size 4KB. We will speak of both virtual pages and physical pages.  
   
  Pagefile Memory  
   
  A pagefile (also called a swap file in Windows 95) is a file that resides on the hard disk. It is used for data and code storage just like physical memory, but is generally larger than physical memory. As we will discuss, Windows uses the pagefile (or pagefiles there can be more than one) to store information that does not fit in RAM swapping pages of information between the pagefile and RAM as needed.  
   
  Thus, a range of virtual addresses can correspond to addresses in a pagefile, rather than to addresses in physical memory (see Figure 13-1). When this happens, we say that the virtual addresses are backed by a pagefile, or pagefile-backed.  
   
  In general, a set of virtual addresses can be backed by physical memory, a pagefile, or indeed any file.  
   
  Memory-Mapped Files  
   
  In Chapter 10, Objects and Their Handles, we briefly discussed memory-mapped files and gave an example of file mapping. Simply put, any file can be used to back virtual memory, just as a pagefile is used in this way. In fact, a pagefile is a file that exists solely for the purpose of backing virtual memory.  
   
  When a file is mapped to memory in this way, the file is called a memory-mapped file. Each of the files in Figure 13-1 is memory-mapped. The corresponding virtual pages are file-backed.  
   
  Recall that the CreateFileMapping function is defined as:  
 
  Private Declare Function CreateFileMapping Lib "kernel32" Alias

"CreateFileMappingA" ( _

   ByVal hFile As Long, _

   ByVal lpSecurityAttributes As Long, _

   ByVal flProtect As Long, _

   ByVal dwMaximumSizeHigh As Long, _

   ByVal dwMaximumSizeLow As Long, _

   ByVal lpName As String _

) As Long

  
   
  This function creates a file-mapping object, using the handle of an open file, and returns a handle to the file-mapping object. This handle can be used with the MapViewOfFile function:  
 
  Private Declare Function MapViewOfFile Lib "kernel32" ( _

   ByVal hFileMappingObject As Long, _

   ByVal dwDesiredAccess As Long, _

   ByVal dwFileOffsetHigh As Long, _

 

  

Page 212
 
     ByVal dwFileOffsetLow As Long, _

   ByVal dwNumberOfBytesToMap As Long _

) As Long

  
   
  to map the file to virtual memory. The starting address of the file mapping in virtual memory is returned by the MapViewOfFile function. We can also say that the view is backed by the file with handle hFile.  
   
  Note that if the hFile parameter to CreateFileMapping is set to -1, then the file-mapping object (and any views created from this object) are backed by the pagefile, instead of a specified file.  
   
  Shared Physical Memory  
   
  Physical memory is said to be shared when it is mapped into the virtual address space of more than one process, although the virtual addresses may be different in each process. Figure 13-2 illustrates this concept.  
   
   
   
  Figure 13-2.

Shared physical memory

  
   
  If a file, such as a DLL, is in shared physical memory, we may refer to the file as being shared.  
Page 213
   
  One of the advantages of memory-mapped files is that they are easily shared. As we saw in Chapter 10, naming a file-mapping object makes it easy for more than one process to share the file. In this case, the file's contents are mapped to shared physical memory. Figure 13-3 illustrates this. Note that it is also possible to share pagefile contents through the use of file mappings. In particular, we can create a file-mapping object that is backed by a pagefile simply by setting the hFile parameter of the CreateFileMapping function to -1.  
   
   
   
  Figure 13-3.

A shared file mapping

  
 
  The Address Space of a Process  
   
  Every Win32 process gets a virtual address space (also called the address space or process space) of 4GB. Thus, the code within a process can refer to addresses &H0000 0000 through &HFFFF FFFF (or 0 through 232-1 = 4,294,967,295 in decimal). Of course, since a virtual address is just a number, the statement that each process gets its own virtual address space is pretty much meaningless. (It is like saying that each person gets his or her own age range, say from 0 to 150.)  
Page 214
   
  However, this statement is intended to mean that Windows does not draw any relationship between Process A's use of the virtual address &H4000 0000, for example, and Process B's use of the same virtual address &H4000 0000. In particular, Windows may (or may not) associate a different physical address to this virtual address for each process.  
   
  Windows NT and Windows 9x use virtual memory differently.  
   
  Windows NT Address Space Usage  
   
  Figure 13-4 shows the general use of a process address space under Windows NT.  
   
  Region A  
   
  Referring to Figure 13-4, Windows NT reserves the first 64KB of virtual memory for a special purpose and marks it as inaccessible to user mode code. When working with pointers, which VC++ programmers must do quite often, it is easy to forget to initialize a pointer. For instance, consider the following code:  
 
  int *pi;   // declare a pointer to an integer

*pi = 5;   // point the pointer to an integer value

  
   
  This code will not work. The problem is that a pointer must point to a variable, not a value. The first line of code declares a pointer, which initially contains the value 0. That is, it is a NULL pointer. The second line of code is thus trying to place the value 5 in memory location 0!  
   
  To alert programmers to this commonly made mistake, the lower portion of memory is reserved by Windows (both Windows NT and Windows 9x). As a result, code such as the previous example will cause a GPF, thus alerting the programmer to the error.  
   
  Region B  
   
  Referring to Figure 13-4, region B starts at the 64KB boundary and reaches up to 64 KB short of the 2GB mark. Thus, region B has size 2GB - 128KB (essentially half the entire address space). Portions of this region are mapped to the main EXE for the application, any application-specific DLLs, and system DLLs, such as KERNEL32.DLL. USER32.DLL, GDI32.DLL, and more.  
   
  To illustrate. Figure 13-5 shows the address space of an application called Designer (from Micrografx, Inc.). Note the following:  
   
  The main EXE is located at &H40 0000, which is the default base address for a program written in VC++. (I do not know what development environment was used to write Designer.)  
   
  An application-specific DLL called DS70RES.DLL is located at &H0135 0000, which is somewhat above the EXE.  
Page 215
   
   
   
  Figure 13-4.

Process address space usage under Windows NT

  
   
  The next module (rpcltc.dll) at &H015B 0000 is a Microsoft DLL.  
   
  There follow several Micrografx and Microsoft DLLs and even a Creative Labs DLL (sb16snd.dll)!  
   
  The rest of the DLLs, starting with COMCTR32.DLL, are copyrighted by Microsoft. This includes USER32.DLL at &H77E7 0000, GDI32.DLL at &H77ED 0000, and KERNEL32.DLL at &H77F0 0000.  
Page 216
   
   
   
  Figure 13-5.

An example process space

  
   
  It is worth remarking that this region may contain both shared and unshared files. For instance. Designer's application-specific DLLs are likely to be unshared (unless other copies of Designer are running). On the other hand, the system DLLs, such as KERNEL32.DLL. are shared, in that there is only one copy of these DLLs in physical memory. Generally, this copy is mapped to the same virtual addresses in each process, but this is not a requirement.  
   
  Region C  
   
  Referring to Figure 13-4, region C is the 64KB region just below the 2GB  
   
  mark, where Windows reserved address space begins. It is used as a kind of  
   
  barrier that prevents a programmer from using memory addresses that straddle  
   
  the accessible region B and the inaccessible region D which belongs to  
   
  Windows.  
Page 217
   
  Region D  
   
  This region is reserved for use by Windows NT (the executive, kernel, and device drivers, for example). Applications cannot access memory in this virtual address range. Any attempt to do so will result in an access violation (a GPF).  
   
  Windows 9x Address Space Usage  
   
  Figure 13-6 shows the general use of a process address space under Windows 9x.  
   
  Region A  
   
  Referring to Figure 13-6, Windows 9x reserves region A, which is a mere 4 KB in size, for the same purpose as Windows NT reserves the first 64 KB of memory as an alert for null pointers. This region is protected and will cause an access violation if we attempt to access this memory in user mode code.  
   
  Region B  
   
  The next region of memory (region B) is used to maintain compatibility with DOS and 16-bit Windows applications. Although it is accessible, a programmer should not use this region.  
   
  Region C  
   
  Region C is the address space used by application code and its DLLs. Also, Windows modules are placed in this region. For instance, if an application requires a custom control (OCX), the module will be located in this region.  
   
  Region D  
   
  Windows 9x maps the Win32 system DLLs (KERNEL32.DLL, USER32.DLL, and so on) into this address space. These files are shared, meaning that more than one process has access to a single copy of the files in physical memory.  
   
  This region of memory is accessible to user mode code (although I certainly would not recommend doing so).  
   
  Region E  
   
  This region also contains shared Windows files, such as the Windows executive and kernel, virtual device drivers, file system, and memory management code, and so on.  
   
  This region of memory is also accessible to user mode code.  
Page 218
   
   
   
  Figure 13-6.

Process address space usage under Windows 9x

  
 
  Example: Using GetSystemInfo  
   
  The GetSystemInfo API function retrieves some information about address spaces. Its syntax is:  
 
  VOID GetSystemInfo(

   LPSYSTEM_INFO lpSystemInfo // address of system information structure

);

  
Page 219
   
  or, in VB:  
 
  Declare Sub GetSystemInfo Lib "kernel32" Alias "GetSystemInfo" ( _

   lpSystemInfo As SYSTEM_INFO)

  
   
  The system information structure is defined as:  
 
  struct _SYSTEM_INFO {

   union { DWORD dwOemId;

      struct {

         WORD wProcessorArchitecture;

         WORD wReserved;

      };

   };

   DWORD dwPageSize;

   LPVOID lpMinimumApplicationAddress;

   LPVOID lpMaximumApplicationAddress;

   DWORD dwActiveProcessorMask;

   DWORD dwNumberOfProcessors;

   DWORD dwProcessorType;

   DWORD dwAllocationGranularity;

   WORD wProcessorLevel;

   WORD wProcessorRevision;

}

  
   
  A union is a VC++ construct that means that the variable in question can hold more than one data type at different times. For instance, a variable whose type is the following union:  
 
  union uExample {

   int i;

   char ch;

}

  
   
  can be either an integer or a character. Of course, the system must set aside enough space for the largest possibility, in this case, 4 bytes for a VC++ integer. Naturally, we must do the same.  
   
  The SYSTEM_INFO structure contains a union whose members are a DWORD and another structure consisting of two WORDs. Thus, the size of the union is that of a DWORD (or two WORDs), which is 4 bytes.  
   
  We can define this structure in VB as:  
 
  Type SYSTEM_INFO

   wProcessorArchitecture As Long

   dwPageSize as Long

   lpMinimumApplicationAddress As Long

   lpMaximumApplicationAddress As Long

   dwActiveProcessorMask As Long

   dwNumberOfProcessors As Long

   dwProcessorType As Long

   dwAllocationGranularity As Long

   wProcessorLevel As Integer

   wProcessorRevision As Integer

End Type

  
Page 220
   
  Here is some code that calls GetSystemInfo, along with its output on my system:  
 
  Public Sub GetSystemInfoExample()

Dim si As SYSTEM_INFO

GetSystemInfo si

Debug.Print "Page size: " & si.dwPageSize

Debug.Print "Max app address: " & Hex(si.lpMaximumApplicationAddress)

Debug.Print "Min app address: " & Hex(si.lpMinimumApplicationAddress)

Debug.Print "Application Granularity: " & si.dwAllocationGranularity

Debug.Print "Processor architecture: " & si.wProcessorArchitecture

Debug.Print "Processor count: " & si.dwNumberOfProcessors

Debug.Print "Processor type: " & si.dwProcessorType

Debug.Print "Processor level: " & si.wProcessorLevel

Debug.Print "Processor revision: " & Hex(si.wProcessorRevision)

End Sub

Page size: 4096

Max app address: 7FFEFFFF

Min app address: 10000

Application Granularity: 65536

Processor architecture: 0

Processor count: 1

Processor type: 586

Processor level: 6

Processor revision: 501

  
   
  This reports, as expected, that the page size is 4KB. Also, under Windows NT (which I am running), the minimum address for an application is &H1000 (= 64 KB) as expected from Figure 13-4. Similarly, the maximum application address is &H7FFE FFFF, as expected.  
   
  We will explain application granularity later.  
   
  According to the documentation, the wProcessorArchitecture variable is set to 0 to represent PROCESSOR_ARCHITECTURE_INTEL. (The dwOemID parameter is obsolete.) The processor level of 6 indicates a Pentium II processor. The processor revision number 501 is interpreted as Model 5 stepping 1. (For more details, please see the GetSystemInfo documentation.)  
 
  Allocating Virtual Memory  
   
  Each page of virtual address space can be in one of three states:  
 
  Reserved

The page has been reserved for use.

  
 
  Committed

Physical storage is allocated for this virtual page. The storage is either in a pagefile or in a memory-mapped file.

  
Page 221
 
  Free

The page is neither reserved nor committed and is therefore currently not accessible to the process.

  
   
  Virtual memory can be reserved or committed by calling the API function VirtualAlloc:  
 
  LPVOID VirtualAlloc(

   LPVOID lpAddress,       // address of region to reserve or commit

   DWORD dwSize,           // size of region

   DWORD flAllocationType, // type of allocation

   DWORD flProtect         // type of access protection

);

  
   
  The flAllocationType parameter can be set to the following constants (among others):  
 
  MEM_RESERVE

This simply reserves a range of the process's virtual address space without allocating any physical storage. (Storage can be allocated with a subsequent call to this function, however.)

  
 
  MEM_COMMIT

This allocates physical storage in memory or in the paging file on disk for the specified reserved region of pages.

  
   
  The two constants can be combined in order to reserve and commit in one operation.  
   
  Separating the process of reserving and committing memory has some advantages. For example, since reserving memory is a very efficient process, if an application should require a large amount of memory, it can reserve the entire amount but commit only the portion that it currently requires, thus spreading out the more time-consuming process of commitment of physical storage.  
   
  In fact, Windows itself uses this approach when it allocates a stack for each newly created thread. Windows reserves 1MB of virtual memory for each thread's stack but initially commits only two pages (8KB).  
   
  Memory protection  
   
  Note that the flProtect parameter of VirtualAlloc is used to specify the type of access protection to accord the newly committed virtual memory (this does not apply to reserved memory). The choices are:  
 
  PAGE_READONLY

Gives read-only access to the committed virtual memory.

  
 
  PAGE_READWRITE

Gives read-write access to the committed virtual memory.

  
Page 222
 
  PAGE_WRITECOPY

Gives copy-on-write access to the committed virtual memory. We will discuss this a bit later.

  
 
  PAGE_EXECUTE

Enables execute access to the committed virtual memory. Any attempt to read or write to the memory, however, will result in an access violation.

  
 
  PAGE_EXECUTE_READ

Enables execute and read access.

  
 
  PAGE_EXECUTE_READWRITE

Enables execute, read, and write access.

  
 
  PAGE_EXECUTE_WRITECOPY

Enables execute, read, and copy-on-write access.

  
 
  PAGE_NOACCESS

Disables all access to the committed virtual memory.

  
   
  In addition. any of these values except PAGE_NOACCESS can be ORed with the following two flags.  
 
  PAGE_GUARD

Designates the pages as guard pages. Any attempt to access a guard page causes the system to raise a STATUS_GUARD_PAGE exception and turn off the guard page status. Thus, guard pages raise a first-access-only alarm.

  
 
  PAGE_NOCACHE

Prevents caching of the committed memory.

  
   
  Let us explain copy-on-write. Suppose that a certain page of physical memory is being shared by two processes. If the page is marked as read-only, then the two processes can share the page without problems. However, there may be situations in which we want to permit each process to write to this memory but not to affect the other process. By setting the protection to copy-on-write, as soon as one process attempts to write to the shared page, the system makes a copy of that page exclusively for the process that wants to do the writing. Thus, the page is no longer shared and the other process's view of the data remains unchanged.  
   
  We note also that the protection attributes of a page can be changed using the VirtualProtect API function.  
   
  Allocation granularity  
   
  Note that if the lpAddress parameter is not a multiple of 64KB, then the system will round down the requested address to the nearest multiple of 64KB. Windows always reserves virtual memory starting on an allocation granularity boundary, which, in the case of Intel-style processors, is a multiple of 64KB. In other words, the starting address of any block of reserved memory is a multiple of 64KB.  
Page 223
   
  In addition, the amount of memory that is allocated is always a multiple of the system's page size, that is, of 4KB. Hence, VirtualAlloc will round up any memory request to the nearest multiple of the page size.  
   
  Incidentally, there are times when Windows allocates virtual memory on behalf of a process, but for Windows' own use. In these cases, Windows does not always follow the rules about allocation granularity boundaries that it follows when allocating memory requested by a programmer (although page size requirements are honored).  
   
  The Virtual Address Descriptor  
   
  Finally, in case you are wondering, the system keeps track of which virtual pages are reserved in a structure called a Virtual Address Descriptor or VAD. This is essential because there is no other way to tell when virtual addresses have been reserved.  
 
  Example: Using GlobalMemoryStatus  
   
  The API function GlobalMemoryStatus:  
 
  Declare Sub GlobalMemoryStatus Lib "kerne132" Alias "GlobalMemoryStatus" ( _

   lpBuffer As MEMORYSTATUS)

  
   
  reports on a variety of memory-related values by filling in the following structure:  

struct _MEMORYSTATUS { DWORD dwLength; // sizeof(MEMORYSTATUS) DWORD dwMemoryLoad; // percent of memory in use DWORD dwTotalPhys; // bytes of physical memory DWORD dwAvailPhys; // free physical memory bytes DWORD dwTotalPageFile; // bytes of paging file DWORD dwAvailPageFile; // free bytes of paging file DWORD dwTotalVirtual; // user bytes of address space DWORD dwAvailVirtual; // free user bytes }

   
  In VB, this is:  

Type MEMORYSTATUS dwLength As Long sizeof(MEMORYSTATUS) dwMemoryLoad As Long percent of memory in use dwTotalPhys As Long bytes of physical memory dwAvailPhys As Long free physical memory bytes dwTotalPageFile As Long bytes of paging file dwAvailPageFile As Long free bytes of paging file dwTotalVirtual As Long user bytes of address space dwAvailVirtual As Long free user bytes End Type

Page 224
   
  For example, the code:  
 
  Dim ms As MEMORYSTATUS

GlobalMemoryStatus ms

Debug.Print "Virtual memory: " & ms.dwTotalVirtual

Debug.Print "Short of 2 GB: " & (2 ^ 31 - ms.dwTotalVirtual) / 1024 & " KB"

Debug.Print "Available Virtual: " & ms.dwAvailVirtual

  
   
  might produce:  
 
  Virtual memory: 2147352576

Short of 2 GB: 128 KB

Available Virtual: 1965658112

  
   
  Note that the available virtual memory is 128KB less than the 2GBs in the lower half of the total address space of a process, as we would expect from Figure 13-4.  
 
  Virtual Memory Management  
   
  We want to take a look next at how the Windows virtual memory manager (abbreviated VMM) actually interprets virtual memory addresses as physical addresses.  
   
  Translating Virtual Addresses to Physical Addresses: Page Hits  
   
  Figure 13-7 shows the translation process when the virtual address is mapped to a physical address. This situation is referred to as a (physical) page hit.  
   
   
   
  Figure 13-7.

Virtual-to-physical address resolution for a page hit

  
   
  All virtual addresses are divided into three parts. The leftmost part (bits 22 31) contains an index into the process's page directory. Windows maintains a separate  
Page 225
   
  page directory for each process. The address of this page directory is placed in a CPU register called CR3. (Incidentally, part of the task-switching process is to switch this register so that it points to the incoming process's page directory.) A page directory contains 1024 4-byte entries.  
   
  Windows also maintains, for each process, a collection of page tables. Each page directory entry contains a page table number, which identifies a particular page table. Thus, Windows supports up to 1024 page tables. (Actually, page tables are created only when needed, that is, when an attempt is made to access data or code at a given virtual address, not simply when the virtual memory is allocated.)  
   
  The second part of the virtual address (bits 12 21) is used as an index into the page table corresponding to the page directory entry. The page table entry at this index contains, in its leftmost 20 bits, a page frame number that identifies a particular page frame in physical memory.  
   
  The third and final part of the virtual address (bits 0 11) is an offset into that page frame. The combination of page frame number and offset into that page frame constitutes a physical address in physical memory.  
   
  Since each page table has 1024 entries and there are 1024 page tables, the total number of page frames that can be identified in this manner is 1024 1024 = 210 210 = 220. Since each page frame is 4KB = 4 210 bytes in size, the theoretical limit of physical addresses is 4 230 = 4GB.  
   
  This rather complex-looking translation scheme has several important advantages. One immediate advantage is that page frames are very small and can easily be kept in memory. It is much easier to find 4KB of consecutive physical memory than, say, 64KB.  
   
  However, the main advantage is that a virtual memory address from two processes can be deliberately made to point to different physical addresses or be deliberately made to point to the same physical address!  
   
  To illustrate, suppose that Process1 and Process2 both refer in code to the same virtual address. Process1's virtual-to-physical address resolution uses its page directory, and Process2's translation uses its page directory. Thus, while the indices into the page directories are the same in both cases, these indices are indices into different directories. In this way, the VMM can assure that the virtual addresses in each process resolve to different physical addresses.  
   
  On the other hand, the VMM can also ensure that virtual addresses in the two processes, whether they be the same virtual address or not, can resolve to the same physical address. One way to do this would be to have both page directory entries point to the same page table, and thus to the same page frames. In this way, processes can share physical memory.  
Page 226
   
  System page directory and page tables  
   
  We should also mention that Windows maintains a system page directory to deal with virtual memory reserved for Windows, as well as a corresponding set of system page tables.  
   
  Format of a valid page table entry  
   
  A page table entry is said to be valid when it points to a physical page frame, as in Figure 13-7. Figure 13-8 shows the format of a valid page table entry (some data is omitted).  
   
   
   
  Figure 13-8.

Valid page table entry

  
   
  Note that a page table entry contains flags that describe various properties of the page frame, such as whether or not it has been written to or read from and whether or not access is permitted to this memory from user mode code. The least significant bit is the valid bit (also called a present bit), signaling that this entry is valid.  
   
  Translating Virtual Addresses to Physical Addresses: Page Faults  
   
  If a virtual page is not backed by physical memory, but is backed instead by a pagefile, then the translation process will come a cropper, resulting in a page fault. The situation is pictured in Figure 13-9.  
Page 227
   
   
   
  Figure 13-9.

Virtual-to-physical address resolution for a page fault

  
   
  In this case, the system will see that the page table entry is invalid, having its least significant bit set to 0. The format of an invalid page table entry whose page is backed by a pagefile is shown in Figure 13-10.  
   
   
   
  Figure 13-10.

A page table entry backed by a pagefile

  
   
  The fact that the valid bit is 0 tells the system that the page table entry contains a pagefile number between 0 and 15 that identifies the pagefile that backs the virtual address. The page table entry also contains the number of the page within the pagefile that backs the virtual address. The system can then copy that pagefile page into physical memory, change the page table entry from an invalid one to a valid one, and use the 12 bits of the virtual address as an offset into the physical page frame, just as before. This process is referred to as paging.  
   
  Shared Pages  
   
  Our discussion so far concerns unshared physical memory only. The situation for shared physical memory is considerably more complex, and we will not go into the details here, except to say that the VMM uses a concept called prototype page table entries. The idea is simply to point the ordinary page table entry in each process  
Page 228
   
  with shared memory to a single prototype page table entry, instead of to physical memory. In this way, a single prototype page table entry can point to shared physical memory.  
   
  Working Sets  
   
  We have discussed the fact that each virtual page in the 4GB virtual address space of a process exists in one of three states free, reserved, or committed. We can now also say that each committed page is either valid or invalid. Those pages that are valid, that is, backed by physical memory, are referred to collectively as the working set of the process. Of course, the working set is constantly changing, as pages are paged in and out of physical memory. Recall that the rpiEnumProcsNT utility also displays the current size of the working set for a process.  
   
  The system working set describes the virtual pages of system memory that are currently mapped to physical memory.  
   
  The size of a process's working set is limited by minimum and maximum settings that Windows makes, based on the size of physical memory. The settings are described in Table 13-1.  
Table 13-1. Process Working Set Size Limits
Memory Model Memory Size Minimum Process Working Set Size Maximum Process Working Set Size
Small <=19MB 20 pages (80KB) 45 pages (180KB)
Medium 20 32MB 30 pages (120KB) 145 pages (580KB)
Large >=32MB 50 pages (200KB) 345 pages (1380KB)

   
  Note that these limits can be changed using the API function SetProcessWorkingSetSize:  

BOOL SetProcessWorkingSetSize( HANDLE hProcess, // open handle to the process of interest DWORD dwMinimumWorkingSetSize, // specifies minimum working set size in bytes DWORD dwMaximumWorkingSetSize, // specifies maximum working set size in bytes );

   
  Incidentally, setting both of the size parameters to -1 will cause the function to reduce the working set to size 0, thus temporarily removing the process from physical memory.  
   
  We also note that the actual size of a process's working set will vary with time, since Windows will increase the working set if it notices that the process is incurring a large number of page faults.  
   
  The system working set size limits are described in Table 13-2.  
Page 229
Table 13-2. System Working Set Size Limits
Memory Model Memory Size Minimum Process Working Set Size Maximum Process Working Set Size
Small <=19MB 388 pages (1.5MB) 500 pages (2.0MB)
Medium 20 32MB 688 pages (2.7MB) 1150 pages (4.5MB)
Large >=32MB 1188 pages (4.6MB) 2050 pages (8MB)

   
  The Page Frame Database  
   
  Windows keeps track of the state of each physical page of memory in a data structure called the Page Frame Database. Each physical page can be in one of eight different states:  
 
  Active (or valid)

This page is currently mapped to virtual memory. Thus, it is the target of a working set page.

  
 
  Transition

On its way to becoming an active page.

  
 
  Standby

The page was just removed from active status, but it was not modified while active.

  
 
  Modified

The page has been removed from active status. Its contents were modified while it was active, but have not yet been written to disk.

  
 
  Modified No Write

Similar to modified but marked so that the contents will not be flushed to disk. Used by the Windows file system drivers.

  
 
  Free

The page is free but has been written to and is thus not eligible for use by a process.

  
 
  Zeroed

The page is free and initialized to all 0s by the zero page thread. This page can be allocated to a process.

  
 
  Bad

The page has incurred parity errors or some other hardware problem and will not be used.

  
 
  Heaps of Memory  
   
  Simply put, a heap is a region of virtual memory that has been reserved by Windows. The Win32 API supplies functions for allocating memory from a heap,  
Page 230
   
  freeing heap memory, and resizing blocks of allocated memory. In short, these functions provide a more intuitive method for manipulating application memory than VirtualAlloc and are especially suited for manipulating a large number of small blocks of memory. Although VB programmers don't have any direct use for heaps, the concept does appear quite often in documentation, so we should discuss it briefly.  
   
  Heaps Under 32-Bit Windows  
   
  When a process is created, Windows assigns it a default heap. That is, Windows initially reserves a 1MB region of virtual memory. However, the system will adjust the size of the default heap as necessary. The default heap is used by Windows itself in a variety of ways. For instance, as discussed in Chapter 6, Strings, Visual Basic translates Unicode strings to ANSI for passing to the ANSI entry points of a DLL. Memory for these temporary ANSI strings is obtained from the default heap. The API function GetProcessHeap is used to obtain a handle to the default heap. A programmer can also create additional heaps using the HeapCreate function, which returns a heap handle.  
   
  There are several reasons why a programmer might want to create additional heaps rather than using the default heap. For example, heaps that are dedicated to a particular task are often more efficient. Also, errors made in writing data to heap memory allocated from a dedicated heap will not affect data in other heaps. Finally, allocating memory from a dedicated heap will generally mean that the data is packed more closely together in memory, which may reduce the need for paging data from the pagefile. We should also mention that access to a heap is serialized, that is, the system makes each thread attempting access to heap memory wait its turn until other threads are finished. In this way, only one thread at a time can allocate or free heap memory, thus avoiding unpleasant conflicts.  
   
  Incidentally, the 16-bit version of Windows supported both a global heap and a local heap. Accordingly, 16-bit Windows implemented the functions GlobalAlloc and LocalAlloc. These functions still work, but are not very efficient and should be avoided under Win32. (We will see, however, that they still must be used for certain purposes, such as creating a Clipboard viewer.)  
   
  Heap Functions  
   
  The following functions are used with heaps:  
 
  GetProcessHeap

Returns a handle to the default heap for the process.

  
 
  GetProcessHeaps

Returns a list of the handles for all heaps

  
 
  currently used by a process.  
Page 231
 
  HeapAlloc

Allocates a block of memory from the specified heap.

  
 
  HeapCompact

Defragments a heap by combining free blocks. It can also decommit unused pages of heap memory.

  
 
  HeapCreate

Creates a new heap in the process's address space.

  
 
  HeapDestroy

Destroys the specified heap.

  
 
  HeapFree

Frees a previously allocated heap memory block.

  
 
  HeapLock

Locks a heap so that only a single thread has access to the heap. Other threads requesting access are suspended until the owning thread unlocks the heap. This provides a form of thread synchronization. This is also how the system enforces thread serialization.

  
 
  HeapReAlloc

Reallocates a block of memory on the heap. Used to change a block's size.

  
 
  HeapSize

Returns the size of an allocated heap memory block.

  
 
  HeapUnlock

Unlocks a heap that was previously locked using the HeapLock function.

  
 
  HeapValidate

Verifies that a heap (or block within a heap) is valid; that is, that it has not been corrupted.

  
 
  HeapWalk

Allows the programmer to walk the contents of a heap. Used mostly for debugging purposes.

  
 
  Example: Mapping Virtual Memory  
   
  The Win32 API function VirtualQuery can be used to get information about the state of virtual memory addresses. The syntax is:  

DWORD VirtualQuery( LPCVOID lpAddress, // address of region PMEMORY_BASIC_INFORMATION lpBuffer, // address of information buffer DWORD dwLength // size of buffer );

   
  In VB, this can be declared as:  
 
  Declare Function VirtualQuery Lib "kernel32" (

   ByVal lpAddress As Long, _

 

  

Page 232
 
     lpBuffer As MEMORY_BASIC_INFORMATION, _

   ByVal dwLength As Long _

) As Long

  
   
  There is also VirtualQueryEx, an extended version of VirtualQuery that permits looking at foreign virtual address spaces:  

DWORD VirtualQueryEx( HANDLE hProcess, // handle to process LPCVOID lpAddress, // address of region PMEMORY_BASIC_INFORMATION lpBuffer, // address of information buffer DWORD dwLength // size of buffer );

   
  or, in VB:  
 
  Declare Function VirtualQueryEx Lib "kernel32" ( _

   ByVal hProcess As Long, _

   ByVal lpAddress As Long, _

   lpBuffer As MEMORY_BASIC_INFORMATION, _

   ByVal dwLength As Long _

) As Long

  
   
  As expected, the hProcess parameter requires a process handle. The lpAddress parameter is the starting address on which to report, but it will be rounded down to the closest multiple of the page size (4KB). Both functions return information in the structure:  

struct _MEMORY_BASIC_INFORMATION { PVOID BaseAddress; // base address of region PVOID AllocationBase; // allocation base address DWORD AllocationBase; // initial access protection DWORD RegionSize; // size, in bytes, of region DWORD State; // committed, reserved, free DWORD Protect; // current access protection DWORD Type; // type of pages }

   
  or, in VB:  

Type MEMORY_BASIC_INFORMATION BaseAddress As Long base address of region AllocationBase As Long allocation base address AllocationProtect As Long initial access protection RegionSize As Long size, in bytes, of region State As Long committed, reserved, free Protect As Long current access protection Type As Long type of pages End type

   
  To understand the members of this structure, we need to describe what the function does. Unfortunately, the documentation does a rotten job of explaining this function, but it appears to perform as follows. To aid our discussion, let us refer to the page containing the address lpAddress as the specified page. Figure 13-11 will help define our terms.  
Page 233
   
   
   
  Figure 13-11.

The VirtualQuery(Ex) function

  
   
  The following members of MEMORY_BASIC_INFORMATION are always set by VirtualQuery(Ex):  
 
  BaseAddress

Returns the base address of the specified page.

  
 
  RegionSize

The number of bytes from the start of the specified page to the top of the specified region. Despite the name, this is not generally the size of the entire specified region. We will define the term specified region in a moment.

  
   
  If the page containing lpAddress is free (neither reserved nor committed), the State member contains the symbolic constant MEM_FREE. No other member (except BaseAddress and RegionSize) has meaning.  
   
  If the page containing lpAddress is not free, the function identifies the allocation region containing the specified page, that is, the region of virtual memory that contains the specified page and was originally allocated by a call to VirtualAlloc.  
   
  Starting at the base address of the specified page, the function examines each successive page in the allocation region, checking to see if it matches the specified page in allocation type and protection type. The collection of all consecutive matching pages is the specified region. The values in the MEMORY_BASIC_INFORMATION structure refer to this region. Specifically, a page matches the specified page if it satisfies the following two conditions.  
Page 234
   
  The page has the same allocation type as the initial page, as reflected by the flag values: MEM_COMMIT, MEM_RESERVE, MEM_FREE, MEM_PRIVATE, MEM_MAPPED, or MEM_IMAGE.  
   
  The page has the same protection type as the initial page as reflected by the flag values: PAGE_READONLY, PAGE_READWRITE, PAGE_NOACCESS, PAGE_WRITECOPY. PAGE_EXECUTE, PAGE_EXECUTE_READ, PAGE_EXECUTE_READWRITE. PAGE_EXECUTE_WRITECOPY, PAGE_GUARD, or PAGE_NOCACHE.  
   
  Now we can describe the other members of the MEMORY_BASIC_INFORMATION structure:  
 
  AllocationBase

The base address of the allocation region.

  
 
  AllocationProtect

The initial protection type of the allocation region.

  
 
  State

Either MEM_FREE, MEM_RESERVE, or MEM_COMMIT. Refers to the specified region.

  
 
  Protect

The current protection type of the specified region.

  
 
  Type

Either MEM_IMAGE, MEM_MAPPED, or MEM_PRIVATE. Refers to the specified region. The meanings of these constants are as follows: MEM_IMAGE indicates that the region is mapped to an image (that is, executable) file; MEM_MAPPED indicates that the region is mapped to a nonexecutable memory-mapped file (such as a data file); MEM_PRIVATE indicates that the region is private, that is, not shared by more than one process.

  
   
  Finally, we note that these functions return the size of the MEMORY_BASIC_INFORMATION structure.  
   
  The following procedure fills a list box with a virtual memory map. You will find this procedure in the rpiEnumProcsNT and rpiEnumProcs95 applications on the CD.  
 
  Sub MapMemory(lProcessID As Long)

Dim hProcess As Long

Dim lret As Long

Dim meminfo As MEMORY_BASIC_INFORMATION

Dim lCurRegion As Long

Dim sName As String

Dim sItem As String

Dim Tabstops (1 To 5) As Long

Dim bIsFree As Boolean

 

  

Page 235
 
  lstMain.FontName = "Courier New"

lstMain.FontSize = 9

' Get process handle from process ID

hProcess = ProcHndFromProcID(lProcessID)

If hProcess = 0 Then Stop

lstMain.Clear

' Set tab stops for list box

Tabstops(1) = 4 * 22

Tabstops(2) = Tabstops(1) + 4 * 10

Tabstops(3) = Tabstops(2) + 4 * 10

Tabstops(4) = Tabstops(3) + 4 * 12

SendMessage lstMain.hWnd, LB_SETTABSTOPS, 5, Tabstops(1)

Do

   ' Get info on this region

   lret = VirtualQueryEx(hProcess, lCurRegion, meminfo, LenB(meminfo))

' Is it free?

   bIsFree = (meminfo.State = MEM_FREE)

   ' Build item for list box

   sItem = Hex(lCurRegion) & " - " & Hex(lCurRegion + meminfo.RegionSize - 1) _

       & vbTab & meminfo.RegionSize / 4096 & " pp" _

       & vbTab & MemType(meminfo.State)

   If Not bIsFree Then

        sItem = sItem & vbTab & MemType(meminfo.Type) _

   m302

1

      & vbTab & AccessType(meminfo.Protect)

   End If

   ' Get module name. Skip 0, since this gives EXE name.

   If lCurRegion > 0 And Not bIsFree Then

      sName = String$(MAX_PATH + 1, 0)

      lret = GetModuleFileName(lCurRegion, sName, MAX_PATH)

      If lret <> 0 Then sItem = sItem & vbTab & Trim0 (sName)

   End If

  

     ' Display

   lstMain.AddItem sItem

  

   ' Next start of region

   lCurRegion = lCurRegion + meminfo.RegionSize

  

   If lCurRegion >= &H7FFEFFFF Then Exit Do

Loop

  

CloseHandle hProcess

  

End Sub

  
Page 236
   
  A portion of the output for one process is shown here:  
   
  0 - FFFF 16 pp FREE  
     
   
  10000 - 10FFF  
   
  1 pp     
   
  COMMIT  
   
  PRIVATE  
   
  READWRIRE  
   
  11000 - 1 FFFF  
   
  15 pp  
   
  FREE  
   
   
  20000 - 20FFF  
   
  1 pp  
   
  COMMITE  
   
  PRIVATE  
   
  READWRITE  
   
  21000 - 2FFFF  
   
  15 pp  
   
  FREE  
   
   
  30000 - 12DFFF  
   
  253 pp  
   
  RESERVE  
   
  PRIVATE  
   
  0  
   
  12D000 - 12FFFF  
   
  1 pp  
   
  COMMIT  
   
  PRIVATE  
   
  260  
   
  12E000 - 12FFFF  
   
  2 pp  
   
  COMMIT  
   
  PRIVATE  
   
  READWRITE  
   
  130000 - 130FFF  
   
  1 pp  
   
  COMMIT  
   
  PRIVATE  
   
  READWRITE  
   
  131000 - 13FFFF  
   
  15 pp  
   
  FREE  
   
   
  140000 - 158FFF  
   
  25 pp  
   
  COMMITE  
   
  PRIVATE  
   
  READWRITE  
   
  159000 - 23FFFF  
   
  231 pp  
   
  RESERVE  
   
  PRIVATE  
   
  0  
   
  240000 - 240FFF  
   
  1 pp  
   
  COMMIT  
   
  MAPPED  
   
  READWRITE  
   
  241000 - 24FFFF  
   
  15 pp  
   
  RESERVE  
   
  MAPPED  
   
  0  
   
  250000 - 265FFFF  
   
  22 pp  
   
  COMMIT  
   
  MAPPED  
   
  READONLY  
   
  266000 - 26FFFF  
   
  10 pp  
   
  FREE  
   
   
  270000 - 293FFF  
   
  36 pp  
   
  COMMIT  
   
  MAPPED  
   
  READONLY  
   
  294000 - 29FFFF  
   
  12 PP  
   
  FREE  
   
   
  2A0000 - 2E0FFF  
   
  65 pp  
   
  COMMIT  
   
  MAPPED  
   
  READONLY  
   
  2E1000 - 2EFFFF  
   
  15 pp  
   
  FREE  
   
   
  2F0000 - 2F2FFF  
   
  3 pp  
   
  COMMIT  
   
  MAPPED  
   
  READONLY  
   
  2F3000 - 2FFFFF  
   
  13 pp  
   
  FREE  
   
   
  300000 - 305FFF  
   
  6 pp  
   
  COMMIT  
   
  MAPPED  
   
  EXECUTE_READ  
   
  306000 - 3BFFFF  
   
  186 pp  
   
  RESERVE  
   
  MAPPED  
   
  0  
   
  3C0000 - 3C0FFF  
   
  1 pp  
   
  COMMIT  
   
  MAPPED  
   
  EXECUTE_READ  
   
  3C1000 - 3C7FFF  
   
  7 pp  
   
  RESERVE  
   
  MAPPED  
   
  0  
   
  3C8000 - 3FFFFF  
   
  56 pp  
     
   
  400000 - 400FFF  
   
  1 pp  
   
  COMMIT  
   
  IMAGE  
   
  READONLY G:\Visual Studio\VB98\VB6.  
   
  EXE  
       
   
  401000 - 40BFFF  
   
  11 pp  
 
   
  IMAGE  
   
  EXECUTE_READ  
   
  40C000 - 40DFFF  
   
  2 pp  
   
  COMMIT  
   
  IMAGE  
   
  READWRITE  

   
  Two utility functions used in MapMemory are:  
 
  Function MemType(vValue As Variant) As String

' Function returns name of constant for given value.

Dim sName As String

Select Case vValue

   Case &H1000

      sName = "COMMIT"

   Case &H2000

      sName = "RESERVE"

   Case &H4000

      sName = "DECOMMIT"

   Case &H8000

      sName = "RELEASE"

   Case &H10000

      sName = "FREE"

   Case &H20000

      sName = "PRIVATE"

   Case &H40000

      sName = "MAPPED"

   Case &H80000

      sName = "RESET"

 

  

Page 237
 
     Case &H100000

      sName = "TOP_DOWN"

   Case &H1000000

      sName = "IMAGE"

   Case Else

      sName = vValue

End Select

MemType = sName

End Function

' ------------------

Function AccessType(vValue As Variant) As String

' Function returns name of constant for given value.

Dim sName As String

Select Case vValue

   Case &H1

      sName = "NOACCESS"

   Case &H2

      sName = "READONLY"

   Case &H4

      sName = "READWRITE"

   Case &H8

      sName = "WRITECOPY"

   Case &H10

      sName = "EXECUTE"

   Case &H20

      sName = "EXECUTE_READ"

   Case &H40

      sName = "EXECUTE_READWRITE"

   Case &H80

      sName = "EXECUTE_WRITECOPY"

   Case &H100

      sName = "GUARD"

   Case &H200

      sName = "NOCACHE"

   Case Else

      sName = vValue

End Select

AccessType = sName

End Function
Related

Категории