Disassembling Code: IDA Pro and SoftICE

4.1. Basic Information about Working with SoftIce

This section covers the basic information required to start working with this powerful instrument.

4.1.1. Getting Started

Main SoftIce Window

When the SoftIce window pops up, all system functions are "frozen." So, to obtain a screenshot of the SoftIce window, I had to use two computers (Fig. 4.1).

Figure 4.1: The SoftIce main window

The SoftIce main window appears in the following four cases:

• When the user presses the <Ctrl>+<D> keyboard combination. This command pops up the debugger window when executing any program. Thus, you can view the state of the operating system and executed applications at anytime.

• When loading some application into the memory using the loader32.exe program. In this case, the loading process is interrupted exactly when it encounters the entry point into the executable module. You'll be able to continue application execution in any debugger mode from that point.

• When one of the breakpoint conditions is satisfied. You set breakpoint beforehand in the debugger window. The debugger will show exactly the location where the interrupt should be placed. One of the greatest advantages of SoftIce is its capability for working with several applications in parallel. You can set breakpoints to several applications simultaneously.

• The SoftIce window can appear in case of a system error or a system crash (Blue Screen of Death, or BSOD).

The debugger window is shown in Fig. 4.1. The main window contains several windows that show different information. The number of such windows might vary. For instance, you, at your discretion, can add or remove these windows to or from the main window. The illustration demonstrates the most frequently used windows of the debugger. Note that you not only can view information in these windows but also can change their contents, such as the contents of the processor registers. However, this should be carried out carefully, because changing these registers might result in unpredictable behavior of the application or, perhaps, of the entire system. Thus, consider the debugger windows one by one, from top to bottom (see Fig 4.1):

• Registers window — This window lists all registers, including the segment registers (except for FPU registers) and their contents. The flags register is also shown, each flag designated by a separate letter. If the flag was changed during the last operation, it is marked by an uppercase letter and highlighted in the different color.

• FPU registers window— This window shows the contents of the eight FPU registers.

• Data window — This window is intended for displaying the contents of a specific memory region in both byte format and ASCII format. You can scroll this window, viewing arbitrary memory regions.

• Code window — This window contains the disassembled code, which also can be scrolled. If the application that you have loaded contains debug information recognizable by SoftIce, then the window would display the program's source code in a high-level programming language.

• Stack window — The stack window doesn't present the entire stack contents. On the contrary, it displays only the stack frame directly related to the operation of the given application.

• Command window — In this window, you can enter various SoftIce commands. In particular, from the illustration it is possible to see that by entering the H command you can display the help pane, a list of the debugger commands. To obtain information about an individual command, it is necessary to enter H followed by the command name, for example: H hwnd.

When working in the main (command) window, it is possible to issue commands to control the debugger. Along with the commands issued in the main window, it is possible to use keyboard shortcuts. In addition, provision has been made for using the standard mouse and the context menu.

Also note the lowest window — the help pane. When you issue a command, the help pane can assist you correctly enter that command and its parameters. In particular, the window will list all commands, starting from the characters that you have already entered. Besides, the debugger always displays the current process in the top right corner of the window. Always pay attention to this important issue to avoid confusing applications. This topic will be covered in more detail in Section 4.1.3.

In addition to the preceding windows, it is possible to use the trace window, where the values of all variables listed in the WATCH command are traced, as well as other windows, including the MMX registers window and the local variables window.

Operating Modes of the Debugger

After installing the SoftIce debugger, you can choose from the following five methods of start-up:

• Disable — The debugger doesn't start.

• Manual — The debugger doesn't start automatically. To start the debugger, issue the Net start ntice command. The directory, into which you install SoftIce, contains the ntice.bat file that holds this command. This mode is the safest one; however, it doesn't allow driver debugging at start-up.

• Automatic — The debugger starts up automatically. However, in this mode you cannot debug kernel-mode drivers.

• System and Boot — In both cases, the debugger starts automatically. The difference between these two modes is the order of loading the system and boot drivers.

4.1.2. The Loader

The main window of the loader32.exe program is shown in Fig. 4.2. As mentioned before, this program is intended for loading executable modules into the debugger. This utility also can retrieve debug information from the modules being debugged (provided that it is present there) and pass this information to SoftIce. When loading the module being debugged, the loader sets a breakpoint to the program entry point.

Figure 4.2: The loader32.exe program window

Loading the Executable Module

To load an executable module into the debugger, it is necessary to proceed as follows:

1. Open the module using the File | Open... menu option. You can use the Open button on the toolbar for the same purpose.

2. Then, choose the Module | Load menu item. It is also possible to use the Load Symbols toolbar button. The debugger would first translate the detected symbolic information into the file with the same name as that of the module being loaded and with the NMS file name extension, after which it would load the module, along with the debug information, into SoftIce. If the debugging information is missing, the loader would inform you about this and provide you with the choice whether or not to continue loading the module being investigated into the debugger. Translation of the debug information into the file with the NMS extension can be carried out by a separate command: either use the Module | Translate menu or click the Translate toolbar button.

The Loaded Symbols list contains the list of loaded modules. Pay special attention to the SYM= column. When loading the executable module, this column contains the size of the loaded symbolic information. Modules that do not contain such information are not displayed in the Loaded Symbols list.

Loading Parameters

After loading the module being investigated into SoftIce, it is possible to specify the start-up settings. To achieve this, use the Module | Settings... menu. The window, in which you can specify these settings is shown in Fig. 4.3. This window has four tabs, which must be considered in more detail.

Figure 4.3: The Settings window allows you to set the loading parameters for the modules to be debugged

• The General tab contains the following controls:

  • The Command line arguments field allows you to specify the command-line parameters, with which the program being debugged must be started under the debugger.

  • The Source file search path field lets you specify the search paths for the files related to the module being debugged.

  • The Default source file search path field allows you to specify the main search path for the files. The debugger always searches according to the path specified in the Source file search path field, and only after that uses this field.

  • If the Prompt for missing source files flag is set, then the debugger will inform you if all files required for debugging the executable file are not available. In particular, if the debugging information is missing, you'll be prompted to specify whether to continue loading the executable file into the debugger.

  • The Minimize Loader on successful load flag is used for minimizing the loader size in the memory after the executable program is loaded into the debugger.

• The Debugging tab allows you to modify some current debugging parameters:

  • The Load symbol information only and Load executable checkboxes allow you to load only debug information into the debugger and to load debug information with the executable module, respectively.

  • The Stop at WinMain, main, DllMain etc. flag allows you to set a breakpoint to the starting point of the user part of the executable module. If the debug information is not available, the breakpoint is set to the starting point of program execution.

• Using the Translation tab, you can set the translation parameters of the debug information of the executable module. The switches are as follows:

  • Publics only — Translate external names only.

  • Type information only — Translate only information about variable types.

  • Symbols only — Translate symbolic names only.

  • Symbols and source code — Translate all debugging information.

  • Package source with symbol table — Save the translated information in the NMS file.

• The Modules and Files tab allows you to list all files and their locations. These files will be loaded with the executable module. You can list all files that contain the debug information here. Loading of specific files can be temporarily blocked using a special switch.

4.1.3. Techniques of Working with SoftIce

Getting Started with Processes

Consider the main issues related to working with SoftIce:

• You are working in a multitasking operating system. The program that you are going to investigate using SoftIce after loading will become only one of the multiple processes running in the system. You must know exactly, with which process you are working. Do not confuse processes, because this might freeze the entire system. The debugger shows the current process in the bottom right corner of the help window.

• When loading an application using the loader32.exe program, the breakpoint is set to the start of program execution. The newly-created process becomes the current one. Thus, you can comfortably trace the newly-loaded application (see Section 4.2.2). However, when you close the debugger window by pressing <F5>, and then call it again, the process you were tracing earlier won't be the current process any longer. Each newly-running process has its own virtual address space. This address space is called the process context. For example, the D Ds:004080AF command will output the memory contents for a specific virtual address space, and that address space will represent the context of the current process. To work with the addresses of an individual process, it is necessary to ensure that this process is the current one. To achieve this, use the ADDR command (a description of this command will be provided in Section 4.2.2, see "Main Informational Commands"). The following illustrates the use of the ADDR command: : ADDR 058. Here, 058 is the process identifier (PID). The PID value for the current process can be determined by using the ADDR command without parameters.

• Breakpoints represent the main tool of investigating executable code. You should clearly understand where the breakpoint is set (in other words, to which process or thread a specific breakpoint relates). In particular, this relates to setting breakpoints to API calls. When you create such a breakpoint, always use conditional constructs to specify, to which process that breakpoint relates. For this purpose, use the PID function that returns the current PID. As relates to the PID value, it can be obtained using the previously-mentioned ADDR command. The following illustrates how to set a breakpoint to the CreateWindowEx API function: : BPX CreateWindowEx if (PID == 0x58). It is necessary to point out again that the PID value for the required process can be determined using the ADDR or PROC commands. To set such a breakpoint, it is not necessary to make the required process the current one.

Breakpoints

When investigating specific executable code, one task always consists of searching for the required location within the program being investigated. When program code written in one of the high-level programming languages is not available (which most often is the case, unless you are debugging your own program), breakpoints are indispensable in research.

Nonpermanent Breakpoints

Nonpermanent (one-shot) breakpoints operate only once. Actually, nonpermanent breakpoint is a line in the code window, to which the cursor (highlighted string) points. To move the cursor, use the U command. The HERE command (or the <F7> shortcut) runs the executable code from the current command to the line of code marked using this method. Bear in mind that the HERE command is issued from the code window; it is necessary to switch to this window by pressing <F6> before issuing the command. It is also possible to use the G address command, in which case the code will execute to the specified address.

Persistent Breakpoints

A typical example of a persistent (sticky) breakpoint is a breakpoint set to a specific command (a specific virtual address of the process). To set a persistent breakpoint, it is necessary to switch to the code window and use the BPX command without parameters. You can scroll the code and set breakpoints at required addresses. In this case, the lines of code, to which the breakpoints are set, will be highlighted. The same result can be achieved by using the <F9> shortcut. To remove the existing breakpoint, either move the cursor to the existing breakpoint and issue the BPX command, or press <F9>.

The general method of controlling breakpoints is applicable to the following breakpoints: BL to find the numbered list of breakpoints, BC n to delete the breakpoint with the specified number, BC * to delete all breakpoints, and BE n to edit the breakpoint with the specified number. Finally, if you know the address, at which you need to set a breakpoint, you can use that address in the BPX command, for example: BPX 0008:806CEFAB. If you issue the BPX command with the same address again, the breakpoint with the specified address will be deleted. Do not forget that the breakpoint set to the command address is related to a specific address space — in other words, to a specific process.

Conditional Breakpoints

Conditional breakpoints are activated if the conditions specified for them are satisfied. It is impossible to set two different breakpoints to the same address or to the same API function; however, you can use conditional constructs to take into account different variants of calling the same breakpoints. Here are typical examples of using conditional breakpoints.

Example 1

The breakpoint set to a specific address is activated only if the content of the EAX register takes the specified value:

: BPX 0008:806CEFAB if(EAX == 406090)

Example 2

Consider a small investigation for the breakpoint set to the MessageBox function call (the application under consideration is the WinRar archiving utility). Start the WinRar application, open the SoftIce window, and determine the application identifier using the ADDR command. The identifier turned out to be equal to 0x328. Then, issue the following command to create a conditional breakpoint:

: BPX MessageBoxA if(PID == 0x328)

Issue the BL command to make sure that the breakpoint is set as desired. Note that the A suffix has been specified. This suffix is mandatory, because SoftIce distinguishes API functions by their actual names.

Exit the debugger by pressing <F5>, and execute one of the application's commands that must cause the MessageBox window to appear. The SoftIce window will pop up immediately. The command window will display the message informing you why the SoftIce window has popped up. In this example, this message will appear as follows:

Break due to BP 00: USER32!MessageBoxA IF(PID = 0x328) (ET = 2.65 seconds)

Now, consider the code window. The first line of the entry into the MessageBox procedure will be highlighted there:

USER32!MessageBoxA 001B:77D56471 CMP DWORD PTR [77D8C3DO], 0

You can easily investigate the stack of the MessageBoxA function call and find the return address and parameter values. Having executed the ? * (ESP + 4) command, you'll obtain the value of the window handle for the window that initialized the MessageBox call (if something is unclear to you, return to Section 3.2.1, "Stack Structures"). The HWND value turns out to be equal to 100EC. View the list of windows opened by the WinRar application using the HWND 328 command, and you'll see that such a window exists and that it corresponds to the WinRarWindow class. By the way, in the same table you'll see the address of the window function of this window. Thus, it is possible to dive into studying the operation of this window. However, return to the first line of the MessageBox function call and find the return address. The ESP register points to the return address, Thus, by issuing the ? *(ESP) command, you'll discover that this address is equal to 43C76D.

There is another method of obtaining the return address. To use this method, press <F11>. After the MessageBox window appears, click one of the buttons in this window. The SoftIce window will appear, and you'll find yourself on the line that directly follows the MessageBox function call.

Example 3

Consider how the register contents can be traced:

: BPX EIP IF(EAX == 0x10)

The breakpoint specified by this command will be activated when the value of the EAX register becomes equal to 0x10, regardless of in which thread this event takes place.

Breakpoints to Windows Messages

As you know, the main events in GUI applications take place in window functions. Discovering the reaction of the window function to a specific message is the most important goal of program code investigation. Here, breakpoints set to Windows messages are indispensable. An example of the command that sets such a breakpoint is as follows:

: BMSG 100EC WM_CREATE

The first parameter of this command is the window handle, the function of which must receive the message. By the window handle value, the debugger determines the thread that has created this window, so the investigator doesn't need to care about solving this problem. When the required message arrives, the SoftIce debugger is activated and the code window will display the start of the window function. An interesting point is that the same result can be achieved using a standard BPX command, for example:

: BPX 43C76D IF((ESP -> 8) == WM_CREATE)

The first parameter is the address of the first command of the window function. Later, the command exploits the fact that the second parameter of the window function is located 8 bytes from the stack top.

Searching for a Window Procedure

How is it possible to locate a window procedure? Here are some helpful and easy tips:

• View the list of application windows. This list can be displayed by the HWND n command, where n is the application identifier. As you already know, the application identifier can be obtained using the ADDR command. The list of application windows contains their names, using which it is sometimes easy to locate the required window and, consequently, the window procedure address.

• If the list of windows is small, you can easily test all procedures by setting a breakpoint to the start of the window procedure (to be precise, at one of the first commands). If the breakpoint is activated when the window is activated, this means that you have found the required window.

• Analyze the window operation to find out, which API function can be called when working with this window (this was method I chose in Example 2 from the previous section). Set a breakpoint to that function and carry out some operations in that window. In case of an interrupt, determine from which location was that function called. This will be the window function. In addition, bear in mind that most API functions accept the window handle as the first parameter.

Working with Applications that Contain Debug Information

SoftIce is a full-featured debugger, which means that it can load debug information and supply it with the executable code. Thus, when debugging custom applications, SoftIce can be used instead of the standard debugger built into the integrated development environment (IDE). Consider this approach on the example of debugging user applications written in C++ using Visual Studio .NET.

When the "add debug info" option is chosen (for this purpose, the best approach is using the DEBUG project configuration), the debug information database is created with the executable module. This database represents a file that by default has the same name as the executable module and the PDB file name extension (see Section 1.6.1). Information stored in that file is enough to represent the structure of the source program, along with the names of local and global variables, and to map this structure to the machine code.

When the loader32.exe program loads the executable module, it also loads the debug information and passes this to the debugger. By default, if the debug information is available for the program, SoftIce presents the program source code without Assembly commands in the source code window. Later, you can use the SRC command to switch to the mixed program representation (the source code of the program and machine code) or to pure machine code. In the first case, step-by-step program execution means that the program is executed one operator at a time. When using mixed representation, one step is equivalent to one machine command. Accordingly, it is possible to set breakpoints both to the operators of a high-level programming language and to machine commands. Listing 4.1 demonstrates several lines from the SoftIce code window when mixed representation was used.

Listing 4.1: Several lines from the Softlce code window when mixed representation was used

00006 a = 10; 001B:00411A2E MOV DWORD PTR [EBP - a], 0000000A 00007 b = 11; 001B:00411A35 MOV DWORD PTR [EBP - b], 0000000B 00008 c = 10; 001B:00411A3C MOV DWORD PTR [EBP - c], 0000000C 00009 printf("%d\n", max(a, b, c)); 001B:00411A43 MOV EAX, [EBP - c] 001B:00411A46 PUSH EAX ...

You should understand that in expressions such as [EBP - a] the a value is the address of the a variable in the stack (to be precise, the offset in relation to the address where the old EBP is stored — in other words, simply four).