[Martyn.Rae]

Microsoft - Using The Linker Map As Part Of The Debug Process

Introduction

The Microsoft linker has a linker option that can be set in the 'debugging' properties window entitled 'Generate Map File'. When set to Yes(/MAP), the linker will generate a map file which can be very useful in scenarios where a program is throwing an exception that only appears in the 'Release' version.

This tutorial is aimed at showing you how given the address that a program goes illegal, the linker map and a hex calculator, you can determine approximately where in your code that illegal is occurring.

The Code and Generated Map File

Here is a very simple windows application and the map file generated by the linker. (The code is generated by Visual Studio as part of a non-empty Win32 API project, with a small number of modifications).

#include <Windows.h>

// Global Variables:
HINSTANCE hInst;					// current instance
CHAR szTitle[] = "This application";			// The title bar text
CHAR szWindowClass[] = "MainWindowClass";		// the main window class name

// Forward declarations of functions included in this code module:
ATOM				MyRegisterClass(HINSTANCE hInstance);
BOOL				InitInstance(HINSTANCE, int);
LRESULT CALLBACK	WndProc(HWND, UINT, WPARAM, LPARAM);

int APIENTRY WinMain(HINSTANCE hInstance,
                     HINSTANCE hPrevInstance,
                     LPTSTR    lpCmdLine,
                     int       nCmdShow)
{

	MSG msg;

	// Initialize global strings
	MyRegisterClass(hInstance);

	// Perform application initialization:
	if (!InitInstance (hInstance, nCmdShow))
	{
		return FALSE;
	}

	// Main message loop:
	while (GetMessage(&msg, NULL, 0, 0))
	{	
		TranslateMessage(&msg);
		DispatchMessage(&msg);
	}

	return (int) msg.wParam;
}



//
//  FUNCTION: MyRegisterClass()
//
//  PURPOSE: Registers the window class.
//
//  COMMENTS:
//
//    This function and its usage are only necessary if you want this code
//    to be compatible with Win32 systems prior to the 'RegisterClassEx'
//    function that was added to Windows 95. It is important to call this function
//    so that the application will get 'well formed' small icons associated
//    with it.
//
ATOM MyRegisterClass(HINSTANCE hInstance)
{
	WNDCLASSEX wcex;

	wcex.cbSize = sizeof(WNDCLASSEX);

	wcex.style			= CS_HREDRAW | CS_VREDRAW;
	wcex.lpfnWndProc	= (WNDPROC)((INT)WndProc + 2); // Force illegal
	wcex.cbClsExtra		= 0;
	wcex.cbWndExtra		= 0;
	wcex.hInstance		= hInstance;
	wcex.hIcon			= NULL;
	wcex.hCursor		= LoadCursor(NULL, IDC_ARROW);
	wcex.hbrBackground	= (HBRUSH)(COLOR_WINDOW+1);
	wcex.lpszMenuName	= NULL;
	wcex.lpszClassName	= szWindowClass;
	wcex.hIconSm		= NULL;

	return RegisterClassEx(&wcex);
}

//
//   FUNCTION: InitInstance(HINSTANCE, int)
//
//   PURPOSE: Saves instance handle and creates main window
//
//   COMMENTS:
//
//        In this function, we save the instance handle in a global variable and
//        create and display the main program window.
//
BOOL InitInstance(HINSTANCE hInstance, int nCmdShow)
{
   HWND hWnd;

   hInst = hInstance; // Store instance handle in our global variable

   hWnd = CreateWindow(szWindowClass, szTitle, WS_OVERLAPPEDWINDOW,
      CW_USEDEFAULT, 0, CW_USEDEFAULT, 0, NULL, NULL, hInstance, NULL);

   if (!hWnd)
   {
      return FALSE;
   }

   ShowWindow(hWnd, nCmdShow);
   UpdateWindow(hWnd);

   return TRUE;
}

//
//  FUNCTION: WndProc(HWND, UINT, WPARAM, LPARAM)
//
//  PURPOSE:  Processes messages for the main window.
//
//  WM_COMMAND	- process the application menu
//  WM_PAINT	- Paint the main window
//  WM_DESTROY	- post a quit message and return
//
//
LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
	PAINTSTRUCT ps;
	HDC hdc;

	switch (message)
	{
	case WM_PAINT:
		hdc = BeginPaint(hWnd, &ps);
		// TODO: Add any drawing code here...
		EndPaint(hWnd, &ps);
		break;
	case WM_DESTROY:
		PostQuitMessage(0);
		break;
	default:
		return DefWindowProc(hWnd, message, wParam, lParam);
	}
	return 0;
}

and the map file:

 MapFile

 Timestamp is 4d7260a3 (Sat Mar 05 16:11:15 2011)

 Preferred load address is 0000000140000000

 Start         Length     Name                   Class
 0001:00000000 00000918H .text                   CODE
 0001:00000920 00000062H .text$x                 CODE
 0002:00000000 00000170H .idata$5                DATA
 0002:00000170 00000008H .CRT$XCA                DATA
 0002:00000178 00000008H .CRT$XCAA               DATA
 0002:00000180 00000008H .CRT$XCZ                DATA
 0002:00000188 00000008H .CRT$XIA                DATA
 0002:00000190 00000008H .CRT$XIAA               DATA
 0002:00000198 00000008H .CRT$XIY                DATA
 0002:000001a0 00000008H .CRT$XIZ                DATA
 0002:000001b0 0000001cH .rdata                  DATA
 0002:000001cc 0000003bH .rdata$debug            DATA
 0002:00000208 00000008H .rtc$IAA                DATA
 0002:00000210 00000008H .rtc$IZZ                DATA
 0002:00000218 00000008H .rtc$TAA                DATA
 0002:00000220 00000008H .rtc$TZZ                DATA
 0002:00000228 000000c8H .xdata                  DATA
 0002:000002f0 0000003cH .idata$2                DATA
 0002:0000032c 00000014H .idata$3                DATA
 0002:00000340 00000170H .idata$4                DATA
 0002:000004b0 000002d6H .idata$6                DATA
 0002:00000786 00000000H .edata                  DATA
 0003:00000000 00000048H .data                   DATA
 0003:00000048 00000068H .bss                    DATA
 0004:00000000 000000e4H .pdata                  DATA

  Address         Publics by Value              Rva+Base               Lib:Object

 0000:00000000       ___safe_se_handler_count   0000000000000000     <absolute>
 0000:00000000       ___safe_se_handler_table   0000000000000000     <absolute>
 0000:00000000       __ImageBase                0000000140000000     <linker-defined>
 0001:00000338       WinMainCRTStartup          0000000140001338 f   MSVCRT:crtexew.obj
 0001:0000034c       ?__CxxUnhandledExceptionFilter@@YAJPEAU_EXCEPTION_POINTERS@@@Z 000000014000134c f   MSVCRT:unhandld.obj
 0001:00000390       __CxxSetUnhandledExceptionFilter 0000000140001390 f   MSVCRT:unhandld.obj
 0001:000003a8       _amsg_exit                 00000001400013a8 f   MSVCRT:MSVCR100.dll
 0001:000003b0       _onexit                    00000001400013b0 f   MSVCRT:atonexit.obj
 0001:00000460       atexit                     0000000140001460 f   MSVCRT:atonexit.obj
 0001:00000478       _RTC_Initialize            0000000140001478 f   MSVCRT:_initsect_.obj
 0001:000004b0       _RTC_Terminate             00000001400014b0 f   MSVCRT:_initsect_.obj
 0001:000004e8       __C_specific_handler       00000001400014e8 f   MSVCRT:MSVCR100.dll
 0001:000004ee       _XcptFilter                00000001400014ee f   MSVCRT:MSVCR100.dll
 0001:00000500       _ValidateImageBase         0000000140001500 f   MSVCRT:pesect.obj
 0001:00000530       _FindPESection             0000000140001530 f   MSVCRT:pesect.obj
 0001:00000580       _IsNonwritableInCurrentImage 0000000140001580 f   MSVCRT:pesect.obj
 0001:000005c2       _initterm                  00000001400015c2 f   MSVCRT:MSVCR100.dll
 0001:000005c8       _initterm_e                00000001400015c8 f   MSVCRT:MSVCR100.dll
 0001:000005d0       _matherr                   00000001400015d0 f   MSVCRT:merr.obj
 0001:000005d0       _setargv                   00000001400015d0 f   MSVCRT:dllargv.obj
 0001:000005d4       __security_init_cookie     00000001400015d4 f   MSVCRT:gs_support.obj
 0001:00000688       ?terminate@@YAXXZ          0000000140001688 f   MSVCRT:MSVCR100.dll
 0001:0000068e       _unlock                    000000014000168e f   MSVCRT:MSVCR100.dll
 0001:00000694       __dllonexit                0000000140001694 f   MSVCRT:MSVCR100.dll
 0001:0000069a       _lock                      000000014000169a f   MSVCRT:MSVCR100.dll
 0001:000006a0       WndProc                    00000001400016a0 f   MapFile.obj
 0001:00000700       InitInstance               0000000140001700 f   MapFile.obj
 0001:000007a0       MyRegisterClass            00000001400017a0 f   MapFile.obj
 0001:00000820       WinMain                    0000000140001820 f   MapFile.obj
 0002:00000000       __imp_GetSystemTimeAsFileTime 0000000140002000     kernel32:KERNEL32.dll
 0002:00000008       __imp_GetCurrentProcessId  0000000140002008     kernel32:KERNEL32.dll
 0002:00000010       __imp_GetCurrentThreadId   0000000140002010     kernel32:KERNEL32.dll
 0002:00000018       __imp_GetTickCount         0000000140002018     kernel32:KERNEL32.dll
 0002:00000020       __imp_QueryPerformanceCounter 0000000140002020     kernel32:KERNEL32.dll
 0002:00000028       __imp_DecodePointer        0000000140002028     kernel32:KERNEL32.dll
 0002:00000030       __imp_SetUnhandledExceptionFilter 0000000140002030     kernel32:KERNEL32.dll
 0002:00000038       __imp_EncodePointer        0000000140002038     kernel32:KERNEL32.dll
 0002:00000040       __imp_GetStartupInfoW      0000000140002040     kernel32:KERNEL32.dll
 0002:00000048       __imp_Sleep                0000000140002048     kernel32:KERNEL32.dll
 0002:00000050       \177KERNEL32_NULL_THUNK_DATA 0000000140002050     kernel32:KERNEL32.dll
 0002:00000058       __imp__onexit              0000000140002058     MSVCRT:MSVCR100.dll
 0002:00000060       __imp__amsg_exit           0000000140002060     MSVCRT:MSVCR100.dll
 0002:00000068       __imp___getmainargs        0000000140002068     MSVCRT:MSVCR100.dll
 0002:00000070       __imp___C_specific_handler 0000000140002070     MSVCRT:MSVCR100.dll
 0002:00000078       __imp__XcptFilter          0000000140002078     MSVCRT:MSVCR100.dll
 0002:00000080       __imp__exit                0000000140002080     MSVCRT:MSVCR100.dll
 0002:00000088       __imp__ismbblead           0000000140002088     MSVCRT:MSVCR100.dll
 0002:00000090       __imp__cexit               0000000140002090     MSVCRT:MSVCR100.dll
 0002:00000098       __imp_exit                 0000000140002098     MSVCRT:MSVCR100.dll
 0002:000000a0       __imp__acmdln              00000001400020a0     MSVCRT:MSVCR100.dll
 0002:000000a8       __imp__initterm            00000001400020a8     MSVCRT:MSVCR100.dll
 0002:000000b0       __imp__initterm_e          00000001400020b0     MSVCRT:MSVCR100.dll
 0002:000000b8       __imp__configthreadlocale  00000001400020b8     MSVCRT:MSVCR100.dll
 0002:000000c0       __imp___setusermatherr     00000001400020c0     MSVCRT:MSVCR100.dll
 0002:000000c8       __imp__commode             00000001400020c8     MSVCRT:MSVCR100.dll
 0002:000000d0       __imp__fmode               00000001400020d0     MSVCRT:MSVCR100.dll
 0002:000000d8       __imp___set_app_type       00000001400020d8     MSVCRT:MSVCR100.dll
 0002:000000e0       __imp_?terminate@@YAXXZ    00000001400020e0     MSVCRT:MSVCR100.dll
 0002:000000e8       __imp__unlock              00000001400020e8     MSVCRT:MSVCR100.dll
 0002:000000f0       __imp___dllonexit          00000001400020f0     MSVCRT:MSVCR100.dll
 0002:000000f8       __imp__lock                00000001400020f8     MSVCRT:MSVCR100.dll
 0002:00000100       \177MSVCR100_NULL_THUNK_DATA 0000000140002100     MSVCRT:MSVCR100.dll
 0002:00000108       __imp_EndPaint             0000000140002108     user32:USER32.dll
 0002:00000110       __imp_GetMessageA          0000000140002110     user32:USER32.dll
 0002:00000118       __imp_RegisterClassExA     0000000140002118     user32:USER32.dll
 0002:00000120       __imp_PostQuitMessage      0000000140002120     user32:USER32.dll
 0002:00000128       __imp_BeginPaint           0000000140002128     user32:USER32.dll
 0002:00000130       __imp_TranslateMessage     0000000140002130     user32:USER32.dll
 0002:00000138       __imp_CreateWindowExA      0000000140002138     user32:USER32.dll
 0002:00000140       __imp_DefWindowProcA       0000000140002140     user32:USER32.dll
 0002:00000148       __imp_ShowWindow           0000000140002148     user32:USER32.dll
 0002:00000150       __imp_DispatchMessageA     0000000140002150     user32:USER32.dll
 0002:00000158       __imp_UpdateWindow         0000000140002158     user32:USER32.dll
 0002:00000160       __imp_LoadCursorA          0000000140002160     user32:USER32.dll
 0002:00000168       \177USER32_NULL_THUNK_DATA 0000000140002168     user32:USER32.dll
 0002:00000170       __xc_a                     0000000140002170     MSVCRT:cinitexe.obj
 0002:00000180       __xc_z                     0000000140002180     MSVCRT:cinitexe.obj
 0002:00000188       __xi_a                     0000000140002188     MSVCRT:cinitexe.obj
 0002:000001a0       __xi_z                     00000001400021a0     MSVCRT:cinitexe.obj
 0002:00000208       __rtc_iaa                  0000000140002208     MSVCRT:_initsect_.obj
 0002:00000210       __rtc_izz                  0000000140002210     MSVCRT:_initsect_.obj
 0002:00000218       __rtc_taa                  0000000140002218     MSVCRT:_initsect_.obj
 0002:00000220       __rtc_tzz                  0000000140002220     MSVCRT:_initsect_.obj
 0002:000002f0       __IMPORT_DESCRIPTOR_USER32 00000001400022f0     user32:USER32.dll
 0002:00000304       __IMPORT_DESCRIPTOR_MSVCR100 0000000140002304     MSVCRT:MSVCR100.dll
 0002:00000318       __IMPORT_DESCRIPTOR_KERNEL32 0000000140002318     kernel32:KERNEL32.dll
 0002:0000032c       __NULL_IMPORT_DESCRIPTOR   000000014000232c     user32:USER32.dll
 0003:00000000       __native_dllmain_reason    0000000140003000     MSVCRT:natstart.obj
 0003:00000004       __native_vcclrit_reason    0000000140003004     MSVCRT:natstart.obj
 0003:00000008       __globallocalestatus       0000000140003008     MSVCRT:xthdloc.obj
 0003:0000000c       __defaultmatherr           000000014000300c     MSVCRT:merr.obj
 0003:00000010       __security_cookie          0000000140003010     MSVCRT:gs_cookie.obj
 0003:00000018       __security_cookie_complement 0000000140003018     MSVCRT:gs_cookie.obj
 0003:00000020       szTitle                    0000000140003020     MapFile.obj
 0003:00000038       szWindowClass              0000000140003038     MapFile.obj
 0003:00000070       _dowildcard                0000000140003070     MSVCRT:wildcard.obj
 0003:00000074       _newmode                   0000000140003074     MSVCRT:_newmode.obj
 0003:00000078       _commode                   0000000140003078     MSVCRT:xncommod.obj
 0003:0000007c       _fmode                     000000014000307c     MSVCRT:xtxtmode.obj
 0003:00000080       __native_startup_state     0000000140003080     <common>
 0003:00000088       __native_startup_lock      0000000140003088     <common>
 0003:00000090       __onexitend                0000000140003090     <common>
 0003:00000098       __onexitbegin              0000000140003098     <common>
 0003:000000a0       __dyn_tls_init_callback    00000001400030a0     <common>
 0003:000000a8       hInst                      00000001400030a8     <common>

 entry point at        0001:00000338

 Static symbols

 0001:00000000       pre_cpp_init               0000000140001000 f   MSVCRT:crtexew.obj
 0001:00000068       __tmainCRTStartup          0000000140001068 f   MSVCRT:crtexew.obj
 0001:00000268       pre_c_init                 0000000140001268 f   MSVCRT:crtexew.obj
 0001:00000920       __tmainCRTStartup$filt$0   0000000140001920 f   MSVCRT:crtexew.obj
 0001:0000093e       _onexit$fin$0              000000014000193e f   MSVCRT:atonexit.obj
 0001:00000960       _IsNonwritableInCurrentImage$filt$0 0000000140001960 f   MSVCRT:pesect.obj

Now, referring to the map file above, the first and most important thing to check is that the timestamp of the generated map file as provided on line 3 matches the creation timestamp of the executable that is going illegal. It is all to easy to use an old map file and end up wasting a lot of time looking at code that does not contain the error.

Next, we need to determine the base address of the executable. If you look at the map file, you will notice that line 5 states:-

Preferred load address is 0000000140000000

but this is not normally used by the loader!

So, we use the debugger to run the application, by pressing F11 (step into). You will be told that there is no debugging information in the executable and asked if you would like to continue. Press yes, and then select that you would like to see the disassembly.

You will get something like the image shown below:

    000000013FC61326  or          ecx,0FFFFFFFFh  
    000000013FC61329  call        qword ptr [13FDD20B8h]  
    000000013FC6132F  xor         eax,eax  
    000000013FC61331  add         rsp,28h  
    000000013FC61335  ret  
    000000013FC61336  int         3  
    000000013FC61337  int         3
->  000000013FC61338  sub         rsp,28h
    000000013FC6133C  call        000000013FDD15D4  
    000000013FC61341  add         rsp,28h  
    000000013FC61345  jmp         000000013FDD1068  
    000000013FC6134A  int         3  
    000000013FC6134B  int         3  
    000000013FC6134C  sub         rsp,28h  
    000000013FC61350  mov         rax,qword ptr [rcx]

Line 142 of the map file tells us that:

entry point at 0001:00000338

which is read as segment 0x0001 offset address 0x00000338. Each segment is 0x1000 bytes in length, so the entry point of the executable is 0x00001338 bytes from it's base. Then, referring to the debugger, the first instruction to be executed is at 0x13FC61338. So the executable has been loaded at:

0x13FC61338 - 0x00001338 = 0x13FC60000.

Now, press F5 to continue the debugger. When the program goes illegal, the debugger will provide the address. In the code example above, I have forced an illegal by setting the window procedure address 2 bytes further on than it should be. When run, you get :

First-chance exception at 0x3fc616a2 in MapFile.exe: 0xC0000005: Access violation at location 0x000000003fc616a2.
First-chance exception at 0x3fc616a2 in MapFile.exe: 0xC0000005: Access violation at location 0x000000003fc616a2.
First-chance exception at 0x3fc616a2 in MapFile.exe: 0xC0000005: Access violation at location 0x000000003fc616a2.

in the output window. You will notice that there is a bug in visual studio such that it does not print the top 32 bits of the 64 bit address. So the value 0x3fc616a2 is actually 0x13fc616a2. To progress this further, subtract the base address from the access violation address:

0x13fc616a2 - 0x13FC60000 = 0x16a2.

So the executable has gone illegal at an offset address of 0x16a2 from the base address. Now, add the map preferred base address of 0x140000000 to get 0x1400016a2.
Looking down the map, you can see that this address lies somewhere between the following two lines.

0001:000006a0 WndProc 00000001400016a0 f MapFile.obj
0001:00000700 InitInstance 0000000140001700 f MapFile.ob

In fact, you can see that the executable has gone illegal 2 bytes into WndProc. (This makes sense as I added 2 to the address of WndProc and passed it to the RegisterClass function to force an illegal.)

Conclusion

Hopefully, you will not find yourself in a situation where the debug version of your application works perfectly, but the release version does not. The technique I have shown above may make the difference between many a happy hour of head scratching, and being able to shed some light on approximately where in the executable your illegal is occurring.