DLL Advanced Techniques

For a thread to call a function in a DLL module, the DLL’s file image must be mapped into the address space of the calling thread’s process. You can accomplish this in two ways. The first way is to have your application’s source code simply reference symbols contained in the DLL. This causes the loader to implicitly load (and link) the required DLL when the application is invoked.

The second way is for the application to explicitly load the required DLL and explicitly link to the desired exported symbol while the application is running. In other words, while the application is running, a thread within it can decide that it wants to call a function within a DLL. That thread can explicitly load the DLL into the process’ address space, get the virtual memory address of a function contained within the DLL, and then call the function using this memory address. The beauty of this technique is that everything is done while the application is running.

Figure below shows how an application explicitly loads a DLL and links to a symbol within it

If you pass NULL to GetModuleHandle, the handle of the application executable is returned.

You can explicitly load a DLL through the WinAPI LoadLibrary(Ex)

You can explicitly unload a DLL through WInAPI FreeLibrary or FreeLibraryAndExitThread

You can explicitly link to an exported symbol from a DLL using WinAPI GetProcAddress

The Platform SDK documentation states that your DllMain function should perform only simple initialization, such as setting up thread-local storage, creating kernel objects, and opening files. You must also avoid calls to User, Shell, ODBC, COM, RPC, and socket functions (or functions that call these functions) because their DLLs might not have initialized yet or the functions might call LoadLibrary(Ex) internally, again creating a dependency loop.

If a process terminates because some thread in the system calls TerminateProcess, the system does not call the DLL’s DllMain function with a value of DLL_PROCESS_DETACH. This means that any DLLs mapped into the process’ address space do not have a chance to perform any cleanup before the process terminates. This can result in the loss of data. You should use the TerminateProcess function only as a last resort.

Figure below explains what happens when a thread call LoadLibrary

And this figure when FreeLibrary is called

Normally, you don’t even think about this DllMain serialization. The reason I’m making a big deal out of it is that you can have a bug in your code caused by DllMain serialization. This code looked something like the following code:

BOOL WINAPI DllMain(HINSTANCE hInstDll, DWORD fdwReason, PVOID fImpLoad) {
   HANDLE hThread;
   DWORD dwThreadId;

   switch (fdwReason) {

   case DLL_PROCESS_ATTACH:

      // The DLL is being mapped into the process' address space.

      // Create a thread to do some stuff.

      hThread = CreateThread(NULL, 0, SomeFunction, NULL, 0, &dwThreadId);

      // Suspend our thread until the new thread terminates.
      WaitForSingleObject(hThread, INFINITE);
      // We no longer need access to the new thread.
      CloseHandle(hThread);
      break;
   case DLL_THREAD_ATTACH:
      // A thread is being created.
      break;
   case DLL_THREAD_DETACH:
      // A thread is exiting cleanly.
      break;
   case DLL_PROCESS_DETACH:
      // The DLL is being unmapped from the process' address space.
      break;
   }

   return(TRUE);

}

 

It may took several hours to discover the problem with the code. Can you see it? When DllMain receives a DLL_PROCESS_ATTACH notification, a new thread is created. The system must call DllMain again with a value of DLL_THREAD_ATTACH. However, the new thread is suspended because the thread that caused the DLL_PROCESS_ATTACH notification to be sent to DllMain has not finished processing. The problem is the call to WaitForSingleObject. This function suspends the currently executing thread until the new thread terminates. However, the new thread never gets a chance to run, let alone terminate, because it is suspended—waiting for the current thread to exit the DllMain function. What we have here is a deadlock situation. Both threads are suspended forever!

Delay Loading a DLL

Microsoft Visual C++ offers a fantastic feature to make working with DLLs easier: delay-load DLLs. A delay-load DLL is a DLL that is implicitly linked but not actually loaded until your code attempts to reference a symbol contained within the DLL. Delay-load DLLs are helpful in these situations:

  • If your application uses several DLLs, its initialization time might be slow because the loader maps all the required DLLs into the process’ address space. One way to alleviate this problem is to spread out the loading of the DLLs as the process executes. Delay-load DLLs let you accomplish this easily.
  • If you call a new function in your code and then try to run your application on an older version of the system in which the function does not exist, the loader reports an error and does not allow the application to run. You need a way to allow your application to run and then, if you detect (at run time) that the application is running on an older system, you don’t call the missing function. For example, let’s say that an application wants to use the new Thread Pool functions when running on Windows Vista and the old functions when running on older versions of Windows. When the application initializes, it calls GetVersionEx to determine the host operating system and properly calls the appropriate functions. Attempting to run this application on versions of Windows older than Windows Vista causes the loader to display an error message because the new Thread Pool functions don’t exist on these operating systems. Again, delay-load DLLs let you solve this problem easily.

However, a couple of limitations are worth mentioning:

  • It is not possible to delay-load a DLL that exports fields.
  • The Kernel32.dll module cannot be delay-loaded because it must be loaded for LoadLibrary and GetProcAddress to be called.
  • You should not call a delay-load function in a DllMain entry point because the process might crash.

Let’s start with the easy stuff: getting delay-load DLLs to work. First, you create a DLL just as you normally would. You also create an executable as you normally would, but you do have to change a couple of linker switches and relink the executable. Here are the two linker switches you need to add:

  • /Lib:DelayImp.lib
  • /DelayLoad:MyDll.dll

To unload a delay-loaded DLL, you must do two things. First, you must specify an additional linker switch (/Delay:unload) when you build your executable file. Second, you must modify your source code and place a call to the __FUnloadDelayLoadedDLL2 function at the point where you want the DLL to be unloaded:

BOOL __FUnloadDelayLoadedDLL2(PCSTR szDll);

You can take advantage of function forwarders in your DLL module as well. The easiest way to do this is by using a pragma directive, as shown here:

// Function forwarders to functions in DllWork

#pragma comment(linker, "/export:SomeFunc=DllWork.SomeOtherFunc")

This pragma tells the linker that the DLL being compiled should export a function called SomeFunc. But the actual implementation of SomeFunc is in another function called SomeOtherFunc, which is contained in a module called DllWork.dll. You must create separate pragma lines for each function you want to forward.

Applications can depend on a specific version of a shared DLL and start to fail if another application is installed with a newer or older version of the same DLL. There are two ways to ensure that your application uses the correct DLL: DLL redirection and side-by-side components. Developers and administrators should use DLL redirection for existing applications, because it does not require any changes to the application. If you are creating a new application or updating an application and want to isolate your application from potential problems, create a side-by-side component.

Rebasing Modules

 

When this executable module is invoked, the operating system loader creates a virtual address for the new process. Then the loader maps the executable 
module at memory address 0x00400000 and the DLL module at 0x10000000. Why is this preferred base address so important? Let's look at this code:
int g_x;

 

void Func() {

   g_x = 5; // This is the important line.

}

When the compiler processes the Func function, the compiler and linker produce machine code that looks something like this:

MOV   [0x00414540], 5

In other words, the compiler and linker have created machine code that is actually hard-coded in the address of the g_x variable: 0x00414540. This address is in the machine code and absolutely identifies the location of the g_x variable in the process’ address space. But, of course, this memory address is correct if and only if the executable module loads at its preferred base address: 0x00400000.

What if we had the exact same code in a DLL module? In that case, the compiler and linker would generate machine code that looks something like this:

MOV   [0x10014540], 5

Again, notice that the virtual memory address for the DLL's g_x variable is hard-coded in the DLL file's image on the disk drive. And again, this memory address is absolutely correct as long as the DLL does in fact load at its preferred base address.

Relocating an executable (or DLL) module is an absolutely horrible process, and you should take measures to avoid it. Let's see why. 
Suppose that the loader relocates the second DLL to address 0x20000000. In that case, the code that changes the g_x variable to 5 should be

MOV   [0x20014540], 5

But the code in the file’s image looks like this:

 

MOV   [0x10014540], 5

 

If the code from the file’s image is allowed to execute, some 4-byte value in the first DLL module will be overwritten with the value 5. This can’t possibly be allowed. The loader must somehow fix this code. When the linker builds your module, it embeds a relocation section in the resulting file. This section contains a list of byte offsets. Each byte offset identifies a memory address used by a machine code instruction. If the loader can map a module at its preferred base address, the module’s relocation section is never accessed by the system. This is certainly what we want—you never want the relocation section to be used.

If, on the other hand, the module cannot be mapped at its preferred base address, the loader opens the module’s relocation section and iterates though all the entries. For each entry found, the loader goes to the page of storage that contains the machine code instruction to be modified. It then grabs the memory address that the machine instruction is currently using and adds to the address the difference between the module’s preferred base address and the address where the module actually got mapped.

So, in the preceding example, the second DLL was mapped at 0x20000000 but its preferred base address is 0x10000000. This yields a difference of 0x10000000, which is then added to the address in the machine code instruction, giving us this:

MOV   [0x20014540], 5

Now this code in the second DLL will reference its g_x variable correctly.

There are two major drawbacks when a module cannot load at its preferred base address:

  • The loader has to iterate through the relocation section and modify a lot of the module’s code. This produces a major performance hit and can really hurt an application’s initialization time.
  • As the loader writes to the module’s code pages, the system’s copy-on-write mechanism forces these pages to be backed by the system’s paging file.

By the way, you can create an executable or DLL module that doesn’t have a relocation section in it. You do this by passing the /FIXED switch to the linker when you build the module. Using this switch makes the module smaller in bytes, but it means that the module cannot be relocated. If the module cannot load at its preferred base address, it cannot load at all. If the loader must relocate a module but no relocation section exists for the module, the loader kills the entire process and displays an "Abnormal Process Termination" message to the user.

Preferred base addresses must always start on an allocation-granularity boundary.

Visual Studio ships a tool called rebase utility this utility do the following when you call ReBaseImage:

When you execute Rebase, passing it a set of image filenames, it does the following:

  1. It simulates creating a process’ address space.
  2. It opens all the modules that would normally be loaded into this address space. It thus gets the preferred base address and size of each module.
  3. It simulates relocating the modules in the simulated address space so that none of the modules overlap.
  4. For the relocated modules, it parses that module’s relocation section and modifies the code in the module file on disk.
  5. It updates the header of each relocated module to reflect the new preferred base address.

Binding Modules

Rebasing is very important and greatly improves the performance of the entire system. However, you can do even more to improve performance. Let’s say that you have properly rebased all your application’s modules. The loader writes the symbol’s virtual address into the executable module’s import section. This allows references to the imported symbols to actually get to the correct memory location.

Let’s think about this for a second. If the loader is writing the virtual addresses of the imported symbols into the .exe module’s import section, the pages that back the import section are written to. Because these pages are copy-on-write, the pages are backed by the paging file. So we have a problem that is similar to the rebasing problem: portions of the image file are swapped to and from the system’s paging file instead of being discarded and reread from the file’s disk image when necessary. Also, the loader has to resolve the addresses of all the imported symbols (for all modules), which can be time-consuming.

You can use the technique of binding a module so that your application can initialize faster and use less storage. Binding a module prepares that module’s import section with the virtual addresses of all the imported symbols. To improve initialization time and to use less storage, you must do this before loading the module, of course.

Visual Studio ships a bind.exe utility that do the following when you execute Bind passing it an image name:

  1. It opens the specified image file’s import section.
  2. For every DLL listed in the import section, it opens the DLL file and looks in its header to determine its preferred base address.
  3. It looks up each imported symbol in the DLL’s export section.
  4. It takes the RVA of the symbol and adds to it the module’s preferred base address. It writes the resulting expected virtual address of the imported symbol to the image file’s import section.
  5. It adds some additional information to the image file’s import section. This information includes the name of all DLL modules that the image is bound to and the time stamp of those modules.

OK, so now you know that you should bind all the modules that you ship with your application. But when should you perform the bind? If you bind your modules at your company, you would bind them to the system DLLs that you’ve installed, which are unlikely to be what the user has installed. Because you don’t know if your user is running Windows XP, Windows 2003, or Windows Vista, or whether these have service packs installed, you should perform binding as part of your application’s setup.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s