Debugging Malware with WinDbg
At the Application and Threat Intelligence (ATI) Research Center, we constantly analyze malicious artifacts to harvest their intelligence and use it to keep our customers protected. Over time, a lot of this has become automated through sandboxed analysis frameworks like Cuckoo. Occasionally, we still need to be able to answer questions about a sample of malware that requires manual analysis efforts. There are a lot of specialized tools that exist for performing certain tasks. For example, you may use CFF Explorer to analyze a PE File, unpack an executable, or rebuild an import address table; regshot to identify changes to the registry; and wireshark to closely observe network communication. What they all have in common is they leverage structures and API calls defined within the operating system. This is information that can also be produced by a debugger, provided you know where to look.
In this blog post, we present practical techniques for finding information you may be interested in by stepping through a Locky Ransomware Sample using WinDbg. WinDbg is the debugger of choice by Microsoft, so it should be for us too. There are a lot of features that make WinDbg special, and learning them takes a lot of time and practice. Here are the hash details for this post’s analysis:
* This file is malicious and should be analyzed in an isolated and controlled environment.
To further share our knowledge, you can view our collection of WinDbg scripts on Github (https://github.com/ixiacom/ATI). This folder contains lists of breakpoints and interesting scripts to assist in the analysis of malware. We hope to see collaboration from other contributors to grow and share their useful scripts as well. At anytime during this blog, should you want to know more information about a command, you can type .hh <command> to get more information.
CHANGING APPEARANCES
If you aren't aware, there are themes available for WinDbg that present the inner workings of your target binary in a more consumable manner. Normally, when you start WinDbg, it loads a hideous notepad theme (joke). Okay, not really joking, it truly isn't as appealing as other layouts built into Immunity or OllyDbg. Thankfully, we can change that by applying either of the themes listed below. The one I use can be grabbed from here. There is also another theme on github you may want to experiment with here.
[View with Theme Applied]
WORKING WITH MODULES
The lm command lists modules used in a given process. After loading our Locky Ransomware sample, we can type lmf to get a list of modules loaded and their file paths. Take note of the column names; we see the starting and ending address for each module, its name and location. We may begin inspecting modules closely with lmv m <module_name>, the output below shows the target module (image00400000) being displayed verbosely.
To begin analyzing a PE File in WinDbg, we need to note the base address that signifys where it is loaded in memory. With this, we use the dump type (dt) command to walk through the file structure as other PE tools do. This technique can be very useful if you are trying to analyze a file in memory that does not reside on disk, also known as “fileless malware”. If you are curious what all structures are available for you to dump, you can do so by typing dt *!*. The ones of interest to us live under ntdll and can be listed by typing dt ntdll!*. If you are interested in learning more about PE file structures, you are encouraged to read Peering Inside the PE: A Tour of the Win32 Portable Executable File Format from MSDN.
INSPECTING PE STRUCTURES
dt -r ntdll!_IMAGE_DOS_HEADER <base_address>
This command dumps out the address as an _IMAGE_DOS_HEADER structure. This is the DOS header found in executables and is identified with the magic file header "MZ". Note, that the attribute of importance is the e_lfanew property. Its value indicates how far from the beginning of the module the PE header starts. We can specifically print out this value by typing dt -o ntdlL!_IMAGE_DOS_HEADER e_lfanew <base_address>. Where base_address is from the lm output. The -o parameter omits the offset being displayed for a slightly cleaner look as seen below:
*note: The “n” in 0n128 indicates that the number is base 10, whereas “x” will indicate base 16.
We can now combine the offset of our loaded module with the e_lfanew property to get to the PE header. By using the "?" evaluation expression character, we can quickly calculate this with ? 0x400000 + 0n128 resulting in 0x400080. From here, we can check if this is the correct offset by dumping the address as ascii (da) with da 0x400080 and checking for the "PE" file header:
Now that we've double-checked our work, we can continue dumping the PE file with dt -r ntdll!_IMAGE_NT_HEADERS 0x4000c8. This recursively walks the structure and displays each property inside. Contrasting this with CFF explorer, we validate that the information is indeed the same (truncated output).
[WinDbg Output]
[CFF Explorer Output]
This is quite a bit of information to look at, and you may only want to look at certain attributes like the AddressOfEntryPoint. Thankfully, we can use OO notation to view this by entering dt ntdll!_IMAGE_NT_HEADERS OptionalHeader.AddressOfEntryPoint 0x400080.
This address of entry is also calculated for a special WinDbg variable called $exentry. You may notice when you load an executable in WinDbg, it doesn't stop at the entry point of the target module like other debuggers may do. To get the same experience, we can set a breakpoint with bp $exentry to achieve the same thing. Continuing execution of the debugger with g stops us exactly at the entry point. We can validate this by adding the base address to the AddressOfEntryPoint using ? 0x400000 + 0x2af8 and getting 0x402af8 as a result as shown below:
It's cool to see that we can use dt to dump a given address as a type of structure, but there are two additional ways to dump even more verbose information. The !dh and **!dlls **command can dump additional PE information including section headers, imports, and exports. For example, we can use !dh -s 0x40000 to view all the sections.
Another useful version is !dh -i, it dumps the imports required for the target binary. Here, we can see a list of functions being imported.
(Not shown: shimeng.dll, user32.dll, cmpbk32.dll, azroles.dll)
By understanding what is being imported, you can begin to build an idea of what this sample is capable of and may be doing. Of course this isn't a replacement for awesome utilities like pestudio, but may come in handy if you find a PE header in memory but not on disk.
BYPASSING HURDLES
Malware authors deploy techniques to detect when they are being highly scrutinized so they may change their act and be passed along as harmless or to guard their secrets. As analysts, we consider these techniques to be "hurdles" because they require us to change the flow of code or environment in some fashion to continue analysis. Here are some tips and tricks to help bypass obstacles.
Speeding up Sleep
Without any modification to the binary, you will notice it doesn't seem to be doing much once you start running it. This is because it has detected it is being debugged and decided to sleep before continuing any further. If we are curious what happens after the sleep, we can use the breakpoint below to print how long it tried to sleep, and skip ahead by setting the argument in memory to 0.
bp kernelbase!sleepex ".printf \"Application tried to sleep: %u seconds...\",poi(esp+4)/1000;.echo;ed esp+4 0x0;g"
WinDbg supports executing a series of commands after reaching a breakpoint. In other words, you can have a string of commands to run once you hit a breakpoint. The commands themselves are delimeted with a semicolon and wrapped between double quotes. If you need to use double quotes inside, make sure you escape them with \. Here, we are referencing poi(esp+0x4) and not esp+0x4. This is because poi() grabs the value at the location as opposed to referencing the location. After we print out some interesting information, we use the ed command to edit a dword at esp+0x4 and zero it out, then continue running the malware.
Identifying Debugger Checks
There are many different types of checks used to identify debuggers. A great read on the topic is Nicolas's Windows Anti-Debug Reference. The byte at offset 0x2 in the Process Execution Block (PEB) is often checked by malware via WinAPI calls or direct checks to see if it is currently being debugged. If you were trying to hunt for where this check may have been done, we can use the following breakpoint:
bp $exentry "ba r 1 $peb+0x2 \"ub\";g"
This sets a breakpoint at the target entry point. We have to get here before we can leverage the $peb variable. Once we do, the script sets a break on access (ba) breakpoint when attempting to read that specific byte from the PEB at offset 0x2. The first check on this address from inside the target module occurs at 0x473827 and contains the following code, stripped of any NOP/JMP obfuscation:
movzx eax, byte ptr [eax+2] ; Move the PEB Debugger Byte into eax.
mov dword ptr [ebp-4], eax ; Move the value into a variable.
cmp dword ptr [ebp-4], edx ; Compare it with edx (edx is set to 0)
je image00400000+0x73afe ; Jump if not being debugged.
By analyzing this routine, we can tell that 0x473afe is where this process will continue if it is not being debugged. If you want a closer look, make sure you use a hardware breakpoint (ba) for that address with the syntax of ba e 1 0x473827.
Bypassing IsDebuggerPresent and PEB Checks
We can piggyback on the logic above and implement our own anti-debugging bypass technique by simply zeroing out the memory before we even run the module. Let's prepare our malware for execution by using the following breakpoint:
bp $exentry "eb $peb+0x2 0x0"
The eb seeks to edit a byte of memory located at the $peb, offset 0x2 and sets it to zero. Now, we no longer have to worry about calls to IsDebuggerPresent or direct checks to the memory address because we've zeroed it out in advance.
SYSTEM OPERATIONS
Now that we've bypassed a few checks, we can use additional breakpoints to print arguments to functions of interest. Something to keep in mind with naming conventions is A vs. W. A is expecting an ASCII string while W expects a wide/unicode format. So if you're wondering why nothing is hitting, try switching A for W.
Identifying when Malware Creates Files
This is somewhat difficult feat because the CreateFile function is also used to open a file. The access mask dictates permissions, which is typically a combined read/write request. You can read more about access masks on MSDN here or from a listing at pinvoke.
We try to overcome this confusion by focusing on the CreationDisposition parameter to filter out what isn't being accessed as OPEN_ONLY (0n3). That said, this breakpoint isn't foolproof, but should reduce a lot of false positives where a malware simply attempts to open a file.
bp kernelbase!CreateFileW ".if(poi(esp+0x14) != 0n3){.printf \"Creating File: %mu\",poi(esp+0x4);.echo};g"
bp kernelbase!CreateFileA ".if(poi(esp+0x14) != 0n3){.printf \"Creating File: %mu\",poi(esp+0x4);.echo};g"
Do note there is no support for %s as a format specifier, so we use %ma and it gets passed poi(esp+0x4). poi(esp+0x4) represents a pointer to esp+0x4 -- the filename. Note that we have to escape the double quotes for the printf statement because they reside within the scripting quotes.
Identifying when Malware Writes Data to a File
bp kernelbase!WriteFile ".printf \"Dumping file contents from 0x%p as ASCII: %ma\",poi(esp+0x8),poi(esp+0x8);.echo;g"
Sometimes you may want to see the contents of a file data buffer as it is being written. We can do so by tracing calls to WriteFile and printing out esp+0x8. Interestingly enough, we see an MZ file header as the first two bytes, indicating an executable more than likely exists inside the buffer.
Identifying when Malware Deletes Files
bp kernel32!DeleteFileW ".printf \"Deleting File: %mu\",poi(esp+4);.echo;g"
bp kernel32!DeleteFileA ".printf \"Deleting File: %ma\",poi(esp);.echo;g"
Identifying when Malware Moves Files
bp kernel32!MoveFileExW ".printf \"File moved.\";.echo;.printf \"From: %mu\",poi(esp+0x4);.echo;.printf \"To: %mu\",poi(esp+0x8);.echo;g"
bp kernel32!MoveFileExA ".printf \"File moved.\";.echo;.printf \"From: %ma\",poi(esp+0x4);.echo;.printf \"To: %ma\",poi(esp+0x8);.echo;g"
Identifying when Malware Copies Files
bp kernel32!CopyFileW ".printf \" Copying file: \";.echo;.printf \"\tFrom: %mu\",poi(esp+0x4);.echo;.printf \"\tTo: %mu\",poi(esp+0x8);.echo;g"
bp kernel32!CopyFileA ".printf \" Copying file: \";.echo;.printf \"\tFrom: %ma\",poi(esp+0x4);.echo;.printf \"\tTo: %ma\",poi(esp+0x8);.echo;g"
Identifying when Malware Creates Registry Keys
bp kernel32!RegCreateKeyExA “.printf \”Creating RegKey: %ma\”,poi(esp+0x8);.echo;g”
bp kernel32!RegCreateKeyExW “.printf \”Creating RegKey: %mu\”,poi(esp+0x8);.echo;g”
Identifying when Malware Accesses Registry Keys
bp kernel32!RegOpenKeyExA ".printf \"Accessed RegKey: %ma\",poi(esp+0x8);.echo;g"
bp kernel32!RegOpenKeyExW ".printf \"Accessed RegKey: %mu\",poi(esp+0x8);.echo;g"
Identifying when Malware Accesses Registry Key Values
bp kernel32!RegQueryValueExA ".printf \"\tAccessed RegValue: %ma\",poi(esp+0x8);.echo;g"
bp kernel32!RegQueryValueExW ".printf \"\tAccessed RegValue: %mu\",poi(esp+0x8);.echo;g"
Identifying when Malware Changes Registry Keys
bp kernel32!RegSetValueExA ".printf \"Setting RegKey %ma to value: %ma\",poi(esp+0x8),poi(esp+0x14);.echo;g"
bp kernel32!RegSetValueExW ".printf \"Setting RegKey %mu to value: %mu\",poi(esp+0x8),poi(esp+0x14);.echo;g"
Identifying when Malware Creates Services
bp advapi32!CreateServiceA ".printf \"Creating Service: \";.echo;.printf \"\tService Name: %ma\",poi(esp+0x4);.echo;.printf \"\tDisplay Name: %ma\",poi(esp+0x8);.echo;g"
bp advapi32!CreateServiceW ".printf \"Creating Service: \";.echo;.printf \"\tService Name: %mu\",poi(esp+0x4);.echo;.printf \"\tDisplay Name: %mu\",poi(esp+0x8);.echo;g"
Identifying when Malware Creates Processes
bp kernel32!CreateProcessW ".printf \"Creating Process: %mu\",poi(esp+0x8);.echo;g"
bp kernel32!CreateProcessA ".printf \"Creating Process: %ma\",poi(esp+0x8);.echo;g"
When the sample is done sleeping, it copies itself to the temp directory as svchost.exe and starts the process again.
Identifying when Malware Executes a Command
bp shell32!shellexecuteW ".printf\"Running Command:\";.echo;\"\tOperation: %mu\";.echo;\"\tTarget: %mu\";.echo;\"\tParams: %mu\",poi(esp+0x8),poi(esp+0xC),poi(esp+0x10);.echo;g"
bp shell32!shellexecuteA ".printf\"Running Command:\";.echo;\"\tOperation: %ma\";.echo;\"\tTarget: %ma\";.echo;\"\tParams: %mu\",poi(esp+0x8),poi(esp+0xC),poi(esp+0x10);.echo;g"
bp shell32!shellExecuteExW ".printf\"Running Command:\";.echo;\"\tOperation: %mu\";.echo;\"\tTarget: %mu\";.echo;\"\tParams: %mu\",poi(poi(esp+0x4)+0xC),poi(poi(esp+0x4)+0x10),poi(poi(esp+0x4)+0x14)
bp shell32!shellExecuteExA ".printf\"Running Command:\";.echo;\"\tOperation: %ma\";.echo;\"\tTarget: %ma\";.echo;\"\tParams: %ma\",poi(poi(esp+0x4)+0xC),poi(poi(esp+0x4)+0x10),poi(poi(esp+0x4)+0x14)
Identifying when Malware Loads Libraries
bp kernel32!LoadLibraryA ".printf \"Loading LibraryA: %ma\",poi(esp+0x4);.echo;g"
bp kernel32!LoadLibraryW ".printf \"Loading LibraryW: %mu\",poi(esp+0x4);.echo;g"
Identifying when Malware is Looking for Functions via GetProcAddress:
bp kernel32!GetProcAddress ".printf \"\t Looking up function: %ma\",poi(esp+0x8);.echo;g"
MEMORY ANALYSIS
There are a variety of things you can do in memory that may be helpful to you as an analyst or reverse engineer. Perhaps you want to patch some instructions in memory or dump an executable. These techniques can help you find information.
Dumping ASCII and Unicode Strings
s -[l#]sa <start_address> <end_address>
This will dump all ascii strings where # is the minimum length of string you would like to search for. If you ommit [l#] then the default is 3 characters.
We can search for unicode instead of ascii by specifying su instead of sa.
Searching for ASCII and Unicode Strings
s -a <start_address> <end_address> <pattern>
There is no doubt you've needed to search for strings in memory before. The biggest caveat here is that patterns are expected to be null terminated. So, if we have a URL that's imbedded in a webpage in memory, chances are we're gonna get the whole webpage. Still, we can look for urls like so:
s -a 0x0 L?0x7FFFFFFF http://
Additionally, we can search for unicode instead of ascii strings by specifying -u instead of -a.
s -u 0x0 L?0x7FFFFFFF http://
However, from the screen shots we can't really make out all of the URL. We can leverage loop logic to iterate through memory and print each URL we find, obtaining the full URL.
.foreach (url {s –[1]a 0x0 L?0x7FFFFFFF http://}){.printf “Found URL: %ma\n”,${url}}
.foreach (url {s –[1]u 0x0 L?0x7FFFFFFF http://}){.printf “Found URL: %mu\n”,${url}}
Other Interesting Searches:
Searching for the keyword RSA
Checking out the contents of 0x571516
Searching for Addresses
s -d <start_address> <end_address> <pattern>
This is useful if you are trying to search the stack for references to a particular address. Here is an example searching for an arbitrary address.
s -d esp L?0x2000 0x2798e8
Dumping Executables From Memory
.writemem <file_name> <address_start> <address_end>
Depending on if the executable is loaded in memory or simply stored in memory, you may be wind up dumping based on virtual or physical size on disk. Regardless of which, you can use !dh -a <address> or walk it with dt as we did earlier to obtain the size of the image so we can calculate where to start and stop dumping memory. Once you've calculated your size, you can use the .writemem command to write to an absolute file path based on the starting and ending address. See this reference for calculating sizes based on your needs.
CONCLUSION
In this blog post, we were able to illustrate techniques for obtaining useful information from the Locky sample with WinDbg. We were able to identify some anti-debugging techniques and bypass it with simple breakpoints. This allowed us to detonate the malware and observe interesting information such as: additional loaded libraries, files created, registry keys changed, and processes started. There are tools and utilities available that can also accomplish these tasks, but this is information already available to you while debugging malware once you know where to look.
For or those of you interested in obtaining a concise list of these breakpoints, check out the team's github here. The repository contains WinDbg script files that are useful for debugging malware or assisting with RE in general. Others are encouraged to also contribute with their knowledge if they choose to do so.
Originally written by Devon Greene