DLLs in Rust🦀

CatterPingu — Fri, 20 Oct 2023 16:46:19 +0000

Dynamic Link Libraries or the DLLs are components which are called by a process through the loader when needed during runtime.

DLLs are specially useful so that you're not writing a piece of code again and again and again...

We'll look at writing a DLL in rust just because writing rust is fun. This should be useful whether you're someone starting on the journey of wielding the power of DLLs or looking for some exposure to the Win32 Api in rust.

Executable for loading the DLL

Create a binary rust application package using the cargo package manager. cargo new LoadDll

To load a dynamic library, we'll need the LoadLibrary method from the Win32 Api.
After the DLL has been loaded, we'll need to pass it to the GetProcAddess with the name of the function to be loaded which in our case is a function named 'main'(This function will be present int the DLL).

fn main() {
    if let Ok(hellodll)=unsafe{LoadLibraryA(PCSTR("hellodll.dll\0".as_ptr()))}{
        let dllmain=unsafe{GetProcAddress(hellodll
                                                    , PCSTR("main\0".as_ptr()))};

        if let Some(dllmain)=dllmain{
            unsafe{dllmain()};
        }else{
            println!("NO SUCH FUNC IN DLL\0");
        }
    }else {
        println!("DLL not found\0");
    }




}

Remember that all the methods from the Win32 need to be wrapped in an unsafe block which implies that the rust compiler can't guarantee the memory safety of the functions.

Imports in the cargo.toml file:

[dependencies.windows]
version = "0.51.1"
features = ["Win32_System_LibraryLoader", "Win32_Foundation"]

Now, build the exe using the cargo package manager:
cargo build --release
If everything has been built correctly, on running the executable we should get:Dll not found error message because we have not yet created a DLL!

To be sure that our exe is actually looking for the DLL, fire up the SysInternals' Process Monitor.
To view just the information related to whether our exe is searching for the DLL, we'll set 3 filters. Process Name, Result and Path as:

Now clicking on apply and starting the "monitoring", run loaddll.exe and we should be getting something like:

This gives us the order of the subdirectories the executable tried to search for the DLL and found it at neither. But now we're sure that our exe is looking for a dll named hellodll.

DLL for loaddll.exe

First we create a rust library using: cargo new hellodll --lib. This creates a lib.rs in our rust project without a main.
Then write the necessary information in the cargo.toml file.
We have to specify in the toml file that we're trying to write a DLL.

[lib]
crate-type = ["cdylib"]

This specifies that this is a C Dynamic Library.

Next, we'll make the necessary imports from the windows crate.

[dependencies.windows]
version = "0.51.1"
features = [
    "Win32_Foundation"
    ,"Win32_System_Services"
    ,"Win32_System_Console"
    ,"Win32_Security"
    ,"Win32_System_Memory"
    ,"Win32_System_Threading"
    ,"Win32_System_WindowsProgramming"
    ,"Win32_System_Diagnostics_Debug"
    ,"Win32_UI_WindowsAndMessaging"
]

We'll open a simple MessageBox from the function which is being imported by the loaddll.exe and call it "main"(remember that this can be called anything). In order to have this function called from the executable, it also needs to be an extern "C" function.

extern "C" fn main(){
    unsafe{
        MessageBoxA(
            HWND(0)
            , PCSTR("Hello from DLL\0".as_ptr())
            , PCSTR("HI\0".as_ptr())
            , MESSAGEBOX_STYLE(0))
    };


}

Next, to load a dll into the memory at runtime, it has to have a DllMain function which is like the entry-point to a DLL.

extern "system" fn DllMain(
    dll_module:HANDLE
    ,call_reason:u32
    ,lpv_reserved:&u32
)->BOOL{
    match call_reason{
        _=>{
            return BOOL(1);
        }
    }
}

Since we aren't looking to do something special right now when the DLL is loaded, we'll put a place holder _=> in place of attach and detach.
For Reference: MSDN DllMain Documentation

Now, we also need to specify that the rust compiler should not mess up the function names themselves. This can be done by the #[no_mangle] attribute above both the "main" and the "DllMain".
Now we're all done and we'll cargo build --release to build the DLL inside the release folder of our project directory.
Lastly, to be sure that our DLL is exporting the functions according to our requirements, we can inspect the DLL using the NT Core's CFF Explorer Suite.

Sure enough, out hellodll exports two fucntions main and DllMain.

Finally, place both the executable and the dll in the same directory and on running the the exe, we get a message box indicating the expected behaviour of our DLL called from the exe.

I hope you've found this enjoyable and useful, and I'm look forward for the amazing things you'll build if you decide to delve deeper into Rust and Windows development.

Forem: CatterPingu

DLLs in Rust🦀

Executable for loading the DLL

DLL for loaddll.exe

Reference: