Demystifying Deref Coercion in Rust

Melbourne Baldove — Wed, 13 Dec 2023 11:28:04 +0000

Rust revolutionizes programming by distilling decades of C/C++ memory safety know-how into a core language feature with sane defaults: ownership. It’s a language that not only is blazingly fast but also manages to be a joy to write code in. Despite the generally excellent developer experience, certain language features might seem like ‘magic’ to newcomers. A prime example of this is deref coercion.

Simply put, deref coercion enables ergonomic use of smart pointers (like Box<T>, Rc<T>, etc.)

Smart pointers

Smart pointers play a key role in Rust’s ownership system, offering more functionality than traditional pointers. Unlike pointers in languages like C++, which are primarily used to reference memory locations, Rust’s smart pointers, such as Box<T>, Rc<T>, and RefCell<T>, manage the lifecycle of the data they hold. This management includes features like automatic memory deallocation and reference counting, ensuring memory safety without sacrificing efficiency.

Let’s consider the simplest smart pointer: Box<T>. Box<T> enables you to store the underlying value on the heap so you can move values efficiently to prevent deep copies. Moving a Box<T> means only the pointer is copied around and not the potentially large underlying value. Let’s examine how Rust allows us to dereference Box<T> like a regular pointer.

fn main() {
    // Value is in the stack
    let x = 10;
    // Regular pointer to x
    let x_ptr = &x;
    // Dereferencing for the sake of example
    println!("Value using regular pointer: {}", *x_ptr); 

    // Smart pointer (Box) to an integer on the heap
    let x = Box::new(10);
    // Dereference to underlying value happens 
    // automatically
    println!("Value using smart pointer: {}", *x); 
}

Although Box<T> itself is a lightweight value containing metadata about its heap-stored data, it mimics a regular pointer in that it can be dereferenced with the dereference operator (*) to access the value it holds. This is why Box<T> and friends are called smart pointers.

But what allows a type such as Box<T> to be dereferenced?

Custom smart pointer

Let’s try to create our own Box<T> -like smart pointer. For simplicity, the underlying value will be stored on the stack instead of the heap.

struct MyBox<T>(T);

impl<T> MyBox<T> {
    fn new(x: T) -> MyBox<T> {
      MyBox(x)
    }
}

Here we have a tuple struct MyBox<T> with a constructor. All is well. Let’s naively try to use it like in our previous example.

fn main() {
    let x = MyBox::new(10);
    println!("Value using our smart pointer: {}", *x); // compiler error
}

Upon compiling, we get a type MyBox<{integer}> cannot be dereferenced compiler error. Oof. We knew this would happen. So how do we allow MyBox<T> to be dereferenced?

Deref trait

The Deref trait is provided by the standard library. Implementing this trait on a type will allow that type to be dereferenced. Lets implement it for MyBox<T>.

use std::ops::Deref;

impl<T> Deref for MyBox<T> {
    type Target = T;

    fn deref(&self) -> &Self::Target {
        &self.0
    }
}

A bunch of stuff here. The type Target = T is a language feature called associated types that complement generics and make it more ergonomic for trait definitions. Simply put, we use it here so the signature of deref no longer needs to be generic on T .

The signature of deref says that it takes in an immutable reference to the receiver (&self which would be MyBox<T>) and returns a reference to the underlying value of type T (i32 from our earlier example). Recall that MyBox<T> is a tuple struct. Recompiling our example will now work.

fn main() {
    let x = MyBox::new(10);
    println!("Value using our smart pointer: {}", *x); // prints correctly
}

What actually happens underneath? The Rust book says:

“The deref method gives the compiler the ability to take a value of any type that implements Deref and call the deref method to get an & reference that it knows how to dereference.”

So when we dereferenced x , behind the scenes Rust ran:

*(x.deref())

The initial dereference operator gets substituted with a call to deref() to return a reference that Rust now knows how to dereference finally with (*). Pretty neat.

Implicit deref coercion

Let’s now examine how deref coercion allows us to save some thought and prevent hairy pointer syntax.

fn main() {
    let x = Box::new(String::from("some string literal"));
}

x is a Box<String<&str>> that simply stores a string literal on the heap. Suppose we have a function that takes a string literal and does whatever with it.

fn takes_str(str: &str) {
    // does whatever
}

How would we get a &str from a Box<String<&str>> ? The following are all equivalent.

fn main() {
    let x = Box::new(String::from("some string literal"));

    // Box -> String -> str -> &str
    take_str(&(*(*x)));

    // Box -> String then takes a string slice to get &str
    take_str(&(*x)[..]);

    // Calling deref on Box then on String
    take_str(x.deref().deref());

    // Deref coercion which is equivalent to the above
    take_str(&x);
}

Note that String also implements Deref. So we are able to call deref() on it. But we are interested on why take_str(&x) works.

When the Deref trait is defined for the types involved, Rust will analyze the types and use Deref::deref as many times as necessary to get a reference to match the parameter’s type.

Neat! That means we can also do the following:

fn take_string(string: &String) {
    // Do whatever
}

fn take_str(str: &str) {
    // Do whatever
}

fn main() {
    let x = Box::new(String::from("some string literal"));

    // Rust will resolve to deref only once to match signature
    take_string(&x); 

    // Rust will resolve to two calls of deref to match signature
    take_str(&x); 
}

And all of these conversions resolve at compile time so deref coercion doesn’t incur a runtime penalty!

Deref coercion removes the need to add explicit references and dereferences with & and * which, as you’ve seen in the above example, can quickly get out of hand. It was added to Rust to make our lives easier when working with references and smart pointers.

Summary

Deref coercion is syntactic sugar to make working with references and smart pointers easier.
You allow a type to be dereferenced when you implement the Deref trait on it.
Rust calls Deref::deref as many times as necessary to match the needed type.
Deref coercion happens at compile-time so it doesn’t incur any runtime penalty.