Forem: 0x90

Rust's regex under the hood...

0x90 — Fri, 04 Jul 2025 13:53:02 +0000

Why should you use regex

A regular expression (regex) is a sequence of symbols that identifies a set of strings, numbers, or other characters. They are used for pattern matching and for verifying data

Regular expressions are powerful — and dangerous. In most languages, they can lead to unpredictable performance, surprising behavior, or catastrophic backtracking #d1.

Rust's regex crate is different. It's designed with safety and speed in mind. But how? What makes it so fast? And how should you write your regex patterns to stay in the fast lane?

In this post, we'll dig into how Rust handles regular expressions under the hood, how to write efficient patterns, and when things can go wrong (yes, even in Rust).

Key design choices:

No backtracking engine
Compiled to finite automata (NFA/DFA)
Safe by default: patterns that could cause catastrophic backtracking are impossible

📚 Docs: regex

Regex Compilation and Execution Model in Rust

Rust transforms your regex into a sequence of optimized search structures.

Compilation pipeline:

Parse the pattern into an AST
Convert it into a Thompson NFA (efficient, small-memory execution model)
Optionally compile into a lazy DFA at runtime (fast, but memory-hungry)

The engine dynamically decides which to use depending on the input and pattern.

Lazy DFA:

Built incrementally as needed
Extremely fast matching (no backtracking)
May allocate more memory if input is complex

Theory reference: Regex crate internals – GitHub

Writing Efficient Regex Patterns

Even with a smart engine, writing good regex matters.

Best practices:

Prefer explicit character classes over overly broad wildcards:
- Good: [a-zA-Z0-9_]
- Avoid: .* or .+
Use non-greedy quantifiers when needed (*?, +?)
Avoid ambiguous alternations: foo|foobar is slower than foobar|foo

Things to avoid:

Nested quantifiers: .*.*
Redundant groups: ((abc))
Expecting look-around/backreferences (unsupported)

RegexSet: When You Have Many Patterns

RegexSet allows you to match against many regexes simultaneously.

Instead of testing one-by-one, RegexSet compiles all regexes into a single DFA.

Example use cases:

Firewall filters
Log file scanners
Search engines

Advantages:

Fast matching on large pattern sets
All matches evaluated in a single pass
Avoids allocation and recompilation

Sample

use regex::RegexSet;

fn main(){
    let set = RegexSet::new([
        r"foo",
        r"bar",
        r"baz",
    ]).unwrap();

    let matches = set.matches("bar");

    assert!(matches.matched(1)); // bar is at index 1
}

Regex Overhead: Compile-Time vs Runtime

Calling Regex::new() compiles the regex pattern at runtime — this is fast, but not free.

Costs:

Compilation allocates memory and builds the NFA/DFA
Doing this inside a loop or hot path is a performance killer

Best practices:

Use lazy_static or once_cell to compile once and reuse:

use regex::Regex;
use once_cell::sync::Lazy;

static RE: Lazy<Regex> = Lazy::new(|| {
    Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap()
});

Grouping and Captures in Rust Regex

Rust's regex engine supports capturing groups and non-capturing groups, but not backreferences.

Basic Capturing Groups

Capturing groups let you extract parts of a match.

use regex::Regex;

let re = Regex::new(r"(\d{4})-(\d{2})-(\d{2})").unwrap();
let caps = re.captures("2025-07-04").unwrap();

assert_eq!(&caps[0], "2025-07-04"); // full match
assert_eq!(&caps[1], "2025");       // first group
assert_eq!(&caps[2], "07");         // second group
assert_eq!(&caps[3], "04");         // third group

You can also name your groups:

let re = Regex::new(r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})").unwrap();
let caps = re.captures("2025-07-04").unwrap();

assert_eq!(&caps["year"], "2025");
assert_eq!(&caps["month"], "07");
assert_eq!(&caps["day"], "04");

Docs: Regex::captures

Non-Capturing Groups

Non-capturing groups don't save the match — they’re for grouping logic only:

let re = Regex::new(r"(?:cat|dog)s?").unwrap();
assert!(re.is_match("cats"));

Use (?:...) instead of (...) when you don't need to capture — this can improve performance slightly.

No Backreferences, No Recursion

Rust’s regex engine does not support:

Backreferences (like \1, \k<name>)
Recursion ((?R))
Lookahead/lookbehind assertions

Why?

Because the engine avoids features that require backtracking, to guarantee linear-time execution.

Example (NOT supported):

(.*?)\1     # invalid: backreference not allowed

Using case

In one of my latest projects, I needed to send a request to an apache http server , which returned the HTTP response headers using curl -I.

An example of the returned information:

HTTP/1.1 200 OK

Date: Mon, 21 Apr 2025 09:47:53 GMT

Server: Apache/2.4.37 (AlmaLinux) OpenSSL/1.1.1k mod_auth_gssapi/1.6.1

Last-Modified: Fri, 07 Jan 2022 03:05:20 GMT

My goal was to find the version, such as Apache/2.4.37, so I wrote this regex:

r”(?P<server_str>Server:\s*)(?P<version>[A-Z]?[a-z]*/\d+\.\d+(?:\d+)?)\s(?P<infos>.*)"gm

Let's break it down:

It matches the Server: HTTP header and extracts:

server_str - the literal prefix "Server: "
version - the first server name/version, like Apache/2.4.37
infos - everything else after that (OS, modules, etc.)

Quick part-by-part:

Pattern	Captures
`(?P<server_str>Server:\s*)`	`"Server: "`
`(?P<version>[A-Z]?[a-z]*/\d+\.\d+(?:\.\d+)?)`	`Apache/2.4.37`, `nginx/1.23`, etc.
`\s`	A space separating version/info
`(?P<infos>.*)`	The rest: `(AlmaLinux) OpenSSL...`

So it will return 3 named groups:

server_str: "Server: "
version: "Apache/2.4.37"
infos: "(AlmaLinux) OpenSSL/1.1.1k mod_auth_gssapi/1.6.1"

Practicing

If you want to practice regex101 is really useful, you can see time, performance, test unit make it for various languages and a lot more...

Definitions

d1 Catastrophic backtracking happens when certain regex patterns cause exponential time complexity due to nested quantifiers — Rust avoids this by design.

💡 any particular request?

☕ Was this helpful?

Treat me to a coffee on Ko-fi https://ko-fi.com/riccardoadami

Sled, the Fast and Lightweight Rust Database

0x90 — Fri, 04 Jul 2025 09:44:27 +0000

Reliable storage with minimal footprint, tailored for Rust developers

Sled vs SQL vs Other Rust DB Crates (RocksDB, SeaORM, Rbatis)

Recently, I needed to integrate a database into a Rust application. Exploring the ecosystem, I encountered several options: from traditional SQL-based crates like _Diesel and SQLx, to embedded key-value stores such as RocksDB, and more lightweight solutions like Sled. Each option offers different trade-offs in terms of performance, ease of use, scalability, and flexibility.

After testing and researching, I realized why Sled stands out for embedded, high-performance, and safe data storage in Rust applications. Below is a comparison of Sled against other common Rust database crates to help clarify the differences and why Sled could be the ideal choice depending on your project needs.

Sled guarantees crash-safety by using a lock-free, append-only log structure and supports atomic batch operations, making it ideal for concurrent embedded use

Feature	Sled	SQL-based Crates (Diesel, SQLx)	Other Rust DB Crates (RocksDB, SeaORM, Rbatis)
Performance	High for embedded use; optimized for fast writes and reads	Depends on DB backend (Postgres, SQLite), generally it's good	Varies: RocksDB is very fast for key-value, ORM layers add overhead
Speed	Very fast in-memory and disk-based operations	Moderate to fast, depends on SQL backend and query complexity	RocksDB fast for key-value, ORMs slower due to abstraction
Safety	Memory safe, leverages Rust ownership; crash-safe design	Depends on backend; Rust crates provide compile-time query safety	Varies: RocksDB bindings safe but lower-level; ORMs type-safe
Scalability	Designed for embedded/single-node; not distributed	Scales well with DB backend (e.g., Postgres scales well)	RocksDB scales for local data; ORM scalability depends on backend
Size/Weight	Very lightweight (~1 MB crate size); minimal dependencies	Heavier; depends on features and SQL backend libraries	RocksDB larger due to C++ dependency; ORMs add abstraction layers
Flexibility	Key-value store, no SQL, supports trees and complex data structures	Full SQL support, complex queries, joins	Varies: RocksDB key-value; ORMs offer flexible schema and querying
Ease of Use	Simple API for embedded use, less boilerplate	Steeper learning curve; need to understand SQL and ORM paradigms	Varies: RocksDB lower-level, ORMs easier for relational models
Concurrency	Lock-free design, supports concurrent reads/writes	Depends on backend and crate support	RocksDB supports concurrent access; ORMs depend on DB
Use Case	Embedded apps, local storage, fast persistent KV store	Web apps, complex querying, relational DB needs	Embedded key-value, ORM abstraction for relational DBs
Maturity	Actively developed, modern Rust idioms	Mature ecosystems (Diesel, SQLx widely used)	Varies widely; RocksDB bindings stable, some ORMs newer

Code snippet

Here’s a simple example showing how easy it is to open a DB, and read/write data using Sled.

use sled::Db;

fn main() -> sled::Result<()> {
    let db: Db = sled::open("my_db")?;
    db.insert(b"key", b"value")?;
    let value = db.get(b"key")?;
    println!("Got: {:?}", value);
    Ok(())
}

Instead, a SQL example from the official documentation looks like this:

use sqlx::postgres::PgPoolOptions;
// use sqlx::mysql::MySqlPoolOptions;
// etc.

#[async_std::main] // Requires the `attributes` feature of `async-std`
// or #[tokio::main]
// or #[actix_web::main]
async fn main() -> Result<(), sqlx::Error> {
    // Create a connection pool
    //  for MySQL/MariaDB, use MySqlPoolOptions::new()
    //  for SQLite, use SqlitePoolOptions::new()
    //  etc.
    let pool = PgPoolOptions::new()
        .max_connections(5)
        .connect("postgres://postgres:password@localhost/test").await?;

    // Make a simple query to return the given parameter (use a question mark `?` instead of `$1` for MySQL/MariaDB)
    let row: (i64,) = sqlx::query_as("SELECT $1")
        .bind(150_i64)
        .fetch_one(&pool).await?;

    assert_eq!(row.0, 150);

    Ok(())
}

As you can see, it requires more configuration and async functions.

A direct comparison between sqlx and sled

Feature	SQLx	Sled
Async support	✅	❌
SQL queries	✅	❌
Embedded use	❌	✅
Performance	✅	✅
Type safety	✅	✅
Scalability	✅	❌
Ease of use	❌	✅
Weight	❌ (heavier, async & DB drivers)	✅ (lightweight, minimal dependencies)

When (and When Not) to Use Sled

When to use Sled:

You need a fast, embedded key-value store for local storage in Rust applications.
Your project requires crash-safe, concurrent data access without complex setup.
You want a lightweight, minimal-dependency database that’s easy to integrate.
Your application doesn’t require SQL queries or relational data modeling.
You’re building embedded systems, desktop apps, or small services with local persistence.

When not to use Sled:

You need full SQL support with complex queries, joins, and transactions.
Your application requires a distributed or scalable multi-node database.
You want to leverage existing relational database backends like Postgres or MySQL.
You’re working in a no_std or bare-metal embedded environment.
Your project demands extensive ORM features or schema migrations.

To summarize

Sled is a powerful option when you need reliable, lightweight, and high-performance local storage for Rust applications. It shines in scenarios where simplicity, speed, and safety matter more than SQL features or scalability. If your project doesn't need relational models or complex queries, sled might be the perfect tool for the job.

Link

You can find other info at official GitHub page

💡 Other cool stuff?

☕ Was this helpful?

Treat me to a coffee on Ko-fi https://ko-fi.com/riccardoadami

Speed Up Rust Code Easily with Rayon

0x90 — Thu, 03 Jul 2025 15:20:38 +0000

Maximize Rust’s Speed: Achieve Smooth Parallelism Using Rayon

Why Performance Is Hard

I wanted to speed things up in Rust and let's be honest, threads are one of the best tools to improve performance in Rust.. But using the Tokio crate can be quite unintuitive and difficult to use, all those await and features...

Here's where rayon helps us, allows us to parallelize tasks without having to think about threads. It's simple to use, fast, lightweight, and just works.

A simple definition

Rayon is a library that helps you run code in parallel, making it easy to turn slow, step-by-step computations into faster ones that use multiple CPU cores.

It's a small and easy-to-use tool that lets you add parallelism. It makes sure your code runs safely without data races, and it only uses parallelism when it makes sense, depending on the amount of work at runtime.

For example, we can simply turn this line:

let results: Vec<_> = data.iter().map(|x| x.do_something()).collect();

into:

use rayon::prelude::*;

let results: Vec<_> = data.par_iter().map(|x| x.do_something()).collect();

Using the prelude is the easiest way to do parallelism using rayon in rust.

Let's break down performance

Without rayon

I ran the following code which iterates from 1 to 1,000,000, computes the cube (x.pow(3)) and the square (x.pow(2)) of each number, takes the remainder of both results using modulo 97,531, then sums those two remainders. I ran it using cargo run without any optimization:

- `Finished dev profile [unoptimized + debuginfo] target(s) in 0.86s 
-  `Running target\debug\ry.exe 
- `2, 12, 36, 80, 150, 252, 392, 576, 810, 1100`

These are the CPU specs:

1. CPU Name: Intel(R) microarchitecture code named Alderlake-S
2. Frequency: 2.5 GHz
3. Logical CPU Count: 12

fn main() {
    let data: Vec<u64> = (1..1_000_000).collect();
    let results: Vec<u64> = data.iter()
        .map(|x| x.pow(3) % 97_531 + x.pow(2) % 97_531)
        .collect();
    println!("{:?}", &results[..10]);
}

I measured performance using Intel Vtune profiler and we can see that without using rayon it needs 0.041s using 1 single thread

and the function which needs more time is the main, because we iterate, calculate and collect the result

With rayon

The computation level is the same as before, but this time we use rayon:

use rayon::prelude::*;

fn main() {
    let data: Vec<u64> = (1..1_000_000).collect();
    let results: Vec<u64> = data.par_iter()
        .map(|x| x.pow(3) % 97_531 + x.pow(2) % 97_531)
        .collect();
    println!("{:?}", &results[..10]);
}

adding rayon = "1.10.0" to your Cargo.toml dependencies

I compiled without optimizations:

- Finished dev profile [unoptimized + debuginfo] target(s) in 0.02s
    - Running target\debug\ry.exe`
- [2, 12, 36, 80, 150, 252, 392, 576, 810, 1100]`

Already now we can see that the program ran in 0.02 seconds, compared to 0.86 seconds without Rayon, but let's see in detail:

First, we can see it uses 8 threads instead of just one
We see that it took 0.029 seconds instead of 0.041s
CPU status is constantly in idle mode instead of poor as before

As before all the effective time is used by one single function which is the last called

When (and When Not) to Use Rayon

The ideal use cases are CPU-bound work, large datasets, pure functions, sorting, etc.

Instead for small workloads, shared mutable state or I/O-heavy tasks it's better to use the Tokio runtime if you really need it. The Tokio module supports fs, time, command execution, net and a lot more using multithreading, but that's another topic I'll write about...

Other stuff Rayon does

Beyond .map and .par_iter Rayon also includes .filter(), .reduce(), .for_each(), join() for parallel sorting

To sum up

Rayon isn't always the best choice. Still, it's a smart and safe way to add parallelism. It helps you scale workloads with minimal code changes, making it a solid choice for performance-critical applications.

💡 Got another crate in mind?

☕ Was this helpful?

Treat me to a coffee on Ko-fi https://ko-fi.com/riccardoadami

Zeroize: The Tiny and Memory-Safe Rust Crate

0x90 — Thu, 03 Jul 2025 12:44:01 +0000

Sensitive Data in Memory: A Hidden Threat

In a secure environment, one of the most overlooked threats is the presence of sensitive data in memory, such as passwords, tokens, cryptographic keys, or card numbers.

Even when using Rust, where we emphasize ownership and thread safety, there's another crucial question to consider:

What remains in the heap or stack after we no longer need it?

Why Memory Zeroing Matters

Zeroize helps us address this. It’s a lightweight but powerful crate that allows us to zero out memory securely and efficiently.

Imagine this scenario:

Your program reads a password from user input
It uses it for authentication or to decrypt a file
The variable goes out of scope and Rust "cleans it up"... right?

Not quite. Rust does not guarantee that memory will be overwritten.

So your password might remain in memory, in plain text, until it's overwritten by something else. In the case of a memory dump or cold boot attack, it could be exposed.

What Can You Do About It?

That’s where Zeroize comes in. It lets you explicitly clear memory when you're done using sensitive data.

Instead of relying on the runtime or assuming safety, you take control.

It provides a trait Zeroize which can be derive or implemented manually for your types.

Using it is literally one line:

use zeroize::Zeroize;
fn main() {
    let mut password = String::from("hunter2");
    // Use it for authentication or anything else

     // now clean memory
    password.zeroize();
}

The last line will overwrite the contents of the string with zero. If you use other types like Vec<u8> it will work in the same way

Automate the process

If you want to automate everything, you can wrap your data with Zeroizing, which will automatically reset it when it goes out of scope:

use zeroize::Zeroizing;

fn main() {
    let password = Zeroizing::new(String::from("hunter2"));
    // No need to call zeroize, auto-zero on drop
}

Easy, elegant, secure. It adds almost no overhead to your final binary

When should you use it?

Every time you handle:

Password
Auth token (eg. JWT, OAuth)
Encryption keys (AES, ChaCha20, etc)
Any data that shouldn't live in RAM longer than necessary

In synthesis

Zeroize is not magic.But it's a good habit. It won't protect you from every attack, but it does make your code more resilient, especially in environments where security ≠ optional (embedded, mobile, backend auth, etc.)

💡 Do you know any other crate that you'd like to explore? Let me know

Originally published on my Hashnode blog

☕ Was this helpful?

Treat me to a coffee on Ko-fi https://ko-fi.com/riccardoadami