Forem: Douglas Makey Mendez Molero

MU Online in Rust: Hardening the Foundation, runtime, Timeouts, and Packet Size Limits

Douglas Makey Mendez Molero — Thu, 12 Mar 2026 16:41:59 +0000

We are not implementing login yet, there is cleanup worth doing first. A few things were intentionally left behind in the previous article, and they are worth addressing before we go deeper.

Before implementing login, handling encrypted packets, and authenticating real players, we must ensure the foundation can withstand basic abuse. This means enforcing timeouts, limiting packet sizes, and laying the groundwork for future enhancements. Clean up first, then harden.

Currently, both servers duplicate the same TCP listener setup. Although the function is small and a bit of code duplication is tolerable, extracting this logic into a dedicated crate provides a single, clean place to apply these protections.

Introducing mu-runtime

We are going to introduce a new crate called mu-runtime. The idea is simple: one generic server loop, any handler you plug in. Taking advantage of Rust's generics, we can implement a Server struct that accepts a generic handler for client connections:

pub type PacketStream = Framed<TcpStream, PacketCodec>;
pub struct Server {...}

impl Server {
    pub async fn run_tcp_listener<H, F>(&self, handler: H) -> Result<()>
    where
        H: Fn(PacketStream, SocketAddr) -> F + Send + 'static,
        F: Future<Output = Result<()>> + Send + 'static,
    {
        let listener = TcpListener::bind(self.bind_addr).await.with_context(|| {
            format!("failed to bind to {bind_addr}", bind_addr = self.bind_addr)
        })?;

        let mut shutdown = Box::pin(tokio::signal::ctrl_c());
        loop {
            tokio::select! {
                _ = &mut shutdown => {
                    break;
                }
                accepted = listener.accept() => {
                    let (socket, peer_addr) = match accepted {
                        Ok(conn) => conn,
                        Err(e) => {...}
                    };

                    let stream = Framed::new(socket, PacketCodec);
                    let h = handler(stream, peer_addr);
                    tokio::spawn(async move {
                        if let Err(e) = h.await {...}
                    });
                }
            }
        }

        Ok(())
    }
}

If you compare this to the previous implementation on both servers, it's nearly identical. The only real difference is that instead of calling a hardcoded handle_client function, we now receive a generic handler. The magic is in the where clause.

Method signature: run_tcp_listener accepts a handler closure H that gets called for each inbound connection. The two generic parameters work together:

H: Fn(PacketStream, SocketAddr) -> F: The handler receives a framed stream and the client's address, and returns a future. Fn (not FnOnce) means the same handler is reused for every connection.
F: Future<Output = Result<()>>: The returned future resolves to Result<()>, meaning each connection handler is async and can fail independently.

Trait bounds explained:

Send + 'static on H: the handler can be shared across threads and doesn't hold any temporary references that might become invalid. Required because we spawn a tokio::spawn per connection.
Send + 'static on F: the future returned by the handler can be sent to a Tokio task (again, tokio::spawn requirement).

The reason we use Fn and not FnOnce is that FnOnce would consume the handler on the first connection, leaving nothing for the second accept(). The compiler would catch this, but the intent matters: we need a handler we can call repeatedly.

Now both GameServer and ConnectServer can use this Server struct to start listening:

#[tokio::main]
async fn main() -> Result<()> {
    tracing_subscriber::fmt::init();
    let server = Server::new("GameServer".to_string(), "0.0.0.0:55901".parse().unwrap());

    server
        .run_tcp_listener(
            move |stream, peer_addr| async move { handle_client(stream, peer_addr).await },
        )
        .await
}

This simplifies both servers considerably and gives us a single, central place to evolve the TCP layer. The workspace crate structure now looks like this:

 ┌──────────────────────┐   ┌──────────────────────┐
 │    connect-server    │   │     game-server      │
 │  (handlers, config)  │   │  (handlers, config)  │
 └──────────┬───────────┘   └──────────┬───────────┘
            │                          │
            └─────────────┬────────────┘
                          │
            ┌─────────────▼────────────┐
            │        mu-runtime        │
            │  Server, PacketStream,   │
            │         timeouts         │
            └─────────────┬────────────┘
                          │
            ┌─────────────▼────────────┐
            │        mu-protocol       │
            │  Packet, PacketCodec,    │
            │  size limits, errors     │
            └──────────────────────────┘

Both servers depend on mu-runtime for the TCP loop, and mu-runtime depends on mu-protocol for framing. The servers themselves only contain what is unique to them: their handlers and config. Now that we have a shared runtime, this is exactly the right place to enforce the protections we deliberately skipped in the first article.

Timeouts: The Idle Client Problem

In the previous article, I mentioned some security measures that we intentionally skipped for simplicity. Let's start fixing that.

We use plain TCP as our transport. TCP provides reliability at the byte-stream level (retransmission, ordering, flow control), but it gives us no protection at the application level. Without additional safeguards, a single misbehaving or malicious client can tie up server resources indefinitely.

The first line of defense is timeouts. We need two:

Read timeout: limits how long we wait for a client to send data. Without this, a client that connects but never sends anything (intentionally or due to a network issue) holds a connection and a spawned task open forever.

Read Timeout — Client sends partial data then goes silent
═══════════════════════════════════════════════════════════

  Client                          Server
    │                               │
    │──── [C1 04 00 01] ───────────►│  Valid packet header
    │                               │  Server expects remaining bytes...
    │                               │
    │          (silence)            │  ⏱ 1s...
    │                               │  ⏱ 2s...
    │                               │  ⏱ 3s...
    │                               │  ⏱ READ_TIMEOUT reached
    │                               │
    │       ◄──── [connection closed]
    │                               │
    Without timeout: server task blocks here forever,
    holding memory and a file descriptor hostage.

Write timeout: limits how long we wait for a client to drain data we're sending. A client that stops reading (or reads extremely slowly) causes backpressure that can stall our write side indefinitely.

Write Timeout — Client connects but never reads responses
═══════════════════════════════════════════════════════════

  Client                          Server
    │                               │
    │──── [packet request] ────────►│
    │                               │──── [packet response] ────► TCP buffer
    │                               │
    │    (client stops reading)     │  Server sends next packet...
    │                               │──── [data packet] ──────► TCP buffer
    │                               │                           TCP buffer full!
    │                               │
    │                               │  .send() blocks waiting
    │                               │  for client to drain buffer
    │                               │  ⏱ 1s...
    │                               │  ⏱ 2s...
    │                               │  ⏱ WRITE_TIMEOUT reached
    │                               │
    │       ◄──── [connection closed]
    │                               │
    Without timeout: server task stalls on .send(),
    backpressure from one client blocks its entire task.

Without both of these, our server is vulnerable to resource exhaustion. Even a handful of bad clients could starve legitimate connections.

To put it simply:

Read timeout: for a client that won't talk.
Write timeout: for a client that won't listen.

These two timeouts protect against well-known attack patterns. The read timeout defends against Slowloris, an attack where a client opens many connections and keeps them alive by sending data just slowly enough to prevent the server from closing them, eventually exhausting the connection pool. The write timeout defends against backpressure exhaustion, where a client deliberately stops reading, causing TCP send buffers to fill up and server tasks to stall indefinitely waiting on a .send() that can never complete.

Implementing Timeouts in PacketStream

To add timeouts, we are going to evolve our PacketStream. Instead of a plain type alias for Framed, we are going to make it a proper wrapper with recv and send methods that layer tokio::time::timeout on top:

pub struct PacketStream {
    inner: Framed<TcpStream, PacketCodec>,
    read_timeout: Duration,
    write_timeout: Duration,
}

impl PacketStream {
    pub async fn recv(&mut self) -> Option<Result<RawPacket, ConnectionError>> {
        match tokio::time::timeout(self.read_timeout, self.inner.next()).await {
            Ok(result) => result.map(|r| r.map_err(ConnectionError::from)),
            Err(_) => Some(Err(ConnectionError::ReadTimeout)),
        }
    }

    pub async fn send(&mut self, packet: RawPacket) -> Result<(), ConnectionError> {
        tokio::time::timeout(self.write_timeout, self.inner.send(packet))
            .await
            .map_err(|_| ConnectionError::WriteTimeout)?
            .map_err(ConnectionError::from)
    }
}

And in run_tcp_listener, we replace the old stream with the new wrapper:

// Old
let stream = Framed::new(socket, PacketCodec);

// New
let stream = PacketStream::new(
    socket,
    self.config.read_timeout,
    self.config.write_timeout,
);

A small change with a real impact.

Seeing It in Action

The best part about these protections is how trivially easy it is to trigger the attack and then watch the server shut it down. No special tools needed, just nc.

Simulating a read-timeout attack (Slowloris):

Open a terminal and connect to the ConnectServer without sending anything:

$ nc 127.0.0.1 44405
# just sit here and do nothing

That's it. Before our fix, that connection would hold a Tokio task and a file descriptor open indefinitely. With the read timeout in place, the server cuts it off after 30 seconds:

RUST_LOG=info cargo run -p connect-server
2026-02-20T19:57:18.632733Z  INFO mu_runtime: ConnectServer listening bind_addr=0.0.0.0:44405
2026-02-20T19:57:19.536570Z  INFO mu_runtime: client connected peer=127.0.0.1:55927
2026-02-20T19:57:19.536892Z DEBUG connect_server: sent hello packet peer_addr=127.0.0.1:55927
2026-02-20T19:57:49.538663Z  WARN connect_server: Packet read error error=read timed out
2026-02-20T19:57:49.538787Z  INFO connect_server: connect-server client disconnected peer=127.0.0.1:55927

The nc session gets terminated on the client side too. The connection just drops. One idle connection, handled cleanly, no leaked resources.

Simulating a write-timeout attack (backpressure exhaustion):

Honestly, the write timeout is impossible to demonstrate with the ConnectServer as it currently stands, and it's worth understanding why. For a write to block, the client's TCP receive buffer must be completely full while the server is actively trying to send. The ConnectServer's packets are tiny, the Hello is 4 bytes, a ServerListResponse for two servers is ~15 bytes. The OS enforces a minimum receive buffer of around 4KB regardless of what you set on the socket. Those tiny packets will never come close to filling it, so .send() always completes instantly and the write timeout never gets a chance to fire.

The protection becomes real once the GameServer starts streaming larger payloads such as: position updates, inventory data, map state. A client that connects, triggers a large response, and then stops reading will fill that buffer quickly. At that point, .send() blocks, and without a write timeout the server task stalls indefinitely.

So for now, trust the code more than the demo: the mechanism is correct, the ConnectServer just doesn't produce enough data to make the failure mode visible yet. We'll see it in action when the GameServer gets more interesting.

Max Packet Size: Starving the Allocator

Another vector worth closing is unbounded memory allocation. A malicious client can send a crafted header claiming an excessively large packet size, forcing the server to allocate memory it will never use before any parsing occurs. Because this happens at the framing layer, that’s also the ideal point to block it.

The ConnectServer, for example, only ever receives C1 packets meaning a maximum of 255 bytes. It has no business accepting a C2/C4 packet, which can be up to 65,535 bytes. If we blindly allocate memory for whatever length the client claims, a malicious client can send a crafted header claiming a massive packet size and force us to allocate memory we'll never actually use. That's a memory allocation DoS vector, and it happens before any parsing even occurs.

The GameServer legitimately receives C2/C4 packets, so it needs a higher ceiling. But we don't know yet if a real game packet comes close to the 65KB protocol maximum. We can set a tighter limit something like 12–24KB and adjust as real packets are implemented.

The right place to enforce this is the codec, which is already responsible for framing the byte stream. The change is minimal:

pub struct PacketCodec {
    max_packet_size: usize,
}

fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
    ...
    if declared_len > self.max_packet_size {
        return Err(ProtocolError::PacketTooLarge {
            max: self.max_packet_size,
            actual: declared_len,
        });
    }
    ...
}


// Each server enforces its own limit:
let server = Server::new(ServerConfig {
    name: "ConnectServer".to_string(),
    bind_addr: "0.0.0.0:44405".parse().unwrap(),
    read_timeout: Duration::from_secs(15),
    write_timeout: Duration::from_secs(15),
    max_packet_size: SMALL_PACKET_MAX_SIZE, // C1 only no need for larger packets
});

We reject oversized packets at the codec level, before we touch the payload. No parsing, no allocation beyond the declared limit.

Seeing It in Action

This one is easy to trigger with just nc and printf. A C2 packet header is only 3 bytes: the type byte plus the 2-byte length field. That is all the codec needs to make its decision. We never have to send the actual payload.

# C2 packet claiming to be 1000 bytes:
#   \xC2        — C2 type (large packet, 2-byte length)
#   \x03\xE8    — length 1000 in big-endian
#   \xF4\x06    — code/sub-code (doesn't matter, codec rejects before parsing)
$ printf '\xC2\x03\xE8\xF4\x06' | nc 127.0.0.1 44405

The ConnectServer is configured with SMALL_PACKET_MAX_SIZE = 255. As soon as the codec reads the 3-byte prefix and sees declared_len = 1000 > 255, it returns PacketTooLarge. No allocation, no parsing, connection closed:

2026-02-20T20:18:42.764238Z  INFO mu_runtime: ConnectServer listening bind_addr=0.0.0.0:44405
2026-02-20T20:18:45.055888Z  INFO mu_runtime: client connected peer=127.0.0.1:56865
2026-02-20T20:18:45.056244Z DEBUG connect_server: sent hello packet peer_addr=127.0.0.1:56865
2026-02-20T20:18:45.056295Z  WARN connect_server: Packet read error error=protocol error: packet too large: max 255 bytes, got 1000
2026-02-20T20:18:45.056312Z  INFO connect_server: connect-server client disconnected peer=127.0.0.1:56865

The attack is dead before it even starts. The same packet sent to the GameServer which has a higher limit would pass through, because it legitimately handles large packets. Each server enforces the right limit for its own protocol role.

Where We Are Now

With these changes, our server is meaningfully more robust. To summarize what we added:

mu-runtime: A centralized, generic TCP listener that both servers share.
Read timeout: Disconnects clients that connect but never send data (Slowloris protection).
Write timeout: Disconnects clients that connect but never read responses (backpressure exhaustion protection).
Max packet size: Rejects oversized packets at the codec level before any allocation happens (memory DoS protection).

There are more mechanisms we could add such as: rate limiting, max concurrent connections, per-IP throttling, but we will revisit those when the need arises. For now, the foundation is solid enough to move forward.

In the next article, I promise you that we will finally get to the login flow. That means dealing with XOR32-encrypted client packets and implementing our first real game logic.

All the code can be found in the repository. Feel free to explore, experiment, and leave comments! Happy coding!

From Nostalgia to Code - Building a MU Online Server in Rust

Douglas Makey Mendez Molero — Fri, 06 Mar 2026 17:33:41 +0000

From Nostalgia to Code: Building a MU Online Server in Rust

Somewhere between 2005 and 2009, my younger self spent almost all the time in front of a computer playing MU Online, one of the best MMORPGs in history if you ask me. Cybercafes with friends, deploying private servers from guides I barely understood, staying up way too late. Even many years later, every now and then the nostalgia kicks in and I end up playing again with friends to relive the good old times.

Last year, out of pure curiosity, I was looking on GitHub for MU Online projects and found a very interesting repository: OpenMU. These guys created a customizable server for MU Online using C# from scratch. That is amazing, hats off to them. So I thought, why not use this repo as a reference to write a server in Rust? My intention isn't to fully rewrite their project. I just want to learn how one of the best games of my childhood works under the hood and have some fun in the process.

I started by cloning their project and analyzing the repo to understand how everything works, then trying to replicate minimal features. How much will I replicate? I don't know, we are going to discover that together I guess.

After an initial inspection, the project has a docs folder with very useful information that the team published describing the project, architecture, intentions, and other valuable details for what we are going to do.

The repo has multiple references to the version/protocol of the current implementation being the ENG (English version) of Season 6 Episode 3. Being honest, I don't remember too much about these versions and don't fully understand at this point how they changed. My last version that I played was 0.99 (the best version if you ask me, but biased haha). For now, we are going to focus on the latest one compatible with the Season 6 client.

One of the key docs is the architecture overview, which gives us a brief idea about how this implementation works. Again, we are not going to implement everything, but it's nice to have an idea about where to start.

The other key resource I found is the folder with the packet descriptions, which documents the messages exchanged between client and server. With this, we can get a solid idea of the protocol that MU Online implements. We are definitely going to need and spend time here to implement some minimal features.

In this context, a packet is a structured sequence of bytes that represents a single message in the MU Online protocol. Each packet has a header that identifies what kind of message it is, how long it is, and whether it's encrypted, followed by an optional payload with the actual data. Think of it as an envelope: the header is the addressing and stamp, and the payload is the letter inside.

They already documented the list of which packets are being used in the communication between server and client and vice versa, so pretty nice to have.

One of the key discoveries here is that the protocol utilizes the first byte as a Frame Identifier. As described in the PacketTypes.md documentation, this byte defines the binary contract between the client and server, specifically governing the framing logic (how to calculate packet length) and the cryptographic state (whether XOR32 or Simple Modulus is applied).

Framing is the process of determining where one message ends and the next one begins in a stream of bytes. Since TCP delivers data as a continuous byte stream with no built-in message boundaries, the protocol needs a way to delimit individual packets, that's what the frame identifier and the length fields do.

First byte	Length of the packet	Encrypted Server -> Client	Encrypted Client -> Server
0xC1	Specified by the second byte	No	Yes (XOR32)
0xC2	Specified by the second and third byte	No	Yes (XOR32)
0xC3	Specified by the second byte	Yes (Simple modulus)	Yes (Simple modulus + XOR32)
0xC4	Specified by the second and third byte	Yes (Simple modulus)	Yes (Simple modulus + XOR32)

So basically, the first byte is the frame identifier as we mentioned. It controls two things: how you read the length, and whether the payload is encrypted. The first byte tells the packet parser how to read the rest of the data.

C1 vs C2 is about size capacity. C1 stores the packet length in a single byte (byte index 1), so packets can be at most 255 bytes. C2 uses two bytes for length (indices 1-2), supporting up to 65,535 bytes.

C3 vs C4 mirror C1/C2 in their length encoding, but add Simple Modulus encryption on top. I guess they're used for sensitive data, anything worth protecting from packet sniffing.

Encryption is Asymmetric by Direction

This is a critical detail as we can see in the table:

Server -> Client: C1/C2 are plaintext. C3/C4 use Simple Modulus.
Client -> Server: Everything gets XOR32 encrypted, and C3/C4 additionally get Simple Modulus + XOR32.

So the client always encrypts outbound traffic (at minimum XOR32), while the server only encrypts when using C3/C4. This makes sense, sniffing what the server sends is less dangerous than injecting forged client packets.

What is XOR32

XOR is a bitwise operation: compare two bits, output 1 if they differ, 0 if they match.

1 XOR 1 = 0
1 XOR 0 = 1
0 XOR 1 = 1
0 XOR 0 = 0

XOR32 means we have a 32-byte key (a fixed sequence of 32 bytes, I originally thought it was 32 bits but after reading the implementation I figured out it's the key length). To encrypt, we XOR each byte of our data with the corresponding byte of the key, cycling back to the start of the key every 32 bytes.

So it will be something like:

Data:    [A] [B] [C] [D] ...
Key:     [K0][K1][K2][K3] ...  (repeats every 32 bytes)
Result:  [A^K0] [B^K1] [C^K2] [D^K3] ...

The beauty of XOR is that applying it twice reverses it. Encrypt and decrypt are the same operation with the same key. That's why it's fast and simple. The weakness: if someone knows (or brute forces) the key, it's trivially broken. It's not real security, it's simple obfuscation.

We won't implement XOR32 in this article since the client sends the only payload packet unencrypted for this scope, but we'll tackle it in the next one when we handle login. If you're curious, you can play with a basic XOR32 implementation in the Rust playground.

Simple Modulus (used by C3/C4 packets) is a custom block-based encryption algorithm that deserves its own article, so we'll skip it for now since our initial handshake scope doesn't require it. If you're curious, the OpenMU creator has two great posts about it: A closer look at the MU Online packet encryption and SimpleModulus revisited.

C1 00 01 - The Initial Hello!

Thanks to the great docs and the implementation that the OpenMU team provides, we can understand the initial flow of the communication between the client and the servers: the iconic login screen.

The Hello by server packet describes this initial interaction. The packet is sent by the server after the client connects, and the client will request the server list as soon as it receives the hello packet.

Inspecting the packet:

Index	Length	Data Type	Value	Description
0	1	Byte	0xC1	Packet type
1	1	Byte	4	Packet header - length of the packet
2	1	Byte	0x00	Packet header - packet type identifier
3	1	Byte	0x01	Packet header - sub packet type identifier

Dissecting it:

0xC1 -> Small packet, no encryption server->client, length in byte 1
0x04 -> Total packet length is 4 bytes (header only, no payload)
0x00 -> Main packet type identifier
0x01 -> Sub-type

After inspecting the packet definition headers and the implementation, we can infer that the first byte indicates whether the packet is small (C1/C3) or large (C2/C4). Next comes the one or two byte length field, followed by a byte that serves as a packet or group identifier (for example, the connection or handshake family). And finally, some groups include a subtype byte, such as the hello packet, while others do not. So the Header Size varies (3, 4, or 5 bytes) depending on the first byte and the type/sub-type.

The two-level identification scheme (byte 2 or 3 = group/code, byte 3 or 4 = sub type/code) gives us a clean routing architecture. 0x00 is the connection group, and within it 0x01 means "hello." Think of it like namespace::method, but as we mentioned, some packets just have a type and don't have a sub-type.

As the docs say, after the hello packet is sent to the client, the client will send the ServerListRequest (C1 F4 06) packet to the server. This packet is similar to the hello packet in definition, but the server will respond with the ServerListResponse (C2 F4 06) packet, and this one is worth examining to understand the dynamic headers we discussed earlier.

Index	Length	Data Type	Value	Description
0	1	Byte	0xC2	Packet type
1	2	Short		Packet header - length of the packet
3	1	Byte	0xF4	Packet header - packet type identifier
4	1	Byte	0x06	Packet header - sub packet type identifier
5	2	ShortBigEndian		ServerCount
7	ServerLoadInfo.Length * N	Array of ServerLoadInfo		Servers

ServerLoadInfo Structure (Length: 4 Bytes):

Index	Length	Data Type	Value	Description
0	2	ShortLittleEndian		ServerId
2	1	Byte		LoadPercentage

In this packet we can see the dynamic header in action.

Defining the Scope

After reviewing the implementation, I decided the scope of this first effort: we are going to implement up to the also mythical login screen. That covers the client and server handshake, server list, and client server selection. I believe this will be enough for this first article.

To achieve this, we need to review the implementation and gather all the components and required pieces that we have to put together. As we recently reviewed the packets, which are the foundation of the MU Online protocol, here is the list of packets involved for this scope:

Hello (C1 00 01) - By Server
ServerListRequest (C1 F4 06) - By Client
ServerListResponse (C2 F4 06) - By Server
ConnectionInfoRequest (C1 F4 03) - By Client
ConnectionInfo (C1 F4 03) - By Server
GameServerEntered (C1 F1 00) - By Server

[!NOTE]
The bytes in parentheses after each packet name reflect the packet identification using the packet type, packet group, and packet sub-group. We are omitting the length field and the rest of the payload bytes.

So the flow of this initial communication will be something like:

Architecture Overview

In the diagram above you can identify that we have 3 main components: the Client, the ConnectServer, and the GameServer. MU Online's architecture separates the Discovery/Entry phase from the Gameplay phase. This separation provides some nice benefits:

Load Balancing: A single entry point (ConnectServer) can distribute players across multiple GameServer instances (e.g., Server 1, Server 2, Sub-Server 1).
Scalability: We can add more Game Servers horizontally without changing the client configuration. The client only needs to know the IP of the Connect Server, which as we can see in the diagram is the one responsible for providing Game Server information.
Isolation: If a Game Server crashes or restarts, the Connect Server remains online, allowing players to see the server status and reconnect to other available servers.

ConnectServer - Discovery/Entry Phase

The ConnectServer is the Gateway to our infrastructure. It listens on port 44405 and is the first point of contact for any client.

Core Responsibilities:

Handshake: Handles the initial handshake with the client.
Server List Management: Maintains a registry of available Game Servers.
Client Discovery: Responds to client requests for the Server List.
Redirection: When a user selects a server, it provides the specific IP address and Port of that Game Server.

[!CAUTION]
Something to notice: after the client selects a game server and connects to it, the connection to the ConnectServer is typically closed shortly after.

GameServer - Gameplay Phase

The GameServer is the Engine of the world. It handles all logic once the player has selected a server, and it starts listening on port 55901 by default.

Core Responsibilities:

Authentication: Validates User/Password (Login Packet handling).
World State: Manages maps, player positions, monsters, and items.
Interaction Logic: Handles movement, combat, trading, chat, and skills.
Persistence: Reads/Writes character data to the Database.
Session Management: Maintains the persistent TCP connection for the duration of play.

Let's Start Building

Well, with all of this context we have everything to start playing around and working on our server. We are going to build our simple server, but what about the client? For that, I found a couple of clients also on GitHub that mention they work with OpenMU. The same team has their own client implementation, but I found one that I could compile from my Mac, so I am going to use that one. You can find the client here.

Before we continue, something worth mentioning is that after the initial discovery and review of the flow that we are going to cover in this first effort, I found two important things related to the packets and flow involved:

Only one of the packets has a payload from client to server: the ConnectionInfoRequest packet (which should be encrypted by XOR32 according to the docs, since all client->server traffic uses XOR32 at minimum). But in this case, the client is sending the serverId unencrypted, so we don't need to implement XOR32 for this scope.
For the last packet, GameServerEntered, this is like the "hello packet" that the GameServer sends to the client as soon as it connects. If you inspect the definition, you'll find that this packet has a payload with some values like PlayerId, VersionString, and Version in binary. After some research, I found these values are hardcoded in OpenMU, so we are going to use the same values.

Now we start with our implementation. For that, we can create our new Rust project. We are going to use a Rust workspace. As this involves a fair amount of code, I won't show all of it here. Instead, I will highlight the most significant parts and explain from there. Some snippets are simplified for clarity, the full implementation is in the repo.

The Protocol Crate

One of the central pieces of the architecture will be a crate that we are going to name mu-protocol. Here we are going to define two central pieces for our client and server exchange: our Packet definition and Codec.

#[derive(Debug, PartialEq, Eq, Clone, Copy)]
pub enum RawPacketType {
    C1,
    C2,
    C3,
    C4,
}

impl RawPacketType {
    pub fn header_length(self) -> usize {
        match self {
            Self::C1 | Self::C3 => 2,
            Self::C2 | Self::C4 => 3,
        }
    }
}

As we mentioned before, the protocol uses the first byte as a frame identifier. The protocol names it the PacketType (see PacketTypes.md), and it defines the size of the packet and the encryption. So we create a simple enum to mirror this. header_length applies the logic from the definition for determining the header length: remember C1 and C3 are small (defined by one byte) and C2 and C4 are the larger ones (defined by two bytes).

Then, we have our RawPacket definition. It's a pretty simple struct with some helper methods to make our journey easier. The idea is straightforward: it's a way to create a valid Packet based on the minimal protocol header definition, representing the packets exchanged between client and servers.

pub struct RawPacket {
    bytes: Bytes,
    packet_type: RawPacketType,
}

impl RawPacket {
    pub fn try_new(bytes: Bytes) -> Result<Self, ProtocolError> {
        let declared_len = declared_length_from_prefix(&bytes)?;
        let actual_len = bytes.len();
        if declared_len != actual_len {
            return Err(ProtocolError::LengthMismatch {
                declared: declared_len,
                actual: actual_len,
            });
        }

        let packet_type = bytes[0].try_into()?;
        Ok(Self { bytes, packet_type })
    }

    pub fn header_codes(&self) -> (Option<u8>, Option<u8>) {
        let header_len = self.packet_type.header_length();
        (
            self.bytes.get(header_len).copied(),
            self.bytes.get(header_len + 1).copied(),
        )
    }
}

The Codec

And the codec. But what is a codec in this context? A codec (coder/decoder) is a component that knows how to transform raw bytes from a stream into structured messages (decoding) and back again (encoding). Since TCP gives us a continuous stream of bytes with no message boundaries, we need something to frame individual packets, that's what our codec does. We are using tokio_util::codec, which provides the Decoder and Encoder traits that integrate directly with Tokio's async I/O. This way, we get a clean Stream of RawPacket items coming in and a Sink for sending them out, without having to manually manage byte buffers.

use bytes::BytesMut;
use tokio_util::codec::{Decoder, Encoder};

pub struct PacketCodec;

impl Decoder for PacketCodec {
    type Item = RawPacket;
    type Error = ProtocolError;

    fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
        let Some(declared_len) = declared_length_from_prefix(src)? else {
            return Ok(None);
        };

        if src.len() < declared_len {
            // Tell tokio-util how many bytes we still need so it can size
            // the next read syscall appropriately.
            src.reserve(declared_len - src.len());
            return Ok(None);
        }

        // split_to advances the buffer past this frame; freeze yields an
        // immutable Bytes (zero-copy, Arc-backed) for RawPacket to own.
        let packet = src.split_to(declared_len).freeze();
        Ok(Some(RawPacket::try_new(packet)?))
    }
}

impl Encoder<RawPacket> for PacketCodec {
    type Error = ProtocolError;

    fn encode(&mut self, item: RawPacket, dst: &mut BytesMut) -> Result<(), Self::Error> {
        dst.extend_from_slice(item.as_slice());
        Ok(())
    }
}

The TCP Servers

As we saw before, we are going to need a ConnectServer and a GameServer. These for now will be very simple TCP listeners, and of course we are going to use the well-known Tokio for everything related to async I/O.

For this, we have a simple function on each server component that creates a TCP listener and enters a loop for accepting connections:

async fn run_server(config: Config) -> Result<()> {
    let listener = TcpListener::bind(&config.bind_addr)
        .await
        .with_context(|| format!("failed to bind to {}", &config.bind_addr))?;

    info!("Connect Server is listening");
    let config = Arc::new(config);
    // Pin the future so we can poll it repeatedly across select! iterations.
    let mut shutdown = Box::pin(tokio::signal::ctrl_c());
    loop {
        tokio::select! {
            _ = &mut shutdown => {
                info!("shutting down server");
                break;
            }
            accepted = listener.accept() => {
                let (socket, peer_addr) = accepted.context("failed to accept client connection")?;
                info!(peer = %peer_addr, "Client connected");

                let client_config = Arc::clone(&config);
                tokio::spawn(async move {
                    if let Err(e) = handle_client(socket, peer_addr, client_config).await {
                        warn!(peer = %peer_addr, error = %e, "client handling failed");
                    }
                });
            }
        }
    }

    Ok(())
}

Both ConnectServer and GameServer have an almost identical run_server function that receives some sort of Config relevant to each server. This function launches the TCP listener and keeps things simple for now.

Where ConnectServer and GameServer differ is in the handle_client function, which is responsible for handling each client connection.

Handling the Client Connection

If you remember, the ConnectServer is the entry point and where almost all the communication exchange for this effort happens. So focusing on the ConnectServer, let's look at its handle_client:


async fn handle_client(
    stream: TcpStream,
    peer_addr: SocketAddr,
    config: Arc<ConnectConfig>,
) -> Result<()> {
    let mut framed = Framed::new(stream, PacketCodec);
    let hello_packet = RawPacket::try_from_vec(vec![C1, 0x04, 0x00, 0x01]);
    framed.send(hello_packet).await

    while let Some(result) = framed.next().await {
        let packet = result?;
        let action = handlers::handle_packet(&config, &packet, peer_addr);
        match action {
            handlers::PacketHandling::Reply(response) => {
                framed.send(response).await?;
            }
            handlers::PacketHandling::Ignore => {}
            handlers::PacketHandling::Disconnect => break,
        }
    }

    Ok(())
}

The key here is that as soon as the Client connects to the ConnectServer, we send the Hello packet as we explained in the communication flow. That will cause the client to request the server list, and then we handle the subsequent requests. That part is resolved by handlers::handle_packet.

If we run the ConnectServer and point a client at it, we can see the entire handshake flow playing out in the logs:

RUST_LOG=debug cargo run -p connect-server
2026-02-17T19:25:10.267046Z  INFO connect_server: starting connect server bind_addr=0.0.0.0:44405 server_count=2
2026-02-17T19:25:10.267325Z  INFO connect_server: Connect Server is listening
2026-02-17T19:25:43.970652Z  INFO connect_server: Client connected peer=127.0.0.1:53368
2026-02-17T19:25:43.971023Z DEBUG connect_server: sent hello packet peer_addr=127.0.0.1:53368
2026-02-17T19:25:43.993037Z DEBUG connect_server: received packet packet=RawPacket(len=4, bytes=[C1, 04, F4, 06])
2026-02-17T19:25:43.993071Z  INFO connect_server::handlers: Server list requested peer=127.0.0.1:53368 server_count=2
2026-02-17T19:25:46.543965Z DEBUG connect_server: received packet packet=RawPacket(len=6, bytes=[C1, 06, F4, 03, 01, 00])
2026-02-17T19:25:46.544041Z  INFO connect_server::handlers: Connection info requested peer=127.0.0.1:53368 server_id=1
2026-02-17T19:25:46.544065Z DEBUG connect_server::packet: Building connection info ip=127.0.0.1 port=55901
2026-02-17T19:25:46.598691Z  INFO connect_server: connect-server client disconnected peer=127.0.0.1:53368

You can see the exact flow we described earlier: the server sends the hello, the client requests the server list (C1 04 F4 06), picks a server (C1 06 F4 03 01 00 with server_id=1), gets the connection info back, and disconnects. The raw bytes in the logs match our packet definitions perfectly.

Parsing Packets

For parsing, we are going to use the powerful enum and pattern matching system of Rust to build a packet parser for the packets expected by the ConnectServer:

pub enum ConnectServerPacket {
    ServerListRequest,
    ConnectionInfoRequest {
        server_id: u16,
    },
    Unknown {
        code: Option<u8>,
        sub_code: Option<u8>,
    },
}

impl ConnectServerPacket {
    pub fn parse(packet: &RawPacket) -> Result<Self, ProtocolError> {
        let (code, sub_code) = packet.header_codes();
        match (code, sub_code) {
            (Some(0xF4), Some(0x06)) => Ok(Self::ServerListRequest),
            (Some(0xF4), Some(0x03)) => {
                let data = packet.as_slice();
                let server_id = u16::from_le_bytes([data[4], data[5]]);
                Ok(Self::ConnectionInfoRequest { server_id })
            }
            _ => Ok(Self::Unknown { code, sub_code }),
        }
    }
}

We determine the request type by inspecting the code and sub_code, the two-level identification system that we talked about before. Once we identify the request, we can easily build the response. With this, handle_packet is pretty straightforward:

pub fn handle_packet(
    config: &ConnectConfig,
    packet: &RawPacket,
    peer: SocketAddr,
) -> PacketHandling {
    let parsed = match ConnectServerPacket::parse(packet) {
        Ok(p) => p,
        Err(_) => return PacketHandling::Disconnect,
    };

    match parsed {
        ConnectServerPacket::ServerListRequest => {
            let response = build_server_list_response(&config.servers);
            match response {
                Ok(packet) => PacketHandling::Reply(packet),
                Err(e) => { PacketHandling::Disconnect }
            }
        }
        ConnectServerPacket::ConnectionInfoRequest { server_id } => {
            let server = config.servers.iter().find(|s| s.id == server_id);
            match server {
                Some(server) => {
                    let response = build_connection_info(server);
                    match response {
                        Ok(packet) => PacketHandling::Reply(packet),
                        Err(e) => { PacketHandling::Disconnect }
                    }
                }
                None => { PacketHandling::Ignore }
            }
        }
        ConnectServerPacket::Unknown { code, sub_code } => {
            warn!(?code, ?sub_code, "unknown packet");
            PacketHandling::Ignore
        }
    }
}

Building Responses

And finally, after identifying the request we build the corresponding response, following each packet definition. Again, thanks to the OpenMU team for the great documentation.

// Wire layout constants for C2-F4-06 ServerListResponse.
// See: docs/OpenMU/Packets/C2-F4-06-ServerListResponse_by-server.md
const HEADER_SIZE: usize = 3 + 1 + 1 + 2; // C2 framing(3) + code(1) + sub_code(1) + server_count(2)
const ENTRY_SIZE: usize = 4; // server_id(2) + load_percentage(1) + padding(1)

/// Builds a C1-F4-03 ConnectionInfo response packet.
/// See: docs/OpenMU/Packets/C1-F4-03-ConnectionInfo_by-server.md
///
/// Wire layout (22 bytes total):
///   [0]    C1 header
///   [1]    length (22)
///   [2]    code   (0xF4)
///   [3]    sub    (0x03)
///   [4..20] IP address as a null-terminated ASCII string in a 16-byte field
///   [20..22] port as little-endian u16
pub fn build_connection_info(ip: Ipv4Addr, port: u16) -> Result<RawPacket> {
    let ip_str = ip.to_string();
    let mut buf = BytesMut::with_capacity(22);
    buf.put_u8(C1);
    buf.put_u8(22);
    buf.put_u8(0xF4);
    buf.put_u8(0x03);

    // IPv4 max string is "255.255.255.255" (15 chars), which fits in the
    // 16-byte protocol field. Remaining bytes are null-padded.
    let ip_bytes = ip_str.as_bytes();
    buf.put_slice(ip_bytes);
    for _ in ip_bytes.len()..16 {
        buf.put_u8(0x00);
    }

    buf.put_u16_le(port);
    RawPacket::try_new(buf.freeze())
}

/// Builds a C2-F4-06 ServerListResponse packet.
/// See: docs/OpenMU/Packets/C2-F4-06-ServerListResponse_by-server.md
///
/// Wire layout:
///   [0]      C2 header
///   [1..3]   total length (big-endian u16) — uses C2 because the list can exceed 255 bytes
///   [3]      code   (0xF4)
///   [4]      sub    (0x06)
///   [5..7]   server count (big-endian u16)
///   [7..]    N × 4-byte ServerLoadInfo entries
pub fn build_server_list_response(servers: &[ConfiguredGameServer]) -> anyhow::Result<RawPacket> {
    let payload_len = HEADER_SIZE + servers.len() * ENTRY_SIZE;
    let packet_len: u16 = payload_len
        .try_into()
        .map_err(|_| anyhow::anyhow!("packet length overflow"))?;

    let mut buf = BytesMut::with_capacity(payload_len);
    buf.put_u8(C2);
    buf.put_u16(packet_len);
    buf.put_u8(0xF4);
    buf.put_u8(0x06);

    let server_count: u16 = servers
        .len()
        .try_into()
        .map_err(|_| anyhow::anyhow!("too many servers for u16 count field"))?;

    buf.put_u16(server_count);

    for server in servers {
        buf.put_u16_le(server.id);
        buf.put_u8(server.load_percentage);
        buf.put_u8(0); // padding byte defined by protocol
    }

    RawPacket::try_new(buf.freeze())
}

An important note here is that I am omitting a lot of improvements, features, and security measures that you may notice in our TCP listener on purpose. The idea is to revisit and implement them in future articles, so the code is kept pretty simple for now. As the series advances, the implementation will evolve.

Also, since this is a work in progress, a lot of the implementation will and must change as other features and flows are added and we play around with more complex game mechanisms.

The Result

With this initial implementation, we achieve something great and very fun for me: the mythical login flow!

Our ConnectServer responding with the server list, just like the old days:

After selecting a server, the client connects to our GameServer and we get the login screen:

Seeing that login screen pop up from a server I wrote myself was honestly one of the most satisfying moments I've had in a while. What started as pure nostalgia turned into a deep dive into revisiting binary protocols, packet framing, and async networking in Rust.

A few things that surprised me during this process:

How well-documented the protocol actually is, thanks to the OpenMU team. Without their work, this would have been a much harder journey.
How clean the protocol design is for a game from 2001. The two-level packet identification, the separation between ConnectServer and GameServer, it's a simple and working architecture.

There's a lot left to do: authentication, character selection, and eventually the actual game world. In the next article, we'll tackle the login flow and start dealing with encrypted client packets.

All the code can be found in the repository. Feel free to explore, experiment, and leave comments! Happy coding!

OSDev Bare Bones with Rust - Cross-Compilation and Freestanding

Douglas Makey Mendez Molero — Tue, 10 Feb 2026 20:03:08 +0000

For a while now, I've had OSDev Bare Bones on my todo list—almost two or three years if I'm not wrong. I always wanted to give it a shot but never had the time, and in the last months I was dealing with some burnout. Fortunately, I'm managing to recover from that. Long story for another post, or maybe not.

This post will be the first in a new series where I'll be working through the OSDev Bare Bones tutorial, or at least try to get as far as I can. I'll be posting along the way to share knowledge as usual and also to reinforce my own learning.

The OSDev Bare Bones tutorial is a well-known guide from the OSDev Wiki that walks you through creating a minimal operating system kernel. It covers the fundamentals: setting up a cross-compiler, writing a simple kernel in C, and booting it with GRUB.

Why? I think the OSDev tutorial is a very interesting exercise that can teach you a couple of things. My only thought is that it's a bit old. Also, I'm not a C developer and I'm still fighting to learn more about Rust, so I decided to have a look at the tutorial and try to follow it using Rust instead.

But of course, I want to read and follow the original tutorial first, trying to pair it with modern concepts and Rust along the way.

Cross-Compilation

The first step that the tutorial mentions is cross-compiling, and it talks about the setup of a GCC cross-compiler for i686-elf.

i686-elf is a classic target designed for developing 32-bit (x86) operating systems. The -elf target produces bare binaries with no OS assumptions for bare metal—exactly what we want for kernel development.

What Is a Cross-Compiler?

TL;DR: A cross-compiler produces code for a different platform than the one it runs on (e.g., compiling ARM code on an x86 machine).

A native compiler runs on platform Y and produces binaries for platform Y. A cross-compiler runs on platform X but produces binaries for platform Y.

As I already mentioned, I'm not a C developer—never used C more than for ultra simple basic hello worlds or some lecture. So I didn't know how this process works in C. I have more experience with Go and some experience with Rust, where cross-compilation is a native feature offered out of the box.

Why Cross-Compilers Are Necessary

The compiler must know the correct target platform (CPU, operating system). The compiler that comes with the host system does not know by default that it is compiling something else entirely. In our case, the host platform is our current operating system and the target platform is the operating system we are about to make. It is important to realize that these two platforms are not the same; the operating system we are developing is always going to be different from the operating system we currently use.

As you read before, the guide mentions that we are going to use i686-elf, designed to build 32-bit (i686) executable binaries that run on bare metal, without an underlying operating system. In my case, I have access to an ARM M1 Max as my main machine or a Lima VM running Ubuntu x86_64.

The tutorial mentions i686-elf which is 32-bit, because we will not be able to correctly complete the tutorial with an x86_64-elf cross-compiler, as GRUB 1 "legacy" is only able to load 32-bit multiboot kernels.

So, we will not be able to correctly compile the operating system without a cross-compiler.

Freestanding and Hosted Environments

As the guide mentions, to write an operating system kernel we need code that does not depend on any operating system features. This makes sense since the whole point of the guide is to create our own OS.

Let's look at the two execution environments:

Hosted Environment

This is when we are writing a program that runs on top of an existing Operating System (Linux, macOS).

The Contract: The OS provides a managed environment. It sets up the stack, clears memory, and handles hardware I/O.
The Toolbox: You have the Full Standard Library. You can call printf or malloc because the OS provides the "plumbing" (drivers and memory managers) to make them work.
Safety: If your code crashes, the OS catches it and kills the process. The rest of the computer keeps running.
Target: 99% of software (apps, web servers, CLI tools).

Freestanding Environment

This is when we are writing code for Bare Metal (a CPU with no OS).

The Contract: There is no help. You must manually define the Entry Point and tell the CPU how to use its own RAM.
The Toolbox: You only have Core Language Primitives. No printf, no malloc, no file system, no network. You only get basic types (integers, pointers) and math or core language features.
Safety: If your code crashes, the CPU triggers a "Triple Fault" and the physical machine reboots. There is no "safety net."
Target: The kernel itself, bootloaders, and embedded firmware.

Recap

So basically, a cross-compiler produces code for a different platform than the one it runs on, and a freestanding environment is one where no OS or standard library exists just our code and the hardware. These two concepts go hand-in-hand: when writing an OS kernel or bare metal firmware, we have to cross-compile for our target architecture in freestanding mode.

Zig to the Rescue?

The guide has a link to GCC Cross-Compiler explaining how to set up the required compiler, which I found a bit complex and tedious. I remembered hearing a lot of good things about the Zig compiler and its flexibility, so I thought I could just use Zig as the compiler.

The OSDev tutorial provides the following C code for the kernel.c file. Here's a brief walkthrough of what it does:

The #if defined checks at the top are safeguards that prevent you from compiling with the wrong compiler—they ensure you're using a cross-compiler targeting i386.
It defines a vga_color enum and helper functions to create VGA text-mode entries (each character on screen is a 16-bit value: the character byte + a color byte).
The terminal_* functions manage a simple text terminal by writing directly to VGA memory at 0xB8000—this is the physical address where the VGA text buffer lives in x86 systems.
kernel_main is our entry point: it initializes the terminal and prints "Hello, kernel World!".

Let's look at the full code and then jump to the cross-compilation part:

#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>

/* Check if the compiler thinks you are targeting the wrong operating system. */
#if defined(__linux__)
#error "You are not using a cross-compiler, you will most certainly run into trouble"
#endif

/* This tutorial will only work for the 32-bit ix86 targets. */
#if !defined(__i386__)
#error "This tutorial needs to be compiled with a ix86-elf compiler"
#endif

/* Hardware text mode color constants. */
enum vga_color {
    VGA_COLOR_BLACK = 0,
    VGA_COLOR_BLUE = 1,
    VGA_COLOR_GREEN = 2,
    VGA_COLOR_CYAN = 3,
    VGA_COLOR_RED = 4,
    VGA_COLOR_MAGENTA = 5,
    VGA_COLOR_BROWN = 6,
    VGA_COLOR_LIGHT_GREY = 7,
    VGA_COLOR_DARK_GREY = 8,
    VGA_COLOR_LIGHT_BLUE = 9,
    VGA_COLOR_LIGHT_GREEN = 10,
    VGA_COLOR_LIGHT_CYAN = 11,
    VGA_COLOR_LIGHT_RED = 12,
    VGA_COLOR_LIGHT_MAGENTA = 13,
    VGA_COLOR_LIGHT_BROWN = 14,
    VGA_COLOR_WHITE = 15,
};

static inline uint8_t vga_entry_color(enum vga_color fg, enum vga_color bg)
{
    return fg | bg << 4;
}

static inline uint16_t vga_entry(unsigned char uc, uint8_t color)
{
    return (uint16_t) uc | (uint16_t) color << 8;
}

size_t strlen(const char* str)
{
    size_t len = 0;
    while (str[len])
        len++;
    return len;
}

#define VGA_WIDTH   80
#define VGA_HEIGHT  25
#define VGA_MEMORY  0xB8000

size_t terminal_row;
size_t terminal_column;
uint8_t terminal_color;
uint16_t* terminal_buffer = (uint16_t*)VGA_MEMORY;

void terminal_initialize(void)
{
    terminal_row = 0;
    terminal_column = 0;
    terminal_color = vga_entry_color(VGA_COLOR_LIGHT_GREY, VGA_COLOR_BLACK);

    for (size_t y = 0; y < VGA_HEIGHT; y++) {
        for (size_t x = 0; x < VGA_WIDTH; x++) {
            const size_t index = y * VGA_WIDTH + x;
            terminal_buffer[index] = vga_entry(' ', terminal_color);
        }
    }
}

void terminal_setcolor(uint8_t color)
{
    terminal_color = color;
}

void terminal_putentryat(char c, uint8_t color, size_t x, size_t y)
{
    const size_t index = y * VGA_WIDTH + x;
    terminal_buffer[index] = vga_entry(c, color);
}

void terminal_putchar(char c)
{
    terminal_putentryat(c, terminal_color, terminal_column, terminal_row);
    if (++terminal_column == VGA_WIDTH) {
        terminal_column = 0;
        if (++terminal_row == VGA_HEIGHT)
            terminal_row = 0;
    }
}

void terminal_write(const char* data, size_t size)
{
    for (size_t i = 0; i < size; i++)
        terminal_putchar(data[i]);
}

void terminal_writestring(const char* data)
{
    terminal_write(data, strlen(data));
}

void kernel_main(void)
{
    /* Initialize terminal interface */
    terminal_initialize();

    /* Newline support is left as an exercise. */
    terminal_writestring("Hello, kernel World!\n");
}

As was previously mentioned, if we try to compile this code with a regular gcc on my Lima Ubuntu VM, you will see the safeguards in the code acting up:

$ gcc -c kernel.c -o hosted_kernel.o
kernel.c:7:2: error: #error "You are not using a cross-compiler, you will most certainly run into trouble"
    7 | #error "You are not using a cross-compiler, you will most certainly run into trouble"
      |  ^~~~~
kernel.c:12:2: error: #error "This tutorial needs to be compiled with a ix86-elf compiler"
   12 | #error "This tutorial needs to be compiled with a ix86-elf compiler"
      |  ^~~~~

But if we use Zig and its build magic, we get the output successfully and we can inspect the file:

$ zig cc -target x86-freestanding-none -c kernel.c -o kernel.o
$ readelf -h kernel.o
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           Intel 80386
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          7164 (bytes into file)
  Flags:                             0x0
  Size of this header:               52 (bytes)
  Size of program headers:           0 (bytes)
  Number of program headers:         0
  Size of section headers:           40 (bytes)
  Number of section headers:         22
  Section header string table index: 20

We can see that the class is ELF32 (32-bit) and the OS/ABI is UNIX - System V. If it said Linux, it would not be freestanding; it would have been built with Linux-specific assumptions.

We could also inspect the symbol table, and we should not see _start:

$ nm kernel.o
         U __ubsan_handle_pointer_overflow
         U __ubsan_handle_shift_out_of_bounds
         U __ubsan_handle_type_mismatch_v1
000004f0 T kernel_main
00000000 T strlen
00000024 D terminal_buffer
00000008 B terminal_color
00000004 B terminal_column
000000a0 T terminal_initialize
00000390 T terminal_putchar
000002b0 T terminal_putentryat
00000000 B terminal_row
000002a0 T terminal_setcolor
00000410 T terminal_write
000004c0 T terminal_writestring
00000230 t vga_entry
000001e0 t vga_entry_color

So great, we managed to cross-compile our kernel.c for a different architecture in freestanding mode using Zig. But as I mentioned at the beginning of the article, I want to do the same using Rust.

Rust in Action

Beyond my personal goal of learning Rust, it's actually an ideal candidate for OS development: it gives you the same low-level control as C but with memory safety guarantees at compile time, no garbage collector, and first-class no_std support designed exactly for bare-metal targets.

Let's create a new project:

$ cargo new osdev

A small disclaimer: Rust by default offers x86_64-unknown-none, which means 64-bit instead of the 32-bit target used in the tutorial. We could use nightly and a custom target spec to specify 32-bit, but for simplicity and because I want to try following along using 64-bit later! we are going to use x86_64-unknown-none.

$ rustup target add x86_64-unknown-none

As you can see, both Zig and Rust leverage the Target Triple convention for specifying targets:

Target Component	x86-freestanding-none (Zig)	x86_64-unknown-none (Rust)	Definition
Architecture	`x86` (32)	`x86_64` (64)	The CPU instruction set (32-bit vs. 64-bit).
Vendor	`freestanding`*	`unknown`	Historically the hardware maker. In OS dev, this is a "don't care" field.
OS	`none`	`none`	Tells the compiler there is no OS (no Linux/macOS).

As you may know, when you create a fresh Rust project it comes with its hello world version:

fn main() {
    println!("Hello, world!");
}

If we try to compile using the target we previously installed (x86_64-unknown-none), we get the following error:

$ cargo build --target x86_64-unknown-none
   Compiling osdev v0.1.0 (/Users/douglasmakey/workdir/personal/os-barebones/osdev)
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-none` target may not support the standard library
  = note: `std` is required by `osdev` because it does not declare `#![no_std]`

error: cannot resolve a prelude import

error: cannot find macro `println` in this scope
 --> src/main.rs:2:5
  |
2 |     println!("Hello, world!");
  |     ^^^^^^^

error: `#[panic_handler]` function required, but not found

For more information about this error, try `rustc --explain E0463`.
error: could not compile `osdev` (bin "osdev") due to 4 previous errors

This happens because our current build implicitly links the standard library. Since we specified a freestanding target (meaning no OS context, just core primitives), Rust can't find std. Rust has support for this scenario using #![no_std]:

#![no_std]
fn main() {
    println!("Hello, world!");
}

But if we try to compile this, we get another error:

$ cargo build --target x86_64-unknown-none
   Compiling osdev v0.1.0 (/Users/douglasmakey/workdir/personal/os-barebones/osdev)
error: cannot find macro `println` in this scope
 --> src/main.rs:3:5
  |
3 |     println!("Hello, world!");
  |     ^^^^^^^

error: `#[panic_handler]` function required, but not found

error: could not compile `osdev` (bin "osdev") due to 2 previous errors

The problem is that #![no_std] tells the Rust compiler not to link against the standard library. Instead, it links against the core library, which only provides core primitives. This means macros like println! are gone. We are essentially disabling all OS dependent functionality. The other error is about the panic_handler.

Panic Handler

When a Rust program encounters an unrecoverable error, it "panics." In a regular std environment, this typically prints a backtrace and the OS catches it. In no_std, you must define your own panic handler, as there's no OS to catch the panic. There is no "safety net."

So to make this work, we need to address a few things: add the panic handler, drop println!, and also tell the compiler that we don't have a normal main entry point. In a freestanding environment there's no runtime that calls main for us, so we use #![no_main] to disable that expectation and define our own entry point _start instead. The #[unsafe(no_mangle)] attribute ensures the function name isn't changed by the compiler, so the linker can find it. The -> ! return type means the function never returns—which makes sense because there's no OS to return to, so we just loop forever.

#![no_std]
#![no_main]

use core::panic::PanicInfo;

#[unsafe(no_mangle)]
pub extern "C" fn _start() -> ! {
    loop {}
}

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

If you want to deep dive on no_std, I found this article useful: Rust Without the Standard Library: A Deep Dive into no_std Development

And now if we compile again, this time it will succeed:

$ readelf -h target/x86_64-unknown-none/debug/osdev
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Position-Independent Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x1280
  Start of program headers:          64 (bytes into file)
  Start of section headers:          4216 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         8
  Size of section headers:           64 (bytes)
  Number of section headers:         21
  Section header string table index: 19

As we can notice, the header has similarities with our kernel.c compiled with Zig, of course with the difference of ELF64 (64-bit) and for the same reason the machine shows as AMD X86-64.

We could also inspect the symbol table, this time, we’ll notice the presence of _start. That’s because cargo build automatically invokes the linker (LLD) when it detects a pub fn _start in our code. By default, the linker looks for that symbol and exports it as the program’s entry point.

$ nm target /x86_64-unknown-none/debug/osdev
0000000000002288 d _DYNAMIC
0000000000001280 T _start

I will leave this first article at this point. I hope you find it as interesting as I did. I will continue the series hopefully as soon as possible. I'll be working on the OS at the same time I write the articles, so a couple of things might change or I might make mistakes that I'll fix later on.

In the next post, we'll cover implementing the bootloader and get our kernel running in QEMU—either by following the OSDev guide step by step or by using the Rust bootloader crate.

Bonus: Lima Setup for Mac Users

In case you want to play around with this on a Mac, I'm using Lima to run an x86_64 Ubuntu VM. Here's my current Lima configuration with Zig, Rust, and the build essentials pre-installed:

vmType: "qemu"
arch: "x86_64"

images:
- location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-amd64.img"
  arch: "x86_64"
- location: "https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img"
  arch: "x86_64"

mounts:
- location: "path-to-your-repo"
  writable: true

provision:
- mode: system
  script: |
    DEBIAN_FRONTEND=noninteractive apt-get update
    DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential gdb qemu-system-x86 nasm xorriso mtools
    snap install zig --beta --classic
- mode: user
  script: |
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
    source "$HOME/.cargo/env"
    rustup default stable
    rustup component add rust-src llvm-tools-preview

cpus: 4
memory: "4GiB"

You can start the VM with:

$ limactl start --name osdev osdev.yaml
$ limactl shell osdev

Thanks for reading, and as always, feedback is welcome. Happy coding!

Playing with Rust: Building a Safer rm and Having Fun Along the Way

Douglas Makey Mendez Molero — Sun, 20 Oct 2024 16:39:10 +0000

Welcome to my YOLO series, where I'll be showcasing simple tools and projects that I've built—sometimes for fun, sometimes to solve specific problems, and other times just out of pure curiosity. The goal here isn't just to present a tool; I'll also dive into something interesting related to the process, whether it's a technical insight or a lesson learned while crafting these little experiments.

Introducing rrm: The Command-Line Tool Nobody Asked For

Nobody asked for it, and nobody want it—but here it is anyway. Meet rrm, a tool that solves a problem only I seem to have (but hey, it might be a Layer 8 issue—or, more likely, a skill issue!).

rrm adds a layer of safety to your command-line experience by moving files to a trash bin instead of permanently deleting them. With a customizable grace period, you get the chance to realize, "Oops, I actually needed that!" before it’s too late.

What’s more, rrm doesn’t rely on external configuration files or tracking systems to manage deleted files. Instead, it leverages your filesystem’s extended attributes to store essential metadata—like the original file path and deletion time—directly within the trashed item.

You might be wondering, "Why am I building this tool when there are similar, possibly better tools out there?" Well, the answer is simple:

I wanted to play with Rust. Building small, purposeful tools is a great way to explore a language and sharpen skills.
like developing my own CLI tools as a way to create a mental framework. It helps me consistently approach how I structure command-line utilities for specific technologies. By building these tools, I refine my understanding of what dependencies to use, how to organize the code, and how to adapt each tool to the language’s ecosystem. It’s a way of building a mental playbook for creating CLI tools that suit my needs.
Because YOLO. I enjoy making simple tools or proof-of-concepts around problems I want to solve or things I’m curious about. Sometimes, it’s about experimenting for the sake of learning.

// Detect dark theme var iframe = document.getElementById('tweet-1844834987184410735-190'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1844834987184410735&theme=dark" }

Fun note: While working with std::Path, I found an example in the Rust standard library that uses a folder named laputa. I know it's a reference to Castle in the Sky, but for Spanish speakers, it’s also a curse word, which made it a funny moment for me!

Extended Attributes: Storing Metadata Without Changing the File

When I started building rrm, I needed a way to track the original path of deleted files and the time when they should be permanently removed. I didn’t want to use a JSON file or implement a weird naming format that includes this information—especially if I wanted to store more data later. A database felt like overkill for such a small task.

That’s when I discovered extended attributes.

What Are Extended Attributes?

Now, I don’t know about you, but I didn’t realize there was a built-in mechanism that lets you add custom metadata to files, which is supported by most Linux filesystems and Unix-like systems such as macOS. This feature is called Extended File Attributes. Different systems have their own limitations—like how much data can be added or the specific namespaces used—but they do allow you to store user-defined metadata.

Extended attributes are essentially name:value pairs that are permanently associated with files and directories. As I mentioned earlier, systems differ in how they handle this. For example, in Linux, the name starts with a namespace identifier. There are four such namespaces: security, system, trusted, and user. In Linux, the name starts with one of these, followed by a dot (".") and then a null-terminated string. On macOS, things are a bit different. macOS doesn't require namespaces at all, thanks to its Unified Metadata Approach, which treats extended attributes as additional metadata directly tied to files without needing to be categorized.

In this tiny CLI, I’m using the crate xattr, which supports both Linux and macOS. Regarding the namespaces I mentioned earlier for Linux, we'll be focusing on the user namespace since these attributes are meant to be used by the user. So, in the code, you’ll see something like this:

/// Namespace for extended attributes (xattrs) on macOS and other operating systems.
/// On macOS, this is an empty string, while on other operating systems, it is "user.".
#[cfg(target_os = "macos")]
const XATTR_NAMESPACE: &str = "";
#[cfg(not(target_os = "macos"))]
const XATTR_NAMESPACE: &str = "user.";

...

    fn set_attr(&self, path: &Path, attr: &str, value: &str) -> Result<()> {
        let attr_name = format!("{}{}", XATTR_NAMESPACE, attr);
        ...
    }

The #[cfg(target_os = "macos")] attribute in Rust is used to conditionally compile code based on the target operating system. In this case, it ensures that the code block is only included when compiling for macOS. This is relevant because, as mentioned earlier, macOS doesn’t require a namespace for extended attributes, so the XATTR_NAMESPACE is set to an empty string. For other operating systems, the namespace is set to "user.". This conditional compilation allows the code to adapt seamlessly across different platforms, making the CLI cross-compatible with both Linux and macOS.

One thing I found pretty cool about extended attributes is that they don’t modify the file itself. The metadata lives in a separate disk space, referenced by the inode. This means the file's actual contents remain unchanged. For example, if we create a simple file and use shasum to get its checksum:

The inode (index node) is a data structure in a Unix-style file system that describes a file-system object such as a file or a directory. Link

$ cat a.txt
https://www.kungfudev.com/

$ shasum a.txt
e4c51607d5e7494143ffa5a20b73aedd4bc5ceb5  a.txt

After using rrm to delete the file, we can list the deleted files and see that the file has been moved to the trash bin with its metadata intact:

$ rrm rm a.txt

$ rrm list
╭──────────────────────────────────────────────────────┬──────────────────────────────────────┬──────┬─────────────────────╮
│ Original Path                                        ┆ ID                                   ┆ Kind ┆ Deletion Date       │
╞══════════════════════════════════════════════════════╪══════════════════════════════════════╪══════╪═════════════════════╡
│ /Users/douglasmakey/workdir/personal/kungfudev/a.txt ┆ 3f566788-75dc-4674-b069-0faeaa86aa55 ┆ File ┆ 2024-10-27 04:10:19 │
╰──────────────────────────────────────────────────────┴──────────────────────────────────────┴──────┴─────────────────────╯

As you can see, the file name is changed to a UUID. This is done to avoid name collisions when deleting files with the same name. By assigning a unique identifier to each file, rrm ensures that every deleted file, even if they have identical names, can be tracked and recovered without any issues.

We can navigate to the trash folder and inspect the file to confirm that its contents remain unchanged:

$ shasum 3f566788-75dc-4674-b069-0faeaa86aa55
e4c51607d5e7494143ffa5a20b73aedd4bc5ceb5  3f566788-75dc-4674-b069-0faeaa86aa55

Additionally, by using xattr on macOS, we can verify that the file has its metadata, such as the deletion date and original path:

$ xattr -l 3f566788-75dc-4674-b069-0faeaa86aa55
deletion_date: 2024-10-27T04:10:19.875614+00:00
original_path: /Users/douglasmakey/workdir/personal/kungfudev/a.txt

You can imagine the range of potential use cases for simple validations or actions using this metadata. Since extended attributes work without modifying the file itself, they allow you to check file integrity or perform other operations without affecting the original content.

This is just a small introduction to extended attributes and how they’re used in this project. It’s not meant to be an in-depth explanation, but if you’re interested in learning more, there are plenty of detailed resources out there. Here are a couple of links to the most useful and well-described resources on the topic:

Mocking in Rust: Exploring `mockall` for Testing

I’ve spent a few years working with Go, and I’ve become fond of certain patterns—mocking being one of them. In Go, I typically implement things myself if it avoids unnecessary imports or gives me more flexibility. I’m so used to this approach that when I started writing tests in Rust, I found myself preferring to manually mock certain things, like creating mock implementations of traits.

For example, in this tiny CLI, I created a trait to decouple the trash manager from the way it interacts with the extended attributes. The trait, named ExtendedAttributes, was initially intended for testing purposes, but also because I wasn’t sure whether I would use xattr or another implementation. So, I defined the following trait:

pub trait ExtendedAttributes {
    fn set_attr(&self, path: &Path, key: &str, value: &str) -> Result<()>;
    fn get_attr(&self, path: &Path, key: &str) -> Result<Option<String>>;
    fn remove_attr(&self, path: &Path, key: &str) -> Result<()>;
}

In Go, I would create something like the following, which provides a simple implementation of the previously mentioned interface. The code below is straightforward and generated without much consideration, just for the sake of example:

// ExtendedAttributes allows for passing specific behavior via function fields
type ExtendedAttributes struct {
    SetAttrFunc    func(path string, key string, value string) error
    GetAttrFunc    func(path string, key string) (string, error)
    RemoveAttrFunc func(path string, key string) error
}

// SetAttr calls the injected behavior
func (m *ExtendedAttributes) SetAttr(path string, key string, value string) error{
    return m.SetAttrFunc(path, key, value)
}

// GetAttr calls the injected behavior
func (m *ExtendedAttributes) GetAttr(path string, key string) (string, error) {
    return m.GetAttrFunc(path, key)
}

// RemoveAttr calls the injected behavior
func (m *ExtendedAttributes) RemoveAttr(path string, key string) error {
    return m.RemoveAttrFunc(path, key)
}

Then, I would use my mock and inject the specific behavior needed for each test. Again, this is simple code just for the sake of the example:

func TestSetAttrWithMock(t *testing.T) {
    mock := &ExtendedAttributes{
        SetAttrFunc: func(path string, key string, value string) error {
            if path == "/invalid/path" {
                return errors.New("invalid path")
            }
            return nil
        },
    }

    err := mock.SetAttr("/invalid/path", "key", "value")
    if err == nil {
        t.Fatalf("expected error, got nil")
    }

    err = mock.SetAttr("/valid/path", "key", "value")
    if err != nil {
        t.Fatalf("expected no error, got %v", err)
    }
}

I've gotten used to this pattern in Go, and I plan to keep using it. But I’ve also been doing something similar in Rust. For this project, I decided to try the mockall crate, and I found it really useful.

First, I used the mock! macro to manually mock my structure. I know mockall has an automock feature, but I prefer to define the mock struct directly in my tests where it will be used. Let me know if this is something common or if the community has a different standard for this.

mod test {
...
    mock! {
        pub XattrManager {}
        impl ExtendedAttributes for XattrManager {
            fn set_attr(&self, path: &std::path::Path, key: &str, value: &str) -> crate::Result<()>;
            fn get_attr(&self, path: &std::path::Path, key: &str) -> crate::Result<Option<String>>;
            fn remove_attr(&self, path: &std::path::Path, key: &str) -> crate::Result<()>;
        }
    }
}

I found mockall really useful, allowing me to inject specific behaviors into my tests without the verbosity of my old pattern.

    #[test]
    fn test_trash_items() -> Result<()> {
        ...
        let mut xattr_manager = MockXattrManager::new();
        xattr_manager
            .expect_set_attr()
            .with(
                in_iter(vec![original_path.clone(), original_path2.clone()]),
                in_iter(vec![ORIGINAL_PATH_ATTR, DELETION_DATE_ATTR]),
                in_iter(vec![
                    original_path_str,
                    original_path2_str,
                    deletion_date.to_rfc3339().to_string(),
                ]),
            )
            .times(4)
            .returning(|_, _, _| Ok(()));
        ...
    }

    #[test]
    fn clean_trash_delete_old_file() {
        ...
        xattr_manager
            .expect_get_attr()
            .times(2)
            .returning(move |_, key| match key {
                DELETION_DATE_ATTR => Ok(Some(deletion_date_past.to_rfc3339())),
                _ => Ok(Some("some_path".to_string())),
            });
        ...
    }

As we can see, mockall gives us the capability to define specific behaviors for our tests using its mock methods:

MockXattrManager::new() This creates a new instance of the mock object MockXattrManager, which is used to mock the behavior of XattrManager for testing.
xattr_manager.expect_set_attr() This sets up an expectation that the set_attr method will be called during the test. You define the expected behavior of this method next.
with(...) The with method specifies the expected arguments when set_attr is called. In this case, it’s expecting three arguments and uses in_iter to indicate that each argument should match one of the values in the provided vector. This allows flexibility in the test, as it checks if the arguments are one of the values from the passed vector rather than a single exact match.
times(4) This specifies that the set_attr method is expected to be called exactly four times during the test.
returning(|_, _, _| Ok(())) This tells the mock what to return when set_attr is called. In this case, it returns Ok(()) regardless of the arguments (|_, _, _| means the arguments are ignored). This simulates the successful execution of set_attr.

Some of you might find this super basic or not that interesting, but as I mentioned, in this YOLO series, I’m sharing things that I find interesting or just want to talk about. I wasn’t a big fan of using this kind of library in Go, partly due to Go’s constraints, but in Rust, I found mockall really useful. It even reminded me of my old days with Python.

Again, this section wasn’t meant to explain mocking in Rust or mockall. I’m sure there are plenty of great resources that cover it in detail. I just wanted to mention it briefly.

To conclude

In this post, I’ve shared some of the reasoning behind building rrm and the tools I used along the way. From using extended attributes to simplify metadata handling to experimenting with the mockall crate for testing in Rust, these were just things that piqued my interest.

The goal of this YOLO series is to highlight the fun and learning that comes with building even simple tools. I hope you found something useful here, and I look forward to sharing more projects and insights in future posts. As always, feedback is welcome!

Happy coding!

Simple thoughts

Douglas Makey Mendez Molero — Tue, 02 Jul 2024 19:09:09 +0000

If you read an article that interests you and you enjoy it, or if you watch a video, consider leaving a message or a reaction. I don't consider myself a content creator, but I genuinely enjoy writing my articles and sharing them with others. For me, it's incredibly rewarding if just one person finds my articles helpful or at least interesting to read.

When someone leaves a comment or reacts, it makes all the hours I spend preparing the content feel worthwhile. I know you don't have to do it—it's not mandatory—but next time, think about spending a few seconds to make the hours others spend creating content worth it. ❤️

I'm not the only one who feels this way. I sometimes think my content might not be too interesting, but there's a lot of amazing content out there. So, next time, consider the effort and dedication people put into creating great content. Your engagement will make them happy!

Fooling Port Scanners: Simulating Open Ports with eBPF and Rust

Douglas Makey Mendez Molero — Sat, 29 Jun 2024 21:47:09 +0000

In our previous article, we explored the SYN and accept queues and their crucial role in the TCP three-way handshake. We learned that for a TCP connection to be fully established, the three-way handshake must be successfully completed. Let's recap this process:

The client initiates the connection by sending a SYN packet.
The server responds with a SYN-ACK packet.
The client then sends an ACK packet back to the server.

At this point, the client considers the connection established. However, it's important to note that from the server's perspective, the connection is only fully established when it receives and processes the final ACK from the client.

In this article, we will review the three-way handshake behavior and a related port scanning technique. We will also explore how to use Rust and eBPF to thwart curious individuals attempting to scan our machine using this technique.

Understanding the TCP Three-Way Handshake in Port Scanning

As we delve deeper into TCP connection management, it's crucial to understand the server's behavior when it receives connection requests. Let's break this down:

When a server has a socket listening on a specific port, it's ready to handle incoming connection requests. Upon receiving a SYN packet, the server begins tracking potential connections by allocating resources, such as space in the SYN queue. This behavior, while necessary for normal operation, can be exploited by malicious actors. For instance, the SYN flood attack takes advantage of this resource allocation to overwhelm the server.

If you're interested in learning more about the SYN flood attack, feel free to leave a comment. This attack exploits the TCP handshake process to overwhelm a server with incomplete connection requests. It works by sending a large number of SYN packets to a server. The server allocates resources for each connection request, occupying significant kernel memory, while the cost for the client is just one SYN packet per request, with no memory allocation on their end.

It's important to note the phrase "When a server has a socket listening on a ..." from our previous discussion. This condition is critical because it determines how the server responds to incoming SYN packets. Let's clarify the two scenarios:

Server with a listening socket on the targeted port

Flow:

Receives SYN packet
Responds with SYN-ACK
Allocates resources to track the potential connection

+--------+                      +-----------------------+
| Client |                      | Server with Listening |
|        |                      | Socket on Target Port |
+--------+                      +-----------------------+
     |                                    |
     |--- SYN --------------------------> |
     |                                    |
     |                                    |
     | <--- SYN-ACK --------------------- |
     |                                    |
     |                                    |
     |                                    |
     |                                    |
+--------+                      +-----------------------+
| Client |                      | Allocates Resources to|
|        |                      | Track Potential Conn. |
+--------+                      +-----------------------+

Server without a listening socket on the targeted port

Flow:

Receives SYN packet
Responds with RST-ACK (Reset-Acknowledge)
No resources are allocated for connection tracking

+--------+                      +-----------------------+
|        |                      | Server without        |
| Client |                      | Listening Socket on   |
|        |                      | Target Port           |
+--------+                      +-----------------------+
     |                                    |
     |--- SYN --------------------------> |
     |                                    |
     |                                    |
     | <--- RST-ACK --------------------- |
     |                                    |
     |                                    |
     |                                    |
     |                                    |
+--------+                      +------------------------+
| Client |                      | No Resources Allocated |
|        |                      | for Connection Tracking|
+--------+                      +------------------------+

The RST-ACK response in the second scenario is the server's way of saying, "There's no service listening on this port, so don't attempt to establish a connection." This behavior is a fundamental aspect of TCP/IP networking and plays a crucial role in network security and resource management.

As we can see, by sending a simple SYN packet to a server on a specific port, we can determine if the port is open or not. This is the basis for one of the most popular techniques for port scanning: the Stealth SYN Scan.

The Stealth SYN Scan: A Popular Port Scanning Technique

Indeed, the behavior we've discussed forms the basis for one of the most popular port scanning techniques: the Stealth SYN Scan, also known as a half-open scan. This technique is called a half-open scan because it doesn’t actually open a full TCP connection. Instead, a SYN scan only sends the initial SYN packet and examines the response. If a SYN/ACK packet is received, it indicates that the port is open and accepting connections. This is recorded, and an RST packet is sent to tear down the connection.

To recap:
Stealth SYN Scan Process:

The scanner sends a SYN packet to a target port.
If the port is open (i.e., a service is listening):
- The target responds with a SYN-ACK.
- The scanner immediately sends an RST to terminate the connection.
If the port is closed:
- The target responds with an RST-ACK.

Why it's called "Stealth":

The scan doesn't complete the full TCP handshake.
It's less likely to be logged by basic firewall configurations.
It can potentially bypass certain intrusion detection systems (IDS).

The SYN scan is popular for its speed, capable of scanning thousands of ports per second on a fast network. It's unobtrusive and stealthy, as it never completes TCP connections.

Nmap in Action: Demonstrating SYN Scans

As you may know, Nmap is a powerful network scanning and discovery tool widely used by security professionals and system administrators.

Using Nmap, a SYN scan can be performed with the command-line option -sS. The program must be run as root since it requires raw-packet privileges. This is the default TCP scan when such privileges are available. Therefore, if you run Nmap as root, this technique will be used by default, and you don't need to specify the -sS option.

So these commands are equivalent:

$ sudo nmap -p- 192.168.2.107
$ sudo nmap -sS -p- 192.168.2.107

Here, we are simply instructing Nmap to perform a SYN scan on the target 192.168.2.107 using -p- to scan all ports from 1 through 65535. However, we can also specify individual ports or a range of ports.

sudo nmap -sS -p9000-9500 192.168.2.107
Starting Nmap 7.95 ( https://nmap.org ) at 2024-06-29 14:56 EDT
Nmap scan report for dlm (192.168.2.107)
Host is up (0.0085s latency).
Not shown: 499 closed tcp ports (reset)
PORT     STATE SERVICE
9090/tcp open  ...
9100/tcp open  ...
...

From the output above, we can see that the target machine has two open ports in the specific range from 9000 to 9500. The remaining 499 ports are closed. This is because, as we know, Nmap received RST packets when it sent the initial SYN to those ports.

Crafting Our Defense: eBPF and Rust Implementation

In previous articles, I explained how to start projects with Rust-Aya, including using their scaffolding generator. If you need a refresher, feel free to revisit Harnessing eBPF and XDP for DDoS Mitigation and Uprobes Siblings - Capturing HTTPS Traffic, or check the Rust-Aya documentation.

Setting Up the eBPF Program

As we are going to use XDP to accomplish this trick, the initial part of the code is very similar to the example explained in the article Harnessing eBPF and XDP for DDoS Mitigation. In this code, we first validate if the packet is an IPv4 packet by examining the ether_type in the Ethernet header. If it's not IPv4, the packet is passed through without further processing. Then, we look at the IPv4 header to check if it's a TCP packet. Non-TCP packets are also allowed to pass. This way, our program focuses only on IPv4 TCP packets and then we store the TCP header in tcp_hdr.

fn try_syn_ack(ctx: XdpContext) -> Result<u32, ExecutionError> {
    // Use pointer arithmetic to obtain a raw pointer to the Ethernet header at the start of the XdpContext data.
    let eth_hdr: *mut EthHdr = get_mut_ptr_at(&ctx, 0)?;

    // Check the EtherType of the packet. If it's not an IPv4 packet, pass it along without further processing
    // We have to use unsafe here because we're dereferencing a raw pointer
    match unsafe { (*eth_hdr).ether_type } {
        EtherType::Ipv4 => {}
        _ => return Ok(xdp_action::XDP_PASS),
    }

    // Using Ethernet header length, obtain a pointer to the IPv4 header which immediately follows the Ethernet header
    let ip_hdr: *mut Ipv4Hdr = get_mut_ptr_at(&ctx, EthHdr::LEN)?;

    // Check the protocol of the IPv4 packet. If it's not TCP, pass it along without further processing
    match unsafe { (*ip_hdr).proto } {
        IpProto::Tcp => {}
        _ => return Ok(xdp_action::XDP_PASS),
    }

    // Using the IPv4 header length, obtain a pointer to the TCP header which immediately follows the IPv4 header
    let tcp_hdr: *mut TcpHdr = get_mut_ptr_at(&ctx, EthHdr::LEN + Ipv4Hdr::LEN)?;
    ...
}

Filtering Packets: Targeting Specific Ports

Now that we've confirmed the received packet is a TCP packet, let's implement a simple filter in our eBPF program. This filter will instruct the program to interact only with packets whose destination port falls within the range of 9000 to 9500.

It's important to note that this is a basic implementation. In a production environment, we'd want to implement a more robust and flexible solution. For instance, we could:

Use eBPF to listen for the bind syscall in our server.
Utilize eBPF maps to track all open ports on our server, avoiding interference with legitimate services.
Alternatively, use eBPF maps to maintain a list of ports we want to simulate as open.

Again, we're using unsafe throughout the code, as we're directly manipulating memory through raw pointers. This allows us to alter the packets or inspect them as needed.

For now, let's keep it simple and focus on our port range filter:

// Check the destination port of the TCP packet. If it's not in the range 9000-9500, pass it along without further processing
let port = unsafe { u16::from_be((*tcp_hdr).dest) };
match port {
    9000..=9500 => {}
    _ => return Ok(xdp_action::XDP_PASS),
}

This code snippet does the following:

We extract the destination port from the TCP header, converting it from network byte order (big-endian) to host byte order.
We use a match statement to check if the port falls within our target range.
If the port is in the range 9000-9500, we continue processing.
For any other port, we immediately return XDP_PASS, allowing the packet to continue its normal journey through the network stack.

This filter allows us to focus our eBPF program's attention on a specific range of ports, which could be useful for various network management and security tasks. For example, we could use this to implement a port knocking sequence, set up a honeypot, or monitor for potential port scan attempts within a specific range.

Identifying SYN Packets

Now that we've confirmed the packet is destined for one of our target ports, our next step is to determine if it's a SYN packet. Here's how we can check for a SYN packet:

// Check if it's a SYN packet
let is_syn_packet = unsafe {
    match ((*tcp_hdr).syn() != 0, (*tcp_hdr).ack() == 0) {
        (true, true) => true,
        _ => false,
    }
};

if !is_syn_packet {
    return Ok(xdp_action::XDP_PASS);
}

Let's break down this code:

We define is_syn_packet by checking two conditions:
- The SYN flag is set ((*tcp_hdr).syn() != 0)
- The ACK flag is not set ((*tcp_hdr).ack() == 0)
A valid SYN packet in the initial TCP handshake should have the SYN flag set and the ACK flag unset.
If the packet is not a SYN packet, we immediately return XDP_PASS, allowing non-SYN packets to continue their normal path through the network stack.

Crafting the SYN-ACK Response

Now that we've identified a SYN packet targeting one of our ports of interest, we'll craft a SYN-ACK response. To do this efficiently, we'll modify the incoming packet in-place, transforming it into our response.

// Swap Ethernet addresses
unsafe { core::mem::swap(&mut (*eth_hdr).src_addr, &mut (*eth_hdr).dst_addr) }
// Swap IP addresses
unsafe {
    core::mem::swap(&mut (*ip_hdr).src_addr, &mut (*ip_hdr).dst_addr);
}

Here's what this code accomplishes:

Ethernet Address Swap:
- We exchange the source and destination MAC addresses in the Ethernet header.
- This ensures our response packet will be routed back to the sender at the link layer.
IP Address Swap:
- Similarly, we swap the source and destination IP addresses in the IP header.
- This directs our response to the original sender at the network layer.
core::mem::swap:
- This function efficiently exchanges the values of two mutable references without requiring a temporary variable.
- It's particularly useful here as it keeps our code concise and performant.

By modifying the existing packet, we're essentially "reflecting" it back to the sender, but with crucial changes that we'll make in the next steps to transform it into a valid SYN-ACK response.

After swapping the Ethernet and IP addresses, we now need to modify the TCP header to transform our packet into a valid SYN-ACK response. This step is critical in simulating an open port and continuing the TCP handshake process. Here's how we accomplish this:

// Modify TCP header for SYN-ACK
unsafe {
    core::mem::swap(&mut (*tcp_hdr).source, &mut (*tcp_hdr).dest);
    (*tcp_hdr).set_ack(1);
    (*tcp_hdr).ack_seq = (*tcp_hdr).seq.to_be() + 1;
    (*tcp_hdr).seq = 1u32.to_be();
}

Let's break down these modifications:

Port Swap:
- We exchange the source and destination ports, ensuring our response goes back to the correct client port.
Setting the ACK Flag:
- We set the ACK flag using set_ack(1). This, combined with the existing SYN flag, creates a SYN-ACK packet.
Acknowledgment Number:
- We set the acknowledgment number to the incoming sequence number plus one.
- This acknowledges the client's SYN and informs it of the next sequence number we expect.
- Note the use of to_be() to handle endianness correctly.
Sequence Number:
- We set our initial sequence number to 1 (converted to network byte order).
- In a real TCP stack, this would typically be a random number for security reasons.

It's important to note that in our eBPF program, we're modifying packet headers without recalculating the checksums. In a production environment, this approach could lead to issues as the modified packets might be dropped by network stacks that verify checksums. For the sake of simplicity and focusing on the core concepts, we've omitted checksum recalculation in this demonstration.

Nmap typically uses raw sockets for its SYN scans. Raw sockets bypass much of the normal network stack processing, including checksum verification in many cases.

Also, in a production environment, you might want to consider additional factors. This is just an oversimplified version that demonstrates the basic concept of tricking a simple SYN scan.

These modifications transform our incoming SYN packet into a SYN-ACK response, effectively simulating the behavior of an open port. This is a key step in our port scanning detection or simulation logic.

Sending the Modified Packet

After modifying our packet to create a valid SYN-ACK response, the final step is to send this packet back to the client. We accomplish this using the XDP_TX action. This action instructs the XDP framework to transmit the modified packet back out through the same network interface it arrived on.

This action is particularly useful for applications like our port scan simulation, load balancers, firewalls, and other scenarios where rapid packet inspection and modification are crucial.

Ok(xdp_action::XDP_TX)

By using XDP_TX, we're completing our port scanning response simulation in a highly efficient manner. This approach allows us to respond to SYN packets almost instantaneously, making our simulated open ports "virtually indistinguishable" from real ones in terms of response time.

It's worth noting that this approach, while effective for Syn Scan scenario, doesn't account for more sophisticated scanning techniques that might use non-standard flag combinations. In a production environment, you might want to implement more comprehensive checks to detect various types of port scans.

Putting It All Together: Running Our eBPF Program

Full code:


fn try_syn_ack(ctx: XdpContext) -> Result<u32, ExecutionError> {
    // Use pointer arithmetic to obtain a raw pointer to the Ethernet header at the start of the XdpContext data.
    let eth_hdr: *mut EthHdr = get_mut_ptr_at(&ctx, 0)?;

    // Check the EtherType of the packet. If it's not an IPv4 packet, pass it along without further processing
    // We have to use unsafe here because we're dereferencing a raw pointer
    match unsafe { (*eth_hdr).ether_type } {
        EtherType::Ipv4 => {}
        _ => return Ok(xdp_action::XDP_PASS),
    }

    // Using Ethernet header length, obtain a pointer to the IPv4 header which immediately follows the Ethernet header
    let ip_hdr: *mut Ipv4Hdr = get_mut_ptr_at(&ctx, EthHdr::LEN)?;

    // Check the protocol of the IPv4 packet. If it's not TCP, pass it along without further processing
    match unsafe { (*ip_hdr).proto } {
        IpProto::Tcp => {}
        _ => return Ok(xdp_action::XDP_PASS),
    }

    // Using the IPv4 header length, obtain a pointer to the TCP header which immediately follows the IPv4 header
    let tcp_hdr: *mut TcpHdr = get_mut_ptr_at(&ctx, EthHdr::LEN + Ipv4Hdr::LEN)?;

    // Check the destination port of the TCP packet. If it's not in the range 9000-9500, pass it along without further processing
    let port = unsafe { u16::from_be((*tcp_hdr).dest) };
    match port {
        9000..=9500 => {}
        _ => return Ok(xdp_action::XDP_PASS),
    }

    // Check if it's a SYN packet
    let is_syn_packet = unsafe {
        match ((*tcp_hdr).syn() != 0, (*tcp_hdr).ack() == 0) {
            (true, true) => true,
            _ => false,
        }
    };

    if !is_syn_packet {
        return Ok(xdp_action::XDP_PASS);
    }

    // Swap Ethernet addresses
    unsafe { core::mem::swap(&mut (*eth_hdr).src_addr, &mut (*eth_hdr).dst_addr) }

    // Swap IP addresses
    unsafe {
        core::mem::swap(&mut (*ip_hdr).src_addr, &mut (*ip_hdr).dst_addr);
    }

    // Modify TCP header for SYN-ACK
    unsafe {
        core::mem::swap(&mut (*tcp_hdr).source, &mut (*tcp_hdr).dest);
        (*tcp_hdr).set_ack(1);
        (*tcp_hdr).ack_seq = (*tcp_hdr).seq.to_be() + 1;
        (*tcp_hdr).seq = 1u32.to_be();
    }

    Ok(xdp_action::XDP_TX)
}

Now that we've implemented our eBPF program to simulate open ports within our chosen range, it's time to see it in action. This demonstration will showcase the power of eBPF in manipulating network behavior at a low level.

First, let's run our eBPF program. Open a terminal and execute the following command:

$ RUST_LOG=info cargo xtask run -- -i wlp5s0
[2024-06-29T19:57:33Z INFO  syn_ack] Waiting for Ctrl-C...

With our eBPF program running, let's perform a port scan using nmap to see the effects:

$ sudo nmap -sS -p9000-9500 192.168.2.107
Starting Nmap 7.95 ( https://nmap.org ) at 2024-06-29 15:57 EDT
Nmap scan report for dlm (192.168.2.107)
Host is up (0.0084s latency).

PORT     STATE SERVICE
9000/tcp open  cslistener
9001/tcp open  tor-orport
9002/tcp open  dynamid
9003/tcp open  unknown
9004/tcp open  unknown
9005/tcp open  golem
9006/tcp open  unknown
9007/tcp open  ogs-client
9008/tcp open  ogs-server
...
9491/tcp open  unknown
9492/tcp open  unknown
9493/tcp open  unknown
9494/tcp open  unknown
9495/tcp open  unknown
9496/tcp open  unknown
9497/tcp open  unknown
9498/tcp open  unknown
9499/tcp open  unknown
9500/tcp open  ismserver

As we can see, Nmap reports all ports in the range 9000-9500 as open. This is exactly what our eBPF program is designed to do: respond to SYN packets on these ports with SYN-ACK, simulating open ports.

In a production environment, you would want to implement additional features such as logging, more sophisticated decision-making logic, and possibly integration with other security systems. This example serves as a foundation for understanding how eBPF can be used to manipulate network traffic at a fundamental level.

For a deeper dive and hands-on experience, all the code discussed is available in my repository. Feel free to explore, experiment, and comments !

To conclude

In this article, we've explored the intricacies of TCP handshakes and port scanning techniques, culminating in a practical demonstration of how eBPF and Rust can be used to manipulate network behavior at a fundamental level. By implementing a program that simulates open ports, we've showcased the power and flexibility of eBPF in network security applications. This approach not only provides a means to confuse potential attackers but also opens up possibilities for more advanced network management and security tools.

Thank you for reading along. This blog is a part of my learning journey and your feedback is highly valued. There's more to explore and share regarding eBPF, so stay tuned for upcoming posts. Your insights and experiences are welcome as we learn and grow together in this domain. Happy coding!

Networking and Sockets: Syn and Accept queue

Douglas Makey Mendez Molero — Fri, 21 Jun 2024 22:31:03 +0000

In my previous article, we discussed endianness, its importance, and how to work with it. Understanding endianness is crucial for dealing with data at the byte level, especially in network programming. We examined several examples that highlighted how endianness affects data interpretation.

One key area we focused on was the relationship between endianness and TCP stream sockets. We explained why the bind syscall expects certain information for the INET family to be in a specific endianness, known as network byte order. This ensures that data is correctly interpreted across different systems, which might use different native endianness.

One of the next interesting topics after binding the socket is putting it into listening mode with the listen syscall. In this article, we are going to dive a little deeper into this next phase.

Listen: Handshake and SYN/Accept Queues

Recall that TCP is connection-oriented protocol. This means the sender and receiver need to establish a connection based on agreed parameters through a three-way handshake. The server must be listening for connection requests from clients before a connection can be established, and this is done using the listen syscall.

We won't delve into the details of the three-way handshake here, as many excellent articles cover this topic in depth. Additionally, there are related topics likeTFO (TCP Fast Open)and syncookies that modify its behavior. Instead, we'll provide a brief overview. If you're interested, we can explore these topics in greater detail in future articles.

The three-way handshake involves the following steps:

The client sends a SYN (synchronize) packet to the server, indicating a desire to establish a connection.
The server responds with a SYN-ACK (synchronize-acknowledge) packet, indicating a willingness to establish a connection.
The client replies with an ACK (acknowledge) packet, confirming it has received the SYN-ACK message, and the connection is established.

Do you remember in the first article when we mentioned that the listen operation involves the kernel creating two queues for the socket. The kernel uses these queues to store information related to the state of the connection.

The kernel creates these two queues:

SYN Queue: Its size is determined by a system-wide setting. Although referred to as a queue, it is actually a hash table.
Accept Queue: Its size is specified by the application. We will discuss this further later. It functions as a FIFO (First In, First Out) queue for established connections.

As shown in the previous diagram, these queues play a crucial role in the process. When the server receives a SYN request from the client, the kernel stores the connection in the SYN queue. After the server receives an ACK from the client, the kernel moves the connection from the SYN queue to the accept queue. Finally, when the server makes the accept system call, the connection is taken out of the accept queue.

Accept Queue Size

As mentioned, these queues have size limits, which act as thresholds for managing incoming connections. If the size limit of the queues are exceeded, the kernel may discard or drop the incoming connection request or return an RST (Reset) packet to the client, indicating that the connection cannot be established.

If you recall our previous examples where we created a socket server, you'll notice that we specified a backlog value and passed it to the listen function. This backlog value determines the length of the queue for completely established sockets waiting to be accepted, known as the accept queue.

// Listen for incoming connections
let backlog = Backlog::new(1).expect("Failed to create backlog");
listen(&socket_fd, backlog).expect("Failed to listen for connections");

If we inspect the Backlog::new function, we will uncover some interesting details. Here's the function definition:

    ...
    pub const MAXCONN: Self = Self(libc::SOMAXCONN);
    ...
    /// Create a `Backlog`, an `EINVAL` will be returned if `val` is invalid.
    pub fn new<I: Into<i32>>(val: I) -> Result<Self> {
        cfg_if! {
            if #[cfg(any(target_os = "linux", target_os = "freebsd"))] {
                const MIN: i32 = -1;
            } else {
                const MIN: i32 = 0;
            }
        }

        let val = val.into();

        if !(MIN..Self::MAXCONN.0).contains(&val) {
            return Err(Errno::EINVAL);
        }

        Ok(Self(val))
    }

We can notice a few things:

On Linux and FreeBSD systems, the minimum acceptable value is -1, while on other systems, it is 0. We will explore the behavior associated with these values later.
The value should not exceed MAXCONN, which is the default maximum number of established connections that can be queued. Unlike in C, where a value greater than MAXCONN is silently capped to that limit, in Rust's libc crate, exceeding MAXCONN results in an error.
Interestingly, in the libc crate, MAXCONN is hardcoded to 4096. Therefore, even if we modify maxconn, this won't affect the listening socket, and we won't be able to use a greater value.

The `SOMAXCONN` Parameter

The net.core.somaxconn kernel parameter sets the maximum number of established connections that can be queued for a socket. This parameter helps prevent a flood of connection requests from overwhelming the system. The default value for net.core.somaxconn is defined by the SOMAXCONN constant here .

You can view and modify this value using the sysctl command:

$ sudo sysctl net.core.somaxconn
net.core.somaxconn = 4096

$ sudo sysctl -w net.core.somaxconn=8000
net.core.somaxconn = 8000

Analyzing the Accept Queue with example

If we run the example listen-not-accept, which sets the backlog to 1 without calling accept, and then use telnet to connect to it, we can use the ss command to inspect the queue for that socket.

# server
$ cargo run --bin listen-not-accept
Socket file descriptor: 3

# telnet
$ telnet 127.0.0.1 6797
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

Using ss:

State                      Recv-Q                     Send-Q                                          Local Address:Port                                            Peer Address:Port                     Process                     
LISTEN                     1                          1                                                   127.0.0.1:6797                                                 0.0.0.0:*

Recv-Q: The size of the current accept queue. This indicates the number of completed connections waiting for the application to call accept(). Since we have one connection from telnet and the application is not calling accept, we see a value of 1.
Send-Q: The maximum length of the accept queue, which corresponds to the backlog size we set. In this case, it is 1.

Testing Backlog Behavior on Linux

We are going to execute the following code to dynamically test the backlog configuration:


fn main() {
    println!("Testing backlog...");
    for backlog_size in -1..5 {
        let server_socket = socket(
            AddressFamily::Inet,
            SockType::Stream,
            SockFlag::empty(),
            SockProtocol::Tcp,
        )
        .expect("Failed to create server socket");

        let server_address =
            SockaddrIn::from_str("127.0.0.1:8080").expect("...");
        bind(server_socket.as_raw_fd(), &server_address).expect("...");

        listen(&server_socket, Backlog::new(backlog_size).unwrap()).expect("...");
        let mut successful_connections = vec![];
        let mut attempts = 0;
        loop {
            let client_socket = socket(
                AddressFamily::Inet,
                SockType::Stream,
                SockFlag::empty(),
                SockProtocol::Tcp,
            ).expect("...");

            attempts += 1;
            match connect(client_socket.as_raw_fd(), &server_address) {
                Ok(_) => successful_connections.push(client_socket),
                Err(_) => break,
            }
        }

        println!(
            "Backlog {} successful connections: {} - attempts: {}",
            backlog_size,
            successful_connections.len(),
            attempts
        );
    }
}

The key functions of the code are:

It iterates over a range of backlog values, creating a server socket and putting it in listen mode with each specific backlog value.
It then enters a loop, attempting to connect client sockets to the server socket. If a connection is successful, it adds the client socket to the successful_connections vector. If it fails, it breaks the loop.

This allows us to determine the exact number of successful connections for each backlog configuration.

When we run the program, we will see output similar to the following. Note that this process may take a few minutes:

$ cargo run --bin test-backlog
Backlog -1 successful connections: 4097 - attempts: 4098
Backlog 0 successful connections: 1 - attempts: 2
Backlog 1 successful connections: 2 - attempts: 3
Backlog 2 successful connections: 3 - attempts: 4
Backlog 3 successful connections: 4 - attempts: 5
Backlog 4 successful connections: 5 - attempts: 6

With the above output you can notice two things:

When listen receive the the min acceptable value -1 it will use the default value of SOMAXCONN, we already know if we pass a value greather than SOMAXCONN it will return an error.
And the other "weird" that can notice is that always the connections are 1 more than the limit set by the backlog, for example, backlog is 3 but there are 4 connections,

The last result is due to the behavior of this function in Linux. The sk_ack_backlog represents the current number of acknowledged connections, while the sk_max_ack_backlog represents our backlog value. For example, when the backlog is set to 3, the function checks if sk_ack_backlog > sk_max_ack_backlog. When the third connection arrives, this check (3 > 3) evaluates to false, meaning the queue is not considered full, and it allows one more connection.

static inline bool sk_acceptq_is_full(const struct sock *sk)
{
    return READ_ONCE(sk->sk_ack_backlog) > READ_ONCE(sk->sk_max_ack_backlog);
}

Handling a Full Accept Queue

As we can see from our program's logs, the client will attempt to connect to the server one more time than the specified backlog value. For example, if the backlog is 4, it will attempt to connect 5 times. In the last attempt, the client and server go through the handshake process: the client sends the SYN, the server responds with SYN-ACK, and the client sends an ACK back, thinking the connection is established.

Unfortunately, if the server's accept queue is full, it cannot move the connection from the SYN queue to the accept queue. The server's behavior when the accept queue overflows is primarily determined by the net.ipv4.tcp_abort_on_overflow option.

Normally, this value is set to 0, meaning that when there is an overflow, the server will discard the client's ACK packet from the third handshake, acting as if it never received it. The server will then retransmit the SYN-ACK packet. The number of retransmissions the server will attempt is determined by the net.ipv4.tcp_synack_retries option.

We can reduce or increase the tcp_synack_retries value. By default, this value is set to 5 on Linux.

In our final attempt, when the retransmission limit is reached, the connection is removed from the SYN queue, and the client receives an error. Again, this is an oversimplified overview of the process. The actual process is much more complex and involves additional steps and scenarios. However, for our purposes, we can keep things simpler.

By running the command netstat -s | grep "overflowed", we can check for occurrences of socket listen queue overflows. For example, in my case, the output indicates that "433 times the listen queue of a socket overflowed," highlighting instances where the system's accept queue was unable to handle incoming connection requests. This can lead to potential connection issues.

$ netstat -s | grep "overflowed"
433 times the listen queue of a socket overflowed

Testing Backlog Behavior on Mac

If we execute the same program on a Mac, we will observe some differences:

As mentioned, for other systems, the minimum acceptable value is 0. Using this value gives us the default setting.
The default value for SOMAXCONN is 128, which can be retrieved using the command sysctl kern.ipc.somaxconn.
Lastly, on a Mac, we don't encounter the issue of allowing Backlog + 1 connections. Instead, we accept the exact number of connections specified in the backlog.

$ cargo run --bin test-backlog
Backlog 0 successful connections: 128 - attempts: 129
Backlog 1 successful connections: 1 - attempts: 2
Backlog 2 successful connections: 2 - attempts: 3
Backlog 3 successful connections: 3 - attempts: 4
Backlog 4 successful connections: 4 - attempts: 5

SYN Queue Size Considerations

As the listen man page states, the maximum length of the queue for incomplete sockets (SYN queue) can be set using the tcp_max_syn_backlog parameter. However, when syncookies are enabled, there is no logical maximum length, and this setting is ignored. Additionally, other factors influence the queue's size. As mentioned earlier: its size is determined by a system-wide setting.

You can find the code for this example and future ones in this repo.

To Conclude

In this article, we explored the role of TCP queues in connection management, focusing on the SYN queue and the accept queue. We tested various backlog configurations and examined how different settings impact connection handling. Additionally, we discussed system parameters like tcp_max_syn_backlog and net.ipv4.tcp_abort_on_overflow, and their effects on queue behavior.

Understanding these concepts is crucial for optimizing server performance and managing high traffic scenarios. By running practical tests and using system commands, we gained insights into how to handle connection limits effectively.

As we move forward, keeping these fundamentals in mind will help ensure robust and efficient network communication.

Thank you for reading along. This blog is a part of my learning journey and your feedback is highly valued. There's more to explore and share regarding socket and network, so stay tuned for upcoming posts. Your insights and experiences are welcome as we learn and grow together in this domain. Happy coding!

Introducing Shelldon: A New Rust CLI Tool with GPT Features

Douglas Makey Mendez Molero — Mon, 17 Jun 2024 15:52:17 +0000

I’m a big fan of tools like Warp and Raycast, especially for their AI capabilities. However, I’ve found their free tiers to be somewhat limiting. Since I already have an OpenAI account, I wanted a solution that would allow me to use my own token. That’s why I created Shelldon.

Shelldon is a command-line tool written in Rust. It provides utilities for executing shell commands, managing prompts, and interacting with multiple LLMs.

Yes, another CLI with GPT features. Shelldon is not intended to be a full GPT client from the terminal; there are a couple of CLIs much better for that and also a lot of applications and even the OpenAI ChatGPT apps. Shelldon is to solve some personal use cases and it is very useful for me; I hope it could be useful for you too. Also, I made it to have fun playing with Rust!

One of the features that other tools were missing for my use case is the ability to use custom prompts for specific tasks that I need. For that reason, I created Shelldon with the ability to manage prompts and use them whenever you want for specific tasks. You can read more about this here. Also, I plan to extend it with plugins to integrate more complex workflows.

And if you are like me and spend more time in the terminal than in other apps, this might help you. I hope some of you find it useful too. I’m open to feedback and suggestions, and I’d love to hear how you might use Shelldon in your own setups.

Installation

Shelldon provides Github releases with prebuilt binaries for MacOS and Linux.

Homebrew

brew tap douglasmakey/tap
brew install shelldon

Building from Source

If you prefer to build Shelldon from source, you can clone the repository and build it using cargo:

git clone https://github.com/douglasmakey/shelldon.git
cd shelldon
cargo build --release

Usage

Shelldon supports different AI providers such as Ollama, OpenAI, Gemini, Anthropic, and Cohere. You can control which provider to use with the --model flag. For example, --model claude-3-haiku-20240307 or --model gemini-1.5-flash-latest. By default, Shelldon uses gpt-4o as the model.

To use Shelldon, you need to set your API keys for the mentioned providers. You can do this by setting an environment variable. Here’s how to set it in your terminal:

export OPENAI_API_KEY="api-key"
export ANTHROPIC_API_KEY="api-key"
export COHERE_API_KEY="api-key"
export GEMINI_API_KEY="api-key"

Shelldon allows you to integrate GPT features into your shell commands easily. Here are some examples to get you started:

Running Shell Commands

$ shelldon exec "Show all the graphics ports for the Vagrant machine using Libvirt." --model gpt-4o
Command to execute: vagrant ssh -c "virsh list --all | grep vagrant | awk '{print \$1}' | xargs -I {} virsh domdisplay {}"
? [R]un, [M]odify, [C]opy, [A]bort ›

Analyzing Docker Logs

Use Shelldon to analyze Docker logs and identify errors:

$ docker logs nginx | shelldon ask "check logs, find errors"

Troubleshooting Kubernetes

Shelldon can help you understand why a Kubernetes pod is failing:

$ k describe pod nginx | shelldon ask "why this pod is failing?"
The pod is failing because it was terminated due to an "Out of Memory" (OOM) condition. The `OOMKilled` reason indicates that the container running in the pod exceeded its memory limit, causing the system to kill the process to prevent it from affecting other processes on the node.

Here are some steps you can take to address this issue:
...

Generate configuration files with the help of GPT:

$ shelldon ask "Create a basic nginx configuration file"
Configuration file content:
server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://localhost:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
...

Automate routine system tasks with ease:

$ shelldon exec "Find and delete all log files older than 30 days in /var/log"
Command to execute: find /var/log -name "*.log" -type f -mtime +30 -exec rm {} \;
? [R]un, [M]odify, [C]opy, [A]bort ›

Get help with writing meaningful Git commit messages:

$ git diff | shelldon ask "Generate a commit message" --copy
"Refactor logging system to improve error handling and performance. This change updates the logging library and adjusts the log levels for better clarity."

You can use the --copy command to copy the output directly to your clipboard.

Handling Prompts

Shelldon allows you to create, edit, list, and delete custom prompts to streamline your command-line workflows. Here’s how you can manage your prompts:

Command Overview

$ shelldon prompts -h
Usage: shelldon prompts <COMMAND>

Commands:
  create  Create a new prompt
  edit    Edit an existing prompt
  list    List all prompts
  delete  Delete an existing prompt
  help    Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

Listing Prompts

To view all the prompts you have created, use the list command:

$ shelldon prompts list

╭────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────╮
│ Name       ┆ Content                                                                                                                                                                                                                                                 ┆ Variables │
╞════════════╪═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╪═══════════╡
│ script     ┆ Let’s think step by step and act as a {script:bash} code scripts expert. Provide only the {script} script code as output without any descriptions or explanations. Ensure the output is in plain text format without Markdown formatting or symbols. If ┆ script    │
│            ┆ details are insufficient, provide the most logical solution. You are not allowed to ask for more details. Just print the script directly.                                                                                                               ┆           │
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ translator ┆ Let’s think step by step and act as a translator. Translate the following text from {from:english} to {to:spanish}. Make it sound natural to a native speaker of {to} while keeping the original tone. Do only minimal edits without changing the tone. ┆ from, to  │
│            ┆ Avoid using fancy words. Reply with only the translated text and nothing else. Do not provide explanations.                                                                                                                                             ┆           │
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ note-taker ┆ I am software engineer and I’d like you to look at the following text I wrote and edit it to make it sound more natural to a native English speaker. Do only minimal/minor edits without changing the tone of the text, which should remain the same.   ┆           │
│            ┆ Dont use fancy words and I want you to only reply the correction, the improvements and nothing else, do not write explanations.                                                                                                                         ┆           │
╰────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────╯

You can use the {} notation to add variables to the prompt, and you can override those values using the --set key=value option. Additionally, in the prompt template, you can define default values like {from:spanish}. This allows for flexible and dynamic prompts that can be customized based on user input.

Then, you can run the ask command with a defined template:

In my daily routine, I often need to generate bash and python scripts or cloud-init files. shelldon helps me with this task:

shelldon ask --prompt script "Generate a cloud-init script to set up an Ubuntu server with the following steps:
1. Update Ubuntu.
2. Install Nginx.
3. Create a custom HTML file to be served by Nginx.
4. Ensure Nginx is enabled and started." --set script=cloud-init > cloud-init

As you can see, we can redirect the output directly to a file to create the script.

Or some translations.

alias sat="shelldon ask --prompt translator"
sat "Hey guys, I'm a few minutes late for the meeting, in less than 5 minutes I'll be there."
Hola chicos, voy unos minutos tarde para la reunión, en menos de 5 minutos estaré ahí.

You can also modify the values for the template:

alias sat="shelldon ask --prompt translator"
sat "Chicos voy a llegar 5 minutos tarde a la reunion" --set to=english --set from=spanish
Guys, I'm going to be 5 minutes late to the meeting.

So the ability to handle dynamic prompts with args and use them makes Shelldon a useful tool for me.

Shelldon

Networking and Sockets: Endianness

Douglas Makey Mendez Molero — Fri, 14 Jun 2024 21:11:45 +0000

In my previous article, we introduced some basic concepts about networking and sockets. We discussed different models and created a simple example of network communication using sockets, specifically starting with Stream TCP sockets.

We started with this socket type and family because it is one of the most common implementations and the one that the majority of developers work with most frequently. Stream TCP sockets are widely used due to their reliability and ability to establish a connection-oriented communication channel. This makes them ideal for scenarios where data integrity and order are crucial, such as web browsing, email, and file transfers. By understanding the fundamentals of Stream TCP sockets, developers can build robust network applications and troubleshoot issues more effectively.

In this article, we will continue working with this socket type and dive a bit deeper into socket programming. So far, we have seen how to create a listener socket as a server and how to use another socket to connect to it. Now, we will explore one key detail about this communication, the endianness!

Endianness

Endianness refers to the order in which bytes are arranged in memory. In a big-endian system, the most significant byte is stored at the smallest memory address, while in a little-endian system, the least significant byte is stored first. Understanding endianness is "crucial" in socket programming because data transmitted over a network may need to be converted between different endianness formats to ensure proper interpretation by different systems. This conversion ensures that the data remains consistent and accurate, regardless of the underlying architecture of the communicating devices.

In this example, a 32-bit value 1,200,000 (hex 0x00124F80) shows endianness. In a big-endian system, bytes are stored as 0x00 0x12 0x4F 0x80 from lowest to highest address. In a little-endian system, the order is reversed: 0x80 0x4F 0x12 0x00.

                        Endianness
              (Order of bytes in memory)

                    Big-Endian System
+-----------------+-----------------+-----------------+------------------+
| Most Significant|                 |                 | Least Significant|
| Byte            |                 |                 | Byte             |
| (MSB)           |                 |                 | (LSB)            |
+-----------------+-----------------+-----------------+------------------+
|      0x00       |      0x12       |      0x4F       |      0x80        |
+-----------------+-----------------+-----------------+------------------+
| Address: 0x00   | Address: 0x01   | Address: 0x02   | Address: 0x03    |
+-----------------+-----------------+-----------------+------------------+

                   Little-Endian System
+------------------+-----------------+-----------------+------------------+
| Least Significant|                 |                 | Most Significant |
| Byte             |                 |                 | Byte             |
| (LSB)            |                 |                 | (MSB)            |
+------------------+-----------------+-----------------+------------------+
|      0x80        |      0x4F       |      0x12       |      0x00        |
+------------------+-----------------+-----------------+------------------+
| Address: 0x00    | Address: 0x01   | Address: 0x02   | Address: 0x03    |
+------------------+-----------------+-----------------+------------------+

Imagine you receive 0x00124F80, and you don’t know if they are in big-endian or little-endian format. If you interpret these bytes using the wrong endianness, you’ll end up with a completely different value. In big-endian format, the most significant byte comes first, so 0x00124F80 remains the same (decimal value: 1,200,000). However, in little-endian format, the bytes are reversed, and the value would be 0x804F1200 (decimal value: 2152665600). This discrepancy can lead to significant errors in data processing, making it essential to handle endianness correctly.

To illustrate the concept of endianness with a simpler decimal example, consider the 32-bit value 574. In a big-endian system, the most significant digits are stored first. For 574, the digits are 5, 7, and 4. In a big-endian system, this would be stored as 500 (5 x 100), 70 (7 x 10), and 4 (4 x 1), in that order. In contrast, a little-endian system would store these digits in reverse order: 4 (4 x 1), 70 (7 x 10), and 500 (5 x 100). This reversal can cause the value to be interpreted incorrectly.

Network byte order

So for that it’s crucial for machines to agree on endianness when communicating to ensure data is interpreted correctly. For example, in internet communication, standards like RFC 1700 and RFC 9293 define the use of big-endian format, also known as network byte order. This standardization ensures that all devices on the network interpret the data consistently, preventing errors and miscommunication.

RFC 1700:
The convention in the documentation of Internet Protocols is to express numbers in decimal and to picture data in "big-endian" order ...

RFC 9293:
Source Address: the IPv4 source address in network byte order
Destination Address: the IPv4 destination address in network byte order

https://datatracker.ietf.org/doc/html/rfc768
https://datatracker.ietf.org/doc/html/rfc1700

Machine agreements

If your application defines a protocol that specifies byte order, both the sender and receiver must adhere to this protocol. If the protocol specifies that certain fields in the data should be in network byte order, then you must convert those fields accordingly. Many application data formats that rely on plain text such as JSON and XML are either byte order independent (treating data as strings of bytes) or have their own specifications for encoding and decoding multi-byte values.

Endianness only applies to multi-byte values, meaning it affects how sequences of bytes representing larger data types (like integers and floating-point numbers) are ordered. Single-byte values are not affected by endianness.

For example, in Rust, the crates bincode and postcard use little-endian by default. This means that when you serialize data using these crates, multi-byte values will be ordered in little-endian format unless specified otherwise.

Socket address for binding

If you have read or watched anything about socket programming, it is almost certain that you have encountered some C examples. For instance, you might have seen something like the following code. This is because, as we learned earlier, IP addresses and ports need to be in big-endian format:

address.sin_family = AF_INET;
address.sin_addr.s_addr = htonl(...);
address.sin_port = htons(PORT);
bind(sockfd, (struct sockaddr *)&address, sizeof(struct address))
...

These functions htons and htonl is part of a series of C helper functions that help convert multi-byte values to the host byte order. These functions ensure that data is correctly interpreted regardless of the underlying system’s endianness.

The htonl() function converts the unsigned integer hostlong from host byte order to network byte order.
The htons() function converts the unsigned short integer hostshort from host byte order to network byte order.
The ntohl() function converts the unsigned integer netlong from network byte order to host byte order.
The ntohs() function converts the unsigned short integer netshort from network byte order to host byte order.

Modern programming languages and frameworks typically handle these formats for you and also offer specialized functions to manage them. For example, if we revisit our previous socket server example in Rust, we will see the following when creating the address:

let server_address = SockaddrIn::from_str("127.0.0.1:8080").expect("...");

In the previous line, we don’t have to deal with byte ordering due to the implementation. However, we can inspect the address to determine the default byte ordering on our machine, which is little-endian in my case. We can also convert it to big-endian, as required for the IP address and port.

// Create a socket address
let sock_addr = SockaddrIn::from_str("127.0.0.1:6797").expect("...");

let sockaddr: sockaddr_in = sock_addr.into();
println!("sockaddr: {:?}", sockaddr);
println!("s_addr Default: {}", sockaddr.sin_addr.s_addr);
// big endian
println!("s_addr be: {:?}", sockaddr.sin_addr.s_addr.to_be());
// little endian
println!("s_addr le: {:?}", sockaddr.sin_addr.s_addr.to_le());

When we run this code, we get the following output:

$ cargo run --bin addr
sockaddr: sockaddr_in { sin_len: 16, sin_family: 2, sin_port: 36122, sin_addr: in_addr { s_addr: 16777343 }, sin_zero: [0, 0, 0, 0, 0, 0, 0, 0] }
s_addr Default: 16777343
s_addr be: 2130706433
s_addr le: 16777343

As we mentioned, although Rust and other modern programming languages handle byte ordering i some levels and offer convenient abstractions, it’s always valuable to understand these foundational concepts.

Practical Example: Handling Endianness in Client-Server Communication

Now that we know about byte ordering, we are going to create a simple example to illustrate what we have covered so far. For this, we will create a simple program that receives a file from a socket. The client will first send the size of the file, and then it will send the data of the file. Note that this will be a straightforward implementation, keeping the code simple to illustrate this concept.

Server

We are going to re-use the code from the last article about creating a socket server of type stream and INET family. We will create a simple function to set up this socket server and return the file descriptor. This will help us maintain a clean implementation and reuse it in all future examples. The function is named create_tcp_server_socket, and it is quite simple, as shown below:

pub fn create_tcp_server_socket(addr: &str) -> Result<OwnedFd, nix::Error> {
    let socket_fd = socket(
        nix::sys::socket::AddressFamily::Inet, // Socket family
        nix::sys::socket::SockType::Stream,    // Socket type
        nix::sys::socket::SockFlag::empty(),
        None,
    )?;

    // Create a socket address
    let sock_addr = SockaddrIn::from_str(addr).expect("...");
    // Bind the socket to the address
    bind(socket_fd.as_raw_fd(), &sock_addr)?;
    // Listen for incoming connections
    let backlog = Backlog::new(1).expect("...");
    listen(&socket_fd, backlog)?;
    Ok(socket_fd)
}

Now we can use that function to create a server socket and accept incoming connections.

fn main() {
    let socket_fd = create_tcp_server_socket("127.0.0.1:8000").expect("...");
    // Accept incoming connections
    let conn_fd = accept(socket_fd.as_raw_fd()).expect("...");
}

Next, after accepting the connection, we expect to receive the size of the file that the client will transmit over the network. We expect to receive a 32-bit unsigned integer, so we prepare a 4-byte buffer to receive the size. We then read from the socket and put the data in the buffer:

// Receive the size of the file
let mut size_buf = [0; 4];
recv(conn_fd, &mut size_buf, MsgFlags::empty()).expect("...");
let file_size = u32::from_ne_bytes(size_buf);
println!("File size: {}", file_size);

If you notice, we are using from_ne_bytes here. This function assumes that the bytes are arranged in the native endianness of our machine (ne stands for native endianness). Therefore, we expect that the value is using the same endianness as our machine.

Finally, we will use that size to create an in-memory buffer to receive the file’s data over the connection and print the bytes read. As mentioned, this is a naive and basic implementation just to illustrate the case:

// Receive the file data
let mut file_buf = vec![0; file_size as usize];
let bytes_read = recv(conn_fd, &mut file_buf, MsgFlags::empty()).expect("...");
println!("File data bytes read: {:?}", bytes_read);

Full code:

use nix::sys::socket::{accept, recv, MsgFlags};
use socket_net::server::create_tcp_server_socket;
use std::os::fd::AsRawFd;
fn main() {
    let socket_fd = create_tcp_server_socket("127.0.0.1:8000").expect("...");
    // Accept incoming connections
    let conn_fd = accept(socket_fd.as_raw_fd()).expect("...");

    // Receive the size of the file
    let mut size_buf = [0; 4];
    recv(conn_fd, &mut size_buf, MsgFlags::empty()).expect("...");
    let file_size = u32::from_ne_bytes(size_buf);
    println!("File size: {}", file_size);

    // Receive the file data
    let mut file_buf = vec![0; file_size as usize];
    let bytes_read = recv(conn_fd, &mut file_buf, MsgFlags::empty()).expect("...");
    println!("File data bytes read: {:?}", bytes_read);
}

Client

For our client, the logic is again pretty simple and similar to what we had before, but with some subtle additions, like reading the file and sending its size before sending the actual data.

... socket creation and connection
// Read the file into a buffer
let buffer = std::fs::read("./src/data/data.txt").expect("Failed to read file");

// send the size of the file to the server
let size: u32 = buffer.len() as u32;
send( socket_fd.as_raw_fd(), &size.to_ne_bytes(), MsgFlags::empty()).expect("...");

// send the file to the server
send(socket_fd.as_raw_fd(), &buffer, MsgFlags::empty()).expect("...");

Two things to notice: as we mentioned earlier, we create a u32 for the file size and we are using the function to_ne_bytes to send bytes arranged in the native endianness, which in my case is little-endian.

The data.txt file is a simple text file containing some lorem ipsum data. We can inspect its size using the stat command:

$ stat -c%s ./src/data/data.txt 
574

If you are wondering why we have to deal with endianness for the size but not for the file content itself, remember what we learned earlier:

Many application data formats that relies on plain text, are either byte order independent (treating data as strings of bytes) or ...

Running

Now we can run our server and client to see the output and understand how it works. The size of the file is 574 bytes, and we send the entire file.

# Server
$ cargo run --bin tcp-file-server
Socket file descriptor: 3
Socket bound to address: 127.0.0.1:8000
File size: 574
File data bytes read: 574

# Client
$ cargo run --bin tcp-file-client
Socket file descriptor: 3
Sending file size: 574
Sending file data

So far, so good, right? The behavior is as expected: we are sending a file with 574 bytes, and we are receiving that in the server. This works because both the client and server are using ne (native endianness), and since they are on the same machine, they are both using little-endian. But what if the client uses a different endianness? What if it sends the size using big-endian, for example?

We can simulate what will happen by modifying this line in the client to instruct it to send the data in big-endian instead of little-endian. For that, we use to_be_bytes (be stands for big-endian):

send( socket_fd.as_raw_fd(), &size.to_be_bytes(), MsgFlags::empty()).expect("...");

If we run our programs again, we will see why understanding endianness is important. From the client’s perspective, we are sending the same value, just in a different endianness. However, if you look at our server, you will notice the issue:

# Client
$ cargo run --bin tcp-file-client
Socket file descriptor: 3
Sending file size: 574
Sending file data

The server still treats the size as if it is coming in the same endianness, which is little-endian. But it is not little-endian anymore; it is big-endian. This results in a totally different and incorrect value. In this simple case, it causes the server to allocate much more memory than needed for the buffer to receive the file. You can imagine that in more complex scenarios, this issue could have a much larger and more serious impact.

# Server
$ cargo run --bin tcp-file-server
Socket file descriptor: 3
Socket bound to address: 127.0.0.1:8000
File size: 1040318464
File data bytes read: 574

And if we were using our native OS bit size, like a 64-bit integer in my case, and used a u64 type for the size variable in both the client and server, the issue could be even worse. We can modify the following lines in the client and server and see the result:

// server
let mut size_buf = [0; 8];
...
let file_size = u64::from_ne_bytes(size_buf);

// client
let size: u64 = buffer.len() as u64;

After making these changes, let’s run the server and client again:

# Server
$ cargo run --bin tcp-file-server
Socket file descriptor: 3
Socket bound to address: 127.0.0.1:8000
File size: 4468133780304953344
memory allocation of 4468133780304953344 bytes failed
Aborted (core dumped)

In this case, the multi-byte representation of the integer is larger, and the value representation becomes colossal in the wrong byte order. Here, the server tries to allocate about 4.4 exabytes of memory, which is far more than what’s available on any current machine. This illustrates how critical it is to handle endianness correctly, as incorrect handling can lead to severe issues like memory allocation failures and program crashes.

You can find the code for this example and future ones in this repo.

To Conclude

Understanding endianness is crucial for developing robust network applications. As demonstrated, even a simple task like sending a file over a network can lead to significant issues if endianness is not handled correctly. By adhering to standards and properly managing byte order, we ensure that data is accurately interpreted across different systems, preventing errors and enhancing the reliability of our applications. Modern programming languages like Rust provide helpful abstractions, but a solid grasp of these foundational concepts allows developers to troubleshoot and optimize their code more effectively. Always be mindful of endianness when working with multi-byte values in network communication to avoid potential pitfalls.

Getting Started with Networking and Sockets

Douglas Makey Mendez Molero — Fri, 07 Jun 2024 20:02:59 +0000

In the realm of software development and network engineering, understanding the fundamentals of sockets and networking is invaluable, regardless of your specific focus within the industry. This article aims to provide a comprehensive overview of these essential concepts, facilitating a clearer understanding of their importance and functionality for engineers. Let's delve into the basics and uncover the foundational principles that drive modern networking.

To keep things simple, we will start with basic examples and gradually introduce more complexity. This approach allows us to build a strong foundation without getting overwhelmed. In future articles, we will explore more advanced topics.

Understanding the OSI and TCP/IP Models: Foundations of Networking

OSI model

We can't start discussing basic networking concepts without mentioning our old friend, the OSI model. For two computers to communicate effectively, they need to speak the same language, structured by the OSI model. This foundational framework has been a cornerstone in understanding and implementing network communications. By breaking down the complex process of data transmission into seven manageable layers, the OSI model provides a clear and organized approach to networking that has stood the test of time.

The OSI model simplifies the complex process of data transmission by providing a universal language for networking. Each layer of the OSI model has specific responsibilities and capabilities, ensuring that technologies can interact seamlessly. Higher layers benefit from this abstraction, utilizing lower-layer functions without needing to understand their inner workings.

OSI model layers

Physical Layer: This is the lowest layer, dealing with the physical connection between devices. It is responsible for transmitting raw bit streams over a physical medium, such as cables or wireless signals.
Data-Link Layer: This layer manages the transfer of data between two directly connected nodes. It handles error detection and correction, as well as flow control, ensuring reliable communication over the physical medium.
Network Layer: This layer is responsible for routing data from the source to the destination across multiple networks. It uses logical addressing (such as IP addresses) to determine the best path for data transmission.
Transport Layer: This layer ensures reliable data transfer between systems. It provides error detection and recovery, flow control, and ensures complete data transfer through protocols like TCP (Transmission Control Protocol).
Session Layer: This layer manages sessions between applications. It establishes, maintains, and terminates connections, ensuring data is synchronized and properly sequenced.
Presentation Layer: This layer translates data between the application layer and the network. It handles data encryption, compression, and translation, ensuring that data is presented in a readable format.
Application Layer: This is the topmost layer, which interacts directly with user applications. It provides network services to end-user applications, such as web browsers and email clients.

Have you ever heard the joke about Layer 8? If not, for those who enjoy a bit of networking humor, there's often mention of "Layer 8" — the user layer. It's a playful reminder that no matter how perfect the technical setup, the human element can introduce its own unique challenges!

When data is communicated through protocol layers, it's sent in small segments called packets. Each packet includes implementations from these protocol layers. Starting at the application layer, the data is encapsulated by the presentation layer, which is then encapsulated by the session layer, followed by the transport layer, and so on. This process is known as encapsulation.

Each layer adds a header and a body to the packet. The header contains protocol-specific information necessary for that layer, while the body contains the encapsulated data, including headers from the previous layers. This encapsulation technique can be visualized like the layers of an onion.

The illustration below demonstrates the encapsulation process as data moves from the application layer of one computer, across the Internet, to the application layer of the other computer.

   Computer A Application                Computer B Application
       |                                      ▲
       ▼                                      |
   +---------------------+                +---------------------+
   | 7. Application      |                | 1. Physical         |
   +---------------------+                +---------------------+
       |                                      |
   +---------------------+                +---------------------+
   | 6. Presentation     |                | 2. Data-Link        |
   +---------------------+                +---------------------+
       |                                      |
   +---------------------+                +---------------------+
   | 5. Session          |                | 3. Network          |
   +---------------------+                +---------------------+
       |                                      |
   +---------------------+                +---------------------+
   | 4. Transport        |                | 4. Transport        |
   +---------------------+                +---------------------+
       |                                      |
   +---------------------+                +---------------------+
   | 3. Network          |                | 5. Session          |
   +---------------------+                +---------------------+
       |                                      |
   +---------------------+                +---------------------+
   | 2. Data-Link        |                | 6. Presentation     |
   +---------------------+                +---------------------+
       |                                      |
   +---------------------+                +---------------------+
   | 1. Physical         |                | 7. Application      |
   +---------------------+                +---------------------+
       |                                      |
       +--------------------------------------+
       |              Internet                |
       +--------------------------------------+

TCP/IP as an Alternative to the OSI Model

While the OSI model provides a detailed framework for understanding network communications, TCP/IP is the more practical and widely-used model in real-world networking. TCP/IP simplifies the structure into four layers and serves as the foundation for the Internet, emphasizing robust, scalable communication across diverse networks.

Imagine you want to visit a website by entering a URL into your browser. Here’s how TCP/IP handles this request:

Application Layer: Your web browser (application) generates an HTTP request for the webpage. In the TCP/IP model, the Application layer combines the functions of the Application, Presentation, and Session layers of the OSI model. This means it handles not only the user interface and communication protocols (Application layer in OSI), but also data translation, encryption, and compression (Presentation layer in OSI), as well as managing sessions and dialogues between the devices (Session layer in OSI). This consolidation simplifies the model and aligns it more closely with real-world protocols and applications.
Transport Layer: The HTTP request is handed to the TCP protocol, which breaks it into smaller packets, adds a header with port information, and ensures reliable transmission. This layer matches the Transport layer in the OSI model.
Internet Layer: Each packet is then passed to the IP protocol, which adds another header containing the IP addresses of both the source (your computer) and the destination (the web server). This layer corresponds to the Network layer in the OSI model.
Network Access Layer: The packets are then sent over the physical network (like Ethernet or Wi-Fi), where they travel through various routers and switches to reach the web server. This layer combines the Data Link and Physical layers of the OSI model.

Both the OSI and TCP/IP models are open standards. They’re designed so that anyone can use them, or further build them out to meet specific requirements.

Programs that use networking, such as web browsers and email clients, rely on the operating system to manage network communications. The operating system handles the intricate details of network encapsulation, making it easier for developers to write network programs. They simply need to use the network interface provided by the OS, without worrying about the underlying complexities.

Using Sockets for communication

To interface with the network or enable inter-process communication (IPC), developers often use sockets. Sockets serve as standardized endpoints for sending and receiving data, whether across a network or between processes on the same machine. They provide a way for programs to communicate, abstracting the complexities of the underlying protocol layers. By using socket APIs provided by the operating system, developers can focus on building their application logic while the OS manages the detailed network and IPC operations. This makes tasks like creating a web server, a chat application, or facilitating communication between processes more straightforward and accessible. The most common types of sockets are stream sockets, which provide a reliable connection-oriented service, and datagram sockets, which offer a connectionless service.

For IPC communication offered by the Unix domain socket using the AF_UNIX family, we already explore this in the article below. We may explore it again from another perspective in future parts of this article.
https://www.kungfudev.com/blog/2022/12/05/understanding-unix-domain-sockets-in-golang

Stream sockets and datagram sockets are the two most common types of sockets. Stream sockets, which use the TCP protocol, provide a reliable, connection-oriented service. This means that data is transmitted in a continuous stream, ensuring that it arrives in the correct order and without errors. Examples of stream sockets in action include web servers, where the integrity and order of the data (such as HTML pages) are crucial for proper rendering, and email clients, which require reliable data transmission to ensure messages are received intact. In contrast, datagram sockets use the UDP protocol and offer a connectionless service. This allows for faster data transmission but without the guarantees of order and reliability. An example of datagram sockets is in online gaming or live video streaming, where speed is more critical than perfect data accuracy, and occasional data loss is acceptable.

I would bet that at least once you have seen the image below while learning about networking, sockets, or reading some network articles. This diagram illustrates the basic flow of socket communication between a server and a client, highlighting the key function calls and interactions involved in establishing a connection and exchanging data.

Sockets behave similarly to files because of the Unix philosophy of treating everything as a file. This design choice provides a consistent interface for performing input and output operations across different types of data streams, simplifying the programming model.

Sockets and files both provide a stream of bytes that can be read from or written to. This stream abstraction fits well with many types of I/O operations, whether they involve local files, remote network communication, or inter-process communication.

When a socket is created with the socket() function, it requires parameters such as the domain (e.g., AF_INET for IPv4), the type (e.g., SOCK_STREAM for TCP), and the protocol (usually 0 to select the default protocol for the given type). The socket is then assigned a file descriptor, an integer that uniquely identifies the socket within the operating system.

A file descriptor is a unique integer assigned by the operating system to an open file or socket, serving as an abstract indicator for accessing various I/O resources, such as files, sockets, and pipes. In simple terms, when you open a file, the operating system creates an entry to represent that file and stores information about it. If there are N files open, there will be N corresponding entries in the operating system, each represented by an integer like 20, 21, etc. This integer is the file descriptor. It uniquely identifies an open file within a process, allowing the process to perform operations like reading, writing, and closing files using standardized functions. By managing resources this way, the operating system ensures efficient communication between processes and I/O resources.

More info

The bind() function associates the socket with a specific local address and port, which are provided as arguments. The listen() function marks the socket as a passive socket that will be used to accept incoming connection requests, taking an argument that specifies the maximum number of pending connections. The accept() function extracts the first connection request on the queue of pending connections, creating a new socket file descriptor for the connection.

On the client side, the connect() function is used to establish a connection to the server, requiring the server's address and port as arguments. Both the client and server can then use send() and recv(), or the analogous write() and read(), to transmit and receive data. The close() function is used to close the socket, releasing the file descriptor.

This design simplifies the API and makes it easier for developers to handle network communication using familiar file operations, streamlining the development process and making the code more intuitive and maintainable. By treating sockets as files, the operating system can efficiently manage various types of I/O operations using a unified interface, leveraging existing mechanisms for buffering, blocking, and error handling that are well-established for file systems.

As we mentioned, there are a couple of socket types and families, but for now, for socket families we are going to focus on AF_INET for IPv4, AF_INET6 for IPv6, and as we mentioned the AF_UNIX for local communication within the same host. And regarding type, we will discuss the two primary types of sockets: stream sockets and datagram sockets.

Socket in Actions

To illustrate what we have learned so far, we will write a couple of programs in Rust. We will use the nix crate, which provides friendly unix platform APIs (Linux, Darwin) for working with the socket API. While the Rust standard library offers standard socket functionality in the std, the nix crate provides more comprehensive and idiomatic access to lower-level unix system calls, making it easier to work with advanced socket features and perfect for explaining what we have seen until now.

Server

So as we saw previously, to create a server the sequence of functions are: socket(), bind(), listen(), and accept(). The first step is to create the socket.

let socket_fd = socket(
    nix::sys::socket::AddressFamily::Inet, // Socket family
    nix::sys::socket::SockType::Stream,    // Socket type
    nix::sys::socket::SockFlag::empty(),
    None,
)
.expect("Failed to create socket");

This code snippet creates a new socket using the nix crate in Rust. The socket() function call includes several parameters:

nix::sys::socket::AddressFamily::Inet: Specifies the socket family, in this case, AF_INET, which is used for IPv4 addresses.
nix::sys::socket::SockType::Stream: Specifies the socket type, SOCK_STREAM, which indicates a stream socket using the TCP protocol.
nix::sys::socket::SockFlag::empty(): Indicates that no special flags are set for the socket.
None: Indicates that the default protocol should be used.

After creating the socket, the next step is to bind the socket to a specific address and port. This associates the socket with a particular local endpoint.

// Create a socket address
let sock_addr =
    SockaddrIn::from_str("127.0.0.1:6797").expect("Failed to create socket address");

// Bind the socket to the address
bind(socket_fd.as_raw_fd(), &sock_addr).expect("Failed to bind socket");

This code snippet binds the previously created socket to the local address 127.0.0.1 (localhost) and port 6797:

SockaddrIn::from_str("127.0.0.1:6797"): Creates a new socket address (SockaddrIn) from the string representation of the IPv4 address and port.
bind(socket_fd.as_raw_fd(), &sock_addr): Binds the socket file descriptor to the specified address. This makes the socket listen for incoming connections on 127.0.0.1:6797.

Something to notice here is that we are using IP addresses and ports in a specific format because we are using AddressFamily::Inet. Different protocol families have their own ways of defining endpoint addresses. This means the address format can vary depending on the address family, allowing sockets to handle different networking protocols and address formats properly. For this article, we are focusing on the INET family, but in future articles, we will explore other address families in more detail.

After binding the socket to a specific address and port, the next step is to listen for incoming connections. This prepares the socket to accept connection requests from clients.

// Listen for incoming connections
// The backlog parameter specifies the maximum length of the queue of pending connections
let backlog = Backlog::new(1).expect("Failed to create backlog");
listen(&socket_fd, backlog).expect("Failed to listen for connections");

This code snippet sets up the socket to listen for incoming connections:

let backlog = Backlog::new(1).expect("Failed to create backlog");: This line creates a backlog object that specifies the maximum length of the queue of pending connections. In this case, the backlog is set to 1, meaning the socket can queue up to one pending connection.
listen(&socket_fd, backlog).expect("Failed to listen for connections");: This line calls the listen() function, which puts the socket into listening mode.

We use a backlog of 1 because we are keeping the example simple and synchronous for now. In the future, we will introduce an asynchronous runtime to handle multiple connections more efficiently.

The listen operation involves the kernel creating two queues for this socket: the syn queue and the accept queue. To keep this article concise, we will explore these queues in detail in the next article.

At this point, the server socket is ready and waiting for incoming connection requests from clients. The listen() function allows the server to queue incoming connections, which can be accepted and processed one by one.

Once the server socket is set to listen for incoming connections, the next step is to accept these connections and handle the data communication with the client.

// Accept incoming connections
let conn_fd = accept(socket_fd.as_raw_fd()).expect("Failed to accept connection");
// Read data
let mut buf = [0u8; 1024];
let bytes_read = read(conn_fd, &mut buf).expect("Failed to read from connection");
let received_data =
    std::str::from_utf8(&buf[..bytes_read]).expect("Failed to convert received data to string");
println!(
    "Received {} bytes: {:?} repr: {}",
    bytes_read,
    &buf[..bytes_read],
    received_data
);

This code snippet demonstrates how to accept an incoming connection and read data from it:

let conn_fd = accept(socket_fd.as_raw_fd()).expect("Failed to accept connection");: This line calls the accept() function to accept an incoming connection. It creates a new socket file descriptor for the connection, conn_fd.
let mut buf = [0u8; 1024];: This line initializes a buffer to store the incoming data.
let bytes_read = recv(conn_fd, &mut buf, MsgFlags::empty()).expect("Failed to read from connection");: This line reads data from the accepted connection into the buffer. The recv() function is used for this purpose, with MsgFlags::empty() indicating no special flags. The number of bytes read is stored in bytes_read.
let received_data = std::str::from_utf8(&buf[..bytes_read]).expect("Failed to convert received data to string");: This line converts the received byte data into a UTF-8 string. It slices the buffer up to the number of bytes read and converts it.

At this point, the server has accepted an incoming connection and read the data sent by the client. The server can then process this data, echo it back, or perform other actions based on the application logic.

let bytes_written = send(conn_fd, &buf[..bytes_read], MsgFlags::empty())
    .expect("Failed to write to connection");

This code snippet demonstrates how to send data back to the client:

let bytes_written = send(conn_fd, &buf[..bytes_read], MsgFlags::empty()).expect("Failed to write to connection");: This line sends data from the buffer back to the client using the send() function. The buffer is sliced to the number of bytes read from the client, ensuring only the received data is sent back. MsgFlags::empty() indicates no special flags are used.

For simplicity, we are using MsgFlags::empty(), which indicates no special options are set, allowing the send() and recv() functions to operate in their default mode. To read more about these flags .

With this step, the server echoes the received data back to the client, completing a simple round-trip communication. This demonstrates the basic flow of data from client to server and back to client, showcasing the core operations of socket communication.

So putting it all together, we have the complete example for this simple TCP echo. We’ve explained each step, as discussed previously:

fn main() {
    let socket_fd = socket(
        nix::sys::socket::AddressFamily::Inet, // Socket family
        nix::sys::socket::SockType::Stream,    // Socket type
        nix::sys::socket::SockFlag::empty(),
        None,
    )
    .expect("Failed to create socket");

    // Create a socket address
    let sock_addr =
        SockaddrIn::from_str("127.0.0.1:6797").expect("Failed to create socket address");

    // Bind the socket to the address
    bind(socket_fd.as_raw_fd(), &sock_addr).expect("Failed to bind socket");

    // Listen for incoming connections
    let backlog = Backlog::new(1).expect("Failed to create backlog");
    listen(&socket_fd, backlog).expect("Failed to listen for connections");

    // Accept incoming connections
    let conn_fd = accept(socket_fd.as_raw_fd()).expect("Failed to accept connection");

    // echo back the received data
    let mut buf = [0u8; 1024];
    let bytes_read =
        recv(conn_fd, &mut buf, MsgFlags::empty()).expect("Failed ... connection");
    let received_data =
        std::str::from_utf8(&buf[..bytes_read]).expect("Failed to ...");

    // Echo back the received data
    let bytes_written = send(conn_fd, &buf[..bytes_read], MsgFlags::empty())
        .expect("Failed to write to connection");
}

In Rust, we do not explicitly use the close method to close sockets because Rust's ownership and borrowing system automatically handles resource management. When a socket goes out of scope, Rust's memory safety guarantees ensure that the socket is properly closed and resources are freed. This eliminates the need for explicit close calls, reducing the risk of resource leaks and making the code cleaner and safer.

Most programming languages provide higher-level abstractions to simplify socket programming, making it more accessible and easier to use. These abstractions often wrap the underlying system calls, handling details, and resource management for you. For example, in Rust, the std::net module offers a convenient API for TCP networking:

use std::net::TcpListener;
fn main() {
    let listener = TcpListener::bind("127.0.0.1:7878").unwrap();
    for stream in listener.incoming() {
        let stream = stream.unwrap();
        println!("Connection established!");
    }
}

In this example, TcpListener::bind abstracts the complexity of creating and binding a socket, ensuring that the address is correctly formatted ""and byte-ordered."" The incoming method returns an iterator over incoming connections, managing the accept loop. This abstraction makes the code more readable and easier to maintain, allowing us to focus on the application logic rather than the intricacies of socket communication.

Running the server

Now, if we run the server and use telnet from another terminal, we will see this in action:

$ cargo run part-one-server
Socket file descriptor: 3
Socket bound to address: 127.0.0.1:6797
Listening for incoming connections...

received 22 bytes
bytes: [72, 101, 108, 108, 111, 32, 102, 114, 111, 109, 32, 75, 117, 110, 103, 102, 117, 68, 101, 118, 13, 10]
hex repr: ["0x48", "0x65", "0x6c", "0x6c", "0x6f", "0x20", "0x66", "0x72", "0x6f", "0x6d", "0x20", "0x4b", "0x75", "0x6e", "0x67", "0x66", "0x75", "0x44", "0x65", "0x76", "0x0d", "0x0a"]
str repr: "Hello from KungfuDev\r\n"

Sent 22 bytes back to client

This output indicates that the server is running, bound to the address 127.0.0.1:6797, and successfully received and echoed back data from a client.

And in our telnet(client) we can see the data is replied back.

telnet localhost 6797
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Hello from KungfuDev
Hello from KungfuDev
Connection closed by foreign host.

You can find the code in this repository.

What about the client?

As you saw in the diagram that illustrates the flow of socket communication, the client, instead of calling listen, uses connect to initiate a connection to a server socket.

send(socket_fd.as_raw_fd(), data.as_bytes(), MsgFlags::empty())
.expect("Failed to send data to server");

The client uses the connect function to establish a connection with the server. After successfully connecting, the client can send data to the server using the send function and receive data from the server using the recv function.

You can find the code for this example and future ones in this repo.

To conclude

In this article, we explored the fundamentals of socket programming using Rust and the nix crate. We began by understanding the OSI model and its practical application through the TCP/IP model, laying the groundwork for network communication. We then delved into the sequence of functions necessary to create a server: socket(), bind(), listen(), and accept(). By walking through a complete example, we demonstrated how to set up a simple server that listens for incoming connections, receives data, and echoes it back to the client.

Running the provided Rust program and testing it with telnet illustrated these concepts in action, showing how data is transmitted and received over a network. Although we kept the example synchronous and used a small backlog for simplicity, this foundation paves the way for more advanced topics such as asynchronous programming and handling multiple connections efficiently.

Stay tuned for future articles where we will introduce asynchronous runtimes and explore additional and advanced socket programming techniques to enhance our networking skills.

Turning Problems into Solutions: Provide Alternatives, Not Excuses

Douglas Makey Mendez Molero — Mon, 20 May 2024 21:41:07 +0000

In the journey of software engineering, there are moments when challenges seem insurmountable. These are the moments that test our problem-solving skills, resilience, and creativity. However, the way we approach these challenges can make all the difference. Instead of offering excuses, providing viable alternatives can transform a roadblock into a learning opportunity.

The Importance of Mindset

A crucial aspect of being a successful software engineer is adopting a proactive mindset. When faced with a problem, it's easy to resort to excuses:

"The deadline is too tight."
"The technology is too complex."
"The requirements are unclear."

These statements, while potentially true, don't contribute to progress. Instead, they create a culture of stagnation. A proactive mindset shifts the focus from what can't be done to what can be explored.

           ↑ Increase
       +--------------------+        
       |       +Solutions   |        
       |       +Solutions   |        
       |       +Solutions   |        
       |       +Solutions   |        +----------+
       |       +Solutions   |        | -Problems|
       |       +Solutions   |        | -Problems|
       |       +Solutions   |        | -Problems|
       |       +Solutions   |        | -Problems|
       +--------------------+        +----------+
                                      ↓ Decrease

Analyzing the Situation

When you encounter a problem, start by analyzing the situation thoroughly. Here are some steps to guide you:

Understand the Problem: Take the time to fully understand the problem. Break it down into smaller, manageable parts.
Identify Constraints: Recognize the constraints you are working within, whether they are time, resources, or technical limitations.
Explore Alternatives: Brainstorm possible solutions. Think outside the box and consider unconventional approaches.

Simple Example Scenarios

Obviously, these examples are quite simple, but they help us get a clear idea of what the correct mindset should be.

Tight Deadlines

Situation: The client requests a new feature, but the deadline is tight.
Response with Excuses: "We can't meet this deadline. It's impossible."
Response with Options: "Meeting this deadline is challenging, but we can prioritize the core functionality and add additional features in the next iteration. Alternatively, we can extend the deadline by a week to deliver the complete feature set."

Technical Limitations

Situation: A particular technology doesn't support a requested feature.
Response with Excuses: "This technology doesn't support that feature. We can't do it."
Response with Options: "While this technology doesn't support the feature directly, we can use a workaround or integrate another tool that provides the required functionality."

By providing these alternatives, you are not only demonstrating a solution-oriented mindset but also encouraging a culture of continuous learning and collaboration.

Beyond Alternatives and Excuses

Providing alternatives, not excuses, is more than just offering solutions. It's about how you handle issues or problems.

At the beginning of my career, I was one of those people who, these days, I wouldn't be happy to work with. I was the I told you so guy. I remember approaching the team and mentioning phrases that no one wanted to hear. My boss would say, I don't want you to bring me a problem or say there is a problem when it's clear there is one. I want you to bring possible solutions.

In many cases, I knew we had a problem, and I had possible solutions or something to try, but my communication always started with the problem and not with the solutions. The same principle applies here. We should focus on presenting potential solutions first and foremost.

My great friend Patrick D'appollonio , one of the best engineers I have known and from whom I have the fortune to learn, has always told me: If you come with problems, at least bring a solution. This shows that you are interested in solving the problems and not just criticizing or pointing out something obvious! This advice has stayed with me throughout my career, reinforcing the importance of being proactive and solution-oriented.

I Don't Know

The main idea behind this concept is not to have an answer or proposal for every situation or question immediately. As we often hear, it's okay to say I don't know, but this should be followed with but I will figure it out. The key is to actively involve yourself in the process of finding a solution. Saying you'll figure it out means committing to understanding the problem, researching possible solutions, and engaging with your team to develop and propose viable options.

Relating to the Broken Windows Theory

In my previous article on the Broken Windows theory, we discussed the importance of maintaining quality and addressing small issues before they escalate. This concept ties closely to the mindset of providing alternatives and not excuses. Both approaches emphasize the importance of proactive problem-solving and a commitment to continuous improvement.

When we focus on solutions rather than excuses, we prevent small issues from becoming major problems. Just as the Broken Windows theory suggests that fixing minor issues can prevent larger ones, providing viable alternatives ensures that we address challenges head-on and maintain the integrity of our work. For example, giving excuses like I will revisit this later without the commitment to actually do it can lead to accumulated technical debt. By providing solutions and following through, we maintain a higher standard of quality and reliability in our projects.

Communication is Key

Effective communication is vital when presenting alternatives. Here’s how to ensure your suggestions are well-received:

Be Clear and Concise: Clearly articulate the problem and your proposed solutions.
Provide Evidence: Back up your suggestions with data or examples from past experiences.
Be Open to Feedback: Encourage feedback and be prepared to iterate on your solutions based on input from others.

Building a Resilient Team

As a senior engineer or team leader, it's important to foster an environment where team members feel empowered to suggest alternatives. Encourage open discussions, celebrate innovative ideas, and provide constructive feedback. This not only builds a resilient team but also instills a problem-solving mindset in junior engineers.

Conclusion

In software engineering, challenges are inevitable. However, the way we respond to these challenges defines our growth and success. By providing alternatives instead of excuses, we pave the way for innovation, learning, and progress. Let’s commit to being solution-oriented, proactive, and collaborative in our approach.

Remember, the next time you face a daunting task, think of alternatives. Your proactive approach could be the catalyst for a breakthrough solution.

The Broken Window in Software Projects

Douglas Makey Mendez Molero — Sat, 18 May 2024 22:42:50 +0000

In software development, the Broken Window theory suggests that small issues, if left unfixed, can lead to bigger problems. Imagine a codebase where minor bugs or style inconsistencies are ignored. Over time, these small problems accumulate, making the code harder to maintain and increasing the likelihood of critical errors. However, this theory extends beyond just code issues. It also encompasses bad design choices, poor architectural decisions, and ineffective processes, all of which can degrade the overall quality and maintainability of a software project.

What is the Broken Window Theory? The Broken Window Theory originates from criminology and urban studies, suggesting that visible signs of disorder and neglect (like broken windows) lead to more disorder and crime. Applied to software engineering, it means that neglecting small issues in the codebase can lead to a general decline in code quality and discipline, inviting more significant problems.

Technical Debt

When broken windows are left unattended, they accumulate as technical debt. This debt represents the future cost of rework and refactoring required to address these issues. Just like financial debt, technical debt accrues interest, making the codebase more difficult to work with over time. High technical debt can slow down development, introduce more bugs, and increase the cost of implementing new features.

Simple Example Scenarios

Hacky Solutions for Deadlines

When a project deadline is close, you might implement a hacky solution just to get things working, promising to revisit and fix it later. This often doesn’t happen, and the quick fix becomes part of the codebase, increasing technical debt.

How many times have you been in this situation or made that promise and never looked at that code again? You eventually accept it as part of the program, forgetting the commitment to review and improve it.

Inconsistent Coding Practices

You notice that the code you're writing doesn't fit well with the existing code style or architecture, but you leave it as is, thinking you'll revisit it later. This inconsistency can lead to confusion and errors for future developers working on the code.

You're not a team of one, and even if you are now, it won't always be that way. You must advocate for the practices and standards defined as a team. Everyone should be rowing in the same direction; after all, you're all in the same boat.

Beyond Code: Design and Decision-Making

Design Flaws and Poor Decisions: Broken windows are not limited to just code. They can also appear as bad design choices, poor architectural decisions, or ineffective processes. For example, an ill-thought-out system architecture that doesn't scale well or a design that doesn't consider user experience can be just as detrimental as poor code quality.

Poor Implementation Practices: Even if the design is solid, poor implementation can introduce broken windows. Rushed or sloppy coding, lack of adherence to best practices, or insufficient testing can all contribute to technical debt and a deteriorating codebase.

The Cost of Broken Windows

A few years ago, I joined a small team of engineers working on a project where the initial team had chosen a programming language that no one was proficient in. The decision to use this language was driven by its popularity at the time, not by the team's expertise. This poor decision led to many broken windows in the project. The team rushed through development, leaving many issues unresolved. While the project technically worked, taking it to the next level required a lot of effort. By systematically addressing these broken windows and pushing best practices, we managed to improve the project. However, the time and effort spent fixing these issues were significantly higher than if they had been addressed from the beginning.

Key Takeaway

Regularly address small issues to maintain code quality and prevent larger problems from developing. Keep your codebase clean and organized to minimize technical debt and ensure long-term project success. Create tickets for these issues and enforce you and your team to solve them. Give priority to these tickets to ensure that small problems do not accumulate and become more significant obstacles.

Conclusion

Maintaining a clean and organized codebase is crucial for long-term project success. The Broken Window theory teaches us that small issues, if left unaddressed, can lead to significant problems, including increased technical debt. By understanding and addressing both code-level and design-level broken windows, you can prevent the accumulation of technical debt and ensure a more reliable, maintainable, and scalable software project. Prioritize these issues, create actionable tickets, and foster a culture of continuous improvement within your team. This proactive approach will lead to more efficient development and higher-quality software.

Forem: Douglas Makey Mendez Molero

MU Online in Rust: Hardening the Foundation, runtime, Timeouts, and Packet Size Limits

Introducing mu-runtime

Timeouts: The Idle Client Problem

Implementing Timeouts in PacketStream

Seeing It in Action

Max Packet Size: Starving the Allocator

Seeing It in Action

Where We Are Now

From Nostalgia to Code - Building a MU Online Server in Rust

From Nostalgia to Code: Building a MU Online Server in Rust

Encryption is Asymmetric by Direction

What is XOR32

C1 00 01 - The Initial Hello!

Defining the Scope

Architecture Overview

ConnectServer - Discovery/Entry Phase

GameServer - Gameplay Phase

Let's Start Building

The Protocol Crate

The Codec

The TCP Servers

Handling the Client Connection

Parsing Packets

Building Responses

The Result

OSDev Bare Bones with Rust - Cross-Compilation and Freestanding

Cross-Compilation

What Is a Cross-Compiler?

Why Cross-Compilers Are Necessary

Freestanding and Hosted Environments

Hosted Environment

Freestanding Environment

Recap

Zig to the Rescue?

Rust in Action

Panic Handler

Bonus: Lima Setup for Mac Users

Playing with Rust: Building a Safer rm and Having Fun Along the Way

Introducing rrm: The Command-Line Tool Nobody Asked For

Extended Attributes: Storing Metadata Without Changing the File

What Are Extended Attributes?

Mocking in Rust: Exploring mockall for Testing

To conclude

Simple thoughts

Fooling Port Scanners: Simulating Open Ports with eBPF and Rust

Understanding the TCP Three-Way Handshake in Port Scanning

Server with a listening socket on the targeted port

Server without a listening socket on the targeted port

The Stealth SYN Scan: A Popular Port Scanning Technique

Nmap in Action: Demonstrating SYN Scans

Crafting Our Defense: eBPF and Rust Implementation

Setting Up the eBPF Program

Filtering Packets: Targeting Specific Ports

Identifying SYN Packets

Crafting the SYN-ACK Response

Sending the Modified Packet

Putting It All Together: Running Our eBPF Program

To conclude

Networking and Sockets: Syn and Accept queue

Listen: Handshake and SYN/Accept Queues

Accept Queue Size

The SOMAXCONN Parameter

Analyzing the Accept Queue with example

Testing Backlog Behavior on Linux

Handling a Full Accept Queue

Testing Backlog Behavior on Mac

SYN Queue Size Considerations

To Conclude

Introducing Shelldon: A New Rust CLI Tool with GPT Features

Installation

Homebrew

Building from Source

Usage

Running Shell Commands

Handling Prompts

Networking and Sockets: Endianness

Endianness

Network byte order

Machine agreements

Mocking in Rust: Exploring `mockall` for Testing

The `SOMAXCONN` Parameter