The Nitro Enclave Gotcha That Cost Me 90 Minutes: vsock and a Port That Lied

Muhammad Ablugg — Sat, 09 May 2026 06:08:09 +0000

When you are building inside AWS Nitro Enclaves for the first time, the documentation gives you a clean mental model: parent EC2 instance communicates with the enclave over vsock, the enclave runs your application, everything is isolated and tidy. What the documentation does not tell you is that "connected" and "live" are two different things, and conflating them will cost you time.

This is the story of a silent failure I hit while building Mizan, a legal AI platform running inside Nitro Enclaves where attorney-client privileged data is processed entirely in hardware-isolated memory. The bug took about 90 minutes to resolve. Writing it up takes less than 10 minutes to read. Hopefully this saves you the 90.

The Setup

The architecture is straightforward: an AWS Nitro Enclave runs a raw vsock server on port 5000. The parent EC2 instance communicates with it over vsock, the only communication channel Nitro Enclaves support. No TCP, no network interfaces inside the enclave, just vsock.

The protocol is length-prefixed JSON: the parent sends a 4-byte big-endian message length followed by a JSON payload, the enclave reads it, routes it to the appropriate handler, and sends back a length-prefixed JSON response. Simple in theory.

# vsock framing used by both sides
def recv_message(sock):
    raw_len = sock.recv(4)
    msg_len = struct.unpack(">I", raw_len)[0]
    data = b""
    while len(data) < msg_len:
        chunk = sock.recv(min(4096, msg_len - len(data)))
        data += chunk
    return json.loads(data.decode())

def send_message(sock, message):
    data = json.dumps(message).encode()
    sock.sendall(struct.pack(">I", len(data)) + data)

The enclave binds to VSOCK_CID_ANY (0xFFFFFFFF) on port 5000, accepts connections from the parent, and routes requests by action field.

The Symptom

The parent instance would attempt to connect to the enclave and silently fail. No exception thrown, no error message in the obvious places. The connection appeared to succeed from the parent's perspective, the vsock socket connected without raising an error, but requests into the enclave were going nowhere.

This is the worst category of bug: the thing that fails quietly. If the connection had been refused outright, the error would have pointed directly at the problem. Instead, the parent thought it had a live connection and I thought I had a working system.

Finding It

The first thing I did was add structured logging on both sides of the vsock connection, on the parent before and after the connect call, and inside the enclave at startup. This is where the picture became clear.

The parent logs showed the vsock connection completing successfully. The enclave logs showed something different: the server was crashing on boot before it finished binding to port 5000.

The log sequence that mattered:

[enclave] Starting vsock server on port 5000
[enclave] ERROR: dependency import failed / runtime error during startup
[enclave] Process exited
[parent]  vsock connected to CID:5000

The sequence told the story. The enclave process was dying during startup, the server never fully came up, but the vsock layer had no mechanism to propagate that back to the parent. From vsock's perspective the port existed (it had been registered during the boot sequence), so the connection handshake succeeded. The fact that nothing was actually serving on the other end was invisible to the caller.

Connected but not live.

Why This Happens

Vsock operates at the transport layer. It does not know or care what is running above it. When the enclave image boots, the vsock port becomes available as part of the boot process before the application inside has finished starting. If your application crashes during startup, the port remains technically reachable via vsock while nothing is actually serving requests on it.

This is different from how you might expect TCP to behave on a normal host where a crashed process means the port is immediately unavailable and a connection attempt fails fast. Inside an enclave, the isolation layer introduces a gap between "the enclave is running" and "your application inside the enclave is running."

It also means that any production Nitro Enclave setup that does not explicitly verify application readiness is making an assumption it should not be making.

The Fix

The solution is a health check action that the parent calls before sending real traffic. Since the server already routes requests by action field, adding a health check is a one-liner in handle_request:

# Inside the enclave - server.py
def handle_request(request: dict) -> dict:
    action = request.get("action")

    if action == "health":
        return {"status": "ok"}

    elif action == "chat":
        # ... inference logic
        pass

    # ... other actions

On the parent side, before sending any real request, poll the health action over vsock until it responds or a timeout is reached:

# On the parent EC2 instance - bridge.py
import socket
import struct
import json
import time

ENCLAVE_CID = 16   # replace with your enclave's CID
VSOCK_PORT = 5000
HEALTH_TIMEOUT = 30
RETRY_INTERVAL = 1

def vsock_request(cid, port, payload):
    sock = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
    sock.connect((cid, port))
    data = json.dumps(payload).encode()
    sock.sendall(struct.pack(">I", len(data)) + data)
    raw_len = sock.recv(4)
    msg_len = struct.unpack(">I", raw_len)[0]
    response = b""
    while len(response) < msg_len:
        chunk = sock.recv(min(4096, msg_len - len(response)))
        response += chunk
    sock.close()
    return json.loads(response.decode())

def wait_for_enclave(cid, port, timeout=HEALTH_TIMEOUT):
    deadline = time.time() + timeout
    while time.time() < deadline:
        try:
            response = vsock_request(cid, port, {"action": "health"})
            if response.get("status") == "ok":
                print("[parent] Enclave health check passed")
                return True
        except Exception as e:
            print(f"[parent] Enclave not ready yet: {e}")
            time.sleep(RETRY_INTERVAL)
    raise TimeoutError(f"Enclave did not become healthy within {timeout}s")

# Usage
wait_for_enclave(ENCLAVE_CID, VSOCK_PORT)
# Now safe to send real traffic
result = vsock_request(ENCLAVE_CID, VSOCK_PORT, {"action": "chat", "messages": [...]})

With this in place, the parent will not send sensitive data into the enclave until the enclave has explicitly confirmed it is ready. If the server crashes on boot, the health check times out and raises an error rather than silently dropping requests into a dead connection.

What I Actually Learned

The immediate fix is the health check. The deeper lesson is about the gap between transport-layer success and application-layer readiness in enclave architectures.

Normal distributed systems have this problem too. A service can be "up" from a load balancer's perspective while being in a broken state internally. But inside a Nitro Enclave, the debugging surface is intentionally reduced. You cannot SSH in. You cannot attach a debugger. You cannot inspect the process from the outside. Logging is your primary diagnostic tool, and if you have not built structured logging into your enclave from the start, silent failures become very hard to reason about.

The 90 minutes I spent on this was mostly time I did not have good logging in place. Once I added it, the problem was obvious in under five minutes. Build the logging first.

Summary

Vsock connects at the transport layer. It does not verify that your application is running inside the enclave.
If your enclave server crashes on boot, the parent can still establish a vsock connection and have no idea nothing is listening.
Always implement a health check and poll it from the parent before sending real traffic.
Structured logging inside the enclave is not optional. It is your only debugging surface.
The length-prefixed JSON framing means a crashed server produces no response at all. Make sure your parent handles that case explicitly rather than hanging on recv.

This was written from a production debugging session while building Mizan, a legal AI platform using AWS Nitro Enclaves to protect attorney-client privileged data. The nitro-attestation-verifier tool referenced in other writeups came from the same project.

Muhammad Ablugg - Portfolio - GitHub

Forem: Muhammad Ablugg