Forem: Nikolay Kim

Service pipeline

Nikolay Kim — Mon, 30 Mar 2026 15:45:42 +0000

In the previous post, we described what a Service is and how to define it. Let's check an example:

async fn endpoint(req: HttpRequest) -> Result<HttpResponse, Error> {
    // Load operation from the request
    let oper = load_operation(req).await?;

    // Execute it using our engine
    let result = execute(oper).await?;

    // Convert the result into an HTTP response
    Ok(into_response(result).await?)
}

This is structurally a sequence of fallible async transformations. The natural representation is a composable pipeline using an and_then-style combinator, analogous to Result::and_then():

let pipeline = chain(authn)
    .and_then(load_operation)
    .and_then(authz)
    .and_then(execute)
    .and_then(into_response);

The resulting pipeline is itself a Service<HttpRequest, Response = HttpResponse>. Each stage is independently swappable, and insertion is trivial. For example, introducing throttling becomes a purely local change:

let pipeline = chain(authn)
    .and_then(throttling)
    .and_then(load_operation)
    .and_then(authz)
    .and_then(execute)
    .and_then(into_response);

A throttling service may be as specific or as generic as needed. At one end, it can be Service<HttpRequest, Response = HttpRequest> for request inspection. At the other, it can be fully parametric, Service<R, Response = R> allowing it to act as a reusable concurrency boundary independent of domain types. As you can see, this approach is highly flexible.

Given the Service trait, and_then is straightforward to express generically:

fn and_then<R, A, B>(svc_a: A, svc_b: B) -> impl Service<R, Response = B::Response>
where
    A: Service<R>,
    B: Service<A::Response, Error = A::Error>,
{
    AndThen { svc_a, svc_b }
}

The implementation is just sequential composition with short-circuiting on error:

struct AndThen<A, B> {
    svc_a: A,
    svc_b: B,
}

impl<R, A, B> Service<R> for AndThen<A, B> 
where
    A: Service<R>,
    B: Service<A::Response, Error = A::Error>,
{
    type Response = B::Response;
    type Error = A::Error;

    async fn call(&self, req: R) -> Result<Self::Response, Self::Error> {
        let intermediate = self.svc_a.call(req).await?;
        self.svc_b.call(intermediate).await
    }
}

Other familiar combinators (then, map, map_err) fall out in the same way.

In practice, this machinery is already available in the ntex-service crate. Its ServiceChain acts as a typed builder for pipelines, deferring composition until the chain is finalized. The resulting Pipeline is a distinct type because it encodes an additional constraint over the entire service graph: readiness.

Readiness

So far, Service::call models what a service does. It says nothing about when it is able to accept work.

In practice, services are not always ready to process a request. They may be constrained by in-flight work, internal queues, or external resources. Treating call as always-available either forces unbounded buffering or pushes backpressure into ad hoc, non-composable mechanisms.

To make this explicit, we extend the Service trait with a readiness check:

trait Service<R> {
    type Response;
    type Error;

    async fn ready(&self) -> Result<(), Self::Error>;

    async fn call(&self, req: R) -> Result<Self::Response, Self::Error>;
}

Conceptually, ready() is a gate: it resolves when the service can accept one request without violating its internal constraints.

For a composed service like AndThen<A, B>, readiness is no longer local—it depends on both components.

impl<R, A, B> Service<R> for AndThen<A, B>
where
    A: Service<R>,
    B: Service<A::Response, Error = A::Error>,
{
    type Response = B::Response;
    type Error = B::Error;

    async fn ready(&self) -> Result<(), Self::Error> {
        self.svc_a.ready().await?;
        self.svc_b.ready().await?;
        Ok(())
    }

    async fn call(&self, req: R) -> Result<Self::Response, Self::Error> {
        let intermediate = self.svc_a.call(req).await?;
        self.svc_b.call(intermediate).await
    }
}

This is the simplest correct implementation: the composed service is ready only if both svc_a and svc_b are ready.

Pipeline readiness

Unlike call, readiness is not purely compositional—it requires coordination across the entire chain. A pipeline must determine whether it can admit new work based on the state of all inner services.

Execution is asynchronous, so multiple requests may be in flight at any given time. As a result, readiness is a function of global state—it depends on all currently active executions, not just the local state of an individual service. This makes readiness inherently a cross-cutting concern.

The Pipeline is responsible for enforcing this invariant. It owns readiness across the full chain and must suspend progression at any point if one of the inner services becomes not ready.

This introduces the notion of shared readiness. Each Pipeline instance represents a single readiness slot—i.e., it can admit at most one request at a time. A well-behaved client must either:

clone the pipeline per request, or
use Pipeline::call_static(), which performs the cloning internally.

This constraint ensures that readiness is tracked consistently across concurrent executions without introducing implicit buffering or violating backpressure guarantees.

Because readiness must be enforced at every step in the chain, services cannot directly invoke the next service. Instead, readiness must be re-established before progressing. This requirement is encoded into the Service interface via an explicit execution context.

The ServiceCtx acts as the carrier of pipeline-level state, allowing readiness to be coordinated across service boundaries.

A useful mental model:

Service defines what happens at each step
Pipeline defines the structure
ServiceCtx defines how execution flows through that structure safely

Final Service Definition

trait Service<R> {
    type Response;
    type Error;

    async fn ready(&self, ctx: ServiceCtx<'_, Self>) -> Result<(), Self::Error>;

    async fn call(
        &self,
        req: R,
        ctx: ServiceCtx<'_, Self>,
    ) -> Result<Self::Response, Self::Error>;
}

Notes

Readiness is no longer a purely local property—it is mediated through ServiceCtx<'_, S>.
The context provides the necessary linkage between otherwise independent service instances.
Backpressure becomes explicit, composable, and enforceable without hidden queues.

Service Lifecycle Methods

In addition to call and ready, the Service trait defines two lifecycle methods: poll and shutdown. Each serves a distinct purpose.

At its core, a Service is a passive component—it only makes progress when driven by an external caller. However, some services need to maintain internal asynchronous state (e.g., a connection to a backend, background tasks, or resource pools). This is where poll comes in.

poll()

poll() is the mechanism that allows a service to make progress independently of request execution:

fn poll(&self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>>;

It is expected to be called by the executor (or an owning future) as part of the normal Future::poll() cycle.

A service can use poll to:

drive internal state machines
establish or maintain external resources (e.g., connections)
update readiness based on background progress

Crucially, poll is allowed to influence readiness. For example, a service may only become ready once a connection is established, or may transition back to Pending if capacity is exhausted. This makes poll the active side of the service, complementing the otherwise passive call.

shutdown()

shutdown() defines the termination phase of the service lifecycle:

async fn shutdown(&self);

It must be called by the owner of the service (e.g., the Pipeline) when the service is being dropped or taken out of operation.

The purpose of shutdown is to:

gracefully terminate background tasks
release external resources
flush or complete in-flight work

Unlike poll, which is part of steady-state operation, shutdown is a one-time transition that finalizes the service.

Component / Service Model

Nikolay Kim — Mon, 30 Mar 2026 08:58:03 +0000

The term “component model” is somewhat overloaded. It often brings to mind complex IoC containers, layers of dependency injection, and a fair amount of indirection. That’s not what we’re aiming for here.

What we want instead is something much simpler: a way to build reusable components that are, above all, easy to compose.

Imagine a service whose only responsibility is to execute an operation and return a result. Internally, it might be arbitrarily complex—but from the outside, its interface should remain minimal and predictable.

In Rust, we can express this idea using the most basic building block we have: a function.

async fn execute(op: Operation) -> Result<OperationResult, Err> {
   ...
}

That’s it.

This tiny abstraction already gives us everything we need. It takes a single input and produces a single output. There’s no hidden state, no framework magic, and no ceremony. Just a clear contract.

And importantly, this kind of interface is trivial to test, easy to reason about, and naturally composable.

Now, let’s imagine we want to use our execution engine service inside an http endpoint. What do we actually need in that case?

At a minimum, we need a thin layer responsible for http concerns: deserialization and serialization. In other words, something that can load an Operation from an incoming request and convert an OperationResult into an http response.

Once again, the shape of this can stay very simple.

Our http endpoint might look like just another function:

async fn endpoint(req: HttpRequest) -> Result<HttpResponse, Error> {
    // Load operation from the request
    let oper = load_operation(req).await?;

    // Execute it using our engine
    let result = execute(oper).await?;

    // Convert the result into an HTTP response
    Ok(into_response(result).await?)
}

There’s nothing particularly fancy happening here. The endpoint is just glue code:

translate http request → domain (Operation)
call the service
translate domain → http response (OperationResult)

Each piece has a single responsibility, and the boundaries are explicit. The execution engine knows nothing about http, and the http layer knows nothing about how the operation is executed.

What you might notice is that all of these services are completely independent. None of them depend on each other directly. Each one simply represents a transformation from an input type to an output type.

That’s the key idea.

To compose them, all we need is for the output of one service to match the input of the next. When all input and output types line up, we naturally get a transformation chain.

In our example, we’re ultimately transforming an http request into an http response:

HttpRequest -> HttpResponse

What happens in between is an implementation detail.

We can insert additional steps—like authentication or authorization—without changing the overall shape of the system. It’s still just a sequence of transformations from request to response.

async fn endpoint(req: HttpRequest) -> Result<HttpResponse, Error> {
    // Authentication
    let req = authn(req).await?;

    // Load operation from the request
    let oper = load_operation(req).await?;

    // Authorization
    let oper = authz(oper).await?;

    // Execute it using our engine
    let result = execute(oper).await?;

    // Convert the result into an http response
    Ok(into_response(result).await?)
}

Each step is small, focused, and composable. There’s no shared state, no tight coupling—just a pipeline of transformations.

Once you start thinking in these terms, the “component model” becomes almost trivial: components are just functions, and composition is just wiring outputs to inputs.

And that simplicity scales surprisingly far.

Now that we have the idea, we can start formalizing what a “service” actually is.

At its core, a service is just something that takes an input and produces an output—possibly asynchronously, and possibly failing. That should already sound familiar.

We can capture this idea with a simple Service trait, which looks very similar to Rust’s Fn traits:

trait Service<Request> {
    /// Response produced by the service.
    type Response;

    /// Error produced by the service.
    type Error;

    async fn call(&self, req: Request) -> Result<Self::Response, Self::Error>;
}

This abstraction gives us a uniform way to describe any transformation—from Request to Response.

The important part is what this unlocks.

We’re no longer limited to plain functions. A service can be:

a function,
a struct,
or a more complex composition of other services.

And once everything implements the same Service trait, we can start building generic operations that work over any service—middleware, combinators, pipelines, and so on.

In other words, we move from “just calling functions” to describing a system where everything is composable by design.

Final Notes

If you look closely at network services—especially the generic and reusable parts—you’ll notice something interesting: many of these components naturally fit the service model we’ve been discussing.

For example, consider a TCP connection handler. We can describe it like this:

impl Service<TcpStream, Response = (), Error = io::Error>

At its core, it’s just a function: it takes a TcpStream and either handles it successfully or returns an error. With this abstraction, you can build a generic server capable of handling any kind of TCP connection. In fact, this is exactly how ntex-server works under the hood.

Next, think about a TCP connector—something that creates outbound connections:

impl Service<net::SocketAddr, Response = TcpStream, Error = io::Error>

It takes a socket address and returns a live TCP stream. Simple, composable, and reusable anywhere in your network stack.

Finally, consider a TLS handshake:

impl Service<T: Stream, Response = TlsStream<T>, Error = io::Error>

It transforms a plain stream into a secure TLS stream. The beauty is that it doesn’t care whether the stream comes from a server, a client, or even an in-memory source.

The common thread here is clear: each of these components is just a transformation from input to output. By modeling them as services, we can compose them, reuse them, and swap them in and out without rewriting code. It’s the same idea we explored with HTTP requests earlier—applied to networking in general.

These ideas are implemented in practice in the ntex-service crate, which applies this model to real-world use cases. In fact, the entire ntex framework is built on top of the “service” model—from top to bottom.

Naturally, once you move from a minimal abstraction to a production-ready system, things become more complex than the simplified examples shown here. The crate introduces additional concepts that we haven’t covered yet—such as pipelines, middlewares, service readiness, and configuration.

Each of these plays an important role in making the model practical and scalable in real applications.

The next post covers other parts of the service model: pipeline, readiness, and lifecycle methods.

Building a Error Library

Nikolay Kim — Thu, 26 Mar 2026 11:21:08 +0000

Error handling is one of those things that quietly shapes the entire experience of a system. When it works well, users recover quickly, support teams stay efficient, and developers can focus on real problems. When it doesn’t, even small issues turn into long, frustrating investigations. To improve this, it helps to start with a simple question: what do we actually want from our errors? A big part of the problem with many existing approaches is that they treat all errors more or less the same. In practice, though, not all errors are equal. There’s an important difference between something the client did wrong and something that failed inside the system—and that difference should shape how we handle them.

When a client makes a mistake—bad input, incorrect API usage, missing data—the system should make it obvious what went wrong and how to fix it. That means returning clear, human-readable messages, along with structured error codes for programmatic use. It also means logging enough context and publishing metrics so patterns can be spotted over time. Ideally, clients should be able to resolve these issues on their own, without needing to contact support.

At the same time, if support does get involved, they should have everything they need to understand the issue quickly. The goal is to avoid pulling in developers for cases that aren’t actually system problems.

Service errors are a different story. These are failures inside the system or in its dependencies, and they require a different kind of response. Instead of focusing on user clarity, the focus shifts to fast diagnosis. Errors should automatically trigger alerts, include detailed diagnostic information, and make it easier to identify which component—or even which upstream service—caused the issue.
A good error library connects all of these needs. It gives clients clarity, gives support teams enough context to investigate, and gives developers the signals they need to quickly find and fix real problems.

Type of Error

Every interaction in the system—whether it’s a function call, API request, internal service, or upstream dependency—ultimately results in one of three outcomes: success, a client error, or a service error.

Making this distinction explicit is important because it drives how the system responds, who needs to act, and what kind of information should be surfaced. At its simplest, this classification can be represented with a small enum:

pub enum ResultType {
    Success,
    ClientError,
    ServiceError,
}

With this in place, every response can be consistently tagged with a specific type. That single piece of information becomes a powerful signal: it determines whether we return a user-friendly message, enable self-service debugging, or trigger alerts and deeper diagnostics for engineers.

Error Classification

In practice, errors are rarely simple. A single request can pass through multiple subsystems, and failures can happen at different stages for different reasons. What appears as a single error often represents a combination of underlying issues, each with its own meaning and implications.

For example, a client might send malformed data, which should be treated as a BadRequest. At the same time, a downstream dependency might fail due to a dropped connection—an entirely different category of problem. Even though both issues surface during the same flow, they should be classified differently because they require different responses.

To make this manageable, we can introduce a classification layer using simple enums that describe the nature of the error. These classifications are more specific than the high-level ResultType and help standardize how we reason about failures across the system.

For instance, authentication errors categories might look like this:

pub enum AuthErrorType {
    BadRequest,
    BadAuthenticationMethod,
    AuthenticationDenied,
    Service,
}

This approach establishes a shared vocabulary within a domain while remaining lightweight and easy to reuse. The same classifications can be used across APIs and services, avoiding duplication and ensuring consistency. Instead of reinventing error categories in each place, we build a unified system of error types that behave consistently everywhere.

We can then map these domain-specific classifications back to the higher-level ResultType. This allows us to automatically determine whether an error is a client or service issue based on its classification:

impl ResultKind for AuthErrorType {
    fn tp(&self) -> ResultType {
        match self {
            AuthErrorType::BadRequest
            | AuthErrorType::BadAuthenticationMethod
            | AuthErrorType::AuthenticationDenied => ResultType::ClientError,

            AuthErrorType::Service => ResultType::ServiceError,
        }
    }

    fn signature(&self) -> &'static str {
        match self {
            AuthErrorType::BadRequest => "Auth-BadRequest",
            AuthErrorType::BadAuthenticationMethod => "Auth-BadAuthMethod",
            AuthErrorType::AuthenticationDenied => "Auth-Unauthenticated",
            AuthErrorType::Service => "Auth-Service",
        }
    }
}

This mapping separates what the error means from how it is handled. The classification captures domain-specific detail, while the result type drives behavior—how the system responds, whether the issue is surfaced to the client, and how it is monitored internally.

By keeping classification and outcome distinct, the model remains both expressive and consistent: errors carry precise meaning at the domain level while fitting into a unified system-wide structure.

At its core, this can be expressed with a simple trait:

pub trait ResultKind {
    /// Returns the classification of the result (e.g. success, client error, service error).
    fn tp(&self) -> ResultType;

    /// Returns a stable identifier for the specific error classification.
    fn signature(&self) -> &'static str;
}

Reporting

The final step is reporting. Error reporting is tightly coupled to the error itself, and instead of reinventing the wheel, we can build on top of Rust’s standard Error trait. It already provides most of what we need to make error reporting both useful and consistent.

For public-facing errors, the primary representation should rely on fmt::Display. This is what clients see, so it needs to be clear, concise, and readable. It should communicate what went wrong without exposing unnecessary internal details.

For example, an error with the classification AuthErrorType::BadRequest could be represented like this:

Invalid SAS token signature
type: ClientError
signature: Auth-BadRequest

This format gives the client enough information to understand the issue and act on it, while also providing structured fields (like type and signature) that can be used for debugging or automation.

Service errors require a different approach. Here, the focus is on diagnostics rather than presentation. We want to capture as much useful context as possible, including nested errors and, when available, a backtrace.

This is where Debug and Error::source() come into play. The Debug implementation can be used for internal logging, allowing us to traverse and print the full chain of errors. By following Error::source(), we can expose underlying causes, which is especially important when dealing with failures in upstream services.

It’s worth keeping Debug output focused. There’s no need to dump every field of a struct—only the information that helps explain the failure. If an error is simply wrapping another error, it’s often enough to delegate to the nested error rather than adding redundant noise.

In this model, reporting naturally splits into two layers:  a clean, user-facing view via Display, and a rich, diagnostic view via Debug. Together, they ensure that errors are both understandable to clients and actionable for engineers.

As an additional improvement, we can include a service attribute in our error types. This allows us to explicitly indicate which *upstream service( is responsible for the failure, making it easier to trace issues across system boundaries and speeding up diagnosis in distributed environments.

With this in mind, error types should implement std::error::Error and expose a small amount of structured diagnostic information:

pub trait ErrorDiagnostic: std::error::Error {
    type Kind: ResultKind;

    /// Returns the classification of this error.
    fn kind(&self) -> Self::Kind;

    /// Returns the responsible service, if applicable.
    fn service(&self) -> Option<&'static str>;

    /// Returns a backtrace for debugging, if available.
    fn backtrace(&self) -> Option<&Backtrace>;
}

This keeps the design idiomatic while providing enough structure for classification, observability, and cross-service debugging.

Here is how an AuthError could look:

/// Error for use in authentication routines.
#[derive(Clone, Debug, thiserror::Error)]
pub enum AuthError {
    #[error("Bad request")]
    BadRequest(ErrorMessage),
    #[error("Bad authentication method")]
    BadAuthenticationMethod(ErrorMessage),
    #[error("Authentication denied")]
    AuthenticationDenied(ErrorMessage),
    #[error("Internal server error")]
    Internal(#[source] Box<dyn std::error::Error>),
}

impl ErrorDiagnostic for AuthError {
    type Kind = AuthErrorType;

    fn kind(&self) -> Self::Kind {
        match self {
            AuthError::BadRequest(_) => AuthErrorType::BadRequest,
            AuthError::BadAuthenticationMethod(_) => AuthErrorType::BadAuthenticationMethod,
            AuthError::AuthenticationDenied(_) => AuthErrorType::AuthenticationDenied,
            AuthError::Internal(_) => AuthErrorType::Service,
        }
    }
}

This design allows the system to present appropriate error messages to clients, generate meaningful client and server metrics, and trigger alerts when system-level issues occur.

Final notes

These ideas are reflected in the ntex-error crate, which applies this model in practice.

The approach is primarily suited to application development rather than library or framework design. For that reason, ntex-error focuses on providing building blocks rather than prescribing concrete error categories or types, leaving those decisions to the application domain.

It provides the core building blocks, including the Error container, ErrorDiagnostic and ResultKind traits, and utilities for error formatting, such as this fmt_diag helper.

This model builds on work by Max Gortman, originally developed in the context of private applications.