Forem: Andrey Dudnik

How rising Web3 infrastructure costs pushed me to build my own RPC aggregator

Andrey Dudnik — Thu, 09 Apr 2026 09:39:13 +0000

Hi, my name is Andrey, and I work with blockchain data analytics, mostly in EVM-like networks.

In practice, that means working with things like token prices across exchanges, NFT flows, liquidity depth in DEX pools, lending-related activity, flashloan opportunities, and other on-chain signals that can be turned into useful analytics.

What pulled me into this area was the nature of blockchain data itself.

Unlike traditional fintech systems, blockchain data is open and immutable. Once something is written to the chain, it is usually there forever. That creates a very unusual environment for analytics: you can track how funds move, how wallets behave, how contracts are used, how liquidity appears and disappears, and how entire systems evolve over time.

But there is an important practical detail here.

On-chain analytics does not start with charts or SQL. It starts with access to the network.

To collect data, you need a stable way to talk to blockchains. Very quickly, this turns into an infrastructure problem: nodes, RPC, APIs, sync status, availability, rate limits, and cost.

At a high level, analytics comes down to two things:

knowing which contracts, events, and transactions to inspect;
being able to extract that data reliably and at scale.

The first part is hard, but manageable.

The second part is where things get expensive and painful.

The old model stopped working

The most direct solution is to run your own nodes and use their APIs.

But that is no longer trivial.

For example, running Ethereum execution + consensus infrastructure now requires a serious machine: fast NVMe storage, a lot of RAM, and a decent CPU. In practice, this can easily mean something like 4TB NVMe, 64GB RAM, and a modern processor. And even with the hardware ready, there is still sync time. Sometimes more than 72 hours.

So the market naturally evolved a different offer:

Don’t run your own nodes. Use our RPC. Start on a free tier, then upgrade to paid plans when needed.

A few years ago, this model worked surprisingly well.

Free plans were often enough to build an MVP, test ideas, and even run small products. And if things grew, paid plans or PAYG usually made sense.

But over time, that changed.

If I look back at roughly 2018–2020, that period felt like a very good time for on-chain analytics. Hardware was much cheaper, networks were smaller, and workloads were lighter. Then L2 adoption accelerated, transaction costs dropped, usage exploded, and the amount of chain data grew massively.

As a result, the economics changed too.

What used to be true:

hardware was affordable;
free RPC plans were often enough for early analytics workloads;
self-hosting was still realistic for many developers.

What changed:

NVMe prices increased sharply;
large SSDs became much more expensive;
free RPC plans became heavily restricted;
those free plans became almost useless for serious analytics workloads;
paid plans started at a level that is hard to justify for solo builders and small teams;
and once the workload grows, costs can climb very quickly.

At that point, collecting data from Web3 networks — or building a small analytics startup on top of them — became much less accessible than it used to be.

Why I started self-hosting anyway

I hit this problem myself.

I needed to make a large number of heavy requests across multiple networks. Free providers were no longer enough, and paying commercial rates for the amount of traffic I needed did not feel rational.

That left very few options.

The main one was obvious: run my own nodes.

I did not want to do it. Not because of the hardware alone, but because of everything that comes with it: updating node software, monitoring sync state, dealing with lag, handling failures, and generally maintaining a small fleet of infrastructure.

Still, I did it.

I started with one node. Then another. Then more.

And very quickly I ran into another problem: a node by itself is not a solution.

Nodes need updates. During updates, they can fall behind the network. After updates, they need time to catch up. If production traffic is hitting a lagging node during that period, things stop behaving the way you want them to.

That naturally led to the next step: if I wanted stable access, I needed at least two nodes in HA mode and something in front of them.

HAProxy worked — until it didn’t

I chose HAProxy.

It can route traffic, run health checks, and generally do what a load balancer is supposed to do. On paper, the setup looked fine: two nodes behind HAProxy, traffic balanced between them, one node can go down for updates while the other keeps serving requests.

And yes, that setup worked.

But over time, I realized that HAProxy was not really the right abstraction for the problem I was solving.

What I actually needed was not just balancing. I needed routing logic that understood Web3-specific realities:

whether a node was truly synced;
whether it was still catching up;
whether it should be used for archive-only requests;
whether a consensus endpoint should be handled differently from execution traffic;
whether one provider or node was more appropriate for a specific route or workload.

HAProxy is powerful, but maintaining this kind of logic there became too uncomfortable and too brittle for my use case.

That is when I decided to build a dedicated service for it.

Building my own RPC aggregator

The idea was simple:

build a layer that can aggregate different sources, route and balance requests between them, understand which nodes are healthy, which are behind, which are suitable only for certain request types, and which should not be used at all.

In other words, not just a generic load balancer in front of nodes, but a service built specifically around the realities of Web3 infrastructure.

That is how my RPC aggregator appeared.

At this point it is still in alpha, but it already covers some of the pain points that pushed me to build it in the first place:

aggregation of multiple providers and self-hosted nodes;
separate routing for execution, archive, and consensus traffic;
balancing and request routing;
infrastructure checks that are more practical than using a generic reverse proxy alone.

Why I’m opening it up

At some point I realized something else: this setup is already larger than what I personally need for my own workloads.

So instead of keeping all of that capacity to myself, I decided to open it up in alpha.

Right now I am not trying to sell it.

What I care about more is understanding whether this is actually useful to other people dealing with similar problems:

on-chain analytics;
bots;
infra tooling;
archive-heavy workloads;
consensus or beacon access;
high-volume RPC use cases that do not fit well into free-tier providers.

If that sounds relevant, I’m happy to share access in exchange for honest feedback.

What I want to learn is simple:

is this actually useful in practice?
which networks or routes are missing?
where are the weak points?
what should be improved first?

If there is interest, I can also write a more technical follow-up about the architecture itself: how the routing layer works, why I moved away from a pure HAProxy-based setup, and what kinds of infrastructure problems start appearing once you build this for real analytics workloads.

If this is relevant to your work, feel free to reach out.

Telegram: @ddkand

One of many ways to migrate from NodeJS to Rust

Andrey Dudnik — Wed, 22 Nov 2023 17:11:32 +0000

This post describes my personal approach and the experience I have gained. It may contain some deviations, but they are not critical to understanding and usage.

My goal is to develop a microservice in Rust that matches the speed, safety, and ease of development found in NodeJS and Typescript with NestJS.
Whats more important to me is the utilization of modern system design patterns such as IoC, DDD, CQRS, and Hexagonal architecture, without relying on complex frameworks.

To start with, as I mentioned earlier, this post wont involve complex frameworks such as rocket_rs or actix_web, among others of the sort. This choice is deliberate because delving into such frameworks tends to require learning numerous components that may not significantly contribute to creating clear and comprehensible software. However, if you prefer to use frameworks, its entirely fine — just not the approach I am taking here.

Lets begin preparing to create the Todo microservice in Rust. Initially, we need to decide on the crates that will be utilized in the microservice. For this purpose, I published my own implementations of IoC & CQRS. I chose this route to gain a deeper understanding of these methodologies, as some other crates were either outdated or lacked support from the community.

Links of crates:
IoC container
CQRS implementation
CQRS implementation with IoC wrapper

Other crates of microservice:

tonic - grpc implementation
hyper - http implementation
sqlx - driver for managing PostgreSQL
tokio - handling asynchronous operations

Thats all we will be using.

I lean toward an approach where the application is divided into smaller slices, encapsulated within different packages for reuse. The cargo workspaces package structure will look like this.

API - main app to client access calls
Common - domain area with interfaces for core packages
Config - contain any environment vars to provide it in packages
Core - business logic, such as CRUD operations and etc
Repository - persisted store layer
Schema GRPC - proto files and network contracts to access MS

Lets take a closer look each of them.

API package contain grpc server to launch ours grpc controllers, by current example its will be only one controller with based CRUD operations,
http server with health checks controller its will be used to any probes of kubernetes.

Common package will contains only domain entities - actually its not so good place for them, but now its ok.
And entirely interfaces to implementation of database repositories, business services and etc.

Config package its pretty simple lib which provide store of environment variables. Like database credentials, app hosts and ports.

Repository its more difficult layer which will working only with database, in current case its will be postgres.

Schema GRPC package to accumulate proto files and generating bin files for communication with MS via API

Diagram:

The previous section was more theoretical and now we will dive to code.

We will start by creating a simple todo controller responsible for handling client requests.

pub struct TodoGrpcController {
  context: ContainerContext,
}

impl TodoGrpcController {
  pub fn new(props: ContainerContextProps) -> Self {
    Self {
      context: ContainerContext::new(props),
    }
  }

  fn get_bus(&self) -> Box<CqrsProvider::Provider<AppContext>> {
    self.context.resolve_provider(CqrsProvider::TOKEN_PROVIDER)
  }
}

#[async_trait]
impl TodoService for TodoGrpcController {
  async fn create(
    &self,
    request: Request<CreateTodoRequest>,
  ) -> Result<Response<CreateTodoResponse>, Status> {
    let request = request.into_inner();

    let bus = self.get_bus();

    let command = CreateTodoCase::Command::new(&request.name, &request.description);

    let todo_entity = bus.command(Box::new(command)).await.unwrap();

    Ok(Response::new(CreateTodoResponse {
      status_code: 200,
      message: String::from("CREATED"),
      data: Some(Todo {
        id: todo_entity.get_id().to_string(),
        name: todo_entity.get_name().to_string(),
        description: todo_entity.get_description().to_string(),
        completed: todo_entity.get_completed(),
        created_at: todo_entity.get_created_at().to_string(),
        updated_at: todo_entity.get_updated_at().to_string(),
      }),
    }))
  }

  async fn get_by_id(
    &self,
    request: Request<GetTodoByIdRequest>,
  ) -> Result<Response<GetTodoByIdResponse>, Status> {
    let request = request.into_inner();

    let bus = self.get_bus();

    let query = GetTodoByIdCase::Query::new(&request.id);

    match bus.query(Box::new(query)).await.unwrap() {
      Some(r) => Ok(Response::new(GetTodoByIdResponse {
        status_code: 200,
        message: String::from("SUCCESS"),
        data: Some(Todo {
          id: r.get_id().to_string(),
          name: r.get_name().to_string(),
          description: r.get_description().to_string(),
          completed: r.get_completed(),
          created_at: r.get_created_at().to_string(),
          updated_at: r.get_updated_at().to_string(),
        }),
      })),
      None => return Err(Status::not_found("Not found todo by id.")),
    }
  }
}

The methods are pretty concise, each containing fundamental logic. For instance, lets delve into the creation of a todo item. Initially, I have introduced a method to retrieve the CQRS bus (get_bus), which serves as a pathway for routing commands or queries.

Subsequently, a command is crafted and sent to the command bus using bus.command. Following this, the tasks logic is delegated to the command handler. These handlers operate as layers responsible for executing various business cases, such as creating new todo items.

Lets explore the code for the creation of a todo in the command handler:

#[derive(Clone)]
  pub struct Command {
    name: String,
    description: String,
  }

  impl Command {
    pub fn new(name: &str, description: &str) -> Self {
      Self {
        name: name.to_string(),
        description: description.to_string(),
      }
    }
  }

  #[async_trait]
  impl CommandHandler for Command {
    type Context = AppContext;
    type Output = Result<TodoEntity, Box<dyn Error>>;

    async fn execute(&self, context: Arc<Mutex<Self::Context>>) -> Self::Output {
      let repository = context.lock().unwrap().get_command().get_repository();

      repository
        .create(&TodoEntity::new(
          &Uuid::new_v4().to_string(),
          &self.name,
          &self.description,
          false,
          &Local::now().to_string(),
          &Local::now().to_string(),
        ))
        .await
    }
  }

The logic here is rather straightforward. It involves the creation of a new todo entity and handling the persistence by repository with the PostgreSQL database through sqlx.

In the repository layers, its important to follow a key principle from CQRS, which suggests splitting read and write actions. This means having two separate database pools — one for writing data and the other for reading it. Yet, for simplicity in this example, I have chosen to create two simplified versions: one for executing commands and another for dealing with queries. You can adjust this setup to match your own needs.

The todo query repository code:

#[derive(Clone)]
pub struct SqlxQueryRepository {
  pool: PgPool,
}

impl SqlxQueryRepository {
  pub fn new(pool: PgPool) -> Self {
    Self { pool }
  }
}

#[async_trait]
impl TodoQueryRepository for SqlxQueryRepository {
  async fn get_by_id(&self, id: &str) -> Result<Option<TodoEntity>, Box<dyn Error>> {
    let row = sqlx::query(
      "
      SELECT * FROM todo WHERE id = $1 LIMIT 1
    ",
    )
    .bind(Uuid::parse_str(id).unwrap())
    .fetch_one(&self.pool)
    .await;

    match row {
      Ok(r) => Ok(Some(TodoSqlxMapper::pg_row_to_entity(&r))),
      Err(_) => return Err("Cant find todo by id".into()),
    }
  }
}

And todo command repository code:

#[derive(Clone)]
pub struct SqlxCommandRepository {
  pool: PgPool,
}

impl SqlxCommandRepository {
  pub fn new(pool: PgPool) -> Self {
    Self { pool }
  }
}

#[async_trait]
impl TodoCommandRepository for SqlxCommandRepository {
  async fn create(&self, todo: &TodoEntity) -> Result<TodoEntity, Box<dyn Error>> {
    let created_at = match sqlx_parse_utils::string_to_timestamp(&todo.get_created_at()) {
      Ok(r) => r,
      Err(e) => return Err(e.into()),
    };

    let updated_at = match sqlx_parse_utils::string_to_timestamp(&todo.get_updated_at()) {
      Ok(r) => r,
      Err(e) => return Err(e.into()),
    };

    let row = sqlx::query(
      "
      INSERT INTO todo (id, name, description, completed, created_at, updated_at)
      VALUES ($1, $2, $3, $4, $5, $6) RETURNING *
    ",
    )
    .bind(Uuid::parse_str(&todo.get_id()).unwrap())
    .bind(&todo.get_name())
    .bind(&todo.get_description())
    .bind(&todo.get_completed())
    .bind(created_at)
    .bind(updated_at)
    .fetch_one(&self.pool)
    .await
    .unwrap();

    Ok(TodoSqlxMapper::pg_row_to_entity(&row))
  }
}

I guess these repositories are should be quite clear enough as they implement base logic to manage sql data.

We going to finish, and lets try to make some requests via grpc cli.
In this example I will use the official grpc cli for macOS (brew install grpc), by default tonic supports the server reflection,
so you can use this cli without any problems.

Create the new todo:
$ grpc_cli call localhost:50051 Create "name: 'Read the book', description: 'I would like to read 10 pages'"

connecting to localhost:50051
Received initial metadata from server:
date : Sun, 12 Nov 2023 07:14:53 GMT
status_code: 200
message: "CREATED"
data {
  id: "112e6968-3424-4575-8bde-a16bcf64eeb6"
  name: "Read the book"
  description: "I would like to read 10 pages"
  created_at: "2023-11-12 14:14:53.945656"
  updated_at: "2023-11-12 14:14:53.945855"
}
Rpc succeeded with OK status

Update the todo by id:
$ grpc_cli call localhost:50051 Update "id: '112e6968-3424-4575-8bde-a16bcf64eeb6', completed: true"

connecting to localhost:50051
Received initial metadata from server:
date : Sun, 12 Nov 2023 07:27:05 GMT
status_code: 200
message: "SUCCESS"
data {
  id: "112e6968-3424-4575-8bde-a16bcf64eeb6"
  name: "Read the book"
  description: "I would like to read 10 pages"
  completed: true
  created_at: "2023-11-12 14:14:53.945656"
  updated_at: "2023-11-12 14:26:48.589915"
}
Rpc succeeded with OK status

Get paginated todo list:
$ grpc_cli call localhost:50051 GetPaginated "page: 0, limit: 10"

connecting to localhost:50051
Received initial metadata from server:
date : Sun, 12 Nov 2023 11:17:22 GMT
status_code: 200
message: "SUCCESS"
data {
  id: "c641207e-7c1f-40ec-9735-70b3944bb1b1"
  name: "Buy drinks"
  description: "Going to market and buy some drinks"
  created_at: "2023-11-12 14:18:04.789755"
  updated_at: "2023-11-12 14:18:04.789791"
}
data {
  id: "f65e81f8-1620-4c68-9b58-3936bd250b0f"
  name: "Check the mail"
  description: "Looking for new messages in gmail"
  created_at: "2023-11-12 14:15:43.271947"
  updated_at: "2023-11-12 14:15:43.272056"
}
data {
  id: "112e6968-3424-4575-8bde-a16bcf64eeb6"
  name: "Read the book"
  description: "I would like to read 10 pages"
  completed: true
  created_at: "2023-11-12 14:14:53.945656"
  updated_at: "2023-11-12 14:26:48.589915"
}
Rpc succeeded with OK status

So, thats all.
I finish the minimal microservice in Rust without any complex frameworks and main goal about implementation of clear architecture was done.

I leave the link to github where you can discover all repo.

PS.
My way can be looks like something not finished but if you would like to dive into Rust with NodeJS background, I think this post will help ypu.

Github link to repo