Forem: Mark Gyles

Graph RAG does not need a graph database. It needs a database that does everything.

Mark Gyles — Tue, 07 Apr 2026 15:57:35 +0000

How SurrealDB compares to Neo4j, Amazon Neptune, and ArangoDB for production graph RAG.

Graph RAG is the right idea. Using relationships between entities to scope and improve retrieval produces better results than vector similarity alone. The research is clear on this. What the research does not address is where those operations execute, and that turns out to be the question that actually matters in production. Here is what happens when you try to build production graph RAG across a typical multi-database stack, and what changes when every operation composes in a single system.

The gap: what graph RAG actually needs in production

Most graph RAG implementations follow the pattern from Microsoft's GraphRAG paper: extract entities from documents, build a knowledge graph of those entities and their relationships, then traverse that graph at query time to pull relevant context. The graph is derived from the content. It is an inferred structure that helps you find documents that are semantically connected through shared entities.

That works for document discovery over a static corpus. It does not work for a production agent that needs to serve a specific customer, respect access control, enforce tenancy, check data freshness, and ground its answers in real business relationships rather than inferred ones.

A production graph RAG pipeline needs all of the following in a single retrieval step:

Ground truth relationships. This customer purchased this product. This product has this known defect. This defect maps to these knowledge base articles. These are not inferred from document content. They are transactional facts from your purchase system, your CRM, your product catalog. The graph traversal that scopes retrieval needs to follow these real edges, not entity co-occurrence.

LLM-inferred relationships. This knowledge base article documents a fix for this issue. This article is related to that article. These connections are useful and hard to create manually at scale. An extraction pipeline reads the article and the issue description and decides they are connected. This is exactly what the GraphRAG pattern provides, and it is valuable.

Structured filters. Type constraints, tenant isolation, permission checks. Not every document that is relevant is valid for this agent to return to this customer.

Temporal constraints. The article was updated within 30 days. The product has not been discontinued. The issue is still active.

Hybrid retrieval. Vector similarity and full-text search, blended into a single relevance score, running against the candidates that survived everything above.

None of these are controversial. Every serious production RAG system needs them. The question is where they execute and whether they compose.

The comparison: the same pipeline in SurrealQL and Cypher

Neo4j is the most established graph database. Since version 5.11, it has native vector indexes. It has full-text indexes. It can store properties on nodes and relationships. There is no technical reason you cannot build this entire pipeline in Neo4j.

Here is the retrieval operation in SurrealQL. Same scoping, graph traversal, hybrid search, and result set as you would build in Cypher, but expressed as a single statement with co-equal predicates:

SELECT
  id, title,
  vector::distance::knn() AS vec_dist,
  search::score(1) AS ft_score,
  (1 - vector::distance::knn()) * 0.65
    + search::score(1) * 0.35 AS blend,
  search::highlight('<em>','</em>',1) AS snippet
FROM knowledge_base
WHERE type = 'support'
  AND tenant = $tenant
  AND $agent_principal INSIDE allowed_principals
  AND updated_at > time::now() - 30d
  AND id IN
    $customer->owns->product
      ->has_issue->knowledge_base
  AND content_embedding <|100,COSINE|>
    $query_embedding
  AND content @1@ $query_text
ORDER BY blend DESC
LIMIT 10;

One statement. Co-equal predicates. One consistent snapshot.

In SurrealQL, the graph traversal, structured filters, vector search, and full-text search are co-equal predicates in a single WHERE clause. They compose the same way boolean conditions always compose: with AND. The query reads like a single thought. You can rearrange the predicates, add new ones, or remove them, and the statement still works.

In Cypher, the same operations require separate procedural steps. Vector search is a CALL db.index.vector.queryNodes() block. Full-text search is another CALL db.index.fulltext.queryNodes() block. The graph traversal is a MATCH pattern. The metadata filters are a WHERE clause. The blending is manual arithmetic in the RETURN. You are orchestrating five steps and correlating their results yourself. It works, but it reads like a script, not a query.

The readability gap is the obvious problem. The less obvious one is memory. Each procedural step in the Cypher pipeline YIELDs a result set into the execution context, and those intermediate results stay in memory while the next step runs. The vector CALL yields 100 nodes with scores. Those 100 nodes sit in memory while the MATCH clause runs graph pattern matching against them. The MATCH can fan out fast: if a customer owns 5 products, each product has 10 known issues, and each issue maps to 20 knowledge base articles, the pattern expands to 1,000 intermediate rows before any WHERE filter trims them. All materialized. All held. Then the full-text CALL runs and yields another result set that needs to be correlated with everything already in memory.

At production scale, this is not a theoretical concern. Combinatorial expansion in multi-hop MATCH patterns is one of the most common causes of out-of-memory failures in Neo4j. The intermediate state accumulates faster than the downstream filters can prune it, and the query planner cannot optimize across the boundaries between CALL blocks and MATCH clauses because they are procedurally separate operations. You end up tuning heap sizes, adding LIMIT hints mid-pipeline, or splitting queries into multiple round-trips to keep memory under control. All of which is managing a problem that does not need to exist.

In SurrealQL, because all predicates are co-equal in one WHERE clause, the engine sees every constraint upfront. It can start with the most selective predicate and narrow early before touching the expensive operations. There is no intermediate materialization between steps because there are no steps. The result set is built once, not accumulated across a pipeline.

This matters for day-to-day development too. When retrieval logic is a single composable statement, you can modify it, extend it, and reason about it as a unit. When it is a multi-step procedure, every change requires understanding the data flow between steps. Adding a new filter means figuring out which step it belongs to and whether it changes what gets passed to the next step. Removing a constraint means tracing its effects through the pipeline. The cognitive overhead scales with the number of operations, and production retrieval pipelines have a lot of operations.

The deeper problem: your graph database is not your system of record

Neo4j is almost never where the data originates. The purchase happens in Postgres. The ticket gets created in a CRM. The permission change happens in an auth service. Neo4j receives all of that via sync jobs, CDC pipelines, or batch imports. It is a read replica of graph-shaped data. The same is true of Neptune and, in most deployments, ArangoDB. The graph database is downstream of the systems that produce the data your agent needs to reason over.

That is why the consistency problem exists in the first place. The graph is always behind the source of truth because it is not the source of truth. A permission gets revoked in the auth service, but the graph database does not see it for 50ms and the vector index does not see it for 200ms. An article gets updated in the CMS, but the vector store still has embeddings from the old version for up to 5 minutes. Each system is internally consistent. The inconsistency lives in the spaces between them. Your agent is reasoning over multiple realities simultaneously and does not know it.

SurrealDB is designed as a transactional system of record. The purchase record, the customer relationship, the product graph, the knowledge base articles, the embeddings, the agent's memory, and the LLM-inferred edges from extraction pipelines all live in the same database as first-class data. The ground truth edges are not synced from somewhere else. They are the transaction records:

BEGIN;

-- The purchase record IS the graph edge
RELATE customer:4821->purchased->product:drip_coffee_maker SET
  order_id = 'ORD-29481',
  purchased_at = time::now(),
  source = 'transaction',
  confidence = 1.0;

-- Product-to-known-issue edge (from engineering)
RELATE product:drip_coffee_maker->has_issue->product_issue:gasket_defect SET
  severity = 'moderate',
  confirmed_at = d'2026-02-15',
  source = 'engineering',
  confidence = 1.0;

COMMIT;

The moment that transaction commits, customer:4821->purchased->product:drip_coffee_maker is traversable. There is no sync job. There is no CDC pipeline. There is no replication lag. An agent querying one millisecond later sees it.

The LLM-inferred edges write to the same database. An extraction pipeline reads a knowledge base article and the issue description, decides they are connected, and creates the edge:

RELATE product_issue:gasket_defect->documented_in->kb:fix_base_leaks SET
  source = 'llm_extraction',
  confidence = 0.94,
  model = 'claude-sonnet-4',
  verified_by = NONE,
  created_at = time::now();

A single traversal now crosses both: $customer->purchased->product->has_issue->documented_in->knowledge_base. The first two hops are ground truth from the transaction. The last hop is LLM-inferred from the extraction pipeline. Every predicate in the query evaluates against the same ACID snapshot.

This is not a convenience feature. It eliminates an entire category of infrastructure: the sync jobs, the CDC pipelines, the cache invalidation, the eventual consistency windows. Every one of those is a place where accuracy silently degrades. A database that is both the system of record and the query engine removes the gaps where those failures live.

The landscape: how other graph databases handle this

SurrealDB is not the only database attempting to unify these operations. Every major graph database has recognized that vector search matters for graph RAG, and each has added some form of it. The question is not whether they support it. It is whether the operations compose, and what breaks when they do not.

Neo4j

Neo4j is the most established graph database and the most natural comparison. As of v2026.01, Neo4j introduced a native SEARCH clause with in-index filtering as a preview feature. Before this, vector search could not be pre-filtered at all. You ran the vector index, got results, and filtered afterward. For years, this was a blocking issue for teams building multi-tenant RAG, and community threads going back to 2023 show developers hitting this wall repeatedly.

The new SEARCH clause is a step forward, but the WHERE subclause inside SEARCH supports only a subset of the full Cypher WHERE clause. Full-text search is still a separate procedure call that cannot compose with the SEARCH clause in a single statement. The graph traversal is still a separate MATCH pattern. So even with the improvement, you are still orchestrating procedural steps and correlating results between them, with the memory accumulation problem described earlier in this article. Neo4j is actively closing the gap, but as of today the composition is partial.

Where Neo4j remains the clear winner is deep graph analytics. The Graph Data Science library (community detection, centrality algorithms, pathfinding, node similarity at scale) is purpose-built and has no equivalent in SurrealDB. If your primary workload is analytical graph algorithms rather than transactional agent retrieval, Neo4j is the right tool.

Amazon Neptune

Amazon Neptune is the default choice for teams already on AWS, and it illustrates the composition problem at the product level. Neptune is actually two separate products. Neptune Database is a transactional graph database with no native vector search. Neptune Analytics is an analytics engine with vector search and graph algorithms but designed for analytical workloads, not transactional retrieval. AWS's own "unified" graph solution is itself a two-database architecture.

Neptune Analytics has vector search, but with fundamental constraints. Vector index updates are explicitly not ACID compliant: changes to vector embeddings are non-atomic and become visible to concurrent queries immediately, even if the query that wrote them fails later. If a bulk load with embeddings fails midway, you end up with a partial set of embeddings and need to retry the entire operation. You can only create one vector index per graph, and it must be specified at graph creation time. There is no native full-text search. And because Neptune Analytics is an analytics engine, not a transactional store, it is designed for loading data from S3 or snapshotting from Neptune Database, not for serving live agent queries against data that changes in real time.

ArangoDB

ArangoDB is the closest multi-model competitor to SurrealDB's architecture. It supports documents, graphs, key-value, full-text search (via ArangoSearch), and as of version 3.12.4, vector search via FAISS integration. On paper, it has every primitive under one roof.

ArangoDB's vector index requires a --vector-index startup flag that permanently alters the RocksDB storage engine and cannot be reversed. Since v3.12.6, ArangoDB has supported attribute pre-filtering on vector searches — you can place FILTER operations between FOR and SORT to narrow candidates before the vector index runs:

FOR doc IN knowledge_base
  FILTER doc.tenant == @tenant
  FILTER doc.type == 'support'
  SORT APPROX_NEAR_COSINE(doc.embedding, @query_emb) DESC
  LIMIT 10
  RETURN doc

This addresses the multi-tenant filtering problem for attribute-based constraints and is a genuine improvement over earlier versions. Where the composition breaks down is the full retrieval pipeline. ArangoSearch (the full-text engine) uses a separate SEARCH operation with its own syntax and optimizer path. Graph traversals use FOR v, e, p IN ... GRAPH with a different iteration model. Vector search requires iterating over a collection directly with APPROX_NEAR_* functions. Composing all three — traversing a customer's product graph to scope candidates, running vector similarity against those scoped candidates, and blending in full-text relevance scores — requires orchestrating these as separate operations within AQL rather than expressing them as co-equal predicates in a single statement.

The pattern across all three is the same. Each database has added vector search as a capability. None of them have made it compose natively with graph traversal, structured filters, and full-text search in a single atomic statement. The operations exist. The composition does not. That composition is the specific thing SurrealDB was designed to provide.

Honest tradeoffs: when SurrealDB is not the right choice

If your primary workload is deep graph analytics (community detection, centrality algorithms, pathfinding over billions of edges), Neo4j's Graph Data Science library or TigerGraph's analytical engine are purpose-built for that. SurrealDB does not have an equivalent analytics library.

The honest question is whether your current retrieval accuracy is good enough. If your agents are producing correct results at an acceptable rate and the failure modes described in this article are not showing up in production, there is no reason to migrate. The architecture you have is working.

If you are trying to push accuracy higher and you are hitting a ceiling, the pattern is almost always the same: the individual operations work, but they do not compose. The vector search returns good candidates. The graph traversal follows the right paths. The filters check the right fields. But they run separately, against different snapshots, with intermediate results accumulating in memory, and the accuracy loss lives in the gaps between them. That is not a problem you can solve by tuning parameters. It is a structural limitation of running these operations across systems that were not designed to compose them. Closing that gap requires a database where graph traversal, vector search, full-text search, structured filters, and permission checks are co-equal predicates in a single atomic statement. That is what SurrealDB provides.

Conclusion

Every graph database covered in this article can store graphs, run traversals, and return results. Most of them can now do vector search. The thing none of them can do is compose all of it in a single atomic statement against a single consistent snapshot while also being the transactional system of record for the data your agent reasons over.

That is not a feature gap. It is an architectural one. And it is the gap where retrieval accuracy goes to die.

Exponential cost traps in database architectures: how SurrealDB breaks the cycle

Mark Gyles — Thu, 02 Apr 2026 12:33:01 +0000

Exponential cost traps in database architectures: how SurrealDB breaks the cycle

In the fast-paced world of application development, database choices can make or break an organisation's scalability and budget. Traditional polyglot persistence (relying on multiple specialised databases) often leads to exponential cost growth as systems expand. What starts as a simple setup spirals into a complex, inefficient infrastructure riddled with data duplication, ETL pipelines, and operational silos. SurrealDB, with its multi-model architecture unifying document, graph, vector, full-text, time-series, geospatial, and relational capabilities, offers a consolidated alternative that eliminates these pitfalls. By storing data once and querying it across models, SurrealDB can reduce overall costs by multiple factors, particularly in storage-heavy environments where cloud block storage is in use.

To understand the value of consolidation, consider the common trajectory of database sprawl. Below, we explore a real-world-inspired story of how costs escalate exponentially, helping readers identify if their setup is heading toward inefficiency. This narrative highlights the compounding effects of architectural decisions, setting the stage for how SurrealDB provides a streamlined, cost-effective path forward.

A story of exponentially increasing costs: from simplicity to sprawl

Many teams begin with modest needs but face rapid growth that exposes the limitations of single-model databases. Let's trace a typical app's evolution, illustrating how costs outpace traffic due to added complexity, redundancy, and management overhead.

1. Launching the app: a simple start

Sentiment: "We start simple: a single database, fast and cheap to run."

Your app is new, with straightforward transactional requirements. You deploy a single MySQL instance on AWS, adding one replica for high availability. It's low cost and easy to manage. Database expenses are minimal, infrastructure is straightforward, and personnel needs are light.


Database Costs:	Low
Traffic Handled:	Basic
Infra Cost:	Minimal
Personnel Required:	1–2 engineers
Temp cost:	None yet
Complexity cost:	None yet

Architecture:

MySQL (Primary)
MySQL (Failover Replica)

2. First wave of growth: scaling reads

Sentiment: "Our first scaling challenge is reads. Replicas help, but cost growth slightly outpaces traffic."

User traffic surges, dominated by reads. You add 8 read replicas and scale the writer to a larger instance (where per-vCPU costs rise disproportionately). Costs begin increasing slightly faster than traffic due to over-provisioning and load balancing.


Database costs:	Moderate
Traffic handled:	Increased
Infra cost:	Rising
Personnel required:	Still low, but monitoring grows
Temp cost:	None yet
Complexity cost:	None yet

Architecture:

MySQL Writer
Load Balancer
8x MySQL Read Replicas

3. Write traffic explodes: sharding begins

Sentiment: "As writes explode, we shard the system, but migrations are expensive and disruptive."

Writes overwhelm the single writer. You shard into four MySQL instances, over-provisioning each for spikes. A data access layer handles routing by key, and the migration temporarily doubles costs. Complexity introduces temporary and ongoing expenses.


Database costs:	High
Traffic handled:	Substantial
Infra cost:	Accelerating
Personnel required:	Increasing (sharding expertise needed)
Temp cost:	Spike from migration
Complexity cost:	Emerging

Architecture:

4x MySQL Shards (each with Writer + Multiple Read Replicas)
Load Balancers
CDC (Change Data Capture) for transitioning from old architecture

4. Analytics demands: the first pipeline

Sentiment: "Analytics needs have driven us to add a pipeline and duplicate data with added complexity as a result.”

Product teams demand dashboards and real-time insights. You build an Extract, Transform, Load (ETL) pipeline to an analytical database such as ClickHouse, duplicating data via Change Data Capture (CDC). Applications now query multiple systems, and engineers must manage query routing while maintaining consistency across stores.


Database costs:	Very high
Traffic handled:	Advanced
Infra cost:	Ballooning
Personnel required:	Still increasing (pipeline maintenance)
Temp cost:	Pipeline setup
Complexity cost:	Significant

Architecture:

Existing MySQL Shards
CDC + ETL
ClickHouse
Application querying multiple databases

5. AI ambitions: vector database enters

Sentiment: "AI ambitions require another new system, and more pipelines. The architecture is growing fast."

AI agents need RAG (Retrieval-Augmented Generation) support, querying across systems. You spin up a vector DB like Weaviate or Pinecone, with new pipelines for replication. Data is consolidated in S3 for training, adding sharding as usage grows.


Database costs:	Extreme
Traffic handled:	AI-enhanced
Infra cost:	Exponential
Personnel required:	Specialised teams
Temp cost:	AI integration
Complexity cost:	Overwhelming

Architecture:

Existing Setup
Additional ETL + Kafka
Weaviate
S3 for data lakes

6. Keeping it all running: growing ops

Sentiment: "Now we need a large ops team just to stay afloat. Costs are scaling exponentially, not linearly."

Ongoing tasks include re-sharding MySQL, scaling MongoDB/Weaviate/ClickHouse, upgrades across systems, node recoveries, and pipeline fixes. Teams manage credentials, security, and optimisations in isolation. Ops cycles dominate, slowing innovation and ballooning personnel costs.


Database costs:	Off the charts
Traffic handled:	Peak
Infra cost:	Exponential
Personnel required:	Large teams
Temp cost:	Constant disruptions
Complexity cost:	Debilitating

Architecture:

All systems with balancing, re-sharding, upgrades, and recoveries

Why the cost curve went exponential

Exponential costs are rarely caused by user growth alone. Instead, they often stem from an architecture that acts as a tax on every new login. Between the headaches of manual sharding, the hidden drain of data duplication, and the hours lost to rigid schema changes, many teams end up subsidizing a fragmented mess. If your infrastructure spend is climbing faster than your traffic, you are leaking resources rather than scaling. Moving toward a unified database simplifies the stack and returns your cost curve to a manageable, linear state.

The hidden costs of database sprawl: ETL, layers, and operational overhead

Building on this story, polyglot architectures create fragile ETL pipelines for data movement between systems like MySQL and ClickHouse. These demand resources for processing, debugging, and schema handling. Operational burdens include multiple query languages, fragmented backups, and custom synchronisation.

SurrealDB eliminates this by natively supporting all models in one platform with SurrealQL. No ETL needed. Data stays unified, reducing dependencies and accelerating development, as seen in replacements of PostgreSQL + Neo4j + RabbitMQ stacks.

EBS storage: the #1 cost driver in self-hosted databases

As the story shows, EBS volumes (and Azure and GCP’s equivalent) multiply with replicas and duplications, becoming the dominant expense. With EBS volumes not being eligible for discounts, costs compound in sprawls: 3x replication per cluster, plus copies across databases, over-provisioning, and snapshots typically result in massive redundant data duplication

For example, in a 50-node cluster with 4TB EBS volumes on AWS, storage will be $28,098 monthly, over twice the $12,577 for EC2 (factoring in discounts). Polyglot setups exacerbate this exponential growth.

SurrealDB's compression edge: reducing data size by 1/3 for dramatic savings

SurrealDB's storage engine (TiKV) delivers 70-80% better compression than PostgreSQL's, typically shrinking datasets to one-third. For the 4TB example, this cuts EBS to ~1.33TB, dropping costs to under $10,000, a 65% savings. In addition to the raw savings, smaller data reduces I/O, snapshots, and data transfer cost, countering the sprawl's redundancy.

Eliminating data duplication: organization-wide cost reductions

Storing data once avoids multi-system duplication. A single cluster with 3× replication replaces the primary and secondary copies spread across multiple database systems. Eliminating ETL removes staging costs, while unified data views simplify compliance and backups, reducing S3 storage and egress fees.

Linear scalability: ensuring cost-effective growth

Unlike most other database systems, SurrealDB scales linearly via TiKV, with less than a 5% performance deviation coefficient per node. You get roughly the same increase in capability and performance when adding a node to a 9-node cluster as when adding a node to a 100-node cluster. This keeps costs predictable and prevents exponential cost growth which is typical in sprawling platforms.

Conclusion: escape the exponential trap with SurrealDB

The story of sprawl reveals how polyglot decisions lead to exponential costs through duplication, complexity, and ops overhead. SurrealDB's "one database to rule them all" reverses this, slashing EBS-driven expenses via consolidation and compression. For teams in inefficient systems, migrating promises simplicity and transformative savings.

SurrealMX: In-memory storage with time travel and persistent storage

Mark Gyles — Thu, 26 Mar 2026 15:32:42 +0000

Author: Dave MacLeod

SurrealMX: SurrealDB's new in-memory storage backend

There have been quite a few blog posts since the release of SurrealDB 3.0 to introduce each of the new features we have to show. Today's post is about one called SurrealMX that you likely have yet to hear about - but use every day.

That's because the way you use it is...by typing surreal start. That's it!

That's because SurrealMX is in fact the name of SurrealDB's new in-memory datastore. And since SurrealDB stores data in memory by default, 100% of SurrealDB's users have already given it a try.

What is SurrealMX?

SurrealMX is SurrealDB's in-memory storage engine that was built from the ground up a bit over a year ago. It was merged into the core database during the 3.0 alpha period last November with no fanfare at all in order to try it out for the first time as SurrealDB's in-memory storage engine. Since SurrealDB uses in-memory storage by default, that means that SurrealMX has been every 3.x user's default storage option since then.

But SurrealMX isn't just a more performant way to store data in memory, it comes with new optional functionality too.

If you pay close attention to the SurrealDB source code you might have noticed a PR merged three weeks ago in the runup to 3.0 general availability that included a new way to pass in configuration parameters when starting a database.

If you go to this part of the documentation for the surreal start command, you can see a comparison of how this worked in 2.x versions compared to 3.0.

In 2.x versions, specifying whether SurrealKV is versioned or not was done before the path to the database which led to either writing surrealkv or surrealkv+versioned. Other backends didn't have similar configurations.

surrealkv://my_db
rocksdb://my_db
surrealkv://mydb
surrealkv+versioned://mydb
memory

In 3.0, this has now been changed to pass as many configurations as needed after the path to the database, in the same way that you do the same with a URL. Here are two examples from the documentation showing how this is done.

"surrealkv://path/to/db?versioned=true&sync=every&retention=30d"
"mem://tmp/data?versioned=true&aol=sync&snapshot=60s&sync=5s"
"mem://?versioned=true"

Did you notice that versioned=true is found not just in the surrealkv but also the mem backend? That's because SurrealMX includes versioned queries in the same way that SurrealKV does.

Let's take a closer look!

Time travel using the VERSION clause

With the addition of SurrealMX, SurrealDB now has two storage backgrounds that have versioning: SurrealKV and memory. Versioning is the capability to not just query a table as it stands now, but as it stood before. This feature is exceptionally important in areas like legal compliance in which a database needs to keep past information intact even if it has been overwritten later on.

In SurrealDB, time travel queries are done using the VERSION keyword. Let's give this a quick try with the in-memory backend. First we will start a server with ?versioned=true added:

surreal start --user root --pass secret "mem://?versioned=true"

And then connect via Surrealist or this command in the CLI.

surreal sql --user root --pass secret

Once that is done, we can create a person record or two and make some queries using VERSION followed by time::now() minus how ever long ago we would like to query.

CREATE person;
[{ id: person:g4z4v7yoju9pe21jck6v }]

SELECT * FROM person;
[{ id: person:g4z4v7yoju9pe21jck6v }]

// No 'person' records existed yesterday
SELECT * FROM person VERSION time::now() - 1d
[]

// Add a second record
CREATE person;
[[{ id: person:aczudhy0mvef5bxurcry }]]

// 'person' has two records, but about ten seconds ago
// it had just one
SELECT * FROM person VERSION time::now() - 10s
[{ id: person:g4z4v7yoju9pe21jck6v }]

Since the VERSION keyword has been available via SurrealKV since SurrealDB 2.0, it's very likely that you are already quite familiar with it. Let's now move on to in-memory persistent storage options which were entirely unavailable before SurrealDB 3.0.

In-memory with optional persistent storage and versioning

In-memory storage with optional persistent storage is one of the features of Redis that has made it so popular. SurrealMX offers this as well, via a somewhat more composable model.

Persistence for the memory backend is used for the following:

Append-Only Log (AOL) - Synchronous/asynchronous modes for durability
Snapshots - Periodic full database state capture
Data Recovery - Automatic recovery from snapshots + AOL on startup
AOL Truncation - Automatic cleanup after snapshots

You may be curious at this point how this differs from using SurrealKV (or RocksDB) to store your data. The difference is that SurrealMX persistent storage does not compress the data. It is an append-only-log of database writes, never compacted, until a snapshot occurs. It is in effect an entire database export. You can think of it as a convenience on top of in-memory speed while SurrealKV is used for traditional durable storage for datasets of any size.

Trying it out

Let's grab part of the documentation to see how it works. These are the available parameters to use when starting a database with in-memory storage.

retention (a duration string, e.g. 30d, 24h, 30m)
aol (never, sync, or async), for writing changes to an append-only log file
snapshot (a duration string, e.g. 60s), for periodically writing a snapshot of the database to the file system
sync (never, every, or a duration string like 5s), for specifying when to flush the append-only log file to the file system
never - (default) leave flushing to the OS (least durable)
every - sync on every commit (most durable)
interval - periodic background flushing at the given interval

We can match this up against the example above which shows that aol is set to sync, snapshot is set to 60s, and sync is set to 5s.

"mem://tmp/data?versioned=true&aol=sync&snapshot=60s&sync=5s"

That's quite a bit to think about, but fortunately the SurrealMX repo has a table that lays out exactly when you would want to use one configuration over another. Each of these is balanced against the performance hit, ranging from no persistence at all (the fastest) to Sync AOL + Every fsync which survives both process and system crash at the greatest cost to performance.

To make a choice, just ask yourself the following as you move down the chart: what do you want to happen to your data if the process or system crashes? Then choose the most performant option that offers you the guarantees you need.

Configuration	Survives Process Crash	Survives System Crash	Performance
No persistence	❌	❌	Fastest
Snapshot-only	⚠️ (last snapshot)	⚠️ (last snapshot)	Fastest
Async AOL + No fsync	⚠️ (mostly)	⚠️ (mostly + OS buffers)	Very fast
Async AOL + Interval fsync	⚠️ (mostly)	⚠️ (mostly + since last fsync)	Very fast
Async AOL + Every fsync	⚠️ (mostly)	⚠️ (mostly)	Very fast
Sync AOL + No fsync	✅	⚠️ (OS buffers)	Fast
Sync AOL + Interval fsync	✅	⚠️ (since last fsync)	Fast
Sync AOL + Every fsync	✅	✅	Slow

Looks like the quickest and easiest way to use persistent storage is through periodic snapshots. Since these must be set to a number at least greater than 30 seconds, we'll give it a try with 31.

surreal start --user root --pass secret "mem://tmp/data?snapshot=31s"

We can then connect with Surrealist or the surreal sql --user root --pass secret command and do a CREATE person or two.

After that, use cd tmp/data to go into the folder and wait for the file to show up. After 30 seconds you should see a file called snapshot.bin. That's the snapshot!

!v�4��y��
         /!cgdefault�?H/�y��mainmain/!ndI��� H�
��t����4P���I��� H�
��t��@~����D�&���I��� H�
...snip
*/***person*rh6xmo5z8cqffvp4hbf6��BQ7
                                     ��
/***user*john��T|��
nameJohn%

Once that file is present, you can stop and restart the database, use SELECT * FROM person and see that the person records are still there. Not bad at all!

We suspect that the most popular options will be the following four:

Snapshot only: the fastest option
Async AOL + no fsync: the fastest option that tracks every change the moment one happens
Sync AOL + no fsync: the fastest option if you think you might have a process crash
Sync AOL + every fsync: the most durable option, survives both process and systems crashes

But that remains to be seen.

If you've given persistent in-memory storage a try already or have used it since the publication of this blog post, please get in touch and let us know!

Where SurrealDB fits in your stack

Mark Gyles — Tue, 17 Mar 2026 09:02:54 +0000

Where SurrealDB fits in your stack: a path to multi-model efficiency

In today's data-driven applications, managing multiple specialised databases (often referred to as polyglot persistence) has become a common challenge. Teams juggle relational databases like PostgreSQL for structured data, document stores like MongoDB for flexible schemas, graph databases like Neo4j for relationships, and vector databases for AI workloads.

While powerful once finally set up, this configuration introduces complexity: multiple query languages, data pipelines, security models, and scaling strategies. Enter SurrealDB, an open-source multi-model database that unifies these paradigms into a single system. By consolidating databases into one platform, SurrealDB enables applications with multi-model capabilities, blending relational queries, graph traversals, vector search, time-series and geospatial analysis without the overhead of disparate tools.

This article explores how SurrealDB facilitates database consolidation, drawing from real-world benchmarks, case studies (e.g., Tencent consolidating nine databases into one), and performance comparisons. We'll map current polyglot setups to a consolidated future state, discuss ROI and TCO implications, and provide architecture overviews. If you're a technical evaluator considering SurrealDB as a replacement for your existing stack, this guide assumes basic database knowledge but no deep familiarity with SurrealDB's SurrealQL query language or internals.

Where SurrealDB fits in your deployment

SurrealDB positions itself as a versatile, AI-native database that scales from edge devices to petabyte clusters. Built in Rust, it supports deployment as an embedded library, single-node server, or distributed system. Its layered architecture separates compute (query processing) from storage (e.g., using TiKV for distributed persistence), allowing flexible scaling.

What it replaces

SurrealDB's multi-model design natively handles multiple data paradigms, making it a direct substitute for several specialised databases:

Document stores (e.g., MongoDB): SurrealDB stores data as flexible, JSON-like documents, supporting a default schemaless model that allows as much strictness to be added to your schema as your solution requires. It replaces MongoDB for content management, real-time apps, and unstructured data, with SurrealQL providing aggregation and querying akin to MongoDB's syntax but unified across models.
Graph databases (e.g., Neo4j): with built-in graph functionality, SurrealDB uses record links and graph edges as first-class citizens for traversals and relationships. It has been used to replace Neo4j in recommendation engines, fraud detection, and knowledge graphs, offering SQL-like syntax for graph queries without Cypher's learning curve.
Relational databases (e.g., PostgreSQL): SurrealDB emulates relational tables with ACID transactions, indexes, and joins (via record or graph links rather than traditional SQL JOINs for better scalability). It consolidates Postgres workloads for transactional data, geospatial queries, and time-series.
Vector and Search databases: integrated vector embeddings, full-text search, and hybrid retrieval replace standalone tools like Pinecone or Elasticsearch for AI apps, including RAG (Retrieval-Augmented Generation) pipelines.
Key-Value and Time-Series stores (e.g., Redis, InfluxDB): SurrealDB handles simple key-value pairs and temporal data with live queries for real-time updates along with an in-memory datastore that also includes persistent snapshot or AOL storage.

Case studies highlight replacements: one company swapped Neo4j, RabbitMQ, and Postgres for SurrealDB, while Tencent consolidated nine systems, reducing complexity and costs.

What it does not replace

While versatile, SurrealDB isn't a one-size-fits-all solution:

Pure OLAP or data warehouses (e.g., Snowflake, BigQuery): for massive analytical workloads requiring columnar storage and distributed joins across exabytes, SurrealDB's focus on multi-model OLTP (online transaction processing) may not match specialised OLAP tools' optimisation for ad-hoc queries.
Highly specialised hardware-accelerated databases: it doesn't replace niche systems like GPU-optimised databases for extreme-scale ML training or legacy mainframe databases with proprietary integrations.
Non-database tools: SurrealDB consolidates data storage but not orchestration (e.g., Kubernetes).

The Outcome

A single SurrealDB instance or cluster handles all, with SurrealQL as the query interface. Compute nodes process queries, storage persists data distributively.

Benchmarks show efficiency: SurrealDB often outperforms single-model databases in mixed workloads (e.g., faster than MongoDB in reads, comparable to Neo4j in graphs).

Mapping the customer journey into consolidation: ROI and TCO over time

Adopting SurrealDB follows a phased journey, yielding compounding ROI (return on investment) and reduced TCO (total cost of ownership).

Phase 1: evaluation (months 1-2)

Assess: map current databases to SurrealDB models using migration guides (e.g., from Neo4j/PostgreSQL) and Surreal Sync migration tool.
POC: run SurrealDB in a testing environment as the backend for a single application; test key validation areas such as:
- Setup and integration: install SurrealDB locally or via SurrealDB Cloud, in your application stack, and verify connectivity with existing microservices.
- Data migration: import sample datasets from your current databases and test schema definitions in schemaless or schemafull modes.
- Query performance: execute mixed-model queries (e.g., combining document lookups with graph traversals and vector searches) and benchmark against your existing setup for read/write throughput, latency, and resource usage.
- Feature validation: test real-time capabilities like live queries and change feeds; evaluate AI features such as vector embeddings and hybrid search for RAG use cases; assess geospatial and time-series handling.
- Security and compliance: configure authentication (e.g., JWTs, scopes), test access controls, and ensure alignment with your security best practices like least privilege.
- Scalability and reliability: simulate load with tools like the SurrealDB CLI or Surrealist dashboard, test failover in a small cluster, and monitor performance metrics.
- Developer experience: use the Surrealist dashboard for querying and visualisation; experiment with SurrealQL syntax to rewrite existing queries from MongoDB/Neo4j/PostgreSQL.
Initial ROI: is higher than baseline due to the overhead on the team doing the POC/migration, and temporarily running 2 different databases in parallel until full cut-over.

Phase 2: partial consolidation (months 3-12)

Migrate one workload (e.g., replace MongoDB for documents).
Integrate with AI frameworks (e.g., LangChain for RAG).
ROI: faster feature iteration performance improvements and lower overhead. TCO: 10-50% reduction by eliminating 2-3 databases, cutting licensing/maintenance.

Phase 3: full multi-model shift (year 1+)

Consolidate all: from polyglot deployment model to a unified stack.
Scale: horizontal clustering for high availability.
ROI: 2-3x faster time-to-market; AI apps ship quicker with native vectors/graphs. TCO: Up to 70% savings via fewer systems, no ETL, and simplified ops (e.g., one monitoring setup).

Over time, ROI grows exponentially as complexity drops: enterprises report 3-4x lower costs through consolidation than managing 3-4+ databases.

Conclusion

SurrealDB offers a compelling consolidation path for technical teams, replacing multiple databases while supporting multi-model apps. By mapping your current polyglot setup to its unified models, you can achieve significant ROI through faster development and lower TCO. Start with a POC to evaluate fit. Learn what companies like Tencent, Walmart and Samsung already know. For more, explore SurrealDB's documentation or GitHub, or visit our Discord community.

OpenAI’s Postgres Architecture: A Brilliant Fix for a Billion-Dollar Mistake

Mark Gyles — Wed, 11 Mar 2026 23:33:25 +0000

Author: Matthew Penaroza, Head of AI Solution Architecture at SurrealDB

OpenAI’s write-up on scaling PostgreSQL for ChatGPT is worth reading twice: once as an operations playbook, and once as a case study in what happens when a system outgrows its original shape. The engineering is excellent. But the database they started with leaves a lot to be desired and has almost certainly cost the company millions of dollars in completely unnecessary infrastructure pain.

What I’ve noticed on social media is that many people are treating this article as proof of how amazing the engineering is at OpenAI (which it is) and as a blueprint for how their own organizations should scale. I’d argue the opposite. This architecture is a lesson in what not to do. Just because brilliant engineers can make something work doesn’t mean you should. I wouldn’t eat soup with a fork, and I wouldn't use a single-writer PostgreSQL cluster to serve 800 million users.

Why this architecture exists (and why it shouldn't)

The scaling strategies OpenAI used here are very familiar to me. I've helped design the database architecture for some of the largest transactional systems on the planet, and what OpenAI did mirrors patterns I’ve seen in the field. What’s unusual is the timing. Most companies I’ve seen running this exact playbook are 2010s organizations that started on MySQL or Postgres before the distributed-database space matured. They scaled with read replicas and orchestration because migrating later was impractical. That isn’t OpenAI. They made these architectural decisions presumably in the early 2020s, when far better options already existed. My suspicion is that their technical decisions had a lot to do with the fact that Microsoft has a large stake in OpenAI and wanted them running on Azure services.

(Traditional 2010s Postgres Architecture)

What OpenAI actually built

But enough history. Let's look at what they actually built.

OpenAI runs a single primary Azure PostgreSQL Flexible Server instance with nearly 50 full read replicas distributed globally, handling millions of QPS for ChatGPT and the API. All writes hit the single primary. Reads are aggressively offloaded to replicas. They have explicitly avoided sharding the existing PostgreSQL deployment because of the inherent complexity and time-consuming nature of that adaptation, a fully understandable call at their scale.

A single-writer Postgres instance, even the biggest one Azure offers, obviously can’t handle this kind of write volume. So OpenAI migrated shardable workloads and write-heavy tables to Azure Cosmos DB and blocked all new tables/features on the main PostgreSQL cluster. New workloads now default to sharded alternatives. This also likely means they have applications stitching data together from both Cosmos DB and Postgres.

Because Azure PostgreSQL caps connections at 5,000, OpenAI deployed PgBouncer in Kubernetes. Each read replica gets its own K8s deployment with multiple PgBouncer pods behind a Service for load balancing. This dramatically cuts active connections and connection-setup latency.

On replication: the primary currently streams WAL directly to all ~50 replicas. That’s sustainable today on huge instances with fat pipes, but it has hard limits. To go further, OpenAI is having the Azure Postgres team build out a new feature for them: cascading replication (intermediate replicas relay WAL downstream). But the feature is still in testing.

Other key optimizations:

Aggressive query tuning to kill expensive multi-table joins and ORM anti-patterns
Multi-layer caching with cache locking/leasing to survive cache-miss storms from overloading their PostgreSQL instances
Workload isolation: high-priority vs low-priority traffic (and different products) routed to separate instances to avoid noisy-neighbor issues
Multi-layer rate limiting (application, proxy, connection pooler, and per-query) plus careful retry policies to prevent spikes and retry storms
Restricted schema changes: only lightweight operations allowed (no full table rewrites); backfills are heavily rate-limited; new tables forbidden
High-availability setup on the primary with a hot standby for fast failover, plus enough replica headroom per region to survive individual replica failures

The results: impressive… and concerning

The result is a system that's… kinda stable: double-digit ms p99 client-side latency, claimed five-nines availability, and “only one SEV-0 PostgreSQL incident” in the past year. (I’m questioning how they measure five-nines availability if they’re happy about only having one SEV-0 this past year, but I digress.)

That’s impressive problem-solving, and it’s a credit to OpenAI’s engineering talent that they pulled it off. Many organizations fail at far smaller scale.

But let’s be clear: I have never in my career heard an infrastructure engineer be happy about having “only one SEV-0” on their primary transactional database infrastructure in a year. (Can you imagine what that says about the previous years?) A SEV-0 is the highest-severity outage. This is not normal for a well-architected system (granted, most systems are not well architected). Even in gigantic, high-growth companies, many go years without a SEV-0 on their primary transactional database system, or have never even had one.

The problem with this system is that it currently is and is going to continue to be constantly on the verge of a SEV incident, and it will never stop unless they shard or migrate off of it. Growth is endless (or it is if the company is healthy, which hopefully OpenAI stays healthy), which means more data, QPS, connections, etc. every year forever. And they have put themselves in a situation where there are inherent architectural limits that have required all kinds of architectural workarounds and special arrangements with their cloud hosting provider just to keep things running. Not to mention teams and teams of engineers at both Azure and OpenAI just to keep this alive who could all probably be better utilized to increase functionality instead of keeping a system alive. Remember, Azure Database for PostgreSQL Flexible Server only allows for 5 read replicas, so in order for OpenAI to scale this way they had to have custom bespoke solutions made for them by the Azure engineering team.

The real cost: engineering heroics forever

Virtually every problem they had to solve has already been solved, out of the box, by modern distributed databases. They would have saved millions in infra spend, thousands of engineering hours, and future-proofed the system by choosing something designed for this scale from the start.

But I don’t blame the OpenAI team; maybe this level of growth wasn’t anticipated, or maybe internal constraints forced their hand. No one knows but the OpenAI team.

But the point I want to make with this article is: if you were designing the database layer for this class of workload today, should you do this? And my very strong recommendation would be no. Unless you don't care about time, money, outages, your engineers' sanity, weekends, holidays, sleeping through the night, developer velocity, future-proofing, your remaining hair, or basic human joy.

The alternative: start with distributed architecture

So what's the alternative? You should use a database architecturally designed for this type of scale from the start. In the rest of this piece, I’ll use SurrealDB as the example (full disclosure: they pay me, so I’m biased, at least until the iron fist of the CEO comes down on me for writing this article). That said, the underlying principles I’m about to walk through apply to many modern distributed systems, not just SurrealDB. Bias or not, the technical realities are what matter. And those realities should bias you towards a very different path forward, one that avoids nearly all of the problems OpenAI had to solve through sheer operational heroics. Let’s walk through it.

The problems PostgreSQL forced OpenAI to solve, and how a native distributed system like SurrealDB avoids them entirely

Let’s go through the biggest pain points OpenAI encountered (including ones they didn’t mention) and why almost every one of them disappears with a distributed database.

1. Fundamental scale limitations of a 1990s design

PostgreSQL came out of Berkeley in the mid-1990s, back when a few terabytes felt huge and “big data” was whatever fit on a lab server. And while it's seen a lot of development since then, the core design was never meant to handle planet-scale traffic across regions with millions of QPS and hundreds of millions of users. All the tricks OpenAI threw at it (read replicas, PgBouncer, begging Azure for cascading replication) are basically duct tape on a system that just wasn’t built for this.

Databases like SurrealDB were architected at their core for this level of modern distributed scale. A SurrealDB distributed deployment uses the TiKV storage engine, the same storage engine that some of the biggest banks, insurance companies, Flipkart, Databricks, Pinterest, Plaid, Atlassian, and others use in clusters with hundreds of nodes, pushing millions of QPS and petabytes of data. And these companies are not constantly dealing with SEV-0 events. The fundamental architecture is designed to handle this traffic without the same kind of custom patches and workarounds.

In addition, databases like SurrealDB do not have the same connection limit issues as older relational databases, where you’re capped at 5,000 connections. So the connection pooling problem OpenAI had to overcome becomes mitigated (though it's still a good idea to optimize the system).

2. The insane replica cost

OpenAI has ~50 full read replicas, which means 50 complete copies of the whole dataset. Block storage is the priciest, least-discountable thing on any cloud bill, and copying dozens/hundreds of TBs of data fifty times is throwing away millions a year for no extra value beyond read capacity. On top of that, each replica is its own isolated box that can’t share compute. You have to overprovision every single one to survive its own spikes, so half (or very likely more) of your CPU is probably sitting idle 95% of the time. And when traffic surges, you can’t just add compute instantly. Provisioning a new replica and replicating the full dataset takes a huge amount of time (potentially hours). That’s why OpenAI overprovisions well in advance, to give themselves a buffer while new replicas catch up. But this architecture means that they need accurate forecasting and plenty of spare capacity ahead of time.

And if you don't believe me on the storage pricing analysis, here's the AWS bill calculator for 50 64TB EBS volumes and 50 128vCPU instances. How much more expensive do you think storage is vs. compute?

That’s right. Storage costs ~7× more than compute in this example. This can vary depending on the disk type, how much data and compute you need, and whether you are using managed services. In your situation it might only be 4× or 2×, but you get the gist. And while this example is from AWS, this is true across all cloud hosting providers.

The core reason is that with reserved instances you can get huge discounts on compute with all the cloud providers, up to 72% or even 80%+ with enterprise discount plans. The standard discount for block storage is 0%. I have seen up to 5% with special arrangements on AWS, but that was for one of their largest customers. This is why scaling via read replicas is such a financial disaster. Now, maybe OpenAI gets special treatment here because they are partially owned by Microsoft and this isn’t relevant to them, but it should be relevant to you if your company cares about money.

And feel free to verify these cost figures yourself:

But how does a distributed database like SurrealDB address this? When you store data on SurrealDB, it uses a separate node entirely for storage that replicates it 3 times, auto-shards it into small chunks across the cluster, and distributes it across the cluster for high availability.

And if you need to scale throughput and concurrency, you can do so by adding cheaper stateless compute nodes that all read from the same shared storage pool. No full copies. And on top of that, the stateless compute nodes can be removed and spun up dynamically at any time, which means you don’t have to massively overprovision for any given event. The system is flexible enough to be scaled up and down all the time, saving huge costs on overprovisioning. And in the event you do have a high-growth event and you need more compute, it only takes a few minutes to get the cluster powered up enough to handle the increased workload.

All this means that by going with a distributed architecture you could, depending on your scenario, decrease your current/future total cloud bill by several factors.

3. The single-writer bottleneck and the forced migration to a separate NoSQL system

In PostgreSQL’s single-primary architecture, every single write (updates, inserts, deletes) has to funnel through that one primary instance. No matter how large the machine (and OpenAI is clearly running one of the biggest Azure offers), a sudden write spike can still saturate CPU, I/O, or WAL bandwidth. Latency climbs, queries start timing out, retries pour in, and you’re suddenly in a classic overload death spiral that can degrade ChatGPT and the API for everyone.

OpenAI’s fix was to identify the workloads that were shardable and write-heavy (many of which are probably fundamentally relational in nature), migrate them to Cosmos DB, and accept the permanent cost of running two completely different database systems side-by-side. That means separate query languages, different consistency models, distinct operational tooling, and application code that has to stitch results together across the two stores forever.

A distributed database like SurrealDB is multi-writer by design. Writes are spread across multiple nodes in the cluster, so write capacity grows horizontally as you add machines. There is no single primary choke point, and no need to spin up and maintain an entirely separate database just to handle write load.

4. Expensive multi-table joins killing the cluster

OpenAI repeatedly calls out multi-way joins as a major source of SEV incidents. They had to audit ORM output, break joins into application-side logic, and basically treat joins as dangerous. That’s not a great place to be when your data model is relational.

SurrealDB avoids this class of failure by not relying on traditional relational joins at all. Relationships are modeled explicitly as record links and graph edges, and resolved through traversal inside a single query engine rather than join operators. This eliminates join explosion and centralized join execution while still allowing expressive relationship queries across documents, graphs, and vectors without forcing application-side orchestration or creating single-node saturation risks.

5. Schema changes become impractical

OpenAI now bans anything beyond the lightest schema changes and rate-limits backfills for weeks. That’s a huge velocity tax. Every new column or type change becomes a high-risk, multi-week operation.

SurrealDB reduces schema-change issues by decoupling schema definition from rewriting existing storage. Where relational systems often force heavy DDL and backfills just to keep data and schema synchronized, SurrealDB’s current approach enforces schema primarily as data is written/updated, and the team has explicitly identified richer zero-downtime mechanisms (aliases/versioning/async migrators) as the path to handling large-scale online migrations and reindexing without downtime. This results in a scenario where you can make a change to a table at scale without complexity or waiting.

6. Stability constantly threatened by spikes

In the single-primary PostgreSQL setup, any unexpected surge (cache-miss storms, new feature launches, retry loops, or sudden traffic growth) can quickly saturate a single instance’s CPU, I/O, or connection limits. Latency spikes, requests time out, retries amplify the load, and the whole service risks cascading degradation. The primary is still a true single point of failure (even with a hot standby), and losing a read replica requires pre-planned headroom across every region. OpenAI had to build multiple layers of rate limiting, workload isolation, and cache leasing just to keep the system from regularly tipping over.

A distributed architecture with separated compute and storage handles this differently. Stateless query nodes can be scaled automatically in seconds during spikes, absorbing bursts without saturating any one machine. High availability is native: individual node failures are isolated, and automatic failover and rebalancing occur without any intervention.

7. Astronomical ongoing engineering overhead

This setup requires constant tuning, custom Azure features, cascading replication experiments, query audits, rate-limit adjustments, connection-pool management, and on-call heroics to prevent the next SEV-0. That’s a permanent tax on engineering bandwidth. People who could be building new features are instead keeping Frankenstein alive.

SurrealDB is designed for distributed scale from the start and is intended to be boring at this traffic level. The problems OpenAI is constantly solving just aren’t issues with databases like SurrealDB. Now, every database will encounter problems at hyperscale (and tons of them (and when I say tons, I mean tons; scale is hard)), but they won't be nearly as severe. In fact, with many of the (healthy) companies I’ve worked with, the majority of the time infrastructure engineers spend is more on the optimization and functionality side of things.

8. Inevitable database sprawl

By freezing the core PostgreSQL schema and banning new tables entirely, OpenAI has ensured that any future workload with different performance characteristics or heavier writes will need its own separate database system (Cosmos DB today, likely others tomorrow). That guarantees growing proliferation, more cross-system data movement, and increasing consistency and operational challenges over time, which, by the way, likely means more data transfer fees and more replicas across databases.

(A possible future of OpenAI’s architecture)

SurrealDB is multi-model, which can evolve with the product. It can incorporate relational, document, geospatial, temporal, graph, vector, and time-series data in the same cluster as requirements evolve, avoiding the need to continually spin up new databases.

9. Locking yourself out of future AI workloads on your own data

This is the subtler long-term cost. OpenAI has arguably the most valuable dataset on the planet, but their core transactional store is perpetually at capacity and can’t absorb new workloads. To do anything interesting with that data at scale, they’ll need to replicate it elsewhere, adding cost, latency, time, and consistency risk.

SurrealDB is built for exactly this future: relational + temporal + document + graph + vector + time-series in the same system, on the same dataset, without replication or ETL. You can run transactional workloads, analytics, RAG, and recommendation engines against the same live data without fear of blowing up the cluster.

Database infrastructure shouldn’t require miracles

OpenAI’s infrastructure team pulled off a minor miracle keeping a single-writer PostgreSQL cluster alive at this scale. The optimizations are clever, the operational discipline is elite, and the fact that ChatGPT mostly works for 800 million users is a testament to their skill.

But miracles aren’t scalable. This architecture is a patchwork of workarounds, custom Azure features, and permanent firefighting, held together by constant engineering effort that could be spent building new capabilities instead. All to compensate for fundamental limitations of a 1990s-era database design pushed far beyond its natural habitat.

If you’re building something ambitious in 2026, don’t inherit these problems. Choose a database built for distributed scale, horizontal everything, multi-model flexibility, and reliably boring operations from day one.

Your bill will be lower, your on-call rotation will sleep better, and you’ll have vastly more freedom to experiment with the AI-native workloads that will define the next decade.

Pick the right foundation the first time. Everyone on your team will thank you.

SurrealDB is now available on AWS Marketplace

Mark Gyles — Wed, 11 Mar 2026 18:08:27 +0000

We’re excited to announce that SurrealDB is now officially available on AWS Marketplace.

You can now deploy SurrealDB Cloud with streamlined procurement, simplified billing, and faster time to production - all through your existing AWS accounts.

This launch also reinforces our commitment to meeting customers where they build, and deepens our partnership with AWS to make SurrealDB easier to adopt, deploy, and scale in the cloud.

Why this matters

AWS Marketplace makes it easier for companies to adopt SurrealDB with:

Seamless AWS-native deployment
Consolidated billing
Faster security and procurement approval
Production-ready infrastructure from day one.

If you’re already building on AWS, getting started with SurrealDB just became frictionless.

Why SurrealDB?

SurrealDB combines document, graph, and relational capabilities in one unified database - designed for modern applications, real-time systems, and AI-powered workloads.

With flexible schema, SurrealQL, and built-in real-time features, SurrealDB simplifies your technology stack, reduces Total Cost of Ownership and operational complexity, and allows you to ship products and features in days rather than weeks.

Get started

SurrealDB is available now on AWS Marketplace. We can’t wait to see what you build.

How to use Surrealism to build your own custom SurrealDB extensions

Mark Gyles — Wed, 11 Mar 2026 16:56:48 +0000

Author: Dave MacLeod

How to use Surrealism to build your own custom SurrealDB extensions

Two weeks ago we announced SurrealDB 3.0, along with a new feature called Surrealism that lets you write your own functions in Rust that can then be annotated to call directly from the database.

In this blog post, we are going to take a look at what use cases Surrealism solves and easy it is to get started with it. Let's begin by taking a look at how functions are used in SurrealQL and how Surrealism expands their range of use.

Functions in SurrealDB

SurrealDB has always had user-defined functions that look like this:

DEFINE FUNCTION fn::combine($one: number, $two: number) -> number {
  $one + $two
};

fn::combine(10, 10); -- 20

These functions can get to any complexity and length that you like.

DEFINE FUNCTION fn::relate_all($records: array<record>) {
  IF $records.len() < 2 {
      -- Don't do anything, ending the recursion
  }  ELSE {
      LET $first = $records[0];
      LET $remainder = $records[1..];
      FOR $counterpart IN $remainder {
          RELATE $first->to->$counterpart;
      };
      fn::relate_all($remainder);
  }
};

The next type of function was added in SurrealDB 2.0: closures. Closures are anonymous functions that can work in read operations that can access both arguments and parameters in their scope.

Defining and using a closure

LET $log_level = "DEBUG";

-- Define a closure
LET $split = (|$record: record| {
    table: $record.tb(),
    key: $record.id(),
    log_level: $log_level
});

$split(user:1);
[person:one, cat:two].map($split);

Closure output

-- Calling $split
{
    key: 1,
    log_level: 'DEBUG',
    table: 'user'
}

-- Passing $split into .map() which takes a closure
[
    {
        key: 'one',
        log_level: 'DEBUG',
        table: 'person'
    },
    {
        key: 'two',
        log_level: 'DEBUG',
        table: 'cat'
    }
]

However, both defined functions and closures live inside the SurrealQL query language inside the database.

Surrealism surpasses this limitation by letting you define functions in Rust instead, with all of the available bells and whistles that you get from the Cargo package manager and crates.io ecosystem.

These functions are then compiled to a WASM binary that then can be linked to from the database itself.

Being able to add functions just like that means that new functionality can be added today, to your existing SurrealDB version, with the only setup being two DEFINE statements - one to set up a file bucket, and one more to show the database where inside this bucket the WASM file is located.

Why we made Surrealism

Making Surrealism was in fact an absolute necessity for a project like SurrealDB in which the community plays such a large part. That's because a lot of requests are for features that are nice to have, but don't quite make sense to merge into the code for the database itself. But with an extension system, you can make the feature yourself!

Using Surrealism is basically like installing apps on your phone. You can't modify the OS of the phone itself, but you can install a new app any time you find one that appeals to you. Similarly, you can build and upload an app for other people to try out.

Some examples

Let's look at two case studies where Surrealism allows a functionality to be added that was not possible before.

Case 1: A well-intended PR

Sometimes you will see a well-intended PR that isn't able to be accepted, such as this one. That PR used Rust's fake crate to create a number of database functions that could be used to seed a database with mock data: fake names, addresses, companies, and so on. The function signatures would have looked like this.

fake::address::building_number
fake::address::city
fake::address::country
fake::address::country_code
fake::address::state
fake::address::street
// ... and so on...

While useful (and tempting), it would lead to an increase in the binary size for a functionality that is not core to a database, so it was not able to be merged.

Case 2: A feature request

Another example comes in the form of feature requests, such as this one from a user who would like to see Serbian added to the list of languages that could use the Snowball filter.

First let's look at what the Snowball filter is and what it does. This filter is used inside a DEFINE ANALYZER statement to enable full-text search. The filter uses something called stemming which reduces words to something like a root form. This is best seen through an example. In the two queries below we have defined an analyzer that first splits a string by class (whitespace, dots, numbers vs. letters, etc.) and then passes it through the Snowball filter for English.

DEFINE ANALYZER en_snowball TOKENIZERS class FILTERS snowball(english);
search::analyze(
        "en_snowball",
        "Making Surrealism was in fact an absolute necessity for a project like SurrealDB..."
);

The output shows some words like 'absolut' and 'necess', which are the shortest possible forms that can be used to match on a word. Note that this isn't an exact science as you could argue that 'make' should be 'mak' in order to match on forms like 'making'. But that is a subject for another time.

['make', 'surreal', 'was', 'in', 'fact', 'an', 'absolut', 'necess',
 'for', 'a', 'project', 'like', 'surrealdb', '...']

Now, the interesting part about this feature request is that Serbian is one of the languages with a Snowball stemmer. So adding Serbian is technically doable, except that it would require quite a bit of work.

The reason why is that the crate that SurrealDB uses to implement it hasn't been updated for a while. There was even a PR to add Serbian to it but it was never merged and has since been closed. So adding Serbian to the list of languages would require something like the following:

Try contacting the crate author to bring up the issue again
Wait a while, fork the repo if there is no response
Contacting the author of the PR to add Serbian to see if it's okay to use that PR, or implement it ourself
Follow up with a PR to the SurrealDB source code

Now, if you'd like to put a PR together that accomplishes all that then feel free!

However, thanks to Surrealism, this sort of problem can be resolved today.

Making a Surrealism module

So let's put a Surrealism module together that allows you to create mock data as well as work with Serbian text. This section is a somewhat quicker walkthrough of the same steps as this page in the documentation, so check that page out too to see more details.

There is a bit of boilerplate to start.

The boilerplate

First we need to add a line in Cargo.toml to create a dynamic library that can be loaded from another language.

Add to Cargo.toml

[lib]
crate-type = ["cdylib"]

And then a surrealism.toml file is needed. We can just copy and paste this into the file.

Create surrealism.toml

[package]
organisation = "surrealdb"
name = "demo"
version = "1.0.0"

[attach]
fs = "fs"

[capabilities]
allow_scripting = true
allow_arbitrary_queries = true
allow_functions = ["fn::test"]
allow_net = ["127.0.0.1:8080"]

Annotating functions

Now we can make some functions and annotate them.

The code for the fake data couldn't be simpler, as the fake crate uses a dedicated function for each type of fake data. You do have to choose a language though. Let's go with German for the fun of it.

To make a function a Surrealism function, just add the #[surrealism] annotation to the top. This will allow them to be exposed in the binary once the code is compiled.

// Inside lib.rs
use fake::{
    Fake, 
    faker::{
        address::de_de::{CityName, CountryName}, 
        company::de_de::CompanyName, creditcard::de_de::CreditCardNumber, 
        name::de_de::Name
    }
};
use surrealism::surrealism;

#[surrealism]
pub fn city() -> String {
    CityName().fake()
}

#[surrealism]
pub fn company_name() -> String {
    CompanyName().fake()
}

#[surrealism]
pub fn full_name() -> String {
    Name().fake()
}

Calling these functions will generate an output something like this.

Oberschreiberheim
Schmidt and Götz Stiftung
Gerhard Hoffmann

Now let's move on to Serbian. Implementing Snowball ourselves would be a lot of code, so let's go with something simpler for demonstration: stop words. Stop words is a method by which you remove all the most frequent words from a language that are not very useful when working with text. For English these are words like the, a, an, but, at, on, is, and so on.

Rust has a stop words crate which doesn't have Serbian, but does have Croatian (language code hr). Both Serbian and Croatian are one of the standards of Serbo-Croatian which means that a tool for one works pretty well for another.

This crate has an example of the tool in action here.

fn main() {
    // Read in a file
    let document = std::fs::read_to_string("examples/foreword.txt").expect("Cannot read file");

    // Print the contents
    println!("Original text:\n{}", document);

    // Get the stopwords
    let words = stop_words::get(stop_words::LANGUAGE::English);

    // Remove punctuation and lowercase the text to make parsing easier
    let lowercase_doc = document.to_ascii_lowercase();
    let regex_for_punctuation = human_regex::one_or_more(human_regex::punctuation());
    let text_without_punctuation = regex_for_punctuation
        .to_regex()
        .replace_all(&lowercase_doc, "");

    // Make a regex to match stopwords with trailing spaces and punctuation
    let regex_for_stop_words = human_regex::word_boundary()
        + human_regex::exactly(1, human_regex::or(&words))
        + human_regex::word_boundary()
        + human_regex::one_or_more(human_regex::whitespace());

    // Remove stop words
    let clean_text = regex_for_stop_words
        .to_regex()
        .replace_all(&text_without_punctuation, "");
    println!("\nClean text:\n{}", clean_text);
}

Now we have a small hiccup here: unlike Croatian, Serbian is also written using the Cyrillic alphabet. However, a quick search through crates.io shows that there is a crate to convert Serbian Cyrillic to Latin and the other way around.

Combining these two, we can create a function called serbian_stop() that converts the input to Latin or Cyrillic. Instead of exposing that, we'll make two more functions, serbian_stop_latin() and serbian_stop_cyrillic(), which will get the #[surrealism] annotation so that they can be called directly from the database.

pub enum Alphabet {
    Latin,
    Cyrillic,
}

#[surrealism]
pub fn serbian_stop_latin(document: String) -> String {
    serbian_stop(document, Alphabet::Latin)
}

#[surrealism]
pub fn serbian_stop_cyrillic(document: String) -> String {
    serbian_stop(document, Alphabet::Cyrillic)
}

pub fn serbian_stop(document: String, alphabet: Alphabet) -> String {
    // Make sure it's the Latin alphabet first
    let document = Convertion::from_cyrillic(&document);

    // Get the stopwords
    let words = stop_words::get("hr");

    // Remove punctuation and lowercase the text to make parsing easier
    let lowercase_doc = document.to_ascii_lowercase();
    let regex_for_punctuation = human_regex::one_or_more(human_regex::punctuation());
    let text_without_punctuation = regex_for_punctuation
        .to_regex()
        .replace_all(&lowercase_doc, "");

    // Make a regex to match stopwords with trailing spaces and punctuation
    let regex_for_stop_words = human_regex::word_boundary()
        + human_regex::exactly(1, human_regex::or(words))
        + human_regex::word_boundary()
        + human_regex::one_or_more(human_regex::whitespace());

    // Remove stop words
    let clean_text = regex_for_stop_words
        .to_regex()
        .replace_all(&text_without_punctuation, "");

    // Return as input user requested
    match alphabet {
        Alphabet::Cyrillic => Convertion::from_latin(&clean_text),
        Alphabet::Latin => Convertion::from_cyrillic(&clean_text),
    }
}

Putting all the functions together gives us this code.

use fake::{
    Fake,
    faker::{address::de_de::CityName, company::de_de::CompanyName, name::de_de::Name},
};
use serbian_cyrillic_latin_conversion::Convertion;
use surrealism::surrealism;

#[surrealism]
pub fn city() -> String {
    CityName().fake()
}

#[surrealism]
pub fn company_name() -> String {
    CompanyName().fake()
}

#[surrealism]
pub fn full_name() -> String {
    Name().fake()
}

pub enum Alphabet {
    Latin,
    Cyrillic,
}

#[surrealism]
pub fn serbian_stop_latin(document: String) -> String {
    serbian_stop(document, Alphabet::Latin)
}

#[surrealism]
pub fn serbian_stop_cyrillic(document: String) -> String {
    serbian_stop(document, Alphabet::Cyrillic)
}

pub fn serbian_stop(document: String, alphabet: Alphabet) -> String {
    // Make sure it's the Latin alphabet first
    let document = Convertion::from_cyrillic(&document);

    // Get the stopwords
    let words = stop_words::get("hr");

    // Remove punctuation and lowercase the text to make parsing easier
    let lowercase_doc = document.to_ascii_lowercase();
    let regex_for_punctuation = human_regex::one_or_more(human_regex::punctuation());
    let text_without_punctuation = regex_for_punctuation
        .to_regex()
        .replace_all(&lowercase_doc, "");

    // Make a regex to match stopwords with trailing spaces and punctuation
    let regex_for_stop_words = human_regex::word_boundary()
        + human_regex::exactly(1, human_regex::or(words))
        + human_regex::word_boundary()
        + human_regex::one_or_more(human_regex::whitespace());

    // Remove stop words
    let clean_text = regex_for_stop_words
        .to_regex()
        .replace_all(&text_without_punctuation, "");

    // Return as input user requested
    match alphabet {
        Alphabet::Cyrillic => Convertion::from_latin(&clean_text),
        Alphabet::Latin => Convertion::from_cyrillic(&clean_text),
    }
}

Compiling and accessing the Surrealism module

With the functions done, it's now time to compile the Surrealism module. That can be done with the surreal module build command. Follow this with --out and the file name, which must have the extension .surli.

surreal module build --out serbian_and_fake.surli

You should see the following output.

Building WASM module...
   Compiling random v0.1.0 (/Users/mithr/surrealism_test)
    Finished `release` profile [optimized] target(s) in 2.38s
Optimizing bundle...

Now it's time to start the database with surreal start. It needs two environment variables to make this happen: one to allow the files and surrealism experimental features to be used, and another that shows the location of the .surli file that we just compiled.

SURREAL_CAPS_ALLOW_EXPERIMENTAL=files,surrealism SURREAL_BUCKET_FOLDER_ALLOWLIST="/Users/my_name/my_rust_code/" surreal start --user root --pass secret

Now we'll connect to the database either via Surrealist or the following CLI command.

SURREAL_CAPS_ALLOW_EXPERIMENTAL=files,surrealism surreal sql --user root --pass secret

Once we are connected we just need two more statements: one to define a bucket for the files, and another to define a module, a module being a collection of Surrealism functions.

DEFINE BUCKET my_bucket BACKEND "file:/Users/my_name/my_rust_code";
DEFINE MODULE mod::my_funcs AS f"my_bucket:/serbian_and_fake.surli";

And that's all you need to do! Let's call the first three functions to see what output we get. It'll be different every time, but will look something like this.

mod::my_funcs::full_name();    -- 'Christa Simon'
mod::my_funcs::company_name(); -- 'Götz and Seidel und Partner'
mod::my_funcs::city();         -- 'Kleinfuchsfurt'

Now let's try the Serbian functions. We'll choose this passage from the Serbian Wikipedia:

Неолитско насеље у Винчи удаљено је око 14 km од ушћа Саве у Дунав, што је изузетно повољно место које је омогућило да постане фокална тачка простора југоисточне Европе.

That passage is about a pre-historic neolithic culture that existed in and well beyond present day Serbia. It means:

"The Neolithic settlement in Vinča is about 14 km from the confluence of the Sava and the Danube, which is an extremely favorable place that allowed it to become a focal point of the area of Southeastern Europe."

We'll take the Cyrillic form and see if we can turn it into Latin and vice versa.

mod::my_funcs::serbian_stop_latin(
    "Неолитско насеље у Винчи удаљено је око 14 km од ушћа Саве у Дунав, што је изузетно повољно место које је омогућило да постане фокална тачка простора југоисточне Европе."
);
mod::my_funcs::serbian_stop_cyrillic(
    "Neolitsko naselje u Vinči udaljeno je oko 14 km od ušća Save u Dunav, što je izuzetno povoljno mesto koje je omogućilo da postane fokalna tačka prostora jugoistočne Evrope."
);

Success! Even if you don't know the language, you can compare the input and output to see that the output does indeed lack a number of words. This is clear when you line them up against each other.

-- Original text in Latin
'Neolitsko naselje u Vinči udaljeno je oko 14 km od ušća Save u Dunav, što je izuzetno povoljno mesto koje je omogućilo da postane fokalna tačka prostora jugoistočne Evrope.'

-- Cyrillic turned into Latin with stop words applied
'neolitsko naselje   vinči udaljeno    oko 14 km    ušća save   dunav         izuzetno povoljno mesto         omogućilo    postane fokalna tačka prostora jugoistočne evrope'

-- Latin turned into Cyrillic with stop words applied
'неолитско насеље    винчи удаљено     око 14 км    ушћа саве   дунав         изузетно повољно  место         омогућило    постане фокална тачка простора југоисточне европе'

So there we have it. In the space of a 350-line blog post we've managed to turn SurrealDB from regular SurrealDB 3.0 into one with a few extra "apps" that let you create mock data on the fly, or work with a new language that wasn't available before.

And since a Surrealism plugin is a WASM binary written in Rust, you can compile it yourself or share it for others to use. The ability to share what you've built is what makes a plugin system like this so extensible.

Community example

To finish up this post, let's quickly check out an example of Surrealism in action. SurrealDB user (and ambassador!) Horváth Bálint is the author of an app called recet that he described as follows:

Recet is an open source, work in progress recipe manager application built using Nuxt and SurrealDB. It is motivated from personal needs and the curiosity to see how much functionality can be squeezed out from using SurrealDB as the backend. My goal is to create a web app that feels great on both desktop and mobile using modern technologies like PWA, View Transition API, all while showcasing many features of SurrealDB.

One of the features of this app is a Surrealism function! Let's take a look at it. As you can see, it uses a crate called html2text. Here's part of it.

#[surrealism]
fn scrape_for_recipe(source_type: String, source_text: String) -> Result<RecipeInfo> {
    let text = if source_type == "text" {
        source_text
    } else if source_type == "url" {
        let html: String = sql_with_vars(
            "http::get($url)",
            vars! {
                url: source_text
            },
        )?;
        html2text::from_read(html.as_bytes(), usize::MAX)?
    } else {
        return Err(anyhow::anyhow!("Invalid source type"));
    };
    // ... snip
}

Then we have the same DEFINE BUCKET and DEFINE MODULE statements as above:

// PLUGINS
DEFINE BUCKET OVERWRITE plugins BACKEND $pluginsBucketPath PERMISSIONS NONE;
DEFINE MODULE OVERWRITE mod::recet AS f"plugins:/recet.surli";

And then a query via the JavaScript SDK that calls the function.

async function importRecipe() {
  try {
    importing.value = true

    const [recipe] = await db
      .query(surql`mod::recet::scrape_for_recipe(${importSourceType.value}, ${importSource.value})`)
      .collect<[ImportedRecipe | null]>()

    if (!recipe)
      throw new Error('No recipe found at the provided URL.')
    // ... snip
  }
}

As you can see, it's the same process described in the first examples above. Annotate the function, compile the module, use DEFINE twice to connect to it, and the function(s) are yours to call.

The final app, by the way, looks like this. Personally I think the kókusztejes gombás fehérbab looks particularly good, or the póréhagymás mascarponés gnocchi with the leeks replaced with grated cheese.

Back to you

There's no telling what the SurrealDB community will do with Surrealism now that 3.0 is here and the word is getting out.

We can't wait to see what you build!

Get started with Surrealism

Go to the Surrealism Docs to get started. We are excited to see what extensions you build, be sure to share them in our Discord channel.

Thank you for being on this journey with us!

Introducing JavaScript SDK 2.0

Mark Gyles — Thu, 26 Feb 2026 12:26:43 +0000

Author: Julian Mills

We're excited to announce the SurrealDB JavaScript SDK v2, the most significant update to the SDK to date. We've rebuilt the core internals with a focus on ergonomics, flexibility, and developer experience. Key highlights include full support for SurrealDB 3.0, multi-session support, automatic token refreshing, client-side transactions, a redesigned live query API, and a new query builder pattern that makes working with your data more intuitive than ever.

Welcome WASM and Node!

The existing WebAssembly and Node.js SDKs have been rewritten, updated to support the 2.0 JavaScript SDK, and moved into the JavaScript SDK repository.

Going forward, the JS, WASM, and Node.js SDKs will be published together, so embedded SurrealDB stays up to date. The WASM and Node.js SDK versions align their major and minor versions with SurrealDB, so you can tell at a glance which SurrealDB version you're running.

As a bonus, the WASM SDK now supports running inside a Web Worker, so you can offload database work from the main thread and keep your UI responsive.

Wasm

import { Surreal, createRemoteEngines } from "surrealdb";
import { createWasmEngines } from "@surrealdb/wasm";
import WorkerAgent from "@surrealdb/wasm/worker?worker";

const db = new Surreal({
    engines: {
        ...createRemoteEngines(),
        ...createWasmEngines(),
        // or for Web Worker based engines
        ...createWasmWorkerEngines({
            createWorker: () => new WorkerAgent()
        })
    },
});

Node.js (and Bun & Deno)

import { Surreal, createRemoteEngines } from "surrealdb";
import { createNodeEngines } from "@surrealdb/node";

const db = new Surreal({
    engines: {
        ...createRemoteEngines(),
        ...createNodeEngines(),
    },
});

Official event listeners

The original SDK exposed events through the internal surreal.emitter field. The new SDK provides a type-safe surreal.subscribe() function instead. Calling .subscribe() returns a cleanup function that unsubscribes the listener when invoked.

// Subscribe to events
const unsub = surreal.subscribe("connected", () => {
    ...
});

// Unsubscribe
unsub();

Access internal state

New getters let you read internal state from the Surreal instance:

surreal.namespace and surreal.database for the selected namespace and database
surreal.params for connection parameters
surreal.accessToken and surreal.refreshToken for authentication tokens

await surreal.use({ namespace: surreal.namespace, database: "other-db" });

Automatic token refreshing

The SDK now restores or renews authentication when your access token expires or the connection reconnects. When refresh tokens are available, they are exchanged for a new token pair. Otherwise the SDK reuses the credentials you provided or fires an auth event for custom handling. Read more on GitHub.

For asynchronous authentication, you can pass a callable to the authentication property:

const surreal = new Surreal();

await surreal.connect("http://example.com", {
    namespace: "test",
    database: "test",
    renewAccess: true, // default true
    authentication: () => ({
        username: "foo",
        password: "bar",
    })
});

Multi-session support

You can create multiple isolated sessions on a single connection, each with its own namespace, database, variables, and authentication state. New sessions can be created at any time, or you can fork an existing session and reuse its state.

Simple example

// Create a new session
const session = await surreal.newSession();

// Use the session
session.signin(...);

// Dispose the session
await session.closeSession();

Forking sessions

const freshSession = await surreal.newSession();

// Clone a session including namespace, database, variables, and auth state
const forkedSession = await freshSession.forkSession();

Await using

await using session = await surreal.newSession();

// JavaScript will automatically close the session at the end of the current scope

Redesigned live query API

Live query methods on the Surreal class have been redesigned to be more intuitive. Live select queries can also restart automatically when the driver reconnects.

The record ID is now passed as the third argument to your handlers, so you can identify which record changed when handling patch updates.

// Construct a new live subscription
const live = await surreal.live(new Table("users"));

// Listen to changes
live.subscribe((action, result, record) => {
     ...
});

// Alternatively, iterate messages
for await (const { action, value } of live) {
    ...
}

// Kill the query and stop listening
live.kill();

// Create an unmanaged query from an existing id
const [id] = await surreal.query("LIVE SELECT * FROM users");
const live = await surreal.liveOf(id);

Improved parameter explicitness

Query functions no longer accept plain strings as table names. You must use the Table class, which avoids record IDs being mistaken for table names.

// tables.ts
const usersTable = new Table("users");
const productsTable = new Table("products");
...

// main.ts
await surreal.select(usersTable);

Query builder pattern

A new builder pattern lets you configure RPC calls with optional chaining. All query methods support chainable helpers for filtering, limiting, and fetching.

As a result, update and upsert no longer take content as a second argument. You choose how to change records by calling .content(), .merge(), .replace(), or .patch().

// Select
const record = await db.select(id)
    .fields("age", "firstname", "lastname")
    .fetch("foo");

// Update
await db.update(record).merge({
    hello: "world"
});

Query method overhaul

The .query() function has been overhauled to support more use cases, including picking response indexes, automatic JSON output, and streaming responses.

// Execute a query and return results
const [user] = await db.query<[User]>("SELECT * FROM user:foo");

// Collect specific results
const [foo, bar] = await db.query("LET $foo = ...; LET $bar = ...; SELECT * FROM $foo; SELECT * FROM $bar")
    .collect<[User, Product]>(2, 3);

// Jsonify responses
const [products] = await db.query<[Product[]]>("SELECT * FROM product").json();

// Response objects
const responses = await db.query<[Product[]]>("SELECT * FROM product").responses();

// Stream responses
const stream = surreal.query(`SELECT * FROM foo`).stream();

for await (const frame of stream) {
    if (frame.isValue<Foo>()) {
        //  Process a single value with frame.value typed Foo
    } else if (frame.isDone()) {
        // Handle completion and access stats with frame.stats
    } else if (frame.isError()) {
        // Handle error frame.error
    }
}

SurrealDB does not yet support streaming individual records, but this API is ready for when it does. It works with current SurrealDB versions and is now the only way to obtain query stats.

Expressions API

A new Expressions API works with the .where() function and makes it easier to build dynamic expressions. It integrates with the surql template tag so you can insert parameter-safe expressions anywhere.

const checkActive = true;

// Query method
await db.select(userTable).where(eq("active", checkActive));

// Custom query
await db.query(surql`SELECT * FROM user WHERE ${eq("active", checkActive)}`);

// Expressions even allow raw insertion
await db.query(surql`SELECT * FROM user ${raw("WHERE active = true")}`);

You can also serialize expressions to a string with the expr() function:

const result: BoundQuery = expr(
    or(
        eq("foo", "bar"),
        false && eq("hello", "world"),
        eq("alpha", "beta"),
        and(
            inside("hello", ["hello"]),
            between("number", 1, 10)
        )
    )
);

Value encode/decode visitor API

For advanced use cases where you need to process SurrealDB value types, you can pass encode or decode visitor callbacks in the Surreal constructor. These run for each value sent to or received from the engine, so you can modify or wrap values before they appear in responses.

const surreal = new Surreal({
    codecOptions: {
        valueDecodeVisitor(value) {
            if (value instanceof RecordId) {
                return new RecordId("foo", "bar");
            }

            return value;
        },
    },
});

...

const [result] = await surreal.query(`RETURN hello:world`).collect<[RecordId]>();

console.log(result); // foo:bar

Diagnostics API

The Diagnostics API lets you wrap engines and inspect protocol-level communication. It is useful for debugging queries, analysing SDK behaviour, measuring timings, and similar tasks.

Because it is implemented as a wrapper engine, there is no extra cost unless you use it. We do not recommend using it in production, as it can affect performance and the events may change between versions.

new Surreal({
    driverOptions: {
        engines: applyDiagnostics(createRemoteEngines(), (event) => {
            console.log(event);
        }),
    },
});

Each event has a type, key, and phase. The type identifies the operation, key is a stable id across phases, and phase indicates start, progress, or completion. The query diagnostic exposes the internal chunk stream; you are responsible for stitching queries and batches together. The after phase includes duration, success, and result.

Get started with the new SDK

You can visit the JavaScript SDK documentation for more information on the new SDK, which has been fully updated to reflect all new changes. It contains many new concept guides explaining how to use the new SDK, and now includes a comprehensive API reference for detailed information.

Follow the Quickstart guide to get started.

File support in SurrealDB 3.0

Mark Gyles — Wed, 25 Feb 2026 18:19:04 +0000

File Support

Until now, SurrealDB has always encouraged users to store files externally, referencing them with paths stored in the database. We’re now introducing the first step toward native file support - bringing file storage directly into your database workflow.

File storage is still an experimental feature at this moment, and we would love to hear from you as you give it a try for your own solutions.

Setting up file storage

To begin, we can start a new SurrealDB server with the experimental files capability enabled.

surreal start --user root --pass secret --allow-experimental files

Setting up file storage can be done with a single DEFINE BUCKET statement! A bucket can be a physical location, or even in-memory.

For global usage across all namespaces and databases, you can set a SURREAL_GLOBAL_BUCKET environment variable, with an optional SURREAL_GLOBAL_BUCKET_ENFORCED if you want to ensure that every database uses the same global bucket.

Let’s define a simple in-memory bucket. Just provide a name, and set the backend to "memory":

DEFINE BUCKET my_bucket BACKEND "memory";

The name of the bucket is also your path to the bucket and everything inside.

Storing and accessing files

Once your bucket is defined, you can reference it using a file pointer - a string prefixed with the letter f to indicate a file path.

File pointers have their own functions, like .put() to add bytes, .get() to see them, .rename() to rename a file, and .head() to see its metadata. Let’s use the .get() function to see if our bucket has a file called tour_of_surrealdb.txt.

f"my_bucket:/tour_of_surrealdb.txt".get();

The function returned NONE, so looks like there’s nothing there.

Let’s use the .put() method to add some bytes, creating the file. .put() can take bytes or a string, which it will automatically convert into bytes.

f"my_bucket:/tour_of_surrealdb.txt".put("Welcome to the Tour of SurrealDB!");

Here, we’re adding the string “Welcome to the Tour of SurrealDB!”

Now that the file exists, the .get() function will return its as bytes:

f"my_bucket:/tour_of_surrealdb.txt".get();

-- Output:
b"57656C636F6D6520746F2074686520546F7572206F66205375727265616C444221"

Working with file data

If you prefer to view the data differently, SurrealDB also now supports improved type conversions for byte-related types.

For example, you can cast the result to an array to see each byte as a number:

<array>f"my_bucket:/tour_of_surrealdb.txt".get();

[87, 101, 108, 99, 111, 109, 101, 32, 116, 111, 32, 116, 104, 101, 32, 84, 111, 117, 114, 32, 111, 102, 32, 83, 117, 114, 114, 101,
97, 108, 68, 66, 33]

Or cast to a string:

<string>f"my_bucket:/tour_of_surrealdb.txt".get();

In cases where the bytes include invalid UTF-8, you can use type::string_lossy() - a new function that replaces invalid bytes with the � Unicode replacement character, rather than throwing an error.

This function is especially useful when working with partially corrupted or inconsistently encoded data. Instead of failing outright, it preserves and returns all valid parts of the string.

Here’s an example using type::string, which returns an error because not all the bytes are valid:

type::string(<bytes>[83, 117, 114, 255, 114, 101, 97, 254, 108, 68, 66]);
-- 'Could not cast into `string` using input `b"537572FF726561FE6C4442"`'

And here’s the same input passed through type::string_lossy(). We can see that there are some valid bytes in there - enough to make out the word SurrealDB.

type::string_lossy(<bytes>[83, 117, 114, 255, 114, 101, 97, 254, 108, 68, 66]);
-- 'Sur�rea�lDB'

SurrealDB also has encoding functions that allow you to encode and decode bytes into SurrealDB types. That means that you can take a type like this object and turn it into bytes...

encoding::cbor::encode({ some: "data" });

-- Output:
b"A164736F6D656464617461"

And then turn it back from bytes into an object.

encoding::cbor::decode(b"A164736F6D656464617461");

-- Output:
{
    some: 'data'
}

A takeaway example to get started

Let's take a quick running example to give you some ideas of how you could use files in your own app, in this case an app that stores temporary shopping cart files to disk so that other parts of the app can access them.

This time we'll define a bucket to a file path instead of memory, which requires SURREAL_BUCKET_FOLDER_ALLOWLIST to be set so that the database knows that this folder has been approved as the location for file storage. Note that the SurrealDB instance itself is using memory as the backend, but it's able to save files to this directory, getting the best of both worlds.

SURREAL_BUCKET_FOLDER_ALLOWLIST="/users/your_user_name" surreal start --allow-experimental files --user root --pass secret

After connecting, defining the bucket can be done with a path that begins with file: and holds the path we have allowed.

DEFINE BUCKET shopping_carts BACKEND "file:/users/your_user_name";

Since working with files takes a good amount of code, we'll put some functions together to simplify the process. These three will allow us to save, get, and delete files in the shopping_carts backend.

-- Convenience functions to save, decode back into
-- SurrealQL type, and delete
DEFINE FUNCTION fn::save_file($file_name: string, $input: any) {
    LET $file = type::file("shopping_carts", $file_name);
    $file.put(encoding::cbor::encode($input));
};

DEFINE FUNCTION fn::get_file($file_name: string) -> object {
    encoding::cbor::decode(type::file("shopping_carts", $file_name).get())
};

DEFINE FUNCTION fn::delete_file($file_name: string) {
    type::file("shopping_carts", $file_name).delete();
};

With those functions set up, we can play around with a file for a user with the user number 24567. Once you execute the fn::save_file() function below you will be able to go into the directory yourself to view it. And you can also view and interact with it using SurrealQL via the other functions in the example below.

-- Save current shopping cart
fn::save_file("temp_cart_user_24567", {
    items: ["shirt1"],
    last_updated: time::now()
});

fn::get_file("temp_cart_user_24567");
-- Returns { items: ['shirt1', 'deck_of_cards'], last_updated: d'2026-02-25T01:03:24.141080Z' }

-- User adds item, save over file with newer information
fn::save_file("temp_cart_user_24567", {
    items: ["shirt1", "deck_of_cards"],
    last_updated: time::now()
});

fn::get_file("temp_cart_user_24567");
-- Returns { items: ['shirt1', 'deck_of_cards'], last_updated: d'2026-02-25T01:06:02.752429Z' }

-- Session is over, delete temp file
fn::delete_file("temp_cart_user_24567");

We’re excited to introduce this first stage of native file support in SurrealDB. As the next step in our journey toward multimodality, we’re working on enabling integrations with object storage solutions such as Amazon S3, allowing you to work with files at scale. If you’re building apps that rely on lightweight file storage or embedded assets, give it a try - and let us know what features you’d like to see next. This is just the beginning.

Learn more at: https://surrealdb.com/docs/surrealql/datamodel/files.

Custom API Endpoints: streamlining your architecture

Mark Gyles — Wed, 25 Feb 2026 17:53:23 +0000

Custom API Endpoints: streamlining your architecture

With the release of SurrealDB 3.0, we’re excited to announce the stabilisation of a powerful feature: DEFINE API. This innovative addition allows developers to define database behaviours, set up middleware, and create custom API endpoints directly within the familiar SurrealQL query language.

Traditionally, applications connect to databases through middleware, resulting in a three-layer architecture:

Client → Middleware (API) → Database

With SurrealDB's new DEFINE API, you can simplify your infrastructure significantly:

Client → Database

This streamlined approach not only reduces complexity but also enhances performance and ease of management.

Real-world use case: implementing rate limits for a social app

Consider a scenario: you've built a popular social application that allows users to see recent comments. You want to provide a free API for anonymous users, but you also need to ensure these guest users don't overload your system resources during peak traffic.

Here are some ways how you can achieve this efficiently using SurrealDB's DEFINE API:

Setup: restrict arbitrary queries

To begin, pass in the new --deny-arbitrary-query flag when starting the SurrealDB server.

surreal start --user root --pass root --deny-arbitrary-query guest --allow-experimental define_api

This prevents certain groups, like guests or record users, from putting their own queries together. We'll add guest here so that any guest users will have to use the API endpoint instead.

You can then connect using Surrealist, or with the following command in a new terminal window:

surreal sql --user root --pass root

Defining your API Endpoint

Next, we'll add a DEFINE API statement to define the endpoint. The statement will start with the path /get_latest , and to specify that it can only be used for GET requests, we will include it after the FOR keyword.

DEFINE API OVERWRITE "/get_latest" FOR get

Add middleware functions

We can follow this up with some middleware by passing in some of SurrealDB's API functions. We'll use api::timeout to give guest users up to 50 milliseconds of server time.

MIDDLEWARE
    api::timeout(50ms)

Define the RETURN data

And then we'll finish up the statement by adding what it actually returns: a SELECT statement for all comment records that have been created over the past ten minutes:

THEN {
      {
        body: SELECT * FROM comment:[time::now()-10m].. ORDER BY id DESC
      }
    };

In summary, that gives us a DEFINE API statement that looks like this.

DEFINE API OVERWRITE "/get_latest" FOR get
    MIDDLEWARE
        api::timeout(50ms),
    THEN {
        {
            body: SELECT * FROM comment:[time::now()-10m].. ORDER BY id DESC
        }
};

Testing your endpoint

You can test it from SurrealQL itself by first adding a comment or two and then calling the api::invoke function:

CREATE comment:[time::now()] SET user_says = "Nice blog!";
CREATE comment:[time::now()] SET user_says = "Can't wait for the new version";

api::invoke("/get_latest");

Or externally at: /api/:namespace/:database/:endpoint_name. Here's what our defined endpoint will look like with a namespace called test_ns and a database called test_db.

http://localhost:8000/api/test_ns/test_db/get_latest

That means that a curl command like the following will be enough to get the latest comments.

curl -H "Accept: application/json" \
  http://localhost:8000/api/test_ns/test_db/get_latest

The output should look something like this.

[
  {"id":"comment:[d'2026-02-12T01:10:02.445607Z']","user_says":"Can't wait for the new version"},
  {"id":"comment:[d'2026-02-12T01:08:33.709073Z']","user_says":"Nice blog!"}
]

But that’s not all. The MIDDLEWARE clause in a DEFINE API statement can take custom middleware as well that you define yourself using a regular DEFINE FUNCTION statement. These functions are automatically populated with the user request and a way to get the current state of the response as you put it together.

There are a lot of examples in the documentation to show how it works, but here’s a simple example to get you started.

DEFINE FUNCTION fn::add_prefix($req: object, $next: function, $prefix: string) -> object {
    LET $res = $next($req);
    LET $res = $res + { body: $res.body + { prefix: $prefix + ": " + $res.body.message } };
    $res;
};

DEFINE API "/custom_with_args"
    FOR get
        MIDDLEWARE
            fn::add_prefix("PREFIX")
        THEN {
            {
                status: 200,
                body: {
                    message: "original message"
                }
            };
        };

With DEFINE API, SurrealDB brings the power of API design directly into the database - no external layers, no additional frameworks. Whether you’re enforcing limits, defining access, or building powerful custom endpoints, this feature helps you simplify your architecture and move faster.

Explore further

Dive deeper into SurrealDB's powerful API management capabilities:

Harness the power of DEFINE API and redefine what's possible with SurrealDB 3.0.

SurrealDB 3.0 benchmarks: a new foundation for performance

Mark Gyles — Thu, 19 Feb 2026 21:11:12 +0000

SurrealDB 3.0 benchmarks: a new foundation for performance

When we began our benchmarking journey with SurrealDB 2.0, we set out to answer a simple but important question: how does SurrealDB perform in the real world?

Since then, SurrealDB has seen rapidly growing traction - including enterprises like Tencent and Later.com running in production at significant scale, across high-throughput, business-critical workloads. As adoption has increased, so has the importance of demonstrating not just raw performance, but predictable, production-ready performance.

Last year we shared those early results openly, along with our methodology and learnings, in our first benchmarking report.

With SurrealDB 3.0, we’re taking the next major step in that journey. Today, we’re excited to announce our new benchmarks for SurrealDB 3.0 - and more importantly, the architectural changes that make these results possible.

Benchmarking a multi-model database

As mentioned in our previous benchmarking blog, benchmarking a multi-model database is more complex than comparing single-purpose systems. SurrealDB unifies relational, document, graph, time-series, key-value, vector, geospatial, and full-text workloads in one engine, spanning everything from embedded deployments to distributed clusters.

This versatility makes fair benchmarking challenging, since different databases vary widely in durability guarantees, disk flushing behaviour, and configuration trade-offs. For our benchmarks, we’ve done our best to keep comparisons as balanced and transparent as possible - and we always welcome feedback on how to improve the methodology further.

Because SurrealDB can replace entire stacks of databases and supporting tools, the best way to benchmark SurrealDB is often through a like-for-like proof of concept against multiple existing systems (we have customers like Tencent who reduced nine separate tools down to one with SurrealDB).

Our open-source benchmarking tool

We built our own internal benchmarking tool - crud-bench - in Rust to properly evaluate SurrealDB across the wide range of workloads it supports. Unlike many generic benchmarking suites, crud-bench is designed to test not just basic CRUD operations, but also the broader feature set that makes SurrealDB unique, across embedded, networked, and remote deployments. It allows us to compare SQL, NoSQL, key-value, and embedded systems in a consistent way, while continuously measuring the performance impact of changes inside SurrealDB itself. For more details on how the tool work, check the crud-bench GitHub repo and the explanation in the previous 2.0 benchmark blog.

SurrealDB 3.0: a new execution engine

The foundation

SurrealDB 3.0 has been largely focused on stability and getting the right foundations in place. We rearchitected the internals in ways that would have been nearly impossible under the previous design, and the performance results we're seeing now are a direct consequence of that work.

The biggest change is the query execution engine. We took inspiration from the broader database community and moved SurrealQL's execution model to a more standard pipeline: AST → LogicalPlan → ExecutionPlan, with basic query optimisations applied throughout. The new engine is fully streaming internally, and in a future minor release we will make it stream end-to-end.

Right now, the new execution engine covers read-only statements. In the coming minor releases we'll expand it to handle all of our workloads.

TL;DR
Now let’s get to the benchmarks themselves. The tl;dr for the numbers below are as follows:

Graph queries are between 8-22x faster.
Large table scans with LIMIT, START, and START + LIMIT are 3-6x faster.
The query planner is smarter. Queries like SELECT * FROM table WHERE id = record:42 that previously triggered full table scans now resolve in sub-millisecond time, a 4000x+ improvement. This is especially important for a query language like SurrealQL which is very close to standard SQL in which WHERE id = is one of the most common patterns.
ORDER BY queries are 3-4x faster across all storage engines.
HNSW vector search is up to 8x faster. Indexed vector similarity queries dropped from ~35 seconds to ~4.5 seconds.
The new execution model enables significantly improved concurrency and throughput.
Complex GROUP BY queries with multiple aggregations are up to 55% faster, with the biggest gains on multi-aggregation and deduplication workloads.

Benchmark hardware

The benchmarks below were run on the following system:

Part	Name	Link
CPU	AMD Ryzen Threadripper 9970X	amazon.co.uk
RAM	Samsung DDR5 4800MHz 64GB RDIMM	amazon.co.uk
SSD	Lexar EQ790 4TB NVMe	amazon.co.uk
Motherboard	ASUS PRO WS TRX50-SAGE WIFI A	ballicom.co.uk
CPU Cooler	Arctic Freezer 4U-M	scan.co.uk
Power Supply	CORSAIR RM1000x	amazon.co.uk
GPU	PowerColor AMD Radeon RX 7600 Fighter 8GB	amazon.co.uk
Case	In-Win IW-R400-01N 4U	servercase.co.uk

Benchmarks

In-Memory

Test	v3 vs. v2		SurrealDB 2		SurrealDB 3
	Mean %	OP/s %	Mean (ms)	OP/s	Mean (ms)	OP/s
[C]reate	-62.9%	+168.7%	1.94	147538.6	0.72	396504.5
[R]ead	+56.8%	-35.6%	0.37	757976.6	0.58	488252.1
[U]pdate	-71.7%	+249.6%	2.58	111466.6	0.73	389693.8
[S]can::count_all (2000)	-44.4%	+64.3%	204.97	1277.8	113.99	2100.1
[S]can::limit_id (2000)	-67.5%	+166.8%	5.32	47510.8	1.73	126780.2
[S]can::limit_all (2000)	-85.9%	+550.9%	17.45	15053.5	2.46	97982.9
[S]can::limit_start_id (2000)	+45.1%	-28.2%	7.45	33909.7	10.81	24360.7
[S]can::limit_start_all (2000)	-38.5%	+64.6%	18.11	14419.5	11.14	23733.8
[S]can::select_where_id (2000)	+34.7%	-20.4%	0.49	401422.2	0.66	319614.6
[S]can::select_where_id_eq (2000)	-100.0%	+436179.7%	3935.65	70.9	0.68	309409.5
[S]can::select_where_gt (2000)	+18.9%	-17.2%	5242.39	52.9	6235.40	43.8
[S]can::select_where_in (2000)	-36.2%	+55.8%	9745.80	28.4	6221.91	44.3
[S]can::select_where_multi_and (2000)	+2.8%	-3.2%	6065.92	45.7	6238.46	44.2
[S]can::select_where_order_limit (2000)	+10.9%	-8.7%	5196.36	53.3	5762.22	48.7
[S]can::select_where_order_desc_limit (2000)	+12.5%	-11.8%	5191.97	53.5	5842.42	47.1
[S]can::select_where_multi_order_limit (2000)	-7.0%	+9.2%	6271.61	44.3	5830.96	48.4
[S]can::select_omit_limit (2000)	-84.4%	+465.6%	18.09	14490.0	2.83	81951.6
[S]can::select_fields_where_limit (2000)	+589.8%	-85.7%	16.15	16178.0	111.41	2305.7
[S]can::select_order_by (2000)	-69.5%	+231.8%	17270.90	16.1	5274.94	53.5
[S]can::select_order_by_multi (2000)	-70.1%	+235.3%	17387.92	16.0	5192.82	53.7
[S]can::select_group_count (2000)	-9.2%	+8.7%	5989.63	46.4	5436.81	50.4
[S]can::select_group_sum (2000)	-16.6%	+19.8%	6754.21	41.1	5635.44	49.2
[S]can::select_group_avg (2000)	-18.3%	+20.9%	6766.85	40.9	5529.88	49.4
[S]can::select_group_multi_agg (2000)	-36.3%	+53.4%	9738.26	28.5	6203.12	43.8
[S]can::select_group_all (2000)	-25.8%	+33.7%	7492.08	37.1	5561.68	49.5
[S]can::select_group_order_limit (2000)	-23.7%	+31.6%	7603.12	36.5	5798.60	48.1
[S]can::select_group_where (2000)	-12.8%	+13.8%	7411.37	37.5	6459.82	42.7
[S]can::select_group_dedup_agg (2000)	-35.5%	+54.0%	9735.38	28.5	6278.25	43.9
[S]can::select_split (2000)	-55.1%	+111.3%	19.71	13415.2	8.85	28351.9
[S]can::select_fetch (2000)	-80.6%	+401.8%	39.07	6802.6	7.58	34134.4
[S]can::select_fetch_where_limit (2000)	+291.6%	-74.3%	32.16	8366.1	125.95	2146.3
[S]can::graph_out_depth1 (2000)	-60.0%	+124.2%	2.30	105137.8	0.92	235676.0
[S]can::graph_out_depth2 (2000)	-72.9%	+222.0%	6.16	42125.8	1.67	135637.0
[S]can::graph_out_depth3 (2000)	-83.2%	+449.0%	18.02	15001.7	3.02	82363.1
[S]can::graph_in_depth1 (2000)	-58.8%	+129.5%	2.72	90892.6	1.12	208621.0
[S]can::graph_bidirectional (2000)	-67.1%	+193.4%	4.22	57673.0	1.39	169188.7
[S]can::graph_edge_filter (2000)	-5.9%	-4.2%	1.19	197478.8	1.12	189149.0
[S]can::graph_multi_out (2000)	-84.3%	+504.6%	148.68	1886.3	23.32	11405.3
[S]can::graph_multi_out_where (2000)	-72.0%	+243.5%	51.38	5401.8	14.40	18555.1
[S]can::graph_multi_count (2000)	-72.4%	+247.7%	33.74	8098.3	9.32	28157.6
[S]can::graph_depth2_limit (2000)	-88.1%	+696.2%	78.83	3529.7	9.41	28104.7
[S]can::graph_sub_where (2000)	-24.7%	+32.1%	2.67	91237.9	2.01	120503.8
[S]can::graph_sub_group_all (2000)	-42.6%	+69.2%	3.36	74807.0	1.93	126560.9
[S]can::graph_sub_group_by (2000)	-42.2%	+61.1%	3.25	78504.3	1.88	126496.6
[S]can::subquery_inline (2000)	-49.1%	+95.4%	724.85	377.8	368.70	738.3
[S]can::subquery_count (2000)	+116.6%	-52.4%	28.49	9262.4	61.72	4406.4
[S]can::subquery_from (2000)	-38.3%	+60.9%	2918.72	94.6	1801.28	152.2
[S]can::pipeline_filter_group_order (2000)	-29.2%	+41.6%	9341.45	29.7	6613.75	42.0
[S]can::index_standard (2000)	+34.0%	-26.9%	3910.21	71.0	5241.54	51.9
[I]ndex::index_standard	-49.7%	+98.0%	1003.26	1.0	504.19	2.0
[S]can::index_standard::indexed (2000)	-42.6%	+60.8%	4.11	61150.4	2.36	98303.1
[R]emoveIndex::index_standard	-78.8%	+370.7%	380.54	2.6	80.67	12.4
[S]can::index_composite (2000)	+1.1%	-4.2%	6101.19	45.7	6171.27	43.8
[I]ndex::index_composite	+0.9%	-1.0%	502.14	2.0	506.50	2.0
[S]can::index_composite::indexed (2000)	-50.4%	+107.2%	1504.21	176.7	745.47	366.3
[R]emoveIndex::index_composite	-73.6%	+278.3%	338.82	3.0	89.50	11.2
[S]can::index_range_merged (2000)	-25.7%	+33.5%	7595.90	36.5	5641.66	48.7
[I]ndex::index_range_merged	-50.0%	+99.0%	1004.29	1.0	502.14	2.0
[S]can::index_range_merged::indexed (2000)	-52.4%	+114.4%	478.07	555.2	227.80	1190.4
[R]emoveIndex::index_range_merged	-74.4%	+289.6%	360.32	2.8	92.19	10.8
[S]can::index_in (2000)	-25.3%	+34.1%	7232.84	38.4	5404.26	51.5
[I]ndex::index_in	-50.0%	+99.0%	1004.29	1.0	502.40	2.0
[S]can::index_in::indexed (2000)	-61.6%	+160.3%	25.35	10437.8	9.73	27169.0
[R]emoveIndex::index_in	-77.9%	+352.1%	373.89	2.7	82.78	12.1
[I]ndex::index_fulltext	-93.1%	+1300.0%	51134.46	0.0	3511.30	0.3
[S]can::index_fulltext::indexed (2000)	+237.6%	-69.3%	225.31	1216.5	760.70	373.8
[R]emoveIndex::index_fulltext	+98.8%	-49.2%	845.06	1.2	1679.87	0.6
[I]ndex::index_hnsw	-32.7%	+60.0%	18604.03	0.1	12529.66	0.1
[S]can::index_hnsw::indexed (2000)	-87.4%	+671.0%	38581.61	7.3	4847.09	56.3
[R]emoveIndex::index_hnsw	-82.2%	+456.5%	2190.34	0.5	390.78	2.6

SurrealKV

Test	v3 vs. v2		SurrealDB 2		SurrealDB 3
	Mean %	OP/s %	Mean (ms)	OP/s	Mean (ms)	OP/s
[C]reate	-96.3%	+2579.9%	221.27	1299.7	8.25	34830.7
[R]ead	+76.5%	-42.4%	0.34	828781.5	0.60	477557.1
[U]pdate	-96.2%	+2550.3%	222.01	1295.3	8.37	34330.5
[S]can::count_all (2000)	-53.0%	+89.6%	303.53	917.2	142.73	1738.6
[S]can::limit_id (2000)	-68.2%	+170.0%	6.51	40313.3	2.07	108861.9
[S]can::limit_all (2000)	-84.5%	+460.1%	16.51	16063.4	2.56	89964.2
[S]can::limit_start_id (2000)	-59.2%	+132.9%	14.27	18977.0	5.82	44194.4
[S]can::limit_start_all (2000)	-71.6%	+241.5%	21.48	12560.3	6.10	42894.7
[S]can::select_where_id (2000)	+39.2%	-25.4%	0.51	400808.0	0.71	299184.1
[S]can::select_where_id_eq (2000)	-100.0%	+393137.5%	3478.94	81.6	0.69	320960.5
[S]can::select_where_gt (2000)	+30.1%	-25.9%	4666.15	60.8	6071.58	45.0
[S]can::select_where_in (2000)	-34.5%	+48.6%	9275.42	30.5	6071.93	45.3
[S]can::select_where_multi_and (2000)	+9.7%	-11.1%	5509.79	51.2	6044.50	45.5
[S]can::select_where_order_limit (2000)	+18.4%	-17.6%	4650.07	61.0	5504.01	50.3
[S]can::select_where_order_desc_limit (2000)	+25.0%	-21.4%	4623.38	61.4	5777.77	48.3
[S]can::select_where_multi_order_limit (2000)	-1.8%	+0.5%	5692.97	49.8	5590.58	50.1
[S]can::select_omit_limit (2000)	-76.9%	+316.0%	17.20	15569.4	3.97	64774.7
[S]can::select_fields_where_limit (2000)	+545.0%	-85.3%	16.06	16653.9	103.59	2450.0
[S]can::select_order_by (2000)	-68.1%	+210.7%	15714.24	17.9	5016.71	55.5
[S]can::select_order_by_multi (2000)	-68.3%	+214.1%	15943.91	17.6	5055.99	55.3
[S]can::select_group_count (2000)	+19.3%	-18.1%	4391.26	64.7	5236.94	53.0
[S]can::select_group_sum (2000)	+9.2%	-10.9%	5024.47	56.5	5485.56	50.4
[S]can::select_group_avg (2000)	+8.2%	-10.6%	5023.77	56.5	5437.21	50.5
[S]can::select_group_multi_agg (2000)	-23.4%	+26.5%	7942.62	35.6	6081.36	45.0
[S]can::select_group_all (2000)	-5.9%	+3.6%	5757.57	49.2	5418.85	51.0
[S]can::select_group_order_limit (2000)	-5.1%	+2.0%	5832.72	48.6	5534.30	49.6
[S]can::select_group_where (2000)	+5.1%	-7.2%	6046.34	46.9	6354.95	43.5
[S]can::select_group_dedup_agg (2000)	-23.9%	+26.7%	7959.49	35.5	6055.39	45.0
[S]can::select_split (2000)	-51.9%	+92.3%	18.00	14812.0	8.66	28485.5
[S]can::select_fetch (2000)	-75.1%	+291.8%	35.99	7481.3	8.96	29309.5
[S]can::select_fetch_where_limit (2000)	+271.6%	-72.7%	30.70	8690.9	114.08	2375.0
[S]can::graph_out_depth1 (2000)	-69.0%	+181.5%	5.00	52630.3	1.55	148160.1
[S]can::graph_out_depth2 (2000)	-77.6%	+315.8%	14.70	18594.4	3.29	77306.4
[S]can::graph_out_depth3 (2000)	-80.3%	+382.1%	42.04	6604.0	8.30	31839.6
[S]can::graph_in_depth1 (2000)	-71.1%	+202.3%	4.95	53273.6	1.43	161031.4
[S]can::graph_bidirectional (2000)	-77.2%	+297.3%	11.14	24621.8	2.54	97824.2
[S]can::graph_edge_filter (2000)	-49.0%	+87.9%	2.41	100491.2	1.23	188866.9
[S]can::graph_multi_out (2000)	-83.9%	+508.7%	436.45	643.7	70.48	3918.3
[S]can::graph_multi_out_where (2000)	-78.8%	+361.6%	173.91	1610.2	36.83	7433.1
[S]can::graph_multi_count (2000)	-82.3%	+436.8%	94.20	2968.2	16.70	15934.7
[S]can::graph_depth2_limit (2000)	-80.9%	+413.0%	246.73	1139.2	47.07	5844.6
[S]can::graph_sub_where (2000)	-61.8%	+139.7%	5.50	48232.8	2.10	115625.9
[S]can::graph_sub_group_all (2000)	-61.1%	+142.2%	5.84	45269.5	2.27	109630.7
[S]can::graph_sub_group_by (2000)	-62.5%	+140.8%	5.90	44952.6	2.21	108263.4
[S]can::subquery_inline (2000)	-38.6%	+57.9%	596.32	471.7	366.11	744.9
[S]can::subquery_count (2000)	-43.3%	+75.1%	75.35	3716.0	42.73	6505.7
[S]can::subquery_from (2000)	-36.5%	+54.1%	2723.41	102.0	1730.69	157.1
[S]can::pipeline_filter_group_order (2000)	-17.1%	+17.3%	7624.09	37.0	6318.85	43.4
[S]can::index_standard (2000)	+46.8%	-34.6%	3454.12	82.3	5071.34	53.8
[I]ndex::index_standard	-33.4%	+50.0%	1513.98	0.7	1008.38	1.0
[S]can::index_standard::indexed (2000)	-56.3%	+90.7%	6.04	43641.0	2.64	83230.9
[R]emoveIndex::index_standard	-76.9%	+332.6%	445.31	2.2	102.82	9.7
[S]can::index_composite (2000)	+11.7%	-13.5%	5489.27	51.6	6131.84	44.7
[I]ndex::index_composite	-33.2%	+50.0%	1513.98	0.7	1011.97	1.0
[S]can::index_composite::indexed (2000)	+30.9%	-23.1%	1509.67	183.2	1976.73	141.0
[R]emoveIndex::index_composite	-64.8%	+184.0%	421.76	2.4	148.54	6.7
[S]can::index_range_merged (2000)	-20.9%	+23.4%	7044.30	39.8	5572.12	49.1
[I]ndex::index_range_merged	-33.1%	+50.0%	1513.98	0.7	1012.48	1.0
[S]can::index_range_merged::indexed (2000)	+34.2%	-24.9%	453.72	608.9	608.72	457.5
[R]emoveIndex::index_range_merged	-67.0%	+202.7%	445.06	2.2	146.75	6.8
[S]can::index_in (2000)	-22.1%	+25.5%	6744.01	41.8	5253.10	52.5
[I]ndex::index_in	-33.1%	+50.0%	1512.96	0.7	1011.97	1.0
[S]can::index_in::indexed (2000)	-58.8%	+136.3%	30.92	8937.5	12.73	21115.7
[R]emoveIndex::index_in	-71.2%	+247.0%	456.32	2.2	131.52	7.6
[I]ndex::index_fulltext	+46.8%	-50.0%	55689.22	0.0	81756.16	0.0
[S]can::index_fulltext::indexed (2000)	+45.0%	-29.9%	204.39	1343.2	296.32	942.0
[R]emoveIndex::index_fulltext	+130.9%	-56.3%	969.98	1.0	2239.49	0.5
[I]ndex::index_hnsw	-30.8%	+40.0%	19587.07	0.1	13545.47	0.1
[S]can::index_hnsw::indexed (2000)	-87.1%	+654.4%	35710.91	7.9	4622.45	59.4
[R]emoveIndex::index_hnsw	-67.3%	+205.0%	2509.82	0.4	820.48	1.2

RocksDB

Test	v3 vs. v2		SurrealDB v2		SurrealDB v3
	Mean %	OP/s %	Mean (ms)	OP/s	Mean (ms)	OP/s
[C]reate	-14.7%	+17.3%	9.99	28778.2	8.52	33768.3
[R]ead	+59.5%	-36.1%	0.37	757343.3	0.59	484147.0
[U]pdate	+1.9%	-1.6%	8.34	34370.8	8.50	33828.1
[S]can::count_all (2001)	-13.2%	-2.7%	153.00	1821.8	132.74	1771.7
[S]can::limit_id (2001)	-63.5%	+155.9%	5.32	48199.9	1.94	123328.7
[S]can::limit_all (2001)	-85.0%	+492.6%	16.29	16194.5	2.45	95975.1
[S]can::limit_start_id (2001)	-49.8%	+90.2%	12.08	22395.3	6.07	42606.3
[S]can::limit_start_all (2001)	-69.3%	+210.4%	20.45	13197.6	6.28	40970.0
[S]can::select_where_id (2001)	+19.0%	-12.7%	0.63	334303.5	0.75	291755.1
[S]can::select_where_id_eq (2001)	-100.0%	+352510.7%	3313.50	85.7	0.72	302257.9
[S]can::select_where_gt (2001)	+30.1%	-27.0%	4576.38	62.0	5955.45	45.2
[S]can::select_where_in (2001)	-34.9%	+50.0%	9134.11	30.7	5944.41	46.0
[S]can::select_where_multi_and (2001)	+12.1%	-13.1%	5394.01	52.4	6047.01	45.5
[S]can::select_where_order_limit (2001)	+22.3%	-19.4%	4543.69	62.4	5556.69	50.3
[S]can::select_where_order_desc_limit (2001)	+26.4%	-22.0%	4505.67	63.0	5695.88	49.1
[S]can::select_where_multi_order_limit (2001)	-2.4%	+0.0%	5628.40	50.4	5495.20	50.4
[S]can::select_omit_limit (2001)	-76.9%	+314.9%	16.68	16042.6	3.85	66553.8
[S]can::select_fields_where_limit (2001)	+637.0%	-86.9%	14.19	18728.2	104.58	2460.3
[S]can::select_order_by (2001)	-68.5%	+214.1%	15667.41	17.9	4932.92	56.2
[S]can::select_order_by_multi (2001)	-68.3%	+215.8%	15902.27	17.6	5037.95	55.7
[S]can::select_group_count (2001)	+20.2%	-20.1%	4235.10	67.0	5091.89	53.6
[S]can::select_group_sum (2001)	+8.8%	-11.6%	4881.45	58.1	5309.39	51.4
[S]can::select_group_avg (2001)	+9.6%	-11.3%	4900.75	57.8	5371.33	51.3
[S]can::select_group_multi_agg (2001)	-21.5%	+25.9%	7782.85	36.0	6111.42	45.4
[S]can::select_group_all (2001)	-3.6%	+2.5%	5624.71	50.3	5421.68	51.5
[S]can::select_group_order_limit (2001)	-3.3%	+1.5%	5721.24	49.5	5529.63	50.2
[S]can::select_group_where (2001)	+6.1%	-7.3%	5949.82	47.5	6311.33	44.0
[S]can::select_group_dedup_agg (2001)	-21.6%	+27.4%	7813.47	35.8	6125.32	45.6
[S]can::select_split (2001)	-50.1%	+80.7%	17.45	15307.9	8.70	27658.5
[S]can::select_fetch (2001)	-77.1%	+327.8%	35.19	7564.1	8.06	32357.4
[S]can::select_fetch_where_limit (2001)	+276.7%	-73.1%	30.11	8892.6	113.41	2388.4
[S]can::graph_out_depth1 (2001)	-79.1%	+334.3%	4.78	54273.4	1.00	235728.9
[S]can::graph_out_depth2 (2001)	-88.0%	+612.6%	14.39	19118.8	1.73	136232.2
[S]can::graph_out_depth3 (2001)	-93.3%	+1232.1%	41.42	6724.3	2.78	89577.1
[S]can::graph_in_depth1 (2001)	-77.1%	+254.4%	4.90	54000.1	1.12	191369.2
[S]can::graph_bidirectional (2001)	-89.3%	+654.4%	10.62	25771.5	1.14	194426.5
[S]can::graph_edge_filter (2001)	-57.0%	+122.3%	2.37	102130.8	1.02	227036.8
[S]can::graph_multi_out (2001)	-94.6%	+1651.9%	426.98	659.4	23.02	11551.7
[S]can::graph_multi_out_where (2001)	-91.8%	+1060.6%	170.00	1648.7	13.99	19133.9
[S]can::graph_multi_count (2001)	-90.1%	+833.6%	90.26	3098.7	8.96	28929.8
[S]can::graph_depth2_limit (2001)	-95.9%	+2195.0%	238.54	1176.5	9.82	27000.0
[S]can::graph_sub_where (2001)	-59.8%	+125.1%	5.40	49395.5	2.17	111209.4
[S]can::graph_sub_group_all (2001)	-62.9%	+144.8%	5.72	46438.1	2.12	113671.0
[S]can::graph_sub_group_by (2001)	-65.0%	+158.7%	5.65	47104.9	1.98	121859.2
[S]can::subquery_inline (2001)	-39.8%	+62.5%	585.17	476.7	352.05	774.7
[S]can::subquery_count (2001)	-10.6%	+8.9%	71.02	3947.0	63.50	4299.9
[S]can::subquery_from (2001)	-36.1%	+53.9%	2708.90	102.4	1731.46	157.7
[S]can::pipeline_filter_group_order (2001)	-13.0%	+13.8%	7471.20	37.6	6498.49	42.9
[S]can::index_standard (2001)	+53.5%	-37.3%	3307.64	85.9	5077.27	53.8
[I]ndex::index_standard	+0.2%	+0.0%	1010.43	1.0	1011.97	1.0
[S]can::index_standard::indexed (2001)	-68.1%	+181.6%	6.95	38213.8	2.22	107603.5
[R]emoveIndex::index_standard	-21.0%	+26.8%	275.84	3.6	217.79	4.6
[S]can::index_composite (2001)	+10.0%	-12.3%	5403.80	52.4	5945.42	45.9
[I]ndex::index_composite	+1.2%	-1.0%	1006.34	1.0	1018.11	1.0
[S]can::index_composite::indexed (2001)	-49.0%	+96.1%	1494.50	184.7	762.39	362.3
[R]emoveIndex::index_composite	-70.9%	+243.8%	769.28	1.3	223.81	4.5
[S]can::index_range_merged (2001)	-21.9%	+25.3%	6974.03	40.3	5446.44	50.5
[I]ndex::index_range_merged	+0.2%	+0.0%	1012.48	1.0	1014.53	1.0
[S]can::index_range_merged::indexed (2001)	-47.9%	+92.8%	456.57	595.9	237.86	1148.8
[R]emoveIndex::index_range_merged	-20.5%	+25.5%	271.74	3.7	216.13	4.6
[S]can::index_in (2001)	-24.2%	+27.4%	6644.39	42.3	5037.32	53.8
[I]ndex::index_in	+0.2%	+0.0%	1010.43	1.0	1011.97	1.0
[S]can::index_in::indexed (2001)	-70.1%	+218.6%	30.15	9168.9	9.00	29209.8
[R]emoveIndex::index_in	-18.5%	+22.7%	279.94	3.6	228.03	4.4
[I]ndex::index_fulltext	+28.6%	+0.0%	73302.02	0.0	94273.54	0.0
[S]can::index_fulltext::indexed (2001)	-8.0%	+9.6%	217.18	1267.2	199.83	1388.3
[R]emoveIndex::index_fulltext	+598.3%	-85.7%	435.07	2.3	3038.21	0.3
[I]ndex::index_hnsw	-6.4%	+16.7%	15544.32	0.1	14544.90	0.1
[S]can::index_hnsw::indexed (2001)	-87.3%	+659.9%	35562.02	7.9	4524.91	60.0
[R]emoveIndex::index_hnsw	-16.2%	+18.5%	1546.75	0.7	1295.87	0.8

More performance improvements coming

We have a roadmap of performance-related improvements that we'll be rolling out across the coming minor releases. Here's what to expect:

Expanding the new executor to cover write workloads and all remaining statement types
End-to-end streaming, pushing the streaming model from internal execution all the way to the client
Indexing improvements, including better index planning, reduced scan overhead, expanded index types, and optimised composite index strategies
Continued optimisation of the planning and execution pipeline itself

For now, you can start reaping the benefits of the new executor on read-heavy workloads, and expect it to get better with every release.

What’s next?

These benchmarks are just the beginning. This release focuses on measuring the performance gains in SurrealDB 3.0 compared directly to SurrealDB v2.x, highlighting the impact of the new foundations.

Next, we’ll expand this work with a broader benchmarking iteration that compares SurrealDB against other databases and platforms across a wider range of workloads. Stay tuned - there’s much more to come.

If you have any questions, feedback, or want to dive deeper into the results, feel free to reach out to us or join our Discord community to chat directly with the community and team.

Introducing Surrealism

Mark Gyles — Thu, 19 Feb 2026 18:38:16 +0000

We are excited to introduce Surrealism which offers something developers have wanted for a long time: true extensibility inside SurrealDB itself. Those of you who have participated in the SurrealDB 3.0 beta might have already gotten a glimpse of Surrealism, and we’re excited to announce it more broadly to the world today.

Surrealism is a new open-source extension framework for SurrealDB. It allows you to define modular, programmable logic using functions you write in Rust - and execute them directly within the database at runtime. We intend to expand Surrealism to support other languages including JavaScript and Python.

Traditionally, if you wanted to extend your database, you had to choose between brittle, complex, error-prone, and potentially huge scripts, heavyweight external services, or just push the data to the application layer and perform the logic there. Surrealism changes that. It brings logic into the database in a way that’s fast, secure, testable, and fully integrated with your development workflow. Surrealism redefines the way that developers and organisations work with stored procedures - allowing for custom business logic, and access control layers as version-controlled, testable, and shareable modules. These plugins execute with full transactional guarantees, and can be used to manage everything from dynamic API behaviours to policy enforcement and auditing.

How Surrealism works

Functions are compiled to WebAssembly, allowing them to run in a secure, deterministic, sandboxed execution environment, whilst delivering near-native performance, with strict isolation between functions. This makes it ideal for both single-tenant systems and multi-tenant clusters, where isolation and efficiency are critical.

Plugins can be loaded from local disk, object storage, or uploaded directly into SurrealDB. They can be enabled and executed dynamically - without restarting the database - with fine-grained permission controls to determine who can load or invoke them.

With Surrealism, the development experience has been designed from the ground up for productivity. Developers can use their own libraries, tools, and frameworks. They can write tests in Rust, integrate plugins into CI and CD pipelines, and manage plugin versions just like any other component of their application stack. Plugins are defined in simple projects with metadata in a TOML file, and compiled into portable WebAssembly binaries that can be reused, versioned, and shared.

Built-in AI support and agentic workflows

Surrealism enables direct integration with AI models, both local and remote. Plugins can interact with GPU-accelerated inferencing runtimes, or call out to external APIs for services like text or image generation, classification, tokenisation, embedding, translation, and anything else. Surrealism extensions turn SurrealDB into a programmable data and logic layer - perfect for agentic and AI-heavy workloads.

This means you could define a plugin that takes a user’s input, runs it through a local LLM, and stores the generated result - all within a single SurrealQL query. You can trigger sentiment analysis on inserted documents, run similarity scoring against vector indexes, or use a vision model to extract structured data from uploaded files.

And because Surrealism plugins have full access to SurrealDB’s query engine, schema, and the new file functionality, they can work with structured data, unstructured content, and multi-model assets in a single unified system.

For example, a plugin can transcribe an audio file, extract named entities, enrich the results using a remote model, and save the output - all in one transactional flow.

This allows developers to build intelligent applications entirely within the database. The database becomes more than a data store - it becomes an intelligent, programmable runtime for real-time decision-making, content generation, and AI integration.

Bringing logic and agentic workflows into the database, rather than pushing the data out to external systems, enables seamless integration for AI and unlocks new possibilities for intelligent applications. It’s fast. It’s flexible. It’s secure. And it’s open source.

Surrealism is available today for you to start developing your own custom or business logic, and it opens up a whole new frontier in database development - one where you can bring your own code, your own models, and your own ideas directly into the data layer.

We can’t wait to see what you build.

Get started with Surrealism

Go to the Surrealism Docs to get started. We are excited to see what extensions you build, be sure to share them in our Discord channel.

Thank you for being on this journey with us!