Forem: SurrealDB

Exponential cost traps in database architectures: how SurrealDB breaks the cycle

Mark Gyles — Thu, 02 Apr 2026 12:33:01 +0000

Exponential cost traps in database architectures: how SurrealDB breaks the cycle

In the fast-paced world of application development, database choices can make or break an organisation's scalability and budget. Traditional polyglot persistence (relying on multiple specialised databases) often leads to exponential cost growth as systems expand. What starts as a simple setup spirals into a complex, inefficient infrastructure riddled with data duplication, ETL pipelines, and operational silos. SurrealDB, with its multi-model architecture unifying document, graph, vector, full-text, time-series, geospatial, and relational capabilities, offers a consolidated alternative that eliminates these pitfalls. By storing data once and querying it across models, SurrealDB can reduce overall costs by multiple factors, particularly in storage-heavy environments where cloud block storage is in use.

To understand the value of consolidation, consider the common trajectory of database sprawl. Below, we explore a real-world-inspired story of how costs escalate exponentially, helping readers identify if their setup is heading toward inefficiency. This narrative highlights the compounding effects of architectural decisions, setting the stage for how SurrealDB provides a streamlined, cost-effective path forward.

A story of exponentially increasing costs: from simplicity to sprawl

Many teams begin with modest needs but face rapid growth that exposes the limitations of single-model databases. Let's trace a typical app's evolution, illustrating how costs outpace traffic due to added complexity, redundancy, and management overhead.

1. Launching the app: a simple start

Sentiment: "We start simple: a single database, fast and cheap to run."

Your app is new, with straightforward transactional requirements. You deploy a single MySQL instance on AWS, adding one replica for high availability. It's low cost and easy to manage. Database expenses are minimal, infrastructure is straightforward, and personnel needs are light.


Database Costs:	Low
Traffic Handled:	Basic
Infra Cost:	Minimal
Personnel Required:	1–2 engineers
Temp cost:	None yet
Complexity cost:	None yet

Architecture:

MySQL (Primary)
MySQL (Failover Replica)

2. First wave of growth: scaling reads

Sentiment: "Our first scaling challenge is reads. Replicas help, but cost growth slightly outpaces traffic."

User traffic surges, dominated by reads. You add 8 read replicas and scale the writer to a larger instance (where per-vCPU costs rise disproportionately). Costs begin increasing slightly faster than traffic due to over-provisioning and load balancing.


Database costs:	Moderate
Traffic handled:	Increased
Infra cost:	Rising
Personnel required:	Still low, but monitoring grows
Temp cost:	None yet
Complexity cost:	None yet

Architecture:

MySQL Writer
Load Balancer
8x MySQL Read Replicas

3. Write traffic explodes: sharding begins

Sentiment: "As writes explode, we shard the system, but migrations are expensive and disruptive."

Writes overwhelm the single writer. You shard into four MySQL instances, over-provisioning each for spikes. A data access layer handles routing by key, and the migration temporarily doubles costs. Complexity introduces temporary and ongoing expenses.


Database costs:	High
Traffic handled:	Substantial
Infra cost:	Accelerating
Personnel required:	Increasing (sharding expertise needed)
Temp cost:	Spike from migration
Complexity cost:	Emerging

Architecture:

4x MySQL Shards (each with Writer + Multiple Read Replicas)
Load Balancers
CDC (Change Data Capture) for transitioning from old architecture

4. Analytics demands: the first pipeline

Sentiment: "Analytics needs have driven us to add a pipeline and duplicate data with added complexity as a result.”

Product teams demand dashboards and real-time insights. You build an Extract, Transform, Load (ETL) pipeline to an analytical database such as ClickHouse, duplicating data via Change Data Capture (CDC). Applications now query multiple systems, and engineers must manage query routing while maintaining consistency across stores.


Database costs:	Very high
Traffic handled:	Advanced
Infra cost:	Ballooning
Personnel required:	Still increasing (pipeline maintenance)
Temp cost:	Pipeline setup
Complexity cost:	Significant

Architecture:

Existing MySQL Shards
CDC + ETL
ClickHouse
Application querying multiple databases

5. AI ambitions: vector database enters

Sentiment: "AI ambitions require another new system, and more pipelines. The architecture is growing fast."

AI agents need RAG (Retrieval-Augmented Generation) support, querying across systems. You spin up a vector DB like Weaviate or Pinecone, with new pipelines for replication. Data is consolidated in S3 for training, adding sharding as usage grows.


Database costs:	Extreme
Traffic handled:	AI-enhanced
Infra cost:	Exponential
Personnel required:	Specialised teams
Temp cost:	AI integration
Complexity cost:	Overwhelming

Architecture:

Existing Setup
Additional ETL + Kafka
Weaviate
S3 for data lakes

6. Keeping it all running: growing ops

Sentiment: "Now we need a large ops team just to stay afloat. Costs are scaling exponentially, not linearly."

Ongoing tasks include re-sharding MySQL, scaling MongoDB/Weaviate/ClickHouse, upgrades across systems, node recoveries, and pipeline fixes. Teams manage credentials, security, and optimisations in isolation. Ops cycles dominate, slowing innovation and ballooning personnel costs.


Database costs:	Off the charts
Traffic handled:	Peak
Infra cost:	Exponential
Personnel required:	Large teams
Temp cost:	Constant disruptions
Complexity cost:	Debilitating

Architecture:

All systems with balancing, re-sharding, upgrades, and recoveries

Why the cost curve went exponential

Exponential costs are rarely caused by user growth alone. Instead, they often stem from an architecture that acts as a tax on every new login. Between the headaches of manual sharding, the hidden drain of data duplication, and the hours lost to rigid schema changes, many teams end up subsidizing a fragmented mess. If your infrastructure spend is climbing faster than your traffic, you are leaking resources rather than scaling. Moving toward a unified database simplifies the stack and returns your cost curve to a manageable, linear state.

The hidden costs of database sprawl: ETL, layers, and operational overhead

Building on this story, polyglot architectures create fragile ETL pipelines for data movement between systems like MySQL and ClickHouse. These demand resources for processing, debugging, and schema handling. Operational burdens include multiple query languages, fragmented backups, and custom synchronisation.

SurrealDB eliminates this by natively supporting all models in one platform with SurrealQL. No ETL needed. Data stays unified, reducing dependencies and accelerating development, as seen in replacements of PostgreSQL + Neo4j + RabbitMQ stacks.

EBS storage: the #1 cost driver in self-hosted databases

As the story shows, EBS volumes (and Azure and GCP’s equivalent) multiply with replicas and duplications, becoming the dominant expense. With EBS volumes not being eligible for discounts, costs compound in sprawls: 3x replication per cluster, plus copies across databases, over-provisioning, and snapshots typically result in massive redundant data duplication

For example, in a 50-node cluster with 4TB EBS volumes on AWS, storage will be $28,098 monthly, over twice the $12,577 for EC2 (factoring in discounts). Polyglot setups exacerbate this exponential growth.

SurrealDB's compression edge: reducing data size by 1/3 for dramatic savings

SurrealDB's storage engine (TiKV) delivers 70-80% better compression than PostgreSQL's, typically shrinking datasets to one-third. For the 4TB example, this cuts EBS to ~1.33TB, dropping costs to under $10,000, a 65% savings. In addition to the raw savings, smaller data reduces I/O, snapshots, and data transfer cost, countering the sprawl's redundancy.

Eliminating data duplication: organization-wide cost reductions

Storing data once avoids multi-system duplication. A single cluster with 3× replication replaces the primary and secondary copies spread across multiple database systems. Eliminating ETL removes staging costs, while unified data views simplify compliance and backups, reducing S3 storage and egress fees.

Linear scalability: ensuring cost-effective growth

Unlike most other database systems, SurrealDB scales linearly via TiKV, with less than a 5% performance deviation coefficient per node. You get roughly the same increase in capability and performance when adding a node to a 9-node cluster as when adding a node to a 100-node cluster. This keeps costs predictable and prevents exponential cost growth which is typical in sprawling platforms.

Conclusion: escape the exponential trap with SurrealDB

The story of sprawl reveals how polyglot decisions lead to exponential costs through duplication, complexity, and ops overhead. SurrealDB's "one database to rule them all" reverses this, slashing EBS-driven expenses via consolidation and compression. For teams in inefficient systems, migrating promises simplicity and transformative savings.

SurrealMX: In-memory storage with time travel and persistent storage

Mark Gyles — Thu, 26 Mar 2026 15:32:42 +0000

Author: Dave MacLeod

SurrealMX: SurrealDB's new in-memory storage backend

There have been quite a few blog posts since the release of SurrealDB 3.0 to introduce each of the new features we have to show. Today's post is about one called SurrealMX that you likely have yet to hear about - but use every day.

That's because the way you use it is...by typing surreal start. That's it!

That's because SurrealMX is in fact the name of SurrealDB's new in-memory datastore. And since SurrealDB stores data in memory by default, 100% of SurrealDB's users have already given it a try.

What is SurrealMX?

SurrealMX is SurrealDB's in-memory storage engine that was built from the ground up a bit over a year ago. It was merged into the core database during the 3.0 alpha period last November with no fanfare at all in order to try it out for the first time as SurrealDB's in-memory storage engine. Since SurrealDB uses in-memory storage by default, that means that SurrealMX has been every 3.x user's default storage option since then.

But SurrealMX isn't just a more performant way to store data in memory, it comes with new optional functionality too.

If you pay close attention to the SurrealDB source code you might have noticed a PR merged three weeks ago in the runup to 3.0 general availability that included a new way to pass in configuration parameters when starting a database.

If you go to this part of the documentation for the surreal start command, you can see a comparison of how this worked in 2.x versions compared to 3.0.

In 2.x versions, specifying whether SurrealKV is versioned or not was done before the path to the database which led to either writing surrealkv or surrealkv+versioned. Other backends didn't have similar configurations.

surrealkv://my_db
rocksdb://my_db
surrealkv://mydb
surrealkv+versioned://mydb
memory

In 3.0, this has now been changed to pass as many configurations as needed after the path to the database, in the same way that you do the same with a URL. Here are two examples from the documentation showing how this is done.

"surrealkv://path/to/db?versioned=true&sync=every&retention=30d"
"mem://tmp/data?versioned=true&aol=sync&snapshot=60s&sync=5s"
"mem://?versioned=true"

Did you notice that versioned=true is found not just in the surrealkv but also the mem backend? That's because SurrealMX includes versioned queries in the same way that SurrealKV does.

Let's take a closer look!

Time travel using the VERSION clause

With the addition of SurrealMX, SurrealDB now has two storage backgrounds that have versioning: SurrealKV and memory. Versioning is the capability to not just query a table as it stands now, but as it stood before. This feature is exceptionally important in areas like legal compliance in which a database needs to keep past information intact even if it has been overwritten later on.

In SurrealDB, time travel queries are done using the VERSION keyword. Let's give this a quick try with the in-memory backend. First we will start a server with ?versioned=true added:

surreal start --user root --pass secret "mem://?versioned=true"

And then connect via Surrealist or this command in the CLI.

surreal sql --user root --pass secret

Once that is done, we can create a person record or two and make some queries using VERSION followed by time::now() minus how ever long ago we would like to query.

CREATE person;
[{ id: person:g4z4v7yoju9pe21jck6v }]

SELECT * FROM person;
[{ id: person:g4z4v7yoju9pe21jck6v }]

// No 'person' records existed yesterday
SELECT * FROM person VERSION time::now() - 1d
[]

// Add a second record
CREATE person;
[[{ id: person:aczudhy0mvef5bxurcry }]]

// 'person' has two records, but about ten seconds ago
// it had just one
SELECT * FROM person VERSION time::now() - 10s
[{ id: person:g4z4v7yoju9pe21jck6v }]

Since the VERSION keyword has been available via SurrealKV since SurrealDB 2.0, it's very likely that you are already quite familiar with it. Let's now move on to in-memory persistent storage options which were entirely unavailable before SurrealDB 3.0.

In-memory with optional persistent storage and versioning

In-memory storage with optional persistent storage is one of the features of Redis that has made it so popular. SurrealMX offers this as well, via a somewhat more composable model.

Persistence for the memory backend is used for the following:

Append-Only Log (AOL) - Synchronous/asynchronous modes for durability
Snapshots - Periodic full database state capture
Data Recovery - Automatic recovery from snapshots + AOL on startup
AOL Truncation - Automatic cleanup after snapshots

You may be curious at this point how this differs from using SurrealKV (or RocksDB) to store your data. The difference is that SurrealMX persistent storage does not compress the data. It is an append-only-log of database writes, never compacted, until a snapshot occurs. It is in effect an entire database export. You can think of it as a convenience on top of in-memory speed while SurrealKV is used for traditional durable storage for datasets of any size.

Trying it out

Let's grab part of the documentation to see how it works. These are the available parameters to use when starting a database with in-memory storage.

retention (a duration string, e.g. 30d, 24h, 30m)
aol (never, sync, or async), for writing changes to an append-only log file
snapshot (a duration string, e.g. 60s), for periodically writing a snapshot of the database to the file system
sync (never, every, or a duration string like 5s), for specifying when to flush the append-only log file to the file system
never - (default) leave flushing to the OS (least durable)
every - sync on every commit (most durable)
interval - periodic background flushing at the given interval

We can match this up against the example above which shows that aol is set to sync, snapshot is set to 60s, and sync is set to 5s.

"mem://tmp/data?versioned=true&aol=sync&snapshot=60s&sync=5s"

That's quite a bit to think about, but fortunately the SurrealMX repo has a table that lays out exactly when you would want to use one configuration over another. Each of these is balanced against the performance hit, ranging from no persistence at all (the fastest) to Sync AOL + Every fsync which survives both process and system crash at the greatest cost to performance.

To make a choice, just ask yourself the following as you move down the chart: what do you want to happen to your data if the process or system crashes? Then choose the most performant option that offers you the guarantees you need.

Configuration	Survives Process Crash	Survives System Crash	Performance
No persistence	❌	❌	Fastest
Snapshot-only	⚠️ (last snapshot)	⚠️ (last snapshot)	Fastest
Async AOL + No fsync	⚠️ (mostly)	⚠️ (mostly + OS buffers)	Very fast
Async AOL + Interval fsync	⚠️ (mostly)	⚠️ (mostly + since last fsync)	Very fast
Async AOL + Every fsync	⚠️ (mostly)	⚠️ (mostly)	Very fast
Sync AOL + No fsync	✅	⚠️ (OS buffers)	Fast
Sync AOL + Interval fsync	✅	⚠️ (since last fsync)	Fast
Sync AOL + Every fsync	✅	✅	Slow

Looks like the quickest and easiest way to use persistent storage is through periodic snapshots. Since these must be set to a number at least greater than 30 seconds, we'll give it a try with 31.

surreal start --user root --pass secret "mem://tmp/data?snapshot=31s"

We can then connect with Surrealist or the surreal sql --user root --pass secret command and do a CREATE person or two.

After that, use cd tmp/data to go into the folder and wait for the file to show up. After 30 seconds you should see a file called snapshot.bin. That's the snapshot!

!v�4��y��
         /!cgdefault�?H/�y��mainmain/!ndI��� H�
��t����4P���I��� H�
��t��@~����D�&���I��� H�
...snip
*/***person*rh6xmo5z8cqffvp4hbf6��BQ7
                                     ��
/***user*john��T|��
nameJohn%

Once that file is present, you can stop and restart the database, use SELECT * FROM person and see that the person records are still there. Not bad at all!

We suspect that the most popular options will be the following four:

Snapshot only: the fastest option
Async AOL + no fsync: the fastest option that tracks every change the moment one happens
Sync AOL + no fsync: the fastest option if you think you might have a process crash
Sync AOL + every fsync: the most durable option, survives both process and systems crashes

But that remains to be seen.

If you've given persistent in-memory storage a try already or have used it since the publication of this blog post, please get in touch and let us know!

Where SurrealDB fits in your stack

Mark Gyles — Tue, 17 Mar 2026 09:02:54 +0000

Where SurrealDB fits in your stack: a path to multi-model efficiency

In today's data-driven applications, managing multiple specialised databases (often referred to as polyglot persistence) has become a common challenge. Teams juggle relational databases like PostgreSQL for structured data, document stores like MongoDB for flexible schemas, graph databases like Neo4j for relationships, and vector databases for AI workloads.

While powerful once finally set up, this configuration introduces complexity: multiple query languages, data pipelines, security models, and scaling strategies. Enter SurrealDB, an open-source multi-model database that unifies these paradigms into a single system. By consolidating databases into one platform, SurrealDB enables applications with multi-model capabilities, blending relational queries, graph traversals, vector search, time-series and geospatial analysis without the overhead of disparate tools.

This article explores how SurrealDB facilitates database consolidation, drawing from real-world benchmarks, case studies (e.g., Tencent consolidating nine databases into one), and performance comparisons. We'll map current polyglot setups to a consolidated future state, discuss ROI and TCO implications, and provide architecture overviews. If you're a technical evaluator considering SurrealDB as a replacement for your existing stack, this guide assumes basic database knowledge but no deep familiarity with SurrealDB's SurrealQL query language or internals.

Where SurrealDB fits in your deployment

SurrealDB positions itself as a versatile, AI-native database that scales from edge devices to petabyte clusters. Built in Rust, it supports deployment as an embedded library, single-node server, or distributed system. Its layered architecture separates compute (query processing) from storage (e.g., using TiKV for distributed persistence), allowing flexible scaling.

What it replaces

SurrealDB's multi-model design natively handles multiple data paradigms, making it a direct substitute for several specialised databases:

Document stores (e.g., MongoDB): SurrealDB stores data as flexible, JSON-like documents, supporting a default schemaless model that allows as much strictness to be added to your schema as your solution requires. It replaces MongoDB for content management, real-time apps, and unstructured data, with SurrealQL providing aggregation and querying akin to MongoDB's syntax but unified across models.
Graph databases (e.g., Neo4j): with built-in graph functionality, SurrealDB uses record links and graph edges as first-class citizens for traversals and relationships. It has been used to replace Neo4j in recommendation engines, fraud detection, and knowledge graphs, offering SQL-like syntax for graph queries without Cypher's learning curve.
Relational databases (e.g., PostgreSQL): SurrealDB emulates relational tables with ACID transactions, indexes, and joins (via record or graph links rather than traditional SQL JOINs for better scalability). It consolidates Postgres workloads for transactional data, geospatial queries, and time-series.
Vector and Search databases: integrated vector embeddings, full-text search, and hybrid retrieval replace standalone tools like Pinecone or Elasticsearch for AI apps, including RAG (Retrieval-Augmented Generation) pipelines.
Key-Value and Time-Series stores (e.g., Redis, InfluxDB): SurrealDB handles simple key-value pairs and temporal data with live queries for real-time updates along with an in-memory datastore that also includes persistent snapshot or AOL storage.

Case studies highlight replacements: one company swapped Neo4j, RabbitMQ, and Postgres for SurrealDB, while Tencent consolidated nine systems, reducing complexity and costs.

What it does not replace

While versatile, SurrealDB isn't a one-size-fits-all solution:

Pure OLAP or data warehouses (e.g., Snowflake, BigQuery): for massive analytical workloads requiring columnar storage and distributed joins across exabytes, SurrealDB's focus on multi-model OLTP (online transaction processing) may not match specialised OLAP tools' optimisation for ad-hoc queries.
Highly specialised hardware-accelerated databases: it doesn't replace niche systems like GPU-optimised databases for extreme-scale ML training or legacy mainframe databases with proprietary integrations.
Non-database tools: SurrealDB consolidates data storage but not orchestration (e.g., Kubernetes).

The Outcome

A single SurrealDB instance or cluster handles all, with SurrealQL as the query interface. Compute nodes process queries, storage persists data distributively.

Benchmarks show efficiency: SurrealDB often outperforms single-model databases in mixed workloads (e.g., faster than MongoDB in reads, comparable to Neo4j in graphs).

Mapping the customer journey into consolidation: ROI and TCO over time

Adopting SurrealDB follows a phased journey, yielding compounding ROI (return on investment) and reduced TCO (total cost of ownership).

Phase 1: evaluation (months 1-2)

Assess: map current databases to SurrealDB models using migration guides (e.g., from Neo4j/PostgreSQL) and Surreal Sync migration tool.
POC: run SurrealDB in a testing environment as the backend for a single application; test key validation areas such as:
- Setup and integration: install SurrealDB locally or via SurrealDB Cloud, in your application stack, and verify connectivity with existing microservices.
- Data migration: import sample datasets from your current databases and test schema definitions in schemaless or schemafull modes.
- Query performance: execute mixed-model queries (e.g., combining document lookups with graph traversals and vector searches) and benchmark against your existing setup for read/write throughput, latency, and resource usage.
- Feature validation: test real-time capabilities like live queries and change feeds; evaluate AI features such as vector embeddings and hybrid search for RAG use cases; assess geospatial and time-series handling.
- Security and compliance: configure authentication (e.g., JWTs, scopes), test access controls, and ensure alignment with your security best practices like least privilege.
- Scalability and reliability: simulate load with tools like the SurrealDB CLI or Surrealist dashboard, test failover in a small cluster, and monitor performance metrics.
- Developer experience: use the Surrealist dashboard for querying and visualisation; experiment with SurrealQL syntax to rewrite existing queries from MongoDB/Neo4j/PostgreSQL.
Initial ROI: is higher than baseline due to the overhead on the team doing the POC/migration, and temporarily running 2 different databases in parallel until full cut-over.

Phase 2: partial consolidation (months 3-12)

Migrate one workload (e.g., replace MongoDB for documents).
Integrate with AI frameworks (e.g., LangChain for RAG).
ROI: faster feature iteration performance improvements and lower overhead. TCO: 10-50% reduction by eliminating 2-3 databases, cutting licensing/maintenance.

Phase 3: full multi-model shift (year 1+)

Consolidate all: from polyglot deployment model to a unified stack.
Scale: horizontal clustering for high availability.
ROI: 2-3x faster time-to-market; AI apps ship quicker with native vectors/graphs. TCO: Up to 70% savings via fewer systems, no ETL, and simplified ops (e.g., one monitoring setup).

Over time, ROI grows exponentially as complexity drops: enterprises report 3-4x lower costs through consolidation than managing 3-4+ databases.

Conclusion

SurrealDB offers a compelling consolidation path for technical teams, replacing multiple databases while supporting multi-model apps. By mapping your current polyglot setup to its unified models, you can achieve significant ROI through faster development and lower TCO. Start with a POC to evaluate fit. Learn what companies like Tencent, Walmart and Samsung already know. For more, explore SurrealDB's documentation or GitHub, or visit our Discord community.

OpenAI’s Postgres Architecture: A Brilliant Fix for a Billion-Dollar Mistake

Mark Gyles — Wed, 11 Mar 2026 23:33:25 +0000

Author: Matthew Penaroza, Head of AI Solution Architecture at SurrealDB

OpenAI’s write-up on scaling PostgreSQL for ChatGPT is worth reading twice: once as an operations playbook, and once as a case study in what happens when a system outgrows its original shape. The engineering is excellent. But the database they started with leaves a lot to be desired and has almost certainly cost the company millions of dollars in completely unnecessary infrastructure pain.

What I’ve noticed on social media is that many people are treating this article as proof of how amazing the engineering is at OpenAI (which it is) and as a blueprint for how their own organizations should scale. I’d argue the opposite. This architecture is a lesson in what not to do. Just because brilliant engineers can make something work doesn’t mean you should. I wouldn’t eat soup with a fork, and I wouldn't use a single-writer PostgreSQL cluster to serve 800 million users.

Why this architecture exists (and why it shouldn't)

The scaling strategies OpenAI used here are very familiar to me. I've helped design the database architecture for some of the largest transactional systems on the planet, and what OpenAI did mirrors patterns I’ve seen in the field. What’s unusual is the timing. Most companies I’ve seen running this exact playbook are 2010s organizations that started on MySQL or Postgres before the distributed-database space matured. They scaled with read replicas and orchestration because migrating later was impractical. That isn’t OpenAI. They made these architectural decisions presumably in the early 2020s, when far better options already existed. My suspicion is that their technical decisions had a lot to do with the fact that Microsoft has a large stake in OpenAI and wanted them running on Azure services.

(Traditional 2010s Postgres Architecture)

What OpenAI actually built

But enough history. Let's look at what they actually built.

OpenAI runs a single primary Azure PostgreSQL Flexible Server instance with nearly 50 full read replicas distributed globally, handling millions of QPS for ChatGPT and the API. All writes hit the single primary. Reads are aggressively offloaded to replicas. They have explicitly avoided sharding the existing PostgreSQL deployment because of the inherent complexity and time-consuming nature of that adaptation, a fully understandable call at their scale.

A single-writer Postgres instance, even the biggest one Azure offers, obviously can’t handle this kind of write volume. So OpenAI migrated shardable workloads and write-heavy tables to Azure Cosmos DB and blocked all new tables/features on the main PostgreSQL cluster. New workloads now default to sharded alternatives. This also likely means they have applications stitching data together from both Cosmos DB and Postgres.

Because Azure PostgreSQL caps connections at 5,000, OpenAI deployed PgBouncer in Kubernetes. Each read replica gets its own K8s deployment with multiple PgBouncer pods behind a Service for load balancing. This dramatically cuts active connections and connection-setup latency.

On replication: the primary currently streams WAL directly to all ~50 replicas. That’s sustainable today on huge instances with fat pipes, but it has hard limits. To go further, OpenAI is having the Azure Postgres team build out a new feature for them: cascading replication (intermediate replicas relay WAL downstream). But the feature is still in testing.

Other key optimizations:

Aggressive query tuning to kill expensive multi-table joins and ORM anti-patterns
Multi-layer caching with cache locking/leasing to survive cache-miss storms from overloading their PostgreSQL instances
Workload isolation: high-priority vs low-priority traffic (and different products) routed to separate instances to avoid noisy-neighbor issues
Multi-layer rate limiting (application, proxy, connection pooler, and per-query) plus careful retry policies to prevent spikes and retry storms
Restricted schema changes: only lightweight operations allowed (no full table rewrites); backfills are heavily rate-limited; new tables forbidden
High-availability setup on the primary with a hot standby for fast failover, plus enough replica headroom per region to survive individual replica failures

The results: impressive… and concerning

The result is a system that's… kinda stable: double-digit ms p99 client-side latency, claimed five-nines availability, and “only one SEV-0 PostgreSQL incident” in the past year. (I’m questioning how they measure five-nines availability if they’re happy about only having one SEV-0 this past year, but I digress.)

That’s impressive problem-solving, and it’s a credit to OpenAI’s engineering talent that they pulled it off. Many organizations fail at far smaller scale.

But let’s be clear: I have never in my career heard an infrastructure engineer be happy about having “only one SEV-0” on their primary transactional database infrastructure in a year. (Can you imagine what that says about the previous years?) A SEV-0 is the highest-severity outage. This is not normal for a well-architected system (granted, most systems are not well architected). Even in gigantic, high-growth companies, many go years without a SEV-0 on their primary transactional database system, or have never even had one.

The problem with this system is that it currently is and is going to continue to be constantly on the verge of a SEV incident, and it will never stop unless they shard or migrate off of it. Growth is endless (or it is if the company is healthy, which hopefully OpenAI stays healthy), which means more data, QPS, connections, etc. every year forever. And they have put themselves in a situation where there are inherent architectural limits that have required all kinds of architectural workarounds and special arrangements with their cloud hosting provider just to keep things running. Not to mention teams and teams of engineers at both Azure and OpenAI just to keep this alive who could all probably be better utilized to increase functionality instead of keeping a system alive. Remember, Azure Database for PostgreSQL Flexible Server only allows for 5 read replicas, so in order for OpenAI to scale this way they had to have custom bespoke solutions made for them by the Azure engineering team.

The real cost: engineering heroics forever

Virtually every problem they had to solve has already been solved, out of the box, by modern distributed databases. They would have saved millions in infra spend, thousands of engineering hours, and future-proofed the system by choosing something designed for this scale from the start.

But I don’t blame the OpenAI team; maybe this level of growth wasn’t anticipated, or maybe internal constraints forced their hand. No one knows but the OpenAI team.

But the point I want to make with this article is: if you were designing the database layer for this class of workload today, should you do this? And my very strong recommendation would be no. Unless you don't care about time, money, outages, your engineers' sanity, weekends, holidays, sleeping through the night, developer velocity, future-proofing, your remaining hair, or basic human joy.

The alternative: start with distributed architecture

So what's the alternative? You should use a database architecturally designed for this type of scale from the start. In the rest of this piece, I’ll use SurrealDB as the example (full disclosure: they pay me, so I’m biased, at least until the iron fist of the CEO comes down on me for writing this article). That said, the underlying principles I’m about to walk through apply to many modern distributed systems, not just SurrealDB. Bias or not, the technical realities are what matter. And those realities should bias you towards a very different path forward, one that avoids nearly all of the problems OpenAI had to solve through sheer operational heroics. Let’s walk through it.

The problems PostgreSQL forced OpenAI to solve, and how a native distributed system like SurrealDB avoids them entirely

Let’s go through the biggest pain points OpenAI encountered (including ones they didn’t mention) and why almost every one of them disappears with a distributed database.

1. Fundamental scale limitations of a 1990s design

PostgreSQL came out of Berkeley in the mid-1990s, back when a few terabytes felt huge and “big data” was whatever fit on a lab server. And while it's seen a lot of development since then, the core design was never meant to handle planet-scale traffic across regions with millions of QPS and hundreds of millions of users. All the tricks OpenAI threw at it (read replicas, PgBouncer, begging Azure for cascading replication) are basically duct tape on a system that just wasn’t built for this.

Databases like SurrealDB were architected at their core for this level of modern distributed scale. A SurrealDB distributed deployment uses the TiKV storage engine, the same storage engine that some of the biggest banks, insurance companies, Flipkart, Databricks, Pinterest, Plaid, Atlassian, and others use in clusters with hundreds of nodes, pushing millions of QPS and petabytes of data. And these companies are not constantly dealing with SEV-0 events. The fundamental architecture is designed to handle this traffic without the same kind of custom patches and workarounds.

In addition, databases like SurrealDB do not have the same connection limit issues as older relational databases, where you’re capped at 5,000 connections. So the connection pooling problem OpenAI had to overcome becomes mitigated (though it's still a good idea to optimize the system).

2. The insane replica cost

OpenAI has ~50 full read replicas, which means 50 complete copies of the whole dataset. Block storage is the priciest, least-discountable thing on any cloud bill, and copying dozens/hundreds of TBs of data fifty times is throwing away millions a year for no extra value beyond read capacity. On top of that, each replica is its own isolated box that can’t share compute. You have to overprovision every single one to survive its own spikes, so half (or very likely more) of your CPU is probably sitting idle 95% of the time. And when traffic surges, you can’t just add compute instantly. Provisioning a new replica and replicating the full dataset takes a huge amount of time (potentially hours). That’s why OpenAI overprovisions well in advance, to give themselves a buffer while new replicas catch up. But this architecture means that they need accurate forecasting and plenty of spare capacity ahead of time.

And if you don't believe me on the storage pricing analysis, here's the AWS bill calculator for 50 64TB EBS volumes and 50 128vCPU instances. How much more expensive do you think storage is vs. compute?

That’s right. Storage costs ~7× more than compute in this example. This can vary depending on the disk type, how much data and compute you need, and whether you are using managed services. In your situation it might only be 4× or 2×, but you get the gist. And while this example is from AWS, this is true across all cloud hosting providers.

The core reason is that with reserved instances you can get huge discounts on compute with all the cloud providers, up to 72% or even 80%+ with enterprise discount plans. The standard discount for block storage is 0%. I have seen up to 5% with special arrangements on AWS, but that was for one of their largest customers. This is why scaling via read replicas is such a financial disaster. Now, maybe OpenAI gets special treatment here because they are partially owned by Microsoft and this isn’t relevant to them, but it should be relevant to you if your company cares about money.

And feel free to verify these cost figures yourself:

But how does a distributed database like SurrealDB address this? When you store data on SurrealDB, it uses a separate node entirely for storage that replicates it 3 times, auto-shards it into small chunks across the cluster, and distributes it across the cluster for high availability.

And if you need to scale throughput and concurrency, you can do so by adding cheaper stateless compute nodes that all read from the same shared storage pool. No full copies. And on top of that, the stateless compute nodes can be removed and spun up dynamically at any time, which means you don’t have to massively overprovision for any given event. The system is flexible enough to be scaled up and down all the time, saving huge costs on overprovisioning. And in the event you do have a high-growth event and you need more compute, it only takes a few minutes to get the cluster powered up enough to handle the increased workload.

All this means that by going with a distributed architecture you could, depending on your scenario, decrease your current/future total cloud bill by several factors.

3. The single-writer bottleneck and the forced migration to a separate NoSQL system

In PostgreSQL’s single-primary architecture, every single write (updates, inserts, deletes) has to funnel through that one primary instance. No matter how large the machine (and OpenAI is clearly running one of the biggest Azure offers), a sudden write spike can still saturate CPU, I/O, or WAL bandwidth. Latency climbs, queries start timing out, retries pour in, and you’re suddenly in a classic overload death spiral that can degrade ChatGPT and the API for everyone.

OpenAI’s fix was to identify the workloads that were shardable and write-heavy (many of which are probably fundamentally relational in nature), migrate them to Cosmos DB, and accept the permanent cost of running two completely different database systems side-by-side. That means separate query languages, different consistency models, distinct operational tooling, and application code that has to stitch results together across the two stores forever.

A distributed database like SurrealDB is multi-writer by design. Writes are spread across multiple nodes in the cluster, so write capacity grows horizontally as you add machines. There is no single primary choke point, and no need to spin up and maintain an entirely separate database just to handle write load.

4. Expensive multi-table joins killing the cluster

OpenAI repeatedly calls out multi-way joins as a major source of SEV incidents. They had to audit ORM output, break joins into application-side logic, and basically treat joins as dangerous. That’s not a great place to be when your data model is relational.

SurrealDB avoids this class of failure by not relying on traditional relational joins at all. Relationships are modeled explicitly as record links and graph edges, and resolved through traversal inside a single query engine rather than join operators. This eliminates join explosion and centralized join execution while still allowing expressive relationship queries across documents, graphs, and vectors without forcing application-side orchestration or creating single-node saturation risks.

5. Schema changes become impractical

OpenAI now bans anything beyond the lightest schema changes and rate-limits backfills for weeks. That’s a huge velocity tax. Every new column or type change becomes a high-risk, multi-week operation.

SurrealDB reduces schema-change issues by decoupling schema definition from rewriting existing storage. Where relational systems often force heavy DDL and backfills just to keep data and schema synchronized, SurrealDB’s current approach enforces schema primarily as data is written/updated, and the team has explicitly identified richer zero-downtime mechanisms (aliases/versioning/async migrators) as the path to handling large-scale online migrations and reindexing without downtime. This results in a scenario where you can make a change to a table at scale without complexity or waiting.

6. Stability constantly threatened by spikes

In the single-primary PostgreSQL setup, any unexpected surge (cache-miss storms, new feature launches, retry loops, or sudden traffic growth) can quickly saturate a single instance’s CPU, I/O, or connection limits. Latency spikes, requests time out, retries amplify the load, and the whole service risks cascading degradation. The primary is still a true single point of failure (even with a hot standby), and losing a read replica requires pre-planned headroom across every region. OpenAI had to build multiple layers of rate limiting, workload isolation, and cache leasing just to keep the system from regularly tipping over.

A distributed architecture with separated compute and storage handles this differently. Stateless query nodes can be scaled automatically in seconds during spikes, absorbing bursts without saturating any one machine. High availability is native: individual node failures are isolated, and automatic failover and rebalancing occur without any intervention.

7. Astronomical ongoing engineering overhead

This setup requires constant tuning, custom Azure features, cascading replication experiments, query audits, rate-limit adjustments, connection-pool management, and on-call heroics to prevent the next SEV-0. That’s a permanent tax on engineering bandwidth. People who could be building new features are instead keeping Frankenstein alive.

SurrealDB is designed for distributed scale from the start and is intended to be boring at this traffic level. The problems OpenAI is constantly solving just aren’t issues with databases like SurrealDB. Now, every database will encounter problems at hyperscale (and tons of them (and when I say tons, I mean tons; scale is hard)), but they won't be nearly as severe. In fact, with many of the (healthy) companies I’ve worked with, the majority of the time infrastructure engineers spend is more on the optimization and functionality side of things.

8. Inevitable database sprawl

By freezing the core PostgreSQL schema and banning new tables entirely, OpenAI has ensured that any future workload with different performance characteristics or heavier writes will need its own separate database system (Cosmos DB today, likely others tomorrow). That guarantees growing proliferation, more cross-system data movement, and increasing consistency and operational challenges over time, which, by the way, likely means more data transfer fees and more replicas across databases.

(A possible future of OpenAI’s architecture)

SurrealDB is multi-model, which can evolve with the product. It can incorporate relational, document, geospatial, temporal, graph, vector, and time-series data in the same cluster as requirements evolve, avoiding the need to continually spin up new databases.

9. Locking yourself out of future AI workloads on your own data

This is the subtler long-term cost. OpenAI has arguably the most valuable dataset on the planet, but their core transactional store is perpetually at capacity and can’t absorb new workloads. To do anything interesting with that data at scale, they’ll need to replicate it elsewhere, adding cost, latency, time, and consistency risk.

SurrealDB is built for exactly this future: relational + temporal + document + graph + vector + time-series in the same system, on the same dataset, without replication or ETL. You can run transactional workloads, analytics, RAG, and recommendation engines against the same live data without fear of blowing up the cluster.

Database infrastructure shouldn’t require miracles

OpenAI’s infrastructure team pulled off a minor miracle keeping a single-writer PostgreSQL cluster alive at this scale. The optimizations are clever, the operational discipline is elite, and the fact that ChatGPT mostly works for 800 million users is a testament to their skill.

But miracles aren’t scalable. This architecture is a patchwork of workarounds, custom Azure features, and permanent firefighting, held together by constant engineering effort that could be spent building new capabilities instead. All to compensate for fundamental limitations of a 1990s-era database design pushed far beyond its natural habitat.

If you’re building something ambitious in 2026, don’t inherit these problems. Choose a database built for distributed scale, horizontal everything, multi-model flexibility, and reliably boring operations from day one.

Your bill will be lower, your on-call rotation will sleep better, and you’ll have vastly more freedom to experiment with the AI-native workloads that will define the next decade.

Pick the right foundation the first time. Everyone on your team will thank you.

SurrealDB is now available on AWS Marketplace

Mark Gyles — Wed, 11 Mar 2026 18:08:27 +0000

We’re excited to announce that SurrealDB is now officially available on AWS Marketplace.

You can now deploy SurrealDB Cloud with streamlined procurement, simplified billing, and faster time to production - all through your existing AWS accounts.

This launch also reinforces our commitment to meeting customers where they build, and deepens our partnership with AWS to make SurrealDB easier to adopt, deploy, and scale in the cloud.

Why this matters

AWS Marketplace makes it easier for companies to adopt SurrealDB with:

Seamless AWS-native deployment
Consolidated billing
Faster security and procurement approval
Production-ready infrastructure from day one.

If you’re already building on AWS, getting started with SurrealDB just became frictionless.

Why SurrealDB?

SurrealDB combines document, graph, and relational capabilities in one unified database - designed for modern applications, real-time systems, and AI-powered workloads.

With flexible schema, SurrealQL, and built-in real-time features, SurrealDB simplifies your technology stack, reduces Total Cost of Ownership and operational complexity, and allows you to ship products and features in days rather than weeks.

Get started

SurrealDB is available now on AWS Marketplace. We can’t wait to see what you build.

How to use Surrealism to build your own custom SurrealDB extensions

Mark Gyles — Wed, 11 Mar 2026 16:56:48 +0000

Author: Dave MacLeod

How to use Surrealism to build your own custom SurrealDB extensions

Two weeks ago we announced SurrealDB 3.0, along with a new feature called Surrealism that lets you write your own functions in Rust that can then be annotated to call directly from the database.

In this blog post, we are going to take a look at what use cases Surrealism solves and easy it is to get started with it. Let's begin by taking a look at how functions are used in SurrealQL and how Surrealism expands their range of use.

Functions in SurrealDB

SurrealDB has always had user-defined functions that look like this:

DEFINE FUNCTION fn::combine($one: number, $two: number) -> number {
  $one + $two
};

fn::combine(10, 10); -- 20

These functions can get to any complexity and length that you like.

DEFINE FUNCTION fn::relate_all($records: array<record>) {
  IF $records.len() < 2 {
      -- Don't do anything, ending the recursion
  }  ELSE {
      LET $first = $records[0];
      LET $remainder = $records[1..];
      FOR $counterpart IN $remainder {
          RELATE $first->to->$counterpart;
      };
      fn::relate_all($remainder);
  }
};

The next type of function was added in SurrealDB 2.0: closures. Closures are anonymous functions that can work in read operations that can access both arguments and parameters in their scope.

Defining and using a closure

LET $log_level = "DEBUG";

-- Define a closure
LET $split = (|$record: record| {
    table: $record.tb(),
    key: $record.id(),
    log_level: $log_level
});

$split(user:1);
[person:one, cat:two].map($split);

Closure output

-- Calling $split
{
    key: 1,
    log_level: 'DEBUG',
    table: 'user'
}

-- Passing $split into .map() which takes a closure
[
    {
        key: 'one',
        log_level: 'DEBUG',
        table: 'person'
    },
    {
        key: 'two',
        log_level: 'DEBUG',
        table: 'cat'
    }
]

However, both defined functions and closures live inside the SurrealQL query language inside the database.

Surrealism surpasses this limitation by letting you define functions in Rust instead, with all of the available bells and whistles that you get from the Cargo package manager and crates.io ecosystem.

These functions are then compiled to a WASM binary that then can be linked to from the database itself.

Being able to add functions just like that means that new functionality can be added today, to your existing SurrealDB version, with the only setup being two DEFINE statements - one to set up a file bucket, and one more to show the database where inside this bucket the WASM file is located.

Why we made Surrealism

Making Surrealism was in fact an absolute necessity for a project like SurrealDB in which the community plays such a large part. That's because a lot of requests are for features that are nice to have, but don't quite make sense to merge into the code for the database itself. But with an extension system, you can make the feature yourself!

Using Surrealism is basically like installing apps on your phone. You can't modify the OS of the phone itself, but you can install a new app any time you find one that appeals to you. Similarly, you can build and upload an app for other people to try out.

Some examples

Let's look at two case studies where Surrealism allows a functionality to be added that was not possible before.

Case 1: A well-intended PR

Sometimes you will see a well-intended PR that isn't able to be accepted, such as this one. That PR used Rust's fake crate to create a number of database functions that could be used to seed a database with mock data: fake names, addresses, companies, and so on. The function signatures would have looked like this.

fake::address::building_number
fake::address::city
fake::address::country
fake::address::country_code
fake::address::state
fake::address::street
// ... and so on...

While useful (and tempting), it would lead to an increase in the binary size for a functionality that is not core to a database, so it was not able to be merged.

Case 2: A feature request

Another example comes in the form of feature requests, such as this one from a user who would like to see Serbian added to the list of languages that could use the Snowball filter.

First let's look at what the Snowball filter is and what it does. This filter is used inside a DEFINE ANALYZER statement to enable full-text search. The filter uses something called stemming which reduces words to something like a root form. This is best seen through an example. In the two queries below we have defined an analyzer that first splits a string by class (whitespace, dots, numbers vs. letters, etc.) and then passes it through the Snowball filter for English.

DEFINE ANALYZER en_snowball TOKENIZERS class FILTERS snowball(english);
search::analyze(
        "en_snowball",
        "Making Surrealism was in fact an absolute necessity for a project like SurrealDB..."
);

The output shows some words like 'absolut' and 'necess', which are the shortest possible forms that can be used to match on a word. Note that this isn't an exact science as you could argue that 'make' should be 'mak' in order to match on forms like 'making'. But that is a subject for another time.

['make', 'surreal', 'was', 'in', 'fact', 'an', 'absolut', 'necess',
 'for', 'a', 'project', 'like', 'surrealdb', '...']

Now, the interesting part about this feature request is that Serbian is one of the languages with a Snowball stemmer. So adding Serbian is technically doable, except that it would require quite a bit of work.

The reason why is that the crate that SurrealDB uses to implement it hasn't been updated for a while. There was even a PR to add Serbian to it but it was never merged and has since been closed. So adding Serbian to the list of languages would require something like the following:

Try contacting the crate author to bring up the issue again
Wait a while, fork the repo if there is no response
Contacting the author of the PR to add Serbian to see if it's okay to use that PR, or implement it ourself
Follow up with a PR to the SurrealDB source code

Now, if you'd like to put a PR together that accomplishes all that then feel free!

However, thanks to Surrealism, this sort of problem can be resolved today.

Making a Surrealism module

So let's put a Surrealism module together that allows you to create mock data as well as work with Serbian text. This section is a somewhat quicker walkthrough of the same steps as this page in the documentation, so check that page out too to see more details.

There is a bit of boilerplate to start.

The boilerplate

First we need to add a line in Cargo.toml to create a dynamic library that can be loaded from another language.

Add to Cargo.toml

[lib]
crate-type = ["cdylib"]

And then a surrealism.toml file is needed. We can just copy and paste this into the file.

Create surrealism.toml

[package]
organisation = "surrealdb"
name = "demo"
version = "1.0.0"

[attach]
fs = "fs"

[capabilities]
allow_scripting = true
allow_arbitrary_queries = true
allow_functions = ["fn::test"]
allow_net = ["127.0.0.1:8080"]

Annotating functions

Now we can make some functions and annotate them.

The code for the fake data couldn't be simpler, as the fake crate uses a dedicated function for each type of fake data. You do have to choose a language though. Let's go with German for the fun of it.

To make a function a Surrealism function, just add the #[surrealism] annotation to the top. This will allow them to be exposed in the binary once the code is compiled.

// Inside lib.rs
use fake::{
    Fake, 
    faker::{
        address::de_de::{CityName, CountryName}, 
        company::de_de::CompanyName, creditcard::de_de::CreditCardNumber, 
        name::de_de::Name
    }
};
use surrealism::surrealism;

#[surrealism]
pub fn city() -> String {
    CityName().fake()
}

#[surrealism]
pub fn company_name() -> String {
    CompanyName().fake()
}

#[surrealism]
pub fn full_name() -> String {
    Name().fake()
}

Calling these functions will generate an output something like this.

Oberschreiberheim
Schmidt and Götz Stiftung
Gerhard Hoffmann

Now let's move on to Serbian. Implementing Snowball ourselves would be a lot of code, so let's go with something simpler for demonstration: stop words. Stop words is a method by which you remove all the most frequent words from a language that are not very useful when working with text. For English these are words like the, a, an, but, at, on, is, and so on.

Rust has a stop words crate which doesn't have Serbian, but does have Croatian (language code hr). Both Serbian and Croatian are one of the standards of Serbo-Croatian which means that a tool for one works pretty well for another.

This crate has an example of the tool in action here.

fn main() {
    // Read in a file
    let document = std::fs::read_to_string("examples/foreword.txt").expect("Cannot read file");

    // Print the contents
    println!("Original text:\n{}", document);

    // Get the stopwords
    let words = stop_words::get(stop_words::LANGUAGE::English);

    // Remove punctuation and lowercase the text to make parsing easier
    let lowercase_doc = document.to_ascii_lowercase();
    let regex_for_punctuation = human_regex::one_or_more(human_regex::punctuation());
    let text_without_punctuation = regex_for_punctuation
        .to_regex()
        .replace_all(&lowercase_doc, "");

    // Make a regex to match stopwords with trailing spaces and punctuation
    let regex_for_stop_words = human_regex::word_boundary()
        + human_regex::exactly(1, human_regex::or(&words))
        + human_regex::word_boundary()
        + human_regex::one_or_more(human_regex::whitespace());

    // Remove stop words
    let clean_text = regex_for_stop_words
        .to_regex()
        .replace_all(&text_without_punctuation, "");
    println!("\nClean text:\n{}", clean_text);
}

Now we have a small hiccup here: unlike Croatian, Serbian is also written using the Cyrillic alphabet. However, a quick search through crates.io shows that there is a crate to convert Serbian Cyrillic to Latin and the other way around.

Combining these two, we can create a function called serbian_stop() that converts the input to Latin or Cyrillic. Instead of exposing that, we'll make two more functions, serbian_stop_latin() and serbian_stop_cyrillic(), which will get the #[surrealism] annotation so that they can be called directly from the database.

pub enum Alphabet {
    Latin,
    Cyrillic,
}

#[surrealism]
pub fn serbian_stop_latin(document: String) -> String {
    serbian_stop(document, Alphabet::Latin)
}

#[surrealism]
pub fn serbian_stop_cyrillic(document: String) -> String {
    serbian_stop(document, Alphabet::Cyrillic)
}

pub fn serbian_stop(document: String, alphabet: Alphabet) -> String {
    // Make sure it's the Latin alphabet first
    let document = Convertion::from_cyrillic(&document);

    // Get the stopwords
    let words = stop_words::get("hr");

    // Remove punctuation and lowercase the text to make parsing easier
    let lowercase_doc = document.to_ascii_lowercase();
    let regex_for_punctuation = human_regex::one_or_more(human_regex::punctuation());
    let text_without_punctuation = regex_for_punctuation
        .to_regex()
        .replace_all(&lowercase_doc, "");

    // Make a regex to match stopwords with trailing spaces and punctuation
    let regex_for_stop_words = human_regex::word_boundary()
        + human_regex::exactly(1, human_regex::or(words))
        + human_regex::word_boundary()
        + human_regex::one_or_more(human_regex::whitespace());

    // Remove stop words
    let clean_text = regex_for_stop_words
        .to_regex()
        .replace_all(&text_without_punctuation, "");

    // Return as input user requested
    match alphabet {
        Alphabet::Cyrillic => Convertion::from_latin(&clean_text),
        Alphabet::Latin => Convertion::from_cyrillic(&clean_text),
    }
}

Putting all the functions together gives us this code.

use fake::{
    Fake,
    faker::{address::de_de::CityName, company::de_de::CompanyName, name::de_de::Name},
};
use serbian_cyrillic_latin_conversion::Convertion;
use surrealism::surrealism;

#[surrealism]
pub fn city() -> String {
    CityName().fake()
}

#[surrealism]
pub fn company_name() -> String {
    CompanyName().fake()
}

#[surrealism]
pub fn full_name() -> String {
    Name().fake()
}

pub enum Alphabet {
    Latin,
    Cyrillic,
}

#[surrealism]
pub fn serbian_stop_latin(document: String) -> String {
    serbian_stop(document, Alphabet::Latin)
}

#[surrealism]
pub fn serbian_stop_cyrillic(document: String) -> String {
    serbian_stop(document, Alphabet::Cyrillic)
}

pub fn serbian_stop(document: String, alphabet: Alphabet) -> String {
    // Make sure it's the Latin alphabet first
    let document = Convertion::from_cyrillic(&document);

    // Get the stopwords
    let words = stop_words::get("hr");

    // Remove punctuation and lowercase the text to make parsing easier
    let lowercase_doc = document.to_ascii_lowercase();
    let regex_for_punctuation = human_regex::one_or_more(human_regex::punctuation());
    let text_without_punctuation = regex_for_punctuation
        .to_regex()
        .replace_all(&lowercase_doc, "");

    // Make a regex to match stopwords with trailing spaces and punctuation
    let regex_for_stop_words = human_regex::word_boundary()
        + human_regex::exactly(1, human_regex::or(words))
        + human_regex::word_boundary()
        + human_regex::one_or_more(human_regex::whitespace());

    // Remove stop words
    let clean_text = regex_for_stop_words
        .to_regex()
        .replace_all(&text_without_punctuation, "");

    // Return as input user requested
    match alphabet {
        Alphabet::Cyrillic => Convertion::from_latin(&clean_text),
        Alphabet::Latin => Convertion::from_cyrillic(&clean_text),
    }
}

Compiling and accessing the Surrealism module

With the functions done, it's now time to compile the Surrealism module. That can be done with the surreal module build command. Follow this with --out and the file name, which must have the extension .surli.

surreal module build --out serbian_and_fake.surli

You should see the following output.

Building WASM module...
   Compiling random v0.1.0 (/Users/mithr/surrealism_test)
    Finished `release` profile [optimized] target(s) in 2.38s
Optimizing bundle...

Now it's time to start the database with surreal start. It needs two environment variables to make this happen: one to allow the files and surrealism experimental features to be used, and another that shows the location of the .surli file that we just compiled.

SURREAL_CAPS_ALLOW_EXPERIMENTAL=files,surrealism SURREAL_BUCKET_FOLDER_ALLOWLIST="/Users/my_name/my_rust_code/" surreal start --user root --pass secret

Now we'll connect to the database either via Surrealist or the following CLI command.

SURREAL_CAPS_ALLOW_EXPERIMENTAL=files,surrealism surreal sql --user root --pass secret

Once we are connected we just need two more statements: one to define a bucket for the files, and another to define a module, a module being a collection of Surrealism functions.

DEFINE BUCKET my_bucket BACKEND "file:/Users/my_name/my_rust_code";
DEFINE MODULE mod::my_funcs AS f"my_bucket:/serbian_and_fake.surli";

And that's all you need to do! Let's call the first three functions to see what output we get. It'll be different every time, but will look something like this.

mod::my_funcs::full_name();    -- 'Christa Simon'
mod::my_funcs::company_name(); -- 'Götz and Seidel und Partner'
mod::my_funcs::city();         -- 'Kleinfuchsfurt'

Now let's try the Serbian functions. We'll choose this passage from the Serbian Wikipedia:

Неолитско насеље у Винчи удаљено је око 14 km од ушћа Саве у Дунав, што је изузетно повољно место које је омогућило да постане фокална тачка простора југоисточне Европе.

That passage is about a pre-historic neolithic culture that existed in and well beyond present day Serbia. It means:

"The Neolithic settlement in Vinča is about 14 km from the confluence of the Sava and the Danube, which is an extremely favorable place that allowed it to become a focal point of the area of Southeastern Europe."

We'll take the Cyrillic form and see if we can turn it into Latin and vice versa.

mod::my_funcs::serbian_stop_latin(
    "Неолитско насеље у Винчи удаљено је око 14 km од ушћа Саве у Дунав, што је изузетно повољно место које је омогућило да постане фокална тачка простора југоисточне Европе."
);
mod::my_funcs::serbian_stop_cyrillic(
    "Neolitsko naselje u Vinči udaljeno je oko 14 km od ušća Save u Dunav, što je izuzetno povoljno mesto koje je omogućilo da postane fokalna tačka prostora jugoistočne Evrope."
);

Success! Even if you don't know the language, you can compare the input and output to see that the output does indeed lack a number of words. This is clear when you line them up against each other.

-- Original text in Latin
'Neolitsko naselje u Vinči udaljeno je oko 14 km od ušća Save u Dunav, što je izuzetno povoljno mesto koje je omogućilo da postane fokalna tačka prostora jugoistočne Evrope.'

-- Cyrillic turned into Latin with stop words applied
'neolitsko naselje   vinči udaljeno    oko 14 km    ušća save   dunav         izuzetno povoljno mesto         omogućilo    postane fokalna tačka prostora jugoistočne evrope'

-- Latin turned into Cyrillic with stop words applied
'неолитско насеље    винчи удаљено     око 14 км    ушћа саве   дунав         изузетно повољно  место         омогућило    постане фокална тачка простора југоисточне европе'

So there we have it. In the space of a 350-line blog post we've managed to turn SurrealDB from regular SurrealDB 3.0 into one with a few extra "apps" that let you create mock data on the fly, or work with a new language that wasn't available before.

And since a Surrealism plugin is a WASM binary written in Rust, you can compile it yourself or share it for others to use. The ability to share what you've built is what makes a plugin system like this so extensible.

Community example

To finish up this post, let's quickly check out an example of Surrealism in action. SurrealDB user (and ambassador!) Horváth Bálint is the author of an app called recet that he described as follows:

Recet is an open source, work in progress recipe manager application built using Nuxt and SurrealDB. It is motivated from personal needs and the curiosity to see how much functionality can be squeezed out from using SurrealDB as the backend. My goal is to create a web app that feels great on both desktop and mobile using modern technologies like PWA, View Transition API, all while showcasing many features of SurrealDB.

One of the features of this app is a Surrealism function! Let's take a look at it. As you can see, it uses a crate called html2text. Here's part of it.

#[surrealism]
fn scrape_for_recipe(source_type: String, source_text: String) -> Result<RecipeInfo> {
    let text = if source_type == "text" {
        source_text
    } else if source_type == "url" {
        let html: String = sql_with_vars(
            "http::get($url)",
            vars! {
                url: source_text
            },
        )?;
        html2text::from_read(html.as_bytes(), usize::MAX)?
    } else {
        return Err(anyhow::anyhow!("Invalid source type"));
    };
    // ... snip
}

Then we have the same DEFINE BUCKET and DEFINE MODULE statements as above:

// PLUGINS
DEFINE BUCKET OVERWRITE plugins BACKEND $pluginsBucketPath PERMISSIONS NONE;
DEFINE MODULE OVERWRITE mod::recet AS f"plugins:/recet.surli";

And then a query via the JavaScript SDK that calls the function.

async function importRecipe() {
  try {
    importing.value = true

    const [recipe] = await db
      .query(surql`mod::recet::scrape_for_recipe(${importSourceType.value}, ${importSource.value})`)
      .collect<[ImportedRecipe | null]>()

    if (!recipe)
      throw new Error('No recipe found at the provided URL.')
    // ... snip
  }
}

As you can see, it's the same process described in the first examples above. Annotate the function, compile the module, use DEFINE twice to connect to it, and the function(s) are yours to call.

The final app, by the way, looks like this. Personally I think the kókusztejes gombás fehérbab looks particularly good, or the póréhagymás mascarponés gnocchi with the leeks replaced with grated cheese.

Back to you

There's no telling what the SurrealDB community will do with Surrealism now that 3.0 is here and the word is getting out.

We can't wait to see what you build!

Get started with Surrealism

Go to the Surrealism Docs to get started. We are excited to see what extensions you build, be sure to share them in our Discord channel.

Thank you for being on this journey with us!

Introducing JavaScript SDK 2.0

Mark Gyles — Thu, 26 Feb 2026 12:26:43 +0000

Author: Julian Mills

We're excited to announce the SurrealDB JavaScript SDK v2, the most significant update to the SDK to date. We've rebuilt the core internals with a focus on ergonomics, flexibility, and developer experience. Key highlights include full support for SurrealDB 3.0, multi-session support, automatic token refreshing, client-side transactions, a redesigned live query API, and a new query builder pattern that makes working with your data more intuitive than ever.

Welcome WASM and Node!

The existing WebAssembly and Node.js SDKs have been rewritten, updated to support the 2.0 JavaScript SDK, and moved into the JavaScript SDK repository.

Going forward, the JS, WASM, and Node.js SDKs will be published together, so embedded SurrealDB stays up to date. The WASM and Node.js SDK versions align their major and minor versions with SurrealDB, so you can tell at a glance which SurrealDB version you're running.

As a bonus, the WASM SDK now supports running inside a Web Worker, so you can offload database work from the main thread and keep your UI responsive.

Wasm

import { Surreal, createRemoteEngines } from "surrealdb";
import { createWasmEngines } from "@surrealdb/wasm";
import WorkerAgent from "@surrealdb/wasm/worker?worker";

const db = new Surreal({
    engines: {
        ...createRemoteEngines(),
        ...createWasmEngines(),
        // or for Web Worker based engines
        ...createWasmWorkerEngines({
            createWorker: () => new WorkerAgent()
        })
    },
});

Node.js (and Bun & Deno)

import { Surreal, createRemoteEngines } from "surrealdb";
import { createNodeEngines } from "@surrealdb/node";

const db = new Surreal({
    engines: {
        ...createRemoteEngines(),
        ...createNodeEngines(),
    },
});

Official event listeners

The original SDK exposed events through the internal surreal.emitter field. The new SDK provides a type-safe surreal.subscribe() function instead. Calling .subscribe() returns a cleanup function that unsubscribes the listener when invoked.

// Subscribe to events
const unsub = surreal.subscribe("connected", () => {
    ...
});

// Unsubscribe
unsub();

Access internal state

New getters let you read internal state from the Surreal instance:

surreal.namespace and surreal.database for the selected namespace and database
surreal.params for connection parameters
surreal.accessToken and surreal.refreshToken for authentication tokens

await surreal.use({ namespace: surreal.namespace, database: "other-db" });

Automatic token refreshing

The SDK now restores or renews authentication when your access token expires or the connection reconnects. When refresh tokens are available, they are exchanged for a new token pair. Otherwise the SDK reuses the credentials you provided or fires an auth event for custom handling. Read more on GitHub.

For asynchronous authentication, you can pass a callable to the authentication property:

const surreal = new Surreal();

await surreal.connect("http://example.com", {
    namespace: "test",
    database: "test",
    renewAccess: true, // default true
    authentication: () => ({
        username: "foo",
        password: "bar",
    })
});

Multi-session support

You can create multiple isolated sessions on a single connection, each with its own namespace, database, variables, and authentication state. New sessions can be created at any time, or you can fork an existing session and reuse its state.

Simple example

// Create a new session
const session = await surreal.newSession();

// Use the session
session.signin(...);

// Dispose the session
await session.closeSession();

Forking sessions

const freshSession = await surreal.newSession();

// Clone a session including namespace, database, variables, and auth state
const forkedSession = await freshSession.forkSession();

Await using

await using session = await surreal.newSession();

// JavaScript will automatically close the session at the end of the current scope

Redesigned live query API

Live query methods on the Surreal class have been redesigned to be more intuitive. Live select queries can also restart automatically when the driver reconnects.

The record ID is now passed as the third argument to your handlers, so you can identify which record changed when handling patch updates.

// Construct a new live subscription
const live = await surreal.live(new Table("users"));

// Listen to changes
live.subscribe((action, result, record) => {
     ...
});

// Alternatively, iterate messages
for await (const { action, value } of live) {
    ...
}

// Kill the query and stop listening
live.kill();

// Create an unmanaged query from an existing id
const [id] = await surreal.query("LIVE SELECT * FROM users");
const live = await surreal.liveOf(id);

Improved parameter explicitness

Query functions no longer accept plain strings as table names. You must use the Table class, which avoids record IDs being mistaken for table names.

// tables.ts
const usersTable = new Table("users");
const productsTable = new Table("products");
...

// main.ts
await surreal.select(usersTable);

Query builder pattern

A new builder pattern lets you configure RPC calls with optional chaining. All query methods support chainable helpers for filtering, limiting, and fetching.

As a result, update and upsert no longer take content as a second argument. You choose how to change records by calling .content(), .merge(), .replace(), or .patch().

// Select
const record = await db.select(id)
    .fields("age", "firstname", "lastname")
    .fetch("foo");

// Update
await db.update(record).merge({
    hello: "world"
});

Query method overhaul

The .query() function has been overhauled to support more use cases, including picking response indexes, automatic JSON output, and streaming responses.

// Execute a query and return results
const [user] = await db.query<[User]>("SELECT * FROM user:foo");

// Collect specific results
const [foo, bar] = await db.query("LET $foo = ...; LET $bar = ...; SELECT * FROM $foo; SELECT * FROM $bar")
    .collect<[User, Product]>(2, 3);

// Jsonify responses
const [products] = await db.query<[Product[]]>("SELECT * FROM product").json();

// Response objects
const responses = await db.query<[Product[]]>("SELECT * FROM product").responses();

// Stream responses
const stream = surreal.query(`SELECT * FROM foo`).stream();

for await (const frame of stream) {
    if (frame.isValue<Foo>()) {
        //  Process a single value with frame.value typed Foo
    } else if (frame.isDone()) {
        // Handle completion and access stats with frame.stats
    } else if (frame.isError()) {
        // Handle error frame.error
    }
}

SurrealDB does not yet support streaming individual records, but this API is ready for when it does. It works with current SurrealDB versions and is now the only way to obtain query stats.

Expressions API

A new Expressions API works with the .where() function and makes it easier to build dynamic expressions. It integrates with the surql template tag so you can insert parameter-safe expressions anywhere.

const checkActive = true;

// Query method
await db.select(userTable).where(eq("active", checkActive));

// Custom query
await db.query(surql`SELECT * FROM user WHERE ${eq("active", checkActive)}`);

// Expressions even allow raw insertion
await db.query(surql`SELECT * FROM user ${raw("WHERE active = true")}`);

You can also serialize expressions to a string with the expr() function:

const result: BoundQuery = expr(
    or(
        eq("foo", "bar"),
        false && eq("hello", "world"),
        eq("alpha", "beta"),
        and(
            inside("hello", ["hello"]),
            between("number", 1, 10)
        )
    )
);

Value encode/decode visitor API

For advanced use cases where you need to process SurrealDB value types, you can pass encode or decode visitor callbacks in the Surreal constructor. These run for each value sent to or received from the engine, so you can modify or wrap values before they appear in responses.

const surreal = new Surreal({
    codecOptions: {
        valueDecodeVisitor(value) {
            if (value instanceof RecordId) {
                return new RecordId("foo", "bar");
            }

            return value;
        },
    },
});

...

const [result] = await surreal.query(`RETURN hello:world`).collect<[RecordId]>();

console.log(result); // foo:bar

Diagnostics API

The Diagnostics API lets you wrap engines and inspect protocol-level communication. It is useful for debugging queries, analysing SDK behaviour, measuring timings, and similar tasks.

Because it is implemented as a wrapper engine, there is no extra cost unless you use it. We do not recommend using it in production, as it can affect performance and the events may change between versions.

new Surreal({
    driverOptions: {
        engines: applyDiagnostics(createRemoteEngines(), (event) => {
            console.log(event);
        }),
    },
});

Each event has a type, key, and phase. The type identifies the operation, key is a stable id across phases, and phase indicates start, progress, or completion. The query diagnostic exposes the internal chunk stream; you are responsible for stitching queries and batches together. The after phase includes duration, success, and result.

Get started with the new SDK

You can visit the JavaScript SDK documentation for more information on the new SDK, which has been fully updated to reflect all new changes. It contains many new concept guides explaining how to use the new SDK, and now includes a comprehensive API reference for detailed information.

Follow the Quickstart guide to get started.

File support in SurrealDB 3.0

Mark Gyles — Wed, 25 Feb 2026 18:19:04 +0000

File Support

Until now, SurrealDB has always encouraged users to store files externally, referencing them with paths stored in the database. We’re now introducing the first step toward native file support - bringing file storage directly into your database workflow.

File storage is still an experimental feature at this moment, and we would love to hear from you as you give it a try for your own solutions.

Setting up file storage

To begin, we can start a new SurrealDB server with the experimental files capability enabled.

surreal start --user root --pass secret --allow-experimental files

Setting up file storage can be done with a single DEFINE BUCKET statement! A bucket can be a physical location, or even in-memory.

For global usage across all namespaces and databases, you can set a SURREAL_GLOBAL_BUCKET environment variable, with an optional SURREAL_GLOBAL_BUCKET_ENFORCED if you want to ensure that every database uses the same global bucket.

Let’s define a simple in-memory bucket. Just provide a name, and set the backend to "memory":

DEFINE BUCKET my_bucket BACKEND "memory";

The name of the bucket is also your path to the bucket and everything inside.

Storing and accessing files

Once your bucket is defined, you can reference it using a file pointer - a string prefixed with the letter f to indicate a file path.

File pointers have their own functions, like .put() to add bytes, .get() to see them, .rename() to rename a file, and .head() to see its metadata. Let’s use the .get() function to see if our bucket has a file called tour_of_surrealdb.txt.

f"my_bucket:/tour_of_surrealdb.txt".get();

The function returned NONE, so looks like there’s nothing there.

Let’s use the .put() method to add some bytes, creating the file. .put() can take bytes or a string, which it will automatically convert into bytes.

f"my_bucket:/tour_of_surrealdb.txt".put("Welcome to the Tour of SurrealDB!");

Here, we’re adding the string “Welcome to the Tour of SurrealDB!”

Now that the file exists, the .get() function will return its as bytes:

f"my_bucket:/tour_of_surrealdb.txt".get();

-- Output:
b"57656C636F6D6520746F2074686520546F7572206F66205375727265616C444221"

Working with file data

If you prefer to view the data differently, SurrealDB also now supports improved type conversions for byte-related types.

For example, you can cast the result to an array to see each byte as a number:

<array>f"my_bucket:/tour_of_surrealdb.txt".get();

[87, 101, 108, 99, 111, 109, 101, 32, 116, 111, 32, 116, 104, 101, 32, 84, 111, 117, 114, 32, 111, 102, 32, 83, 117, 114, 114, 101,
97, 108, 68, 66, 33]

Or cast to a string:

<string>f"my_bucket:/tour_of_surrealdb.txt".get();

In cases where the bytes include invalid UTF-8, you can use type::string_lossy() - a new function that replaces invalid bytes with the � Unicode replacement character, rather than throwing an error.

This function is especially useful when working with partially corrupted or inconsistently encoded data. Instead of failing outright, it preserves and returns all valid parts of the string.

Here’s an example using type::string, which returns an error because not all the bytes are valid:

type::string(<bytes>[83, 117, 114, 255, 114, 101, 97, 254, 108, 68, 66]);
-- 'Could not cast into `string` using input `b"537572FF726561FE6C4442"`'

And here’s the same input passed through type::string_lossy(). We can see that there are some valid bytes in there - enough to make out the word SurrealDB.

type::string_lossy(<bytes>[83, 117, 114, 255, 114, 101, 97, 254, 108, 68, 66]);
-- 'Sur�rea�lDB'

SurrealDB also has encoding functions that allow you to encode and decode bytes into SurrealDB types. That means that you can take a type like this object and turn it into bytes...

encoding::cbor::encode({ some: "data" });

-- Output:
b"A164736F6D656464617461"

And then turn it back from bytes into an object.

encoding::cbor::decode(b"A164736F6D656464617461");

-- Output:
{
    some: 'data'
}

A takeaway example to get started

Let's take a quick running example to give you some ideas of how you could use files in your own app, in this case an app that stores temporary shopping cart files to disk so that other parts of the app can access them.

This time we'll define a bucket to a file path instead of memory, which requires SURREAL_BUCKET_FOLDER_ALLOWLIST to be set so that the database knows that this folder has been approved as the location for file storage. Note that the SurrealDB instance itself is using memory as the backend, but it's able to save files to this directory, getting the best of both worlds.

SURREAL_BUCKET_FOLDER_ALLOWLIST="/users/your_user_name" surreal start --allow-experimental files --user root --pass secret

After connecting, defining the bucket can be done with a path that begins with file: and holds the path we have allowed.

DEFINE BUCKET shopping_carts BACKEND "file:/users/your_user_name";

Since working with files takes a good amount of code, we'll put some functions together to simplify the process. These three will allow us to save, get, and delete files in the shopping_carts backend.

-- Convenience functions to save, decode back into
-- SurrealQL type, and delete
DEFINE FUNCTION fn::save_file($file_name: string, $input: any) {
    LET $file = type::file("shopping_carts", $file_name);
    $file.put(encoding::cbor::encode($input));
};

DEFINE FUNCTION fn::get_file($file_name: string) -> object {
    encoding::cbor::decode(type::file("shopping_carts", $file_name).get())
};

DEFINE FUNCTION fn::delete_file($file_name: string) {
    type::file("shopping_carts", $file_name).delete();
};

With those functions set up, we can play around with a file for a user with the user number 24567. Once you execute the fn::save_file() function below you will be able to go into the directory yourself to view it. And you can also view and interact with it using SurrealQL via the other functions in the example below.

-- Save current shopping cart
fn::save_file("temp_cart_user_24567", {
    items: ["shirt1"],
    last_updated: time::now()
});

fn::get_file("temp_cart_user_24567");
-- Returns { items: ['shirt1', 'deck_of_cards'], last_updated: d'2026-02-25T01:03:24.141080Z' }

-- User adds item, save over file with newer information
fn::save_file("temp_cart_user_24567", {
    items: ["shirt1", "deck_of_cards"],
    last_updated: time::now()
});

fn::get_file("temp_cart_user_24567");
-- Returns { items: ['shirt1', 'deck_of_cards'], last_updated: d'2026-02-25T01:06:02.752429Z' }

-- Session is over, delete temp file
fn::delete_file("temp_cart_user_24567");

We’re excited to introduce this first stage of native file support in SurrealDB. As the next step in our journey toward multimodality, we’re working on enabling integrations with object storage solutions such as Amazon S3, allowing you to work with files at scale. If you’re building apps that rely on lightweight file storage or embedded assets, give it a try - and let us know what features you’d like to see next. This is just the beginning.

Learn more at: https://surrealdb.com/docs/surrealql/datamodel/files.

Custom API Endpoints: streamlining your architecture

Mark Gyles — Wed, 25 Feb 2026 17:53:23 +0000

Custom API Endpoints: streamlining your architecture

With the release of SurrealDB 3.0, we’re excited to announce the stabilisation of a powerful feature: DEFINE API. This innovative addition allows developers to define database behaviours, set up middleware, and create custom API endpoints directly within the familiar SurrealQL query language.

Traditionally, applications connect to databases through middleware, resulting in a three-layer architecture:

Client → Middleware (API) → Database

With SurrealDB's new DEFINE API, you can simplify your infrastructure significantly:

Client → Database

This streamlined approach not only reduces complexity but also enhances performance and ease of management.

Real-world use case: implementing rate limits for a social app

Consider a scenario: you've built a popular social application that allows users to see recent comments. You want to provide a free API for anonymous users, but you also need to ensure these guest users don't overload your system resources during peak traffic.

Here are some ways how you can achieve this efficiently using SurrealDB's DEFINE API:

Setup: restrict arbitrary queries

To begin, pass in the new --deny-arbitrary-query flag when starting the SurrealDB server.

surreal start --user root --pass root --deny-arbitrary-query guest --allow-experimental define_api

This prevents certain groups, like guests or record users, from putting their own queries together. We'll add guest here so that any guest users will have to use the API endpoint instead.

You can then connect using Surrealist, or with the following command in a new terminal window:

surreal sql --user root --pass root

Defining your API Endpoint

Next, we'll add a DEFINE API statement to define the endpoint. The statement will start with the path /get_latest , and to specify that it can only be used for GET requests, we will include it after the FOR keyword.

DEFINE API OVERWRITE "/get_latest" FOR get

Add middleware functions

We can follow this up with some middleware by passing in some of SurrealDB's API functions. We'll use api::timeout to give guest users up to 50 milliseconds of server time.

MIDDLEWARE
    api::timeout(50ms)

Define the RETURN data

And then we'll finish up the statement by adding what it actually returns: a SELECT statement for all comment records that have been created over the past ten minutes:

THEN {
      {
        body: SELECT * FROM comment:[time::now()-10m].. ORDER BY id DESC
      }
    };

In summary, that gives us a DEFINE API statement that looks like this.

DEFINE API OVERWRITE "/get_latest" FOR get
    MIDDLEWARE
        api::timeout(50ms),
    THEN {
        {
            body: SELECT * FROM comment:[time::now()-10m].. ORDER BY id DESC
        }
};

Testing your endpoint

You can test it from SurrealQL itself by first adding a comment or two and then calling the api::invoke function:

CREATE comment:[time::now()] SET user_says = "Nice blog!";
CREATE comment:[time::now()] SET user_says = "Can't wait for the new version";

api::invoke("/get_latest");

Or externally at: /api/:namespace/:database/:endpoint_name. Here's what our defined endpoint will look like with a namespace called test_ns and a database called test_db.

http://localhost:8000/api/test_ns/test_db/get_latest

That means that a curl command like the following will be enough to get the latest comments.

curl -H "Accept: application/json" \
  http://localhost:8000/api/test_ns/test_db/get_latest

The output should look something like this.

[
  {"id":"comment:[d'2026-02-12T01:10:02.445607Z']","user_says":"Can't wait for the new version"},
  {"id":"comment:[d'2026-02-12T01:08:33.709073Z']","user_says":"Nice blog!"}
]

But that’s not all. The MIDDLEWARE clause in a DEFINE API statement can take custom middleware as well that you define yourself using a regular DEFINE FUNCTION statement. These functions are automatically populated with the user request and a way to get the current state of the response as you put it together.

There are a lot of examples in the documentation to show how it works, but here’s a simple example to get you started.

DEFINE FUNCTION fn::add_prefix($req: object, $next: function, $prefix: string) -> object {
    LET $res = $next($req);
    LET $res = $res + { body: $res.body + { prefix: $prefix + ": " + $res.body.message } };
    $res;
};

DEFINE API "/custom_with_args"
    FOR get
        MIDDLEWARE
            fn::add_prefix("PREFIX")
        THEN {
            {
                status: 200,
                body: {
                    message: "original message"
                }
            };
        };

With DEFINE API, SurrealDB brings the power of API design directly into the database - no external layers, no additional frameworks. Whether you’re enforcing limits, defining access, or building powerful custom endpoints, this feature helps you simplify your architecture and move faster.

Explore further

Dive deeper into SurrealDB's powerful API management capabilities:

Harness the power of DEFINE API and redefine what's possible with SurrealDB 3.0.

SurrealDB 3.0 benchmarks: a new foundation for performance

Mark Gyles — Thu, 19 Feb 2026 21:11:12 +0000

SurrealDB 3.0 benchmarks: a new foundation for performance

When we began our benchmarking journey with SurrealDB 2.0, we set out to answer a simple but important question: how does SurrealDB perform in the real world?

Since then, SurrealDB has seen rapidly growing traction - including enterprises like Tencent and Later.com running in production at significant scale, across high-throughput, business-critical workloads. As adoption has increased, so has the importance of demonstrating not just raw performance, but predictable, production-ready performance.

Last year we shared those early results openly, along with our methodology and learnings, in our first benchmarking report.

With SurrealDB 3.0, we’re taking the next major step in that journey. Today, we’re excited to announce our new benchmarks for SurrealDB 3.0 - and more importantly, the architectural changes that make these results possible.

Benchmarking a multi-model database

As mentioned in our previous benchmarking blog, benchmarking a multi-model database is more complex than comparing single-purpose systems. SurrealDB unifies relational, document, graph, time-series, key-value, vector, geospatial, and full-text workloads in one engine, spanning everything from embedded deployments to distributed clusters.

This versatility makes fair benchmarking challenging, since different databases vary widely in durability guarantees, disk flushing behaviour, and configuration trade-offs. For our benchmarks, we’ve done our best to keep comparisons as balanced and transparent as possible - and we always welcome feedback on how to improve the methodology further.

Because SurrealDB can replace entire stacks of databases and supporting tools, the best way to benchmark SurrealDB is often through a like-for-like proof of concept against multiple existing systems (we have customers like Tencent who reduced nine separate tools down to one with SurrealDB).

Our open-source benchmarking tool

We built our own internal benchmarking tool - crud-bench - in Rust to properly evaluate SurrealDB across the wide range of workloads it supports. Unlike many generic benchmarking suites, crud-bench is designed to test not just basic CRUD operations, but also the broader feature set that makes SurrealDB unique, across embedded, networked, and remote deployments. It allows us to compare SQL, NoSQL, key-value, and embedded systems in a consistent way, while continuously measuring the performance impact of changes inside SurrealDB itself. For more details on how the tool work, check the crud-bench GitHub repo and the explanation in the previous 2.0 benchmark blog.

SurrealDB 3.0: a new execution engine

The foundation

SurrealDB 3.0 has been largely focused on stability and getting the right foundations in place. We rearchitected the internals in ways that would have been nearly impossible under the previous design, and the performance results we're seeing now are a direct consequence of that work.

The biggest change is the query execution engine. We took inspiration from the broader database community and moved SurrealQL's execution model to a more standard pipeline: AST → LogicalPlan → ExecutionPlan, with basic query optimisations applied throughout. The new engine is fully streaming internally, and in a future minor release we will make it stream end-to-end.

Right now, the new execution engine covers read-only statements. In the coming minor releases we'll expand it to handle all of our workloads.

TL;DR
Now let’s get to the benchmarks themselves. The tl;dr for the numbers below are as follows:

Graph queries are between 8-22x faster.
Large table scans with LIMIT, START, and START + LIMIT are 3-6x faster.
The query planner is smarter. Queries like SELECT * FROM table WHERE id = record:42 that previously triggered full table scans now resolve in sub-millisecond time, a 4000x+ improvement. This is especially important for a query language like SurrealQL which is very close to standard SQL in which WHERE id = is one of the most common patterns.
ORDER BY queries are 3-4x faster across all storage engines.
HNSW vector search is up to 8x faster. Indexed vector similarity queries dropped from ~35 seconds to ~4.5 seconds.
The new execution model enables significantly improved concurrency and throughput.
Complex GROUP BY queries with multiple aggregations are up to 55% faster, with the biggest gains on multi-aggregation and deduplication workloads.

Benchmark hardware

The benchmarks below were run on the following system:

Part	Name	Link
CPU	AMD Ryzen Threadripper 9970X	amazon.co.uk
RAM	Samsung DDR5 4800MHz 64GB RDIMM	amazon.co.uk
SSD	Lexar EQ790 4TB NVMe	amazon.co.uk
Motherboard	ASUS PRO WS TRX50-SAGE WIFI A	ballicom.co.uk
CPU Cooler	Arctic Freezer 4U-M	scan.co.uk
Power Supply	CORSAIR RM1000x	amazon.co.uk
GPU	PowerColor AMD Radeon RX 7600 Fighter 8GB	amazon.co.uk
Case	In-Win IW-R400-01N 4U	servercase.co.uk

Benchmarks

In-Memory

Test	v3 vs. v2		SurrealDB 2		SurrealDB 3
	Mean %	OP/s %	Mean (ms)	OP/s	Mean (ms)	OP/s
[C]reate	-62.9%	+168.7%	1.94	147538.6	0.72	396504.5
[R]ead	+56.8%	-35.6%	0.37	757976.6	0.58	488252.1
[U]pdate	-71.7%	+249.6%	2.58	111466.6	0.73	389693.8
[S]can::count_all (2000)	-44.4%	+64.3%	204.97	1277.8	113.99	2100.1
[S]can::limit_id (2000)	-67.5%	+166.8%	5.32	47510.8	1.73	126780.2
[S]can::limit_all (2000)	-85.9%	+550.9%	17.45	15053.5	2.46	97982.9
[S]can::limit_start_id (2000)	+45.1%	-28.2%	7.45	33909.7	10.81	24360.7
[S]can::limit_start_all (2000)	-38.5%	+64.6%	18.11	14419.5	11.14	23733.8
[S]can::select_where_id (2000)	+34.7%	-20.4%	0.49	401422.2	0.66	319614.6
[S]can::select_where_id_eq (2000)	-100.0%	+436179.7%	3935.65	70.9	0.68	309409.5
[S]can::select_where_gt (2000)	+18.9%	-17.2%	5242.39	52.9	6235.40	43.8
[S]can::select_where_in (2000)	-36.2%	+55.8%	9745.80	28.4	6221.91	44.3
[S]can::select_where_multi_and (2000)	+2.8%	-3.2%	6065.92	45.7	6238.46	44.2
[S]can::select_where_order_limit (2000)	+10.9%	-8.7%	5196.36	53.3	5762.22	48.7
[S]can::select_where_order_desc_limit (2000)	+12.5%	-11.8%	5191.97	53.5	5842.42	47.1
[S]can::select_where_multi_order_limit (2000)	-7.0%	+9.2%	6271.61	44.3	5830.96	48.4
[S]can::select_omit_limit (2000)	-84.4%	+465.6%	18.09	14490.0	2.83	81951.6
[S]can::select_fields_where_limit (2000)	+589.8%	-85.7%	16.15	16178.0	111.41	2305.7
[S]can::select_order_by (2000)	-69.5%	+231.8%	17270.90	16.1	5274.94	53.5
[S]can::select_order_by_multi (2000)	-70.1%	+235.3%	17387.92	16.0	5192.82	53.7
[S]can::select_group_count (2000)	-9.2%	+8.7%	5989.63	46.4	5436.81	50.4
[S]can::select_group_sum (2000)	-16.6%	+19.8%	6754.21	41.1	5635.44	49.2
[S]can::select_group_avg (2000)	-18.3%	+20.9%	6766.85	40.9	5529.88	49.4
[S]can::select_group_multi_agg (2000)	-36.3%	+53.4%	9738.26	28.5	6203.12	43.8
[S]can::select_group_all (2000)	-25.8%	+33.7%	7492.08	37.1	5561.68	49.5
[S]can::select_group_order_limit (2000)	-23.7%	+31.6%	7603.12	36.5	5798.60	48.1
[S]can::select_group_where (2000)	-12.8%	+13.8%	7411.37	37.5	6459.82	42.7
[S]can::select_group_dedup_agg (2000)	-35.5%	+54.0%	9735.38	28.5	6278.25	43.9
[S]can::select_split (2000)	-55.1%	+111.3%	19.71	13415.2	8.85	28351.9
[S]can::select_fetch (2000)	-80.6%	+401.8%	39.07	6802.6	7.58	34134.4
[S]can::select_fetch_where_limit (2000)	+291.6%	-74.3%	32.16	8366.1	125.95	2146.3
[S]can::graph_out_depth1 (2000)	-60.0%	+124.2%	2.30	105137.8	0.92	235676.0
[S]can::graph_out_depth2 (2000)	-72.9%	+222.0%	6.16	42125.8	1.67	135637.0
[S]can::graph_out_depth3 (2000)	-83.2%	+449.0%	18.02	15001.7	3.02	82363.1
[S]can::graph_in_depth1 (2000)	-58.8%	+129.5%	2.72	90892.6	1.12	208621.0
[S]can::graph_bidirectional (2000)	-67.1%	+193.4%	4.22	57673.0	1.39	169188.7
[S]can::graph_edge_filter (2000)	-5.9%	-4.2%	1.19	197478.8	1.12	189149.0
[S]can::graph_multi_out (2000)	-84.3%	+504.6%	148.68	1886.3	23.32	11405.3
[S]can::graph_multi_out_where (2000)	-72.0%	+243.5%	51.38	5401.8	14.40	18555.1
[S]can::graph_multi_count (2000)	-72.4%	+247.7%	33.74	8098.3	9.32	28157.6
[S]can::graph_depth2_limit (2000)	-88.1%	+696.2%	78.83	3529.7	9.41	28104.7
[S]can::graph_sub_where (2000)	-24.7%	+32.1%	2.67	91237.9	2.01	120503.8
[S]can::graph_sub_group_all (2000)	-42.6%	+69.2%	3.36	74807.0	1.93	126560.9
[S]can::graph_sub_group_by (2000)	-42.2%	+61.1%	3.25	78504.3	1.88	126496.6
[S]can::subquery_inline (2000)	-49.1%	+95.4%	724.85	377.8	368.70	738.3
[S]can::subquery_count (2000)	+116.6%	-52.4%	28.49	9262.4	61.72	4406.4
[S]can::subquery_from (2000)	-38.3%	+60.9%	2918.72	94.6	1801.28	152.2
[S]can::pipeline_filter_group_order (2000)	-29.2%	+41.6%	9341.45	29.7	6613.75	42.0
[S]can::index_standard (2000)	+34.0%	-26.9%	3910.21	71.0	5241.54	51.9
[I]ndex::index_standard	-49.7%	+98.0%	1003.26	1.0	504.19	2.0
[S]can::index_standard::indexed (2000)	-42.6%	+60.8%	4.11	61150.4	2.36	98303.1
[R]emoveIndex::index_standard	-78.8%	+370.7%	380.54	2.6	80.67	12.4
[S]can::index_composite (2000)	+1.1%	-4.2%	6101.19	45.7	6171.27	43.8
[I]ndex::index_composite	+0.9%	-1.0%	502.14	2.0	506.50	2.0
[S]can::index_composite::indexed (2000)	-50.4%	+107.2%	1504.21	176.7	745.47	366.3
[R]emoveIndex::index_composite	-73.6%	+278.3%	338.82	3.0	89.50	11.2
[S]can::index_range_merged (2000)	-25.7%	+33.5%	7595.90	36.5	5641.66	48.7
[I]ndex::index_range_merged	-50.0%	+99.0%	1004.29	1.0	502.14	2.0
[S]can::index_range_merged::indexed (2000)	-52.4%	+114.4%	478.07	555.2	227.80	1190.4
[R]emoveIndex::index_range_merged	-74.4%	+289.6%	360.32	2.8	92.19	10.8
[S]can::index_in (2000)	-25.3%	+34.1%	7232.84	38.4	5404.26	51.5
[I]ndex::index_in	-50.0%	+99.0%	1004.29	1.0	502.40	2.0
[S]can::index_in::indexed (2000)	-61.6%	+160.3%	25.35	10437.8	9.73	27169.0
[R]emoveIndex::index_in	-77.9%	+352.1%	373.89	2.7	82.78	12.1
[I]ndex::index_fulltext	-93.1%	+1300.0%	51134.46	0.0	3511.30	0.3
[S]can::index_fulltext::indexed (2000)	+237.6%	-69.3%	225.31	1216.5	760.70	373.8
[R]emoveIndex::index_fulltext	+98.8%	-49.2%	845.06	1.2	1679.87	0.6
[I]ndex::index_hnsw	-32.7%	+60.0%	18604.03	0.1	12529.66	0.1
[S]can::index_hnsw::indexed (2000)	-87.4%	+671.0%	38581.61	7.3	4847.09	56.3
[R]emoveIndex::index_hnsw	-82.2%	+456.5%	2190.34	0.5	390.78	2.6

SurrealKV

Test	v3 vs. v2		SurrealDB 2		SurrealDB 3
	Mean %	OP/s %	Mean (ms)	OP/s	Mean (ms)	OP/s
[C]reate	-96.3%	+2579.9%	221.27	1299.7	8.25	34830.7
[R]ead	+76.5%	-42.4%	0.34	828781.5	0.60	477557.1
[U]pdate	-96.2%	+2550.3%	222.01	1295.3	8.37	34330.5
[S]can::count_all (2000)	-53.0%	+89.6%	303.53	917.2	142.73	1738.6
[S]can::limit_id (2000)	-68.2%	+170.0%	6.51	40313.3	2.07	108861.9
[S]can::limit_all (2000)	-84.5%	+460.1%	16.51	16063.4	2.56	89964.2
[S]can::limit_start_id (2000)	-59.2%	+132.9%	14.27	18977.0	5.82	44194.4
[S]can::limit_start_all (2000)	-71.6%	+241.5%	21.48	12560.3	6.10	42894.7
[S]can::select_where_id (2000)	+39.2%	-25.4%	0.51	400808.0	0.71	299184.1
[S]can::select_where_id_eq (2000)	-100.0%	+393137.5%	3478.94	81.6	0.69	320960.5
[S]can::select_where_gt (2000)	+30.1%	-25.9%	4666.15	60.8	6071.58	45.0
[S]can::select_where_in (2000)	-34.5%	+48.6%	9275.42	30.5	6071.93	45.3
[S]can::select_where_multi_and (2000)	+9.7%	-11.1%	5509.79	51.2	6044.50	45.5
[S]can::select_where_order_limit (2000)	+18.4%	-17.6%	4650.07	61.0	5504.01	50.3
[S]can::select_where_order_desc_limit (2000)	+25.0%	-21.4%	4623.38	61.4	5777.77	48.3
[S]can::select_where_multi_order_limit (2000)	-1.8%	+0.5%	5692.97	49.8	5590.58	50.1
[S]can::select_omit_limit (2000)	-76.9%	+316.0%	17.20	15569.4	3.97	64774.7
[S]can::select_fields_where_limit (2000)	+545.0%	-85.3%	16.06	16653.9	103.59	2450.0
[S]can::select_order_by (2000)	-68.1%	+210.7%	15714.24	17.9	5016.71	55.5
[S]can::select_order_by_multi (2000)	-68.3%	+214.1%	15943.91	17.6	5055.99	55.3
[S]can::select_group_count (2000)	+19.3%	-18.1%	4391.26	64.7	5236.94	53.0
[S]can::select_group_sum (2000)	+9.2%	-10.9%	5024.47	56.5	5485.56	50.4
[S]can::select_group_avg (2000)	+8.2%	-10.6%	5023.77	56.5	5437.21	50.5
[S]can::select_group_multi_agg (2000)	-23.4%	+26.5%	7942.62	35.6	6081.36	45.0
[S]can::select_group_all (2000)	-5.9%	+3.6%	5757.57	49.2	5418.85	51.0
[S]can::select_group_order_limit (2000)	-5.1%	+2.0%	5832.72	48.6	5534.30	49.6
[S]can::select_group_where (2000)	+5.1%	-7.2%	6046.34	46.9	6354.95	43.5
[S]can::select_group_dedup_agg (2000)	-23.9%	+26.7%	7959.49	35.5	6055.39	45.0
[S]can::select_split (2000)	-51.9%	+92.3%	18.00	14812.0	8.66	28485.5
[S]can::select_fetch (2000)	-75.1%	+291.8%	35.99	7481.3	8.96	29309.5
[S]can::select_fetch_where_limit (2000)	+271.6%	-72.7%	30.70	8690.9	114.08	2375.0
[S]can::graph_out_depth1 (2000)	-69.0%	+181.5%	5.00	52630.3	1.55	148160.1
[S]can::graph_out_depth2 (2000)	-77.6%	+315.8%	14.70	18594.4	3.29	77306.4
[S]can::graph_out_depth3 (2000)	-80.3%	+382.1%	42.04	6604.0	8.30	31839.6
[S]can::graph_in_depth1 (2000)	-71.1%	+202.3%	4.95	53273.6	1.43	161031.4
[S]can::graph_bidirectional (2000)	-77.2%	+297.3%	11.14	24621.8	2.54	97824.2
[S]can::graph_edge_filter (2000)	-49.0%	+87.9%	2.41	100491.2	1.23	188866.9
[S]can::graph_multi_out (2000)	-83.9%	+508.7%	436.45	643.7	70.48	3918.3
[S]can::graph_multi_out_where (2000)	-78.8%	+361.6%	173.91	1610.2	36.83	7433.1
[S]can::graph_multi_count (2000)	-82.3%	+436.8%	94.20	2968.2	16.70	15934.7
[S]can::graph_depth2_limit (2000)	-80.9%	+413.0%	246.73	1139.2	47.07	5844.6
[S]can::graph_sub_where (2000)	-61.8%	+139.7%	5.50	48232.8	2.10	115625.9
[S]can::graph_sub_group_all (2000)	-61.1%	+142.2%	5.84	45269.5	2.27	109630.7
[S]can::graph_sub_group_by (2000)	-62.5%	+140.8%	5.90	44952.6	2.21	108263.4
[S]can::subquery_inline (2000)	-38.6%	+57.9%	596.32	471.7	366.11	744.9
[S]can::subquery_count (2000)	-43.3%	+75.1%	75.35	3716.0	42.73	6505.7
[S]can::subquery_from (2000)	-36.5%	+54.1%	2723.41	102.0	1730.69	157.1
[S]can::pipeline_filter_group_order (2000)	-17.1%	+17.3%	7624.09	37.0	6318.85	43.4
[S]can::index_standard (2000)	+46.8%	-34.6%	3454.12	82.3	5071.34	53.8
[I]ndex::index_standard	-33.4%	+50.0%	1513.98	0.7	1008.38	1.0
[S]can::index_standard::indexed (2000)	-56.3%	+90.7%	6.04	43641.0	2.64	83230.9
[R]emoveIndex::index_standard	-76.9%	+332.6%	445.31	2.2	102.82	9.7
[S]can::index_composite (2000)	+11.7%	-13.5%	5489.27	51.6	6131.84	44.7
[I]ndex::index_composite	-33.2%	+50.0%	1513.98	0.7	1011.97	1.0
[S]can::index_composite::indexed (2000)	+30.9%	-23.1%	1509.67	183.2	1976.73	141.0
[R]emoveIndex::index_composite	-64.8%	+184.0%	421.76	2.4	148.54	6.7
[S]can::index_range_merged (2000)	-20.9%	+23.4%	7044.30	39.8	5572.12	49.1
[I]ndex::index_range_merged	-33.1%	+50.0%	1513.98	0.7	1012.48	1.0
[S]can::index_range_merged::indexed (2000)	+34.2%	-24.9%	453.72	608.9	608.72	457.5
[R]emoveIndex::index_range_merged	-67.0%	+202.7%	445.06	2.2	146.75	6.8
[S]can::index_in (2000)	-22.1%	+25.5%	6744.01	41.8	5253.10	52.5
[I]ndex::index_in	-33.1%	+50.0%	1512.96	0.7	1011.97	1.0
[S]can::index_in::indexed (2000)	-58.8%	+136.3%	30.92	8937.5	12.73	21115.7
[R]emoveIndex::index_in	-71.2%	+247.0%	456.32	2.2	131.52	7.6
[I]ndex::index_fulltext	+46.8%	-50.0%	55689.22	0.0	81756.16	0.0
[S]can::index_fulltext::indexed (2000)	+45.0%	-29.9%	204.39	1343.2	296.32	942.0
[R]emoveIndex::index_fulltext	+130.9%	-56.3%	969.98	1.0	2239.49	0.5
[I]ndex::index_hnsw	-30.8%	+40.0%	19587.07	0.1	13545.47	0.1
[S]can::index_hnsw::indexed (2000)	-87.1%	+654.4%	35710.91	7.9	4622.45	59.4
[R]emoveIndex::index_hnsw	-67.3%	+205.0%	2509.82	0.4	820.48	1.2

RocksDB

Test	v3 vs. v2		SurrealDB v2		SurrealDB v3
	Mean %	OP/s %	Mean (ms)	OP/s	Mean (ms)	OP/s
[C]reate	-14.7%	+17.3%	9.99	28778.2	8.52	33768.3
[R]ead	+59.5%	-36.1%	0.37	757343.3	0.59	484147.0
[U]pdate	+1.9%	-1.6%	8.34	34370.8	8.50	33828.1
[S]can::count_all (2001)	-13.2%	-2.7%	153.00	1821.8	132.74	1771.7
[S]can::limit_id (2001)	-63.5%	+155.9%	5.32	48199.9	1.94	123328.7
[S]can::limit_all (2001)	-85.0%	+492.6%	16.29	16194.5	2.45	95975.1
[S]can::limit_start_id (2001)	-49.8%	+90.2%	12.08	22395.3	6.07	42606.3
[S]can::limit_start_all (2001)	-69.3%	+210.4%	20.45	13197.6	6.28	40970.0
[S]can::select_where_id (2001)	+19.0%	-12.7%	0.63	334303.5	0.75	291755.1
[S]can::select_where_id_eq (2001)	-100.0%	+352510.7%	3313.50	85.7	0.72	302257.9
[S]can::select_where_gt (2001)	+30.1%	-27.0%	4576.38	62.0	5955.45	45.2
[S]can::select_where_in (2001)	-34.9%	+50.0%	9134.11	30.7	5944.41	46.0
[S]can::select_where_multi_and (2001)	+12.1%	-13.1%	5394.01	52.4	6047.01	45.5
[S]can::select_where_order_limit (2001)	+22.3%	-19.4%	4543.69	62.4	5556.69	50.3
[S]can::select_where_order_desc_limit (2001)	+26.4%	-22.0%	4505.67	63.0	5695.88	49.1
[S]can::select_where_multi_order_limit (2001)	-2.4%	+0.0%	5628.40	50.4	5495.20	50.4
[S]can::select_omit_limit (2001)	-76.9%	+314.9%	16.68	16042.6	3.85	66553.8
[S]can::select_fields_where_limit (2001)	+637.0%	-86.9%	14.19	18728.2	104.58	2460.3
[S]can::select_order_by (2001)	-68.5%	+214.1%	15667.41	17.9	4932.92	56.2
[S]can::select_order_by_multi (2001)	-68.3%	+215.8%	15902.27	17.6	5037.95	55.7
[S]can::select_group_count (2001)	+20.2%	-20.1%	4235.10	67.0	5091.89	53.6
[S]can::select_group_sum (2001)	+8.8%	-11.6%	4881.45	58.1	5309.39	51.4
[S]can::select_group_avg (2001)	+9.6%	-11.3%	4900.75	57.8	5371.33	51.3
[S]can::select_group_multi_agg (2001)	-21.5%	+25.9%	7782.85	36.0	6111.42	45.4
[S]can::select_group_all (2001)	-3.6%	+2.5%	5624.71	50.3	5421.68	51.5
[S]can::select_group_order_limit (2001)	-3.3%	+1.5%	5721.24	49.5	5529.63	50.2
[S]can::select_group_where (2001)	+6.1%	-7.3%	5949.82	47.5	6311.33	44.0
[S]can::select_group_dedup_agg (2001)	-21.6%	+27.4%	7813.47	35.8	6125.32	45.6
[S]can::select_split (2001)	-50.1%	+80.7%	17.45	15307.9	8.70	27658.5
[S]can::select_fetch (2001)	-77.1%	+327.8%	35.19	7564.1	8.06	32357.4
[S]can::select_fetch_where_limit (2001)	+276.7%	-73.1%	30.11	8892.6	113.41	2388.4
[S]can::graph_out_depth1 (2001)	-79.1%	+334.3%	4.78	54273.4	1.00	235728.9
[S]can::graph_out_depth2 (2001)	-88.0%	+612.6%	14.39	19118.8	1.73	136232.2
[S]can::graph_out_depth3 (2001)	-93.3%	+1232.1%	41.42	6724.3	2.78	89577.1
[S]can::graph_in_depth1 (2001)	-77.1%	+254.4%	4.90	54000.1	1.12	191369.2
[S]can::graph_bidirectional (2001)	-89.3%	+654.4%	10.62	25771.5	1.14	194426.5
[S]can::graph_edge_filter (2001)	-57.0%	+122.3%	2.37	102130.8	1.02	227036.8
[S]can::graph_multi_out (2001)	-94.6%	+1651.9%	426.98	659.4	23.02	11551.7
[S]can::graph_multi_out_where (2001)	-91.8%	+1060.6%	170.00	1648.7	13.99	19133.9
[S]can::graph_multi_count (2001)	-90.1%	+833.6%	90.26	3098.7	8.96	28929.8
[S]can::graph_depth2_limit (2001)	-95.9%	+2195.0%	238.54	1176.5	9.82	27000.0
[S]can::graph_sub_where (2001)	-59.8%	+125.1%	5.40	49395.5	2.17	111209.4
[S]can::graph_sub_group_all (2001)	-62.9%	+144.8%	5.72	46438.1	2.12	113671.0
[S]can::graph_sub_group_by (2001)	-65.0%	+158.7%	5.65	47104.9	1.98	121859.2
[S]can::subquery_inline (2001)	-39.8%	+62.5%	585.17	476.7	352.05	774.7
[S]can::subquery_count (2001)	-10.6%	+8.9%	71.02	3947.0	63.50	4299.9
[S]can::subquery_from (2001)	-36.1%	+53.9%	2708.90	102.4	1731.46	157.7
[S]can::pipeline_filter_group_order (2001)	-13.0%	+13.8%	7471.20	37.6	6498.49	42.9
[S]can::index_standard (2001)	+53.5%	-37.3%	3307.64	85.9	5077.27	53.8
[I]ndex::index_standard	+0.2%	+0.0%	1010.43	1.0	1011.97	1.0
[S]can::index_standard::indexed (2001)	-68.1%	+181.6%	6.95	38213.8	2.22	107603.5
[R]emoveIndex::index_standard	-21.0%	+26.8%	275.84	3.6	217.79	4.6
[S]can::index_composite (2001)	+10.0%	-12.3%	5403.80	52.4	5945.42	45.9
[I]ndex::index_composite	+1.2%	-1.0%	1006.34	1.0	1018.11	1.0
[S]can::index_composite::indexed (2001)	-49.0%	+96.1%	1494.50	184.7	762.39	362.3
[R]emoveIndex::index_composite	-70.9%	+243.8%	769.28	1.3	223.81	4.5
[S]can::index_range_merged (2001)	-21.9%	+25.3%	6974.03	40.3	5446.44	50.5
[I]ndex::index_range_merged	+0.2%	+0.0%	1012.48	1.0	1014.53	1.0
[S]can::index_range_merged::indexed (2001)	-47.9%	+92.8%	456.57	595.9	237.86	1148.8
[R]emoveIndex::index_range_merged	-20.5%	+25.5%	271.74	3.7	216.13	4.6
[S]can::index_in (2001)	-24.2%	+27.4%	6644.39	42.3	5037.32	53.8
[I]ndex::index_in	+0.2%	+0.0%	1010.43	1.0	1011.97	1.0
[S]can::index_in::indexed (2001)	-70.1%	+218.6%	30.15	9168.9	9.00	29209.8
[R]emoveIndex::index_in	-18.5%	+22.7%	279.94	3.6	228.03	4.4
[I]ndex::index_fulltext	+28.6%	+0.0%	73302.02	0.0	94273.54	0.0
[S]can::index_fulltext::indexed (2001)	-8.0%	+9.6%	217.18	1267.2	199.83	1388.3
[R]emoveIndex::index_fulltext	+598.3%	-85.7%	435.07	2.3	3038.21	0.3
[I]ndex::index_hnsw	-6.4%	+16.7%	15544.32	0.1	14544.90	0.1
[S]can::index_hnsw::indexed (2001)	-87.3%	+659.9%	35562.02	7.9	4524.91	60.0
[R]emoveIndex::index_hnsw	-16.2%	+18.5%	1546.75	0.7	1295.87	0.8

More performance improvements coming

We have a roadmap of performance-related improvements that we'll be rolling out across the coming minor releases. Here's what to expect:

Expanding the new executor to cover write workloads and all remaining statement types
End-to-end streaming, pushing the streaming model from internal execution all the way to the client
Indexing improvements, including better index planning, reduced scan overhead, expanded index types, and optimised composite index strategies
Continued optimisation of the planning and execution pipeline itself

For now, you can start reaping the benefits of the new executor on read-heavy workloads, and expect it to get better with every release.

What’s next?

These benchmarks are just the beginning. This release focuses on measuring the performance gains in SurrealDB 3.0 compared directly to SurrealDB v2.x, highlighting the impact of the new foundations.

Next, we’ll expand this work with a broader benchmarking iteration that compares SurrealDB against other databases and platforms across a wider range of workloads. Stay tuned - there’s much more to come.

If you have any questions, feedback, or want to dive deeper into the results, feel free to reach out to us or join our Discord community to chat directly with the community and team.

Introducing Surrealism

Mark Gyles — Thu, 19 Feb 2026 18:38:16 +0000

We are excited to introduce Surrealism which offers something developers have wanted for a long time: true extensibility inside SurrealDB itself. Those of you who have participated in the SurrealDB 3.0 beta might have already gotten a glimpse of Surrealism, and we’re excited to announce it more broadly to the world today.

Surrealism is a new open-source extension framework for SurrealDB. It allows you to define modular, programmable logic using functions you write in Rust - and execute them directly within the database at runtime. We intend to expand Surrealism to support other languages including JavaScript and Python.

Traditionally, if you wanted to extend your database, you had to choose between brittle, complex, error-prone, and potentially huge scripts, heavyweight external services, or just push the data to the application layer and perform the logic there. Surrealism changes that. It brings logic into the database in a way that’s fast, secure, testable, and fully integrated with your development workflow. Surrealism redefines the way that developers and organisations work with stored procedures - allowing for custom business logic, and access control layers as version-controlled, testable, and shareable modules. These plugins execute with full transactional guarantees, and can be used to manage everything from dynamic API behaviours to policy enforcement and auditing.

How Surrealism works

Functions are compiled to WebAssembly, allowing them to run in a secure, deterministic, sandboxed execution environment, whilst delivering near-native performance, with strict isolation between functions. This makes it ideal for both single-tenant systems and multi-tenant clusters, where isolation and efficiency are critical.

Plugins can be loaded from local disk, object storage, or uploaded directly into SurrealDB. They can be enabled and executed dynamically - without restarting the database - with fine-grained permission controls to determine who can load or invoke them.

With Surrealism, the development experience has been designed from the ground up for productivity. Developers can use their own libraries, tools, and frameworks. They can write tests in Rust, integrate plugins into CI and CD pipelines, and manage plugin versions just like any other component of their application stack. Plugins are defined in simple projects with metadata in a TOML file, and compiled into portable WebAssembly binaries that can be reused, versioned, and shared.

Built-in AI support and agentic workflows

Surrealism enables direct integration with AI models, both local and remote. Plugins can interact with GPU-accelerated inferencing runtimes, or call out to external APIs for services like text or image generation, classification, tokenisation, embedding, translation, and anything else. Surrealism extensions turn SurrealDB into a programmable data and logic layer - perfect for agentic and AI-heavy workloads.

This means you could define a plugin that takes a user’s input, runs it through a local LLM, and stores the generated result - all within a single SurrealQL query. You can trigger sentiment analysis on inserted documents, run similarity scoring against vector indexes, or use a vision model to extract structured data from uploaded files.

And because Surrealism plugins have full access to SurrealDB’s query engine, schema, and the new file functionality, they can work with structured data, unstructured content, and multi-model assets in a single unified system.

For example, a plugin can transcribe an audio file, extract named entities, enrich the results using a remote model, and save the output - all in one transactional flow.

This allows developers to build intelligent applications entirely within the database. The database becomes more than a data store - it becomes an intelligent, programmable runtime for real-time decision-making, content generation, and AI integration.

Bringing logic and agentic workflows into the database, rather than pushing the data out to external systems, enables seamless integration for AI and unlocks new possibilities for intelligent applications. It’s fast. It’s flexible. It’s secure. And it’s open source.

Surrealism is available today for you to start developing your own custom or business logic, and it opens up a whole new frontier in database development - one where you can bring your own code, your own models, and your own ideas directly into the data layer.

We can’t wait to see what you build.

Get started with Surrealism

Go to the Surrealism Docs to get started. We are excited to see what extensions you build, be sure to share them in our Discord channel.

Thank you for being on this journey with us!

Introducing SurrealDB 3.0 - the future of AI agent memory

Mark Gyles — Wed, 18 Feb 2026 11:44:48 +0000

Since SurrealDB was first released, we’ve been humbled by the reviews and adoption from our users. The possibility of a single database that does it all, can run anywhere, and can scale from embedded to large scale clusters opened up the imagination of users, all the way from building small applications to large scale enterprise AI production systems.

3.0 looks to take SurrealDB to the next level, with significant improvements across three main areas:

Stability, performance and tooling
Improved developer experience
Building AI agents

Stability, performance and tooling

2.0 was a milestone for SurrealDB in terms of capability, but as usage scaled, some of our internal models made the engine do work it didn’t need to do, and made correctness and predictability harder than they needed to be.

With SurrealDB 3.0, we’ve made major architectural changes to improve stability, performance, and the developer experience. We’ve separated values from expressions, introduced computed fields, moved core metadata to ID‑based storage, made synced writes the default, and re‑designed how documents are represented on disk. The result is a faster, more stable database - and a cleaner path to future optimisation.

Let’s dive into these changes and explore why they matter.

Splitting values and expressions

At the heart of 3.0 is a clean separation between what your data is (values) and how it’s derived (expressions).

In earlier versions these were entangled. A field could be “computed or un‑computed,” but the engine often had to continuously compute to present a result. That meant extra evaluation, extra serialisation, and edge‑case bugs - especially once you stored logic inside records.

By splitting the two, the engine evaluates only when it should, avoiding redundant work and laying a foundation for a more predictable planner and smarter optimisations in future releases.

This corrects the prior model and directly enables a safer, faster way to express logic in your schema: computed fields.

Computed fields

SurrealDB’s future type was used for values that were evaluated at query time. While flexible, they embedded logic into each record and forced per‑row evaluation. In 3.0, they are replaced with computed fields: logic you define once in the schema to be evaluated consistently at query time.

DEFINE FIELD can_drive ON person COMPUTED age > 18;

This approach removes unnecessary per‑record computation, improves stability, and makes behaviour easier to reason about. The result is a reduced workload for the engine and a clearer developer experience, with guardrails that prevent hidden complexity.

Learn more at https://surrealdb.com/docs/surrealql/statements/define/field.

ID‑based storage

3.0 moves core catalog entries like namespaces, databases and indexes from name‑based to ID‑based storage. Internally, these now use compact, fixed‑size identifiers rather than variable‑length names.

The benefits are twofold:

Consistent, smaller keys on disk: reducing storage consumption and improving lookups - especially when names are long.
It lays the groundwork for renaming resources: since references point to stable IDs that don’t change, it will be possible to rename database resources.

This makes the catalog more robust today and more flexible for features in the future.

As an arbitrary example of a smaller key, here’s one of the keys that’s stored in the DB that is used for indexing.

This is what it would look like before at 80 bytes with the previous Rust code:

`/*namespace\0*database\0*order_history\0+order_history__city_time__idx\0!bd\x01\0\0\0\0\0\0\0\x07`

let val = Bd::new(
    "namespace",
    "database",
    "order_history",
    "order_history__city_time__idx",
    Some(7),
);
let enc = Bd::encode(&val).unwrap();
assert_eq!(enc.len(), 80);
assert_eq!(enc, b"/*namespace\0*database\0*order_history\0+order_history__city_time__idx\0!bd\x01\0\0\0\0\0\0\0\x07");

And here it is now at 42 bytes.

`/*\0\0\0\x01*\0\0\0\x02*order_history\0+\0\0\0\x03!bd\0\0\0\0\0\0\0\x07`

let val = Bd::new(NamespaceId(1), DatabaseId(2), "order_history", IndexId(3), 7);
let enc = Bd::encode_key(&val).unwrap();
assert_eq!(enc.len(), 42);
assert_eq!(enc, b"/*\0\0\0\x01*\0\0\0\x02*order_history\0+\0\0\0\x03!bd\0\0\0\0\0\0\0\x07");

That’s a reduction of 38 bytes on this key alone - multiplied across catalogs, it meaningfully trims storage and I/O.

Synced writes by default

Synced writes are now the default in 3.0. Rather than relying on the operating system to flush data “soon,” SurrealDB confirms a write only after it’s durably committed.

This change doesn’t alter what’s possible - it makes the safer behaviour the standard, improving predictability for production workloads.

Document representation

Previously, a record’s value and its metadata lived together, with metadata stored as hidden fields. That made rendering results slower and caused correctness issues - especially with views and table statistics.

3.0 introduces a proper document wrapper type. Record content and record metadata are stored explicitly and separately. That means cleaner responses with no invisible fields, improved stability for views and table statistics, and room to attach system metadata - such as update timestamps or version stamps - without polluting user data.

Bug fixes & SDK improvements

3.0 also includes a large bug-fixing effort. We have closed over 150 bugs, and are now laser-focused on solving any new reported bugs. We have also implemented improved automated testing, and a CRUD bench workflow that runs on every submitted PR. These fixes address long-standing pain points raised by both community users and enterprise deployments, improving correctness and reliability across the board.

Alongside this, the Go and Java SDKs have reached 1.0 production readiness, giving developers stable integration paths in more ecosystems out of the box.

SurrealDB 3.0 represents a significant step forward in the stability and maturity of the database. By separating values from expressions, introducing computed fields, adopting ID‑based storage, enabling synced writes by default, and redesigning document handling, SurrealDB becomes more predictable, more durable, and easier to operate.

As you explore SurrealDB 3.0, you’ll find these improvements make everyday queries faster, behaviour more consistent, and future optimisation substantially easier.

Surreal Sync

Surreal Sync is a new tool that helps you move data from another database or source into SurrealDB. It’s currently in active development, and users are welcome to test it and share feedback or report any issues.

Surreal Sync is run entirely on the command line and allows you to import data directly from a another source straight into a running SurrealDB instance. For example, the following command will sign into a Neo4J database at bolt://localhost:7687 as the user neo4j with the password password into the SurrealDB database graph_data inside the production namespace via a user named root with the password secret.

surreal-sync sync neo4j \
  --source-uri "bolt://localhost:7687" \
  --source-username "neo4j" \
  --source-password "password" \
  --neo4j-timezone "America/New_York" \
  --to-namespace "production" \
  --to-database "graph_data" \
  --surrealdb-username "root" \
  --surrealdb-password "secret"

Learn more at https://surrealdb.com/docs/surrealdb/migrating.

GraphQL support is now stable

GraphQL has become a critical interface layer for modern applications. It gives frontend teams precise control over the data they fetch, reduces over-fetching and under-fetching, and enables faster iteration across web, mobile, and edge clients. For companies, it provides a strongly typed, self-documenting API surface that scales cleanly across teams and services. With SurrealDB 3.0, we are excited to announce that GraphQL support is now stable.

Previously gated behind an experimental flag, the GraphQL interface has undergone a comprehensive overhaul to meet real-world requirements - including full mutation support, built-in authentication flows, permission enforcement, and strong protections against abusive queries through configurable depth and complexity limits.

This release also introduces:

Rich filtering operators and nested object support
First-class geometry and relation resolution
N+1 query optimisations for dramatically fewer database round-trips
Security hardening and safer error handling
GraphQL enabled by default (no experimental flag required)

If you’ve been waiting to run SurrealDB with GraphQL, now’s the time. Explore the GraphQL docs and get started: https://surrealdb.com/docs/surrealdb/querying/graphql

Improved developer experience

2.0 unlocked new possibilities, but as adoption grew, developers asked for a more predictable, extensible, and expressive way to build with SurrealDB. With 3.0, we’ve delivered just that - bringing a refined model that’s simpler to reason about and faster to build on. Developers can now define custom API endpoints directly within the database, manage complex workflows through client-side transactions, and express logic safely with computed fields and record references. These upgrades transform how you handle data relationships and integrate SurrealDB into real-world applications - making the developer experience as seamless as the database itself.

Custom API endpoints

With SurrealDB 3.0, DEFINE API is now fully stabilised - letting you define custom endpoints and middleware directly inside your database, using the same SurrealQL you already know.

DEFINE API allows you to move from:

Client → Middleware (API) → Database

To:

Client → Database

While this was already possible with the standard endpoints, we’ve now improved it by also making custom endpoints possible.

DEFINE API is customisable, so let’s see an example. Let's say you have a social app that lets users see all the latest comments. You'd like to develop a free API to allow anonymous users to see them too, but sometimes the app gets really busy and you don't want guests to take up valuable resources.

In other words, you’d like to implement rate limits.

To begin, pass in the new --deny-arbitrary-query flag when starting the SurrealDB server.

surreal start --user root --pass root --deny-arbitrary-query guest

This prevents certain groups, like guests or record users, from putting their own queries together. We'll add guest here so that any guest users will have to use the API endpoint instead.

Defining your API endpoint

DEFINE API OVERWRITE "/get_latest" FOR get

We can follow this up with some middleware by passing in some of SurrealDB's API functions. We'll use api::timeout to give guest users up to 50 milliseconds of server time.

MIDDLEWARE
    api::timeout(50ms)

And then we'll finish up the statement by adding what it actually returns: a SELECT statement for all comment records that have been created over the past ten minutes. In summary, that gives us a DEFINE API statement that looks like this.

DEFINE API OVERWRITE "/get_latest" FOR get
    MIDDLEWARE
        api::timeout(50ms),
    THEN {
        RETURN {
            body: SELECT * FROM comment:[time::now()-10m].. ORDER BY id DESC
        }
};

You can test it from SurrealQL itself through api::invoke:

api::invoke("/get_latest");

or externally at: /api/:namespace/:database/:endpoint_name. Here's what our defined endpoint will look like with a namespace called test_ns and a database called test_db.

http://localhost:8000/api/test_ns/test_db/get_latest

There are a lot of examples in the documentation to show how it works, but here’s a simple example to get you started.

DEFINE FUNCTION fn::add_prefix($req: object, $next: function, $prefix: string) -> object {
    LET $res = $next($req);
    LET $res = $res + { body: $res.body + { prefix: $prefix + ": " + $res.body.message } };
    $res;
};

DEFINE API "/custom_with_args"
    FOR get
        MIDDLEWARE
            fn::add_prefix("PREFIX")
        THEN {
            {
                status: 200,
                body: {
                    message: "original message"
                }
            };
        };

Dive deeper into SurrealDB's powerful API management capabilities:

Client-side transactions

We’re introducing client-side transactions - a new way of working with transactions that gives developers far more flexibility and control.

In earlier versions, transactions were handled in a single request. That worked well for many use cases, but it also meant that any decision logic - such as whether to commit or cancel - had to be written directly in SurrealQL.

Now, with client-side transactions, that logic can live in your programming language of choice. You can begin a transaction, add operations across multiple requests, and commit when ready - all while keeping the same strong, ACID-compliant guarantees.

How it works

Client-side transactions follow a simple workflow: begin → add operations → commit or cancel.

This enhancement lets developers keep application logic in SDK languages such as Rust, Go, or JavaScript, while SurrealDB manages the transactional consistency behind the scenes.

For developers, this means cleaner, easier-to-read code, and workflows that are simpler to debug and maintain. Complex, multi-step business logic can now be expressed naturally in application code without giving up transactional safety.

And this is just the beginning. Client-side transactions also unlock new possibilities - like multi-step processes that span time or even human intervention - while still relying on SurrealDB for atomic guarantees.

With 3.0, transactions become far more adaptable to the way you build applications.

Learn more at https://surrealdb.com/docs/surrealql/transactions.

Computed fields

As mentioned already, in 3.0 we’re introducing computed fields - a simpler, faster, and more predictable way to define logic directly inside your data.

In previous versions of SurrealDB, you could attach a future to a field. While flexible, futures were too tightly coupled to the records themselves, which made it hard to scale well and often led to unexpected behaviour.

In 3.0, futures are gone. In their place, we now have computed fields - defined with a clear COMPUTED keyword, evaluated consistently whenever you query a record.

-- Before: using a future
DEFINE FIELD can_drive ON person VALUE <future> { time::now() - born > 18y };

-- After: using a computed field
DEFINE FIELD can_drive ON person COMPUTED time::now() - born > 18y;

This new syntax is shorter, clearer, and always up to date. Developers can now trust that computed values will appear in results without having to update records manually.

By consolidating logic into computed fields, SurrealDB reduces complexity in the core engine. This improves stability today - and sets us up to optimise computations at scale in future versions, leading to smarter query planning and faster results.

It also fixes a real pain point: developers no longer need to worry about when or how a future was set. With computed fields, the logic lives in one place, is always evaluated fresh, and returns consistently.

A practical example

Let’s look at how this works in practice. We want to model a driver’s licenses that expire two years after they are issued. We define when the license was issued, who owns it, and then add a computed field that checks if it’s still valid:

DEFINE FIELD since ON license TYPE datetime DEFAULT time::now();
DEFINE FIELD owner ON license TYPE record<person>;
DEFINE FIELD valid ON license COMPUTED time::now() - since < 2y;

The valid field automatically returns whether a license is less than two years old. There’s no update step, no manual reset. Just clean, predictable results every time.

When we query the license, the valid field is computed on the fly and included right in the result:

[
  {

        id: license:buvnfunv21em9zncxe03,
        owner: person:one,
        since: d'2025-09-01T04:23:34.956Z',
        valid: true
    }
]

Guardrails for reliability

Computed fields also introduce guardrails that futures never had. They can only be defined as top-level fields, which avoids hidden complexity in nested objects. And they cannot be combined with clauses like VALUE, DEFAULT, or ASSERT.

These restrictions make computed fields easier to reason about, more consistent to query, and safer to use in production.

Looking ahead

Computed fields aren’t just a replacement for futures. They’re a foundation for the future.

By moving logic into a consistent, centralised system, we’ve opened the door for optimised computation and query planning in upcoming releases. This means faster queries, smarter evaluation, and a simpler mental model for developers.

With SurrealDB 3.0, you can now define logic directly in your schema with confidence - cleaner, safer, and faster than ever before.

Learn more at https://surrealdb.com/docs/surrealql/statements/define/field.

Record references

With the introduction of record references in SurrealDB 3.0, we’re making it possible for simple record links to now work in two directions instead of one.

This is a big improvement for anyone working with record links if you’ve been using ad-hoc queries like this one to simulate bidirectional links.

DEFINE FIELD since ON license TYPE datetime DEFAULT time::now();
-- Link from a license to a person
DEFINE FIELD owner ON license TYPE record<person>;

SELECT 
    name, 
    -- Look through all licenses to find the one linking to this person
    (SELECT type, since FORM license WHERE owner = $parent.id) AS licenses
FROM person;

Record references are defined at the schema level with the new REFERENCE keyword.

DEFINE FIELD owner ON license TYPE record<person> REFERENCE;

This turns a unidirectional link into a bidirectional relationship. With the reference in place, linked records can automatically “see” the records linking to them.

The syntax for traversing them is familiar too. While graph lookups use <- arrows, record references use a tilde - <~ - to signal the reverse lookup.

A practical example

Let’s further develop the license example to see how this works in practice. Here, we’re modelling people and their licenses.

This time though, owner is set to be a REFERENCE, allowing other records to pick up on the incoming link. We have also added two COMPUTED fields. The one called valid checks to see if the license is still valid by being less than two years old, while the one called licenses checks for incoming links from the license table.

DEFINE FIELD since ON license TYPE datetime DEFAULT time::now();
DEFINE FIELD owner ON license TYPE record<person> REFERENCE;
DEFINE FIELD valid ON license COMPUTED time::now() - since < 2y;
DEFINE FIELD licenses ON person COMPUTED <~license;

Let’s test out this new setup by creating a person named Billy, along with two licenses - a current driver’s license and an old pilot’s license:

CREATE person:one SET name = "Billy";
CREATE license SET owner = person:one, type = "driver";
CREATE license SET owner = person:one, type = "pilot", since = d'2003-03-11';

This time when we query a person, all we need to do is select the fields we want to see and the built-in schema does the rest of the work. Both licenses are returned automatically, complete with their validity status.

SELECT 
    name, 
    licenses.{ since, type, valid }
FROM person;

[
    {
        licenses: [
            {
                since: d'2003-03-11T00:00:00Z',
                type: 'pilot',
                valid: false
            },
            {
                since: d'2025-09-03T02:17:04.340Z',
                type: 'driver',
                valid: true
            }
        ],
        name: 'Billy'
    }
]

And since record reference syntax works just like graph queries do, you can use the <~ syntax directly inside a query too. Here’s how we can order Billy’s licenses by date if we decided to use a direct query instead of defining a field using <~ in the schema.

DEFINE FIELD since ON license TYPE datetime DEFAULT time::now();
DEFINE FIELD owner ON license TYPE record<person> REFERENCE;
DEFINE FIELD valid ON license COMPUTED time::now() - since < 2y;

CREATE person:one SET name = "Billy";
CREATE license SET owner = person:one, type = "driver";
CREATE license SET owner = person:one, type = "pilot", since = d'2003-03-11';

SELECT 
    name, 
    (SELECT type, since, valid FROM **<~**license ORDER BY since) AS driving_licenses
FROM person;

[
    {
        driving_licenses: [
            {
                since: d'2003-03-11T00:00:00Z',
                type: 'pilot',
                valid: false
            },
            {
                since: d'2025-09-03T02:18:20.489Z',
                type: 'driver',
                valid: true
            }
        ],
        name: 'Billy'
    }
]

Record references make schemas clearer, queries simpler, and traversals faster. By cutting down redundant lookups, they give you the option to keep adding linked tables without seeing degradation in performance.

For developers working on a schema, it means you can model relationships more naturally, query them more efficiently, and rely on SurrealDB to manage the complexity.

Learn more at https://surrealdb.com/docs/surrealql/datamodel/references.

Surqlize TypeScript ORM (experimental)

We’re excited to introduce Surqlize - our new, type-safe TypeScript ORM for SurrealDB. Surqlize is designed to bring the full power of SurrealDB to TypeScript developers, with end-to-end type safety, zero code generation, and first-class graph support. It embraces SurrealDB’s flexibility while giving you the confidence and developer experience you expect from modern TypeScript tooling.

⚠️ Surqlize is currently experimental. We’re sharing it early so you can try it, push it, and help shape its future.

Why Surqlize?

SurrealDB is powerful - multi-model, graph-native, and highly expressive. Surqlize makes that power ergonomic and type-safe in TypeScript.

With Surqlize, you get:

Type-safe schema definitions: define your database schema using intuitive t.* builders.
Automatic type inference: full TypeScript types - without code generation or separate schema compilation steps.
Fluent, fully typed query builder: chain .select(), .where(), .return(), and more — with complete type safety throughout.
Comprehensive CRUD operations: build SELECT, CREATE, UPDATE, DELETE, and UPSERT queries in a clean, expressive API.
First-class graph relationships: model edges and traverse relationships naturally - fully typed.
Rich type system: objects, arrays, unions, literals, optional fields, and more — all mapped directly to TypeScript.
SurrealDB function support: integrated string, array, and record operations directly within typed queries.

Surqlize isn’t just an abstraction layer - it’s built specifically for SurrealDB’s multi-model architecture, including its graph capabilities and powerful function system.

Try it and help us shape it

Because Surqlize is experimental, your feedback is critical. Try it in your projects, push the limits, and tell us what feels great and what doesn’t. You can learn more here: https://github.com/surrealdb/surqlize

Join us in the SurrealDB Discord server and share your feedback, ideas, and issues. We’re actively iterating and want to build this with the community.

SurrealDB 3.0 is about pushing boundaries - and Surqlize is a big step toward a best-in-class TypeScript developer experience.

Building AI agents

SurrealDB 3.0 moves beyond storing data - it becomes the foundation for agent memory and intelligence. As developers build increasingly autonomous systems, the line between data, logic, and perception starts to blur. With this release, SurrealDB evolves from a multi-model database into a multi-modal platform for ****AI agents - one that can understand, recall, and extend its own context - ideal for context graphs and semantic layers to power complex AI agents and autonomous agentic workflows.

With file support, SurrealDB now handles structured records alongside images, audio, and documents - all queryable within SurrealQL. Enhanced vector search and indexing let agents store and retrieve embeddings with millisecond precision. Surrealism, our new WebAssembly extension system, enables custom logic and model execution directly inside the database. And we have some exciting agent memory functionalities in the roadmap to build better durable, scalable context for long-running AI agents.

Together, these capabilities turn SurrealDB 3.0 into more than a database - it’s a persistent memory engine for intelligent systems.

File support

Setting up file storage

To begin, we can start a new SurrealDB server with the files capability enabled.

surreal start --user root -pass root --allow-experimental files

Setting up file storage can be done with a single DEFINE BUCKET statement! A bucket can be a physical location, or even in-memory.

Let’s define a simple in-memory bucket. Just provide a name, and set the backend to "memory":

DEFINE BUCKET my_bucket BACKEND "memory";

The name of the bucket is also your path to the bucket and everything inside.

Storing and accessing files

Once your bucket is defined, you can reference it using a file pointer - a string prefixed with the letter f to indicate a file path.

f"my_bucket:/tour_of_surrealdb.txt".get();

The function returned NONE, so looks like there’s nothing there.

Let’s use the .put() method to add some bytes, creating the file. .put() can take bytes or a string, which it will automatically convert into bytes.

f"my_bucket:/tour_of_surrealdb.txt".put("Welcome to the Tour of SurrealDB!");

Here, we’re adding the string “Welcome to the Tour of SurrealDB!”

Now that the file exists, the .get() function will return its as bytes:

f"my_bucket:/tour_of_surrealdb.txt".get();

b"57656C636F6D6520746F2074686520546F7572206F66205375727265616C444221"

Working with file data

If you prefer to view the data differently, SurrealDB also now supports improved type conversions for byte-related types.

For example, you can cast the result to an array to see each byte as a number:

<array>f"my_bucket:/tour_of_surrealdb.txt".get();

[ 87, 101, 108, 99, 111, 109, 101, 32, 116, 111, 32, 116, 104, 101, 32, 84, 111, 117, 114, 32, 111, 102, 32, 83, 117, 114, 114, 101,
97, 108, 68, 66, 33 ]

Or cast to a string:

<string>f"my_bucket:/tour_of_surrealdb.txt".get();

This function is especially useful when working with partially corrupted or inconsistently encoded data. Instead of failing outright, it preserves and returns all valid parts of the string.

Here’s an example using type::string, which returns an error because not all the bytes are valid:

type::string(<bytes>[83, 117, 114, 255, 114, 101, 97, 254, 108, 68, 66]);
-- 'Could not cast into `string` using input `b"537572FF726561FE6C4442"`'

And here’s the same input passed through type::string_lossy(). We can see that there are some valid bytes in there - enough to make out the word SurrealDB.

type::string_lossy(<bytes>[83, 117, 114, 255, 114, 101, 97, 254, 108, 68, 66]);
-- 'Sur�rea�lDB'

We’re excited to introduce this first stage of native file support in SurrealDB. If you’re building apps that rely on lightweight file storage or embedded assets, give it a try - and let us know what features you’d like to see next. This is just the beginning.

Learn more at https://surrealdb.com/docs/surrealql/datamodel/files.

Indexing and search

We know that index performance and functionality is important, which is why we’re excited to announce a brand-new indexing foundation - delivering faster queries, higher throughput, and smarter planning across the board.

Until now, SurrealDB’s queries often had to scan entire tables because the planner couldn’t exploit compound or descending indexes, full‑text search only matched all terms, and index updates ran single-threaded. This made searches and analytics painfully slow and blocked high-write workloads, motivating a comprehensive indexing overhaul for 3.0.

Indexing now combines a log‑based, multi‑writer engine with normalised key formats and smarter planners. Compound indexes support prefix + range scans, descending order, and LIMIT-aware iterators, while standard indexes handle duplicates gracefully. These changes let 3.0 answer many queries directly from indexes under heavy concurrency.

Let’s take a closer look at these changes.

Compound indexes with prefix + range scans

In SurrealDB 3.0, compound indexes have been enhanced to support both ****prefix equality and range queries in the same plan.

That means queries like this are now served directly from the index, without touching the entire table:

DEFINE INDEX idx_orders_customer_created
ON orders FIELDS customer, created_at;

SELECT id, total
FROM orders
WHERE customer = "acme"
  AND created_at >= d'2025-01-01'
  AND created_at <  d'2025-02-01'
ORDER BY created_at;

By constraining the left prefix (customer = "acme") and scanning a precise range on created_at, the query planner can narrow down results efficiently, even on large datasets.

Descending order and LIMIT-aware scans

Queries with ORDER BY ... DESC can now be satisfied directly from the index on supported backends, eliminating the need for in-memory sorting. Combined with smarter batching that respects LIMIT, the database returns exactly what you asked for.

SELECT id, created_at, total
FROM orders
WHERE customer = "acme"
ORDER BY created_at DESC
LIMIT 20;

This approach reduces latency, saves memory, and keeps response times consistent.

Normalised numeric keys

Numeric values are now encoded canonically inside indexes. That means that the integer 0, the float 0.0, and 0 decimal are treated as equal in UNIQUE constraints and scan operations, and range queries run seamlessly across all numeric variants.

DEFINE INDEX uniq_price ON products FIELDS price UNIQUE;

CREATE products:one SET price = 0;
CREATE products:two SET price = 0.0;   -- Duplicate error, as expected

This change brings greater predictability to numeric comparisons, ensuring that equality and ordering work the way you’d expect.

Smarter full-text search

Full-text indexing has also been upgraded in 3.0. It now supports OR boolean operations, so you can query for documents that contain any of the specified terms, not just all of them.

DEFINE INDEX idx_docs_ft ON docs FIELDS body FULLTEXT ANALYZER english;

SELECT id
FROM docs
WHERE body @OR@ "vector graph";

On top of that, full-text indexing now supports concurrent writers, using a log-based approach with background compaction. This makes SurrealDB’s search engine more scalable under high-write workloads.

Concurrent index builds

Index creation itself has been improved with concurrent indexing by default. Instead of building an index in one large transaction, 3.0 handles index builds in batches. That means large indexes can be created more efficiently, without running into memory or file space issues.

The impact

Moving from O(N) scans to tight indexed lookups drops latency from seconds to milliseconds on million-row tables, while log‑based multi-writer indexing removes write bottlenecks for full‑text searches.

For developers, the net effect is faster reads, higher throughput, and more predictable tails as your datasets grow.

Indexing in SurrealDB 3.0 is now faster, more flexible, and more powerful, and this foundation sets us up for future distributed and specialised indexes - including vector and geospatial - enabling SurrealDB to scale to ever-larger, multi-tenant workloads.

Introducing Surrealism

We are excited to introduce Surrealism which offers something developers have wanted for a long time: true extensibility inside SurrealDB itself.

Surrealism is a new open-source extension system for SurrealDB. It allows you to define modular, programmable logic using functions you write in Rust - and execute them directly within the database at runtime. We'll also soon be supporting a variety of other languages, including JavaScript and Python too.

This is a big shift.

Traditionally, if you wanted to extend your database, you had to choose between brittle, complex, error-prone, and potentially huge SQL scripts, heavyweight external services, or just push the data to the application layer and perform the logic there. Surrealism changes that. It brings logic into the database in a way that’s fast, secure, testable, and fully integrated with your development workflow. Surrealism redefines the way that developers and organisations work with stored procedures - allowing for custom business logic, and access control layers as version-controlled, testable, and shareable modules. These extensions execute with full transactional guarantees, and can be used to manage everything from dynamic API behaviours to policy enforcement and auditing.

How Surrealism works

Extensions are compiled to WebAssembly, allowing them to run in a secure, deterministic, sandboxed execution environment, whilst delivering near-native performance, with strict isolation between functions. This makes it ideal for both single-tenant systems and multi-tenant clusters, where isolation and efficiency are critical.

Extensions can be loaded from local disk, object storage, or uploaded directly into SurrealDB. They can be enabled and executed dynamically - without restarting the database - with fine-grained permission controls to determine who can load or invoke them.

With Surrealism, the development experience has been designed from the ground up for productivity. Developers can use their own libraries, tools, and frameworks. They can write tests in Rust, integrate extensions into CI and CD pipelines, and manage extension versions just like any other component of their application stack. Extensions are defined in simple projects with metadata in a TOML file, and compiled into portable WebAssembly binaries that can be reused, versioned, and shared.

Built-in AI support and 'datagentic' workflows

Surrealism enables direct integration with AI models, both local and remote. Extensions can interact with GPU-accelerated inferencing runtimes, or call out to external APIs for services like text or image generation, classification, tokenisation, embedding, translation, and anything else.

This means you could define an extension that takes a user’s input, runs it through a local LLM, and stores the generated result - all within a single SurrealQL query. You can trigger sentiment analysis on inserted documents, run similarity scoring against vector indexes, or use a vision model to extract structured data from uploaded files.

And because Surrealism extensions have full access to SurrealDB’s query engine, schema, and the new file functionality, they can work with structured data, unstructured content, and multi-modal assets in a single unified system.

For example, an extension can transcribe an audio file, extract named entities, enrich the results using a remote model, and save the output - all in one transactional flow.

Bringing logic and agentic workflows into the database, rather than pushing the data out to external systems, enables seamless integration for AI, and unlocks new possibilities for intelligent applications. We call this workflow 'datagentic'. It’s fast. It’s flexible. It’s secure. And it’s open source.

What’s next?

Building a database is never finished, and there’s more work we need to do. We’ll continue working on additional performance improvements, more tooling, integrations, and making SurrealDB the best database in the market.

Update your SurrealDB instance to 3.0 with our migration guide, join our Discord server and share with us what you are building.

Thank you for helping us shape the future of databases.