Forem: Romans Malinovskis

SurrealDB test case for SaaS

Romans Malinovskis — Wed, 23 Jul 2025 14:16:44 +0000

I have added a test database for SurrealDB, that looks at the scenario of a SaaS product. It's important to apply the core limitation where an authenticated user can only see records, belonging to that user. An developer-error (or even vibe-coder error) in Graph database can leak data and one of the requirements for vantage is the ability to enforce fundamental entity conditions.

If I want to query "client->placed->order" relation, I need both order and client to belong to my bakery:

SELECT
    name AS product_name,
    inventory.stock AS current_inventory,
    array::first((SELECT count() FROM order WHERE bakery = bakery:hill_valley GROUP ALL)).count AS total_orders
FROM product
WHERE bakery = bakery:hill_valley AND is_deleted = false;

In my latest PR to I have established a use-case that is designed specifically with "production" scenario like this:

https://github.com/romaninsh/vantage/pull/63

The model already existed for PostgreSQL, but next I'm going to extend support into SurrealDB query builder.

Arc and Mutex for SQL Expression parameters

Romans Malinovskis — Sun, 20 Jul 2025 18:18:29 +0000

I have now converted OwnedExpression to use Expressive protocol and started to think about "LazyExpression".

It's not really an expression that's lazy - it is a parameter to the expression that is lazy.

So I ditched LazyExpression entirely, and instead implemented support for Arc and Arc

https://github.com/romaninsh/vantage/pull/61

Before, a parameter to Expression engine could be either a value or a nested expression.

Now, a parameter to Expression can also be Arc or Mutex holding a value or a nested expression.

In other words a query builder (which is mutable) can now be used inside another query. Of course we can still defer things too, which does give us a lot of flexibility:

Suppose you have a custom struct:

struct GreetingQuery {
    name: String,
}

a struct would generate a custom query with current name if it is part of a template:

impl From<&GreetingQuery> for OwnedExpression {
    fn from(greeting: &GreetingQuery) -> OwnedExpression {
        expr!("Hello {}", greeting.name.clone())
    }
}
impl From<GreetingQuery> for IntoExpressive<OwnedExpression> {
    fn from(greeting: GreetingQuery) -> Self {
        IntoExpressive::nested(OwnedExpression::from(&greeting))
    }
}

Lets construct our struct:

let greeting = Arc::new(Mutex::new(GreetingQuery {
    name: "world".to_string(),
}));

this can now be cloned and shared all throughout the app. Some expressions may rely on it:

let expr = expr!("select {}", &greeting);

now every time expression is executed it will acquire a value from a mutex:

let result1 = db.execute(&expr).await;
// select hello "world"

{
    let mut guard = greeting.lock().unwrap();
    guard.name = "vantage".to_string();
}

let result2 = db.execute(&expr).await;
// select helo "vantage"

The "Expressive" protocol

Romans Malinovskis — Sat, 19 Jul 2025 22:56:32 +0000

Followed the introduction of 3 query builders (SQL, MongoDB and SurrealDB query dialects), I still felt that my architecture is still uncertain. While implementing builders, I wasn't sure if I should re-use existing expression engines or build a new one or if I should go for Owned or Lazy expressions. Also I wasn't sure how expression engines would interact with multiple databases.

My latest PR is a step to resolve all those issues, introducing a clean way to implement Expression Engines with the Expressive protocol (https://github.com/romaninsh/vantage/pull/58)

Implementing custom Expression Engines

My PR contains a "mock" expression engine, but I intend to re-shape OwnedExpression into a proper implementation of expressive protocol. It is possible though to create new expression engines relatively easily.

Implement necessary traits:

impl Expressive<MyExpr> for MyExpr {
  fn into_expressive() -> IntoExpressive<MyExpr>;
  fn expr(..) -> MyExpr;
} 
impl DataSource<MyExpr> for MyDB {
  async fn execute(expr) -> Value;
  fn defer(expr) -> (async Fn->Value);
}

Implement or derive Debug trait
Implement From for IntoExpressive

The standard way for Rust to cast types:

impl<MyExpr> From<Identifier> for IntoExpressive<MyExpr> {
 ...
}

.. allows Builders to use a wide range of types to to shift weight of implementation into expression engine

Once your expression engine is ready - you can use your Expressive type. Define my_expr! macro to simplify your code.

let my_expr = my_expr!("select {}", Identifier::new("foo"));

Nesting Expressions

Expressive protocol requires implementations to allow nested templates and also support closures for deferred values. If a particular database is incapable of using expressions - nested templates can also be converted into deferred values.

Creating and manipulating expressions must always be sync(), but it may carry some closures to be evaluated later. Obviously if the database can do logic server-side we prefer that.

Here are the examples:

// nesting expressions:
let expr = my_expr!("NOT({})", my_expr!("true OR false"));

// deferred closures:
let expr = my_expr!("{} + 5", async { json!(5) } );

Executing Queries

To execute a query, you need a database driver that implements DataSource<MyExpr>. Yeah! you can support multiple expression engines inside the same datasource.

let db = DataBase::new(..);
let result = db.execute(expr).await;

Calling execute() will evaluate deferred closures first, then will run expression and return result.

There is however, one more feature - ability to defer expressions:

let db1 = DataBase::new(..);
let db2 = GeoDB::new(..);

let subquery = my_expr!(
  "select country_id from countries where ip={}", 
  ip
);

let expr = my_expr!(
  "select * from data where country = {}",
  db2.defer(subquery)  
)

let result = db1.execute(expr).await;

In this scenario, we are still working with a single await, however query would be performed across 2 databases. Result of subquery will be automatically inserted into expression which will be executed too.

As you might have noticed - we can even use different implementation of expressive protocol here.

Lazy expressions and snapshots

Previously I implemented OwnedExpression and LazyExpression, but now I can make them interact since they both will be implementing the same protocol.

Builder implementations may decide which expressions engine to use and would still be compatible with everything around them.

Into<> can even allow you to cast between different expression engines if they are compatible or automatically defer if they are not.

Now I can now go back to the SelectQuery builders and polish them next.

Rust Query Builder for SQL an SurrealDB

Romans Malinovskis — Thu, 10 Jul 2025 20:37:32 +0000

SurrealDB is a new multi-modal database, which is in active development. And while I am really impressed by the features - users report performance issues.

I also want to use SurrealDB in my project, but if we do run into performance issues - we better have a way to switch to a different database. A custom query language SurrealQL makes it pretty difficult to perform such a change. I've observed many users trying SurrealDB a year ago and having exactly the same issue.

I continue to refactor my project - Vantage, and today I have a very basic support for Query Building, but now it's done in a vendor-independent way.

Query builders

Query builders that I've mentioned in my previous post typically support only a single query language. In Vantage my expression builder supports various vendors and query language dialects and therefore should be able to easily support transition between PostgreSQL and SurrealDB.

A new implementation of Expression engine (0.3) allows this behaviour and makes it much more intuitive to implement a query builder:

https://github.com/romaninsh/vantage/pull/53

As example, here is implementation of "identifiers". In SQL if your column or table is using reserved keyword, you can use backtick to escape it. While SurrealDB supports backticks, it also supports ⟨mytable⟩. Here is a reference implementation for identifier escaping:

#[derive(Debug, Clone)]
pub struct Identifier {
    identifier: String,
}

impl Identifier {
    pub fn new(identifier: impl Into<String>) -> Self {
        Self {
            identifier: identifier.into(),
        }
    }

    fn needs_escaping(&self) -> bool {
        let reserved_keywords = [
            "DEFINE", "CREATE", "SELECT", "UPDATE",
            "DELETE", "FROM", "WHERE", "SET", "ONLY",
            "TABLE",
        ];

        let upper_identifier = self.identifier.to_uppercase();

        // Check if it contains spaces or is a reserved keyword
        self.identifier.contains(' ') 
          || reserved_keywords.contains(&upper_identifier.as_str())
    }
}

impl Into<OwnedExpression> for Identifier {
    fn into(self) -> OwnedExpression {
        if self.needs_escaping() {
            expr!(format!("⟨{}⟩", self.identifier))
        } else {
            expr!(self.identifier)
        }
    }
}

This can then be intuitively used like this:

expr!(
  "SELECT VALUE {} AS {}",
  Identifier::new(base_expr),
  Identifier::new(alias)
),

Now the render_field() construct can easily rely on identifier as needed:

fn render_fields(&self) -> OwnedExpression {
    if self.fields.is_empty() {
        expr!("*")
    } else {
        let field_expressions: Vec<OwnedExpression> = self
            .fields
            .iter()
            .map(|(alias: &Option<String>, field: &OwnedExpression)| {
                if let Some(alias) = alias {
                    expr!("{} AS {}", field.clone(), Identifier::new(alias.clone()))
                } else {
                    field.clone()
                }
            })
            .collect();
        OwnedExpression::from_vec(field_expressions, ", ")
    }
}

Code for each query-builder is clean and can focus on language features of specific query dialect, rather than string formatting.

Next, I'm going to revisit the concept of LazyExpressions some more and possibility to use cross-vendor queries in expressions seamlessly.

SQL Expressions in Rust

Romans Malinovskis — Tue, 08 Jul 2025 11:03:41 +0000

Hello,

I am currently researching a better way to interface between Rust application and SQL (and non-SQL) servers. In the past I have developed SQL expressions as part of Agile Data.

For my new Rust project (Vantage) I am now looking again at the correct approach to render expressions. Current implementation is lacking async support and this makes it impossible to embed cross-database requests.

First - what is an expression?

At some point in life everyone would write this:

let query = format!(
  "SELECT * FROM product WHERE name = \"{}\"",
  user_name
);

A much better way is to rely on a specialised expression language. Here is SQLx for example:

let row: (i64,) = sqlx::query_as("SELECT $1")
        .bind(150_i64)
        .fetch_one(&pool).await?;

A more sophisticated Expression Engine

For my purposes - conventional SQL expressions are too basic. Some of the features they lack:

Limited number of types
Composable
First class citizen
Not database-agnostic
Not recursive
Must take ownership
No async support

For types - SQL databases operate with JSON or Geo primitives as well as 3rd party crates like Chrono have types for duration, that you may also want to use inside queries. What if there is no built-in support for your custom type.

Composability of Expressions

Composability is a big requirement for me. My query builders are literally operate with dozens of expressions, connecting them together and rendering. When it comes to supporting various types in Expression engine - then expression itself should be suitable:

let my_expr = expr!("select id from ({})", expr!("select * from user"));

Having named parameters also is great: select {total_sum} from orders.

First class citizen

Expression should be First class citizen anywhere else. Query builder such as sea_query implements order in a very rigid way:

let query = Query::select()
    .order_by(Glyph::Image, Order::Desc)

SQL languages can order by expression, so query builder should be able to accept expression, not only a field:

let query = query.order_by(expr!("qty * (price - {})", 20));

Identifiers like `table_name`

Expression engines mostly deal with parametric arguments, but they should do a better job with identifiers also. Those are not passed as parameters, but also require validation and possible escaping:

let expr = expr!("select * from {}", Identifier("users"));

Operations

Diesel creates an interesting trend with implementing operations:

let downloads = version_downloads
  .filter(version_id.eq(any(versions)))

Their implementation relies on static field, but operations could work beautifully as an extension to expressions:

let expr = expr.and(expr!("len(name) < {}", 3);

Database Agnostics and Async

Modern reality is that application often have to work with multiple data sources - databases from various vendors, APIs, local caches, etc.

Expression engine should be capable of bridging the gap. The biggest challenge is that a expression usually is a lightweight construct which has no binding to a specific data source. If we bridge between data sources - there should be a possibility to cross this gap.

Consider this query:

select * from user where country_id in (
  select id from country where is_eu
)

What if a "country" table moves elsewhere or is cached differently?

Executing such a query would be an async operation, however manipulating it is not.

Owned vs Shared ownership

Most backend services live for a single request. Rust language works for desktop applications too. Imagine you have a table with a filter controls on top. Changing filter parameter should modify part of a query to fetch table data, but there is no reason really to re-build entire query.

Therefore there should be a concept that would allow part of an expression to be externally modified.

My Current approach

Previously Vantage had "Expr" and "Expr_Arc", however it had no async support and therefore couldn't handle cross-database queries. The implementation worked really well inside query building logic and even with fields, but I needed a more refined approach.

https://github.com/romaninsh/vantage/pull/52

This is a new expression engine implementation and for now it only tackles:

Composable - lazy expressions can contain other queries
Extensible parameters - even async
Full and Lazy ownership

I will continue with the refactor.

Hello Dev.to

Romans Malinovskis — Mon, 07 Jul 2025 11:07:57 +0000

Hello,

I learned PHP in 1998 and fell in love with the language. Never a fan of projects like WordPress or popular frameworks, I appreciated the language in its pure form. My work led me to releasing Agile Toolkit and creating successful consultancies around it, and even though I decided to move on about ten years ago, Agile Toolkit lives on thanks to the community.

One of the greatest advantages of ATK is the ability to operate with data models in a very unusual yet super-efficient way. Real projects which use ATK significantly reduce the number of database queries simply because those queries are built dynamically inside PHP before they are executed. This makes it possible for UI components (like filters or pagination) to naturally integrate with no additional code lines.

Entity Abstraction

There are presently two views on how applications interact with the SQL databases - either through custom-written SQL queries (SQLx for Rust) or with ORM (Diesel or SeaORM). Each has pros and cons.

But what if there is a third way?

Agile Data (https://github.com/atk4/data) implements an approach of DataSet abstractions - implemented in PHP. While it is similar to ORM in syntax - it operates on an entirely different level, allowing you to dynamically build multi-table queries, abstract away expression logic, and introduce extensions (such as soft-delete). All of that integrates into UI directly) making ATK a "low-code" framework that makes you write very little code.

If you are curious how powerful it is - www.sortmybooks.com is a full-featured online accounting suite (much more powerful than Xero), built entirely in ATK and extensively using Agile Data for UI and reports.

Moving On

A few years ago, I started learning Rust, and while I didn't have much time, I decided to write some backend application in Axum. To this day, I could not find a crate to enable pagination across all my data endpoints universally.

I thought if we want Rust to be more "appropriate" for serious business applications— an Entity Framework is absolutely required.

Rust, however, has these unusual ownership and trait systems. On top of that, the desire to apply types to anything makes it even harder to come up with a "good-looking" data abstraction framework.

My next challenge - Vantage

This seemed like a very interesting challenge, and I began working on Vantage. However, as my understanding of Rust continues to evolve, my project will probably undergo several idiomatic rewrites.

I feel that I need to dump my thoughts and ideas somewhere. It helps me reflect and refine my approach, and if this would also be of interest to anyone, GREAT!

What am I planning next?

Vantage cannot be lightweight, otherwise it will fall as "yet-another-light ORM". There are fundamental targets it must achieve in order to advance forward. Here I wanted just to list some of my design goals.

Expressions - Implement an extensible interface for nested expressions, which can operate across multiple data sources and demonstrate great policy towards async execution.
Persistence Mapping - Create a protocol for persistence mapping. Think of this as "serde", which supports multiple data sources, relationships, conditions, optional fields, expressions and tons of other extensions.
Query Builders - This is a glue between the above two components. Provided with a persistence mapping logic - translate operations (such as reading, adding, deleting) into expressions by fully utilising query languages such as SQL or SurrealDB query language.

There are many benefits that can come out of this - but getting those 3 fundamentals correct and aligning them with Rust coding paradigms with lovely syntactic sugar is essential.

If you want to see my current work-in-progress: https://github.com/romaninsh/vantage, but I also plan to post some updates and reflections or my account here.