Forem: Baptiste Parmantier

Manage your users' actions efficiently thanks to a robust, scalable system

Baptiste Parmantier — Sat, 03 May 2025 07:42:02 +0000

Introduction

When we talk about security in the context of an application, we immediately think of three aspects:

Vulnerabilities (XSS…)
Authentication
Roles/permissions

In this article, we will explore different ways to manage user access and actions within a hypothetical platform designed to offer client/prospect management as well as quote/invoice tracking.

Our application allows SMEs (small and medium enterprises) to record billing information of their prospects or clients to easily reuse it for future quotes and invoices.

Considering that in an SME there may be several users needing access to the platform, we want these users to have different permissions. We can identify the following access levels:

Full admin: for the company owner
Semi-restricted access: for HRs

Our business owner should have full access to all the services offered by the platform.

Our HR accounts should have full access to customers as well as quotes and invoices.

At this stage, we identify two business needs requiring different treatments and introducing the notion of access restrictions.

Different approaches

The previously described constraints illustrate a level of complexity that can grow over time and according to the company’s future needs.

To implement a robust and scalable permission management system, let’s first explore several ways to build a solid solution.

Hardcode

This method involves writing the authorization rules directly into the business logic.

Let's imagine a simple structure to model our case.

pub struct User {
  pub firstname: String,
  pub lastname: String,
  pub is_admin: bool,
  pub is_rh: bool
}

fn can_create_document(user: &User) -> bool {
  user.is_admin || user.is_rh
}

This approach may be suited for prototypes aimed at testing a feature’s feasibility but has many issues and should very rarely be used in production:

No flexibility (a change requires a redeployment)
Often duplicates rules
Hard to test or trace

RBAC (Role-Based Access Control)

In the RBAC model, users are assigned one or more roles (e.g., Admin, HR, Sales). Each role is associated with a predefined set of permissions.

pub struct Role {
  pub id: String,
  pub name: String
}

pub struct User {
  pub firstname: String,
  pub lastname: String,
  pub roles: Vec<Role>
}

fn can_create_document(user: &User) -> bool {
  user.roles.iter().any(|role| {
    role.name == "Admin" || role.name == "RH"
  })
}

In our case, note that the name property may change, so it is better to rely on the role's id.

This practice is interesting because it allows storing a set of roles in a persistent storage system, which we can modify.

Adding a new role requires code changes to account for it in our permission management. Also, be very careful not to delete a role if it's referenced by id, as this would break permissions irreversibly.

However, this practice has the advantage of having a simple model to conceptualize and manage if roles are well-defined.

Drawbacks

As mentioned, the essence of the RBAC model becomes inefficient in some cases:

As the business evolves, the number of roles and business rules increases
Temporary access for one person often requires creating a “temporary” role, which involves code changes

ABAC (Attribute-Based Access Control)

ABAC is a more granular approach, based on evaluating N dynamic attributes depending on your needs and business context. However, we generally see two attributes:

The resource
The action

Let’s consider a set of user actions for our platform:

create: allows creating a new document
update: allows editing a document
delete: allows deleting a document

These actions apply to a given resource—here, document—and we scope our actions as <resource>:<action>.

Here is a Rust example:

pub enum Permission {
  CreateDocument,
  UpdateDocument,
  DeleteDocument,
}

impl Permission {
  pub fn serialize(permission: &str) -> Option<Self> {
    match permission {
      "document:create" => Some(Permission::CreateDocument),
      "document:update" => Some(Permission::UpdateDocument),
      "document:delete" => Some(Permission::DeleteDocument),
      _ => None,
    }
  }
}

fn can_create_document(user: &User) -> bool {
  user.permissions.iter().find_map(|element| {
    if let Some(permission) = Permission::serialize(element) {
      permission == Permission::CreateDocument
    } else {
      false
    }
  })
}

With this action-based approach, we get much finer control over user actions, but it takes longer to implement due to the number of permissions we must declare and manage.

We can simplify the above code like this:

pub enum Permission { ... }

impl Permission {
  pub fn serialize(permission: &str) -> Option<Permission> { ... }

  pub fn has(permissions: &Vec<String>, target: Permission) -> bool {
    permissions.iter().find_map(|element| {
      match Permission::serialize(element) {
        Some(permission) if permission == target => Some(true),
        _ => None,
      }
    }).unwrap_or(false)
  }
}

fn my_policy_handler(user: &User) -> bool {
  Permission::has(&user.permissions, Permission::CreateDocument)
}

This method requires associating users with a large number of permissions, which gives optimal control but requires more setup.

What about roles?

We previously mentioned wanting two roles—Admin and RH—to delegate permissions to users.

We can now consider a hybrid approach combining ABAC and RBAC to leverage their respective strengths: atomic control and ease of use.

The hybrid approach

In this section, we’ll use the following data structures:

pub struct Role {
  pub name: String,
  pub permissions: Vec<String>
}

pub struct User {
  pub username: String,
  pub lastname: String,
  pub roles: Vec<Role>
}

This approach “packages” our atomic actions into roles so that assigning a role automatically grants a set of permissions.

We can then create a function to check for a permission in the user’s roles:

pub enum Permission { ... }
pub struct User { ... }

impl User {
  pub fn has_permission(&self, permission: Permission) -> bool {
    self.roles.iter().any(|role| {
      Permission::has(&role.permissions, permission).is_some()
    });
  }
}

fn my_policy_handler(user: &User) -> bool {
  user.has_permission(Permission::CreateDocument)
}

Looks good, but I still can’t give a permission to one specific user without giving it to everyone with that role!

True—and that’s not a mistake. It’s the next improvement 👀

We’ll now allow exceptions by adding a permissions property to the User model:

pub struct User {
  ...
  pub permissions: Vec<String>
}

Now let’s update User::has_permission(...):

pub enum Permission { ... }
pub struct User { ... }

impl User {
  pub fn has_permission(&self, permission: Permission) -> bool {
    if Permission::has(&self.permissions, permission).is_some() {
      return true;
    }

    self.roles.iter().any(|role| {
      Permission::has(&role.permissions, permission).is_some()
    });
  }
}

The check order matters. We check the user’s individual permissions first to avoid unnecessary role checks if already granted.

Congrats, you’ve built a robust, scalable system to manage user actions properly!

Hybrid improved by bitwise

We’ve now explored several approaches, from the simplest to the most robust, capable of scaling with your company’s business evolution.

Here’s an improvement I find particularly interesting for further simplification.

We can define two notable aspects:

Static: permissions are known in advance and used directly in code
Dynamic: permissions are assigned at runtime through roles or directly on the user

If we use a database, do we really need to store our permissions if they’re static?

Simple answer:

Yes to display the permissions in a UI
No for everything else

Storing data has a cost (write, edit, read, serialize).

In a DB context, the structure might be:

User <-HasMany-> Role <-HasMany-> Permission

Permission checking requires:

Iterating over each user role: $O (R)$
For each role, iterating its permissions: $O (P)$

Total complexity: $O (R \times P)$

Knowing this, we can share permissions statically in code instead of storing them in DB—removing a table and simplifying queries.

The new system’s complexity becomes: $O (R)$

But where did the role permissions go ?

Great question 👀 let me introduce: Bitwise

Bitwise

Before continuing, let’s define what bitwise means:

In computer programming, a bitwise operation operates on a bit string, a bit array or a binary numeral (considered as a bit string) at the level of its individual bits...
— Source Wikipedia

Reducing complexity with bitwise

Let’s reintroduce permission persistence. Using bitwise encoding, we can store up to 32 permissions in a single 32-bit unsigned integer!

Exceeding 32 permissions? Use u64 (64 permissions). Beyond that, use PostgreSQL's BIT VARYING.

Let’s now adapt our code.

First, update the enum to support bitwise format:

pub enum Permission {
  CreateDocument = 1 << 0, // 0001
  UpdateDocument = 1 << 1, // 0010
  DeleteDocument = 1 << 2, // 0100
}

impl Permission {
  // Add a permission (set the bit)
  pub fn add(value: &mut u64, permission: Permission) {
    *value |= permission as u64;
  }

  // Remove a permission (clear the bit)
  pub fn remove(value: &mut u64, permission: Permission) {
    *value &= !(permission as u64);
  }

  // Check if a user has a specific permission
  pub fn has(value: u64, permission: Permission) -> bool {
    (value & (permission as u64)) != 0
  }
}

Now update User and Role models:

pub struct Role { 
  ...
  pub permissions: u64
}

pub struct User { 
  ...
  pub permissions: u64
}

Congratulations, you’ve implemented bitwise permission management! 🚀

But every decision has trade-offs. With bitwise:

You’re limited by your integer type size (u32, u64, …)
You must never change the enum order, or you risk assigning incorrect permissions

Conclusion

There are a multitude of ways of effectively managing your users' actions, but don't forget that each of them has its own constraints and limitations.

The approaches discussed in this article relate to some of the best-known methods, such as the RBAC and ABAC models, but there are others that you can use and which in some cases may be more appropriate to your project.

Simplify your environment variable management

Baptiste Parmantier — Sat, 08 Mar 2025 22:26:20 +0000

Introduction

Environment management is an extremely sensitive but almost systematic issue.

Whether you are working on a web, mobile or backend application, environment variables allow you to configure your application according to the environment in which it is being run (development, production, etc...).

However, managing these variables can quickly become complex, especially when you need to validate their format, type or even presence.

Features

Unified API
A single method lets you access your environment variables, so you don't have to deal with multiple getters to retrieve your variables.
Environment variable validation
Define validation schemes for your environment variables. You can specify the expected data type (string, number, boolean, etc..), as well as additional rules (for example, checking whether a number is an integer or a double).
Error management
The package includes a robust error reporting system. If an environment variable does not comply with the defined rules, the package generates detailed errors, making debugging easier.
Flexibility
Designed to be extensible, you can add custom rules to meet specific needs, such as value transformation or enumeration validation.
Support for .env files
The package supports .env files, allowing you to load environment variables from local configuration files. It also manages the priority of environment-specific files (.env, .env.production...).

Environment management in Dart

Following the example of the Javascript environment, the package introduces an environment variable named DART_ENV which will have the same role as NODE_ENV present in Node applications.

This environment variable can be supplied using your application's startup command.

dart run --define=DART_ENV=development bin/entrypoint.dart

Within your Dart application, you will need to write code similar to this.

void main() {
  final dartEnv = String.fromEnvironment('DART_ENV');
  print(dartEnv); // development;
}

When you want to use a variable from the environment, you will have to pass them via command parameters.

It is your responsibility to type your variables correctly when retrieving them, by changing the primitive used to access your variable.

String.fromEnvironment('STRING_VARIABLE');
bool.fromEnvironment('BOOLEAN_VARIABLE');
int.fromEnvironment('INT_VARIABLE');

An exception is thrown if the primitive type doesn't match the environment variable type prefix.

If an environment variable is not set, it defaults to the null value.

User friendly

While the environment variable has not been retrieved yet by the primitive function call (fromEnvironment), there is no validation of the types, nor if the environment variable exists.

The env_guard package solves this problem by providing a builder that allows you to contractually declare your environment variables, ensuring that they are present (or not) but also ensuring their type.

Let's look at our environment variables in a .env file at the root of our project, or injected directly by Kubernetes or another technology.

HOST=127.0.0.1
PORT=3333
DEBUG=true

To validate our environment, we'll use the define() method, which performs two actions :

Load and parse the environment variables
Validate it and persists the types

import 'package:env_guard/env_guard.dart';

void main() {
  env.define({
    'HOST': env.string(),
    'PORT': env.number().integer(),
    'DEBUG': env.boolean(),
  });
}

In the background, the first step is to load the environment variables from the Platform.environment source (with the option of ignoring them) and/or from an environment file on disk if one exists.

As explained above, the package introduces a new DART_ENV variable which will tell us the environment in which our application will be running.

For example, when our environment is production, the package will automatically look at your project to find the .env.production file. If this does not exist, the .env file will be used instead.

If the DART_ENV variable is not defined, it will be assumed as the development value.

Secondly, we will validate our previously extracted data using our validation schemas in order to obtain an error in the event of non-compliance.

We can now retrieve our variables using our environment key.

final host = env.get('HOST');
print(host); // 127.0.0.1

Existence security

In our previous example, we declared a string as a key in our validation scheme and then used another string to retrieve our value from our environment using the env.get(key) method.

To make the existence of our environment keys contractual, the package provides an abstract class called DefineEnvironment which you can implement in one of your classes.

Consider the following environment.

PORT=8080
HOST=localhost
URI={HOST}:{PORT}

Let's define our environment using the abstract class DefineEnvironment.

final class Env implements DefineEnvironment {
  static final String host = 'HOST';
  static final String port = 'PORT';
  static final String uri = 'URI';

  @override
  final Map<String, EnvSchema> schema = {
    host: env.string().optional(),
    port: env.number().integer(),
    uri: env.string(),
  };
}

We can now use our class to define our environment.

void main() {
  env.defineOf(Env.new);
  expect(env.get(Env.uri), 'localhost:8080');
}

It is important to note that the result of the get method is an dynamic type, however this type is persisted during the validation of your environment variables, so its type is defined before accessing it.

So when you perform a get passing the PORT key, the return type will already be integer thanks to the prior validation process.

final port = env.get('PORT');
print(port.runtimeType); // integer

🚧 Error handling

When your application starts and your environment does not meet the defined requirements by the validator, an EnvGuardException is thrown on the following format.

HOST=127.0.0.1
PORT=8080
LOG_LEVEL=trace

enum MyEnum implements Enumerable<String> {
  info('info'),
  error('error'),
  debug('debug');

  @override
  final String value;

  const MyEnum(this.value);
}

env.define({
  'HOST': env.string(),
  'PORT': env.number().integer(),
  'LOG_LEVEL': env.enumerable(MyEnum.values)
});

Since the value of our LOG_LEVEL key is not included in our enumerable, an error is raised.

{
  "errors": [
    {
      "message": "The value must match one of the expected enum values [info, error, debug]",
      "rule": "enum",
      "key": "LOG_LEVEL"
    }
  ]
}

❤️ Credits

My warmest thanks to Pierre Miniggio for proofreading and helping with the translation of this article.

View on Github View on Dart Pub

Validate your data structures with Vine in your Dart projects

Baptiste Parmantier — Thu, 27 Feb 2025 11:33:36 +0000

Origins

Vine is a data validation library developed by Harminder Virk for the Adonis framework.

He was originally an integral part of the framework but was moved to an agnostic package when version 6 was released.

From the documentation available here.

VineJS is a form data validation library for Node.js. You may use it to validate the HTTP request body in your backend applications.

VineJS is one of the fastest validation libraries in the Node.js ecosystem

It provides both runtime and static type safety

Built for validating form data and JSON payloads

Has first-class support for defining custom error messages and formatting errors

Comes with an extensive suite of 50+ validation rules and 12 schema types

Extensible. You can add custom rules and schema types to VineJS. We also make it super easy to test custom validation rules
import vine from '@vinejs/vine'

const schema = vine.object({
  email: vine.string().email(),
  password: vine
    .string()
    .minLength(8)
    .maxLength(32)
    .confirmed()
})

await vine.validate({ schema, data: {...} })

The validation component is essential if you want to have total control over every piece of data entering your backend application.

Never trust your users

It is mainly used to validate 2 data sources.

The contents of the body in the case of POST, PUT or PATCH methods.
Url parameters such as a search property allowing additional behaviour to an SQL query or pagination (limit, page).

const schema = vine.compile(
  vine.object({
    title: vine.string().minLength(3).maxLength(255),
    status: vine.enum(['draft', 'published'])
    content: vine.string().minLength(8)
  })
)

export default class Controller {
  async store({ request }: HttpContext): Promise<Articles> {
    const data = request.validateUsing(schema)
    return Article.create(data)
  }
}

Dart ecosystem overview

The Dart ecosystem is best known for its Flutter cross-platform framework, but the language can also be used in other contexts, such as for web applications (SPA, SSR, static), microservices (although the quality of the packages associated with classic protocols such as AMQP is debatable) or even command guests...

The notion of data validation is very little present in the ecosystem, in fact there are mainly two big leaders for Flutter (simply the tool provided by the framework or form_validation).
In the context of Dart, we can see validators downloaded 107M times this week, but this package has not been updated for 3 years at the time of writing.

The validators package has no fluent API or more complex validation, it's more of a helpers library.

On a purely personal level, and although I appreciate the technology, I find it extremely simplistic to sum up the Dart language as simply Flutter. The language itself is more than sufficient to build almost any application; it meets a need for performance, high loads and, like Javascript, is extremely versatile (backend, mobile, desktop, web, etc...).

Following this guideline, I wanted to use my last 10 years of experience with the Adonis framework to enrich the Dart language ecosystem.

// Detect dark theme var iframe = document.getElementById('tweet-1891367574052495467-478'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1891367574052495467&theme=dark" }

To this end, I asked the package's original creator if I could reuse the name of his project in order to produce a quality library for all the validation needs of any type of application.

Vine in Dart

Like its namesake, this package enables validation operations to be performed on a more or less complex data structure.

View on Github View on Dart Pub

To date, we support an average of 29,000,000 validation operations per second. Benchmarks are available here.

Schema 101

The design of a validation schema follows a set of rules that allow us to structure our validation as well as possible, while avoiding having to continue going through the rules if one of them is not respected, so we can observe several levels of validation.

Description of the rules in schema 101.

A validation schema systematically begins with a form, which must be an object.

final schema = vine.object({...});

To this base, we can add a first property to be validated, for this example we will define a username.

final schema = vine.object({
  'username': vine.string()
});

Now that our property has been declared, we're going to add some additional validation rules (often in line with your column type constraints when validating data before an SQL operation).

final schema = vine.object({
  'username': vine.string()
    .minLength(3)
    .maxLength(255) // 👈 In according to VARCHAR(255) type
});

We're now going to add an extra property which may or may not be present in the payload. To add this last constraint, we'll use a modifier.

final schema = vine.object({
  'username': vine.string()
    .minLength(3)
    .maxLength(255),
  'age': vine.number().optional()
});

Our current scheme now meets the following tests.

void main() {
  final schema = vine.object({
    'username': vine.string()
      .minLength(3)
      .maxLength(255),
    'age': vine.number().optional()
  });

  test('should be valid', () {
    final payload1 = {'username': 'john'};
    expect(() => vine.validate(payload1, schema), returnsNormally);

    final payload2 = {'username': 'John', 'age': 28};
    expect(() => vine.validate(payload2, schema), returnsNormally);
  });

  test('should be invalid', () {
    final payload = {'username': 'jo'};
    expect(() => vine.validate(payload, schema), throwsA(isA<ValidationException>()));
  });
}

Compiling your schema

In our previous example we designed a simple validation schema, but this is not reusable and must be rewritten for each validation operation.

To avoid having to declare your schema N times, the package allows you to transform your dynamic schema into a static schema using a compilation operation.

final validator = vine.compile(
  vine.object({
    'username': vine.string()
      .minLength(3)
      .maxLength(255),
    'age': vine.number().optional()
  });
);

final data = validator.validate({...});
print(data);

It is preferable to transfer your compiled schema to a file dedicated to validators in order to initialise them at the boot time of your application.

Schema

Examples of how to use the package are available in the test/ folder in the repository.

String schema
Character strings are a complex subject because they can represent a multitude of use cases (emails, phone number, etc...).
The validation rules provided will enable you to refine your constraints.

final schema = vine.object({
  'property': vine.string()
    .minLength(3)
    .maxLength(255)
    .fixedLength(4)
    .email()
    .phone()
    .ipAddress()
    .url()
    .alpha()
    .alphaNumeric()
    .startsWith()
    .endsWith()
    .confirmed()
    .trim()
    .normalizeEmail(lowercase: true)
    .toUpperCase()
    .toLowerCase()
    .toCamelCase()
    .uuid()
    .isCreditCard()
    .sameAs('otherProperty')
    .notSameAs('otherProperty')
    .inList(['firstValue', 'secondValue'])
    .notInList(['thirdValue'])
    .regex(...)
    .requiredIfExist(['otherProperty'])
    .requiredIfAnyExist(['otherProperty'])
    .requiredIfMissing(['otherProperty'])
    .requiredIfAnyMissing(['otherProperty'])
    .nullable()
    .optional()
    .transform((ctx, field) => 'foo');
});

Number schema
Numbers are initially expressed using the num language type. You can refine int or double using one of the available modifiers.

final schema = vine.object({
  'property': number()
    .range([1, 2, 3])
    .min(1)
    .max(3)
    .negative()
    .positive()
    .double()
    .integer()
    .requiredIfExist(['otherProperty'])
    .requiredIfAnyExist(['otherProperty'])
    .requiredIfMissing(['otherProperty'])
    .requiredIfAnyMissing(['otherProperty'])
    .nullable()
    .optional()
    .transform((ctx, field) => 1);
});

Boolean schema
This scheme supports various inputs such as 0, 1, on, off, true or false.
You can apply or remove this behaviour to retain Boolean support only.

final schema = vine.object({
  'property': vine.boolean()
    .requiredIfExist(['otherProperty'])
    .requiredIfAnyExist(['otherProperty'])
    .requiredIfMissing(['otherProperty'])
    .requiredIfAnyMissing(['otherProperty'])
    .nullable()
    .optional()
    .transform((ctx, field) => true);
});

Date schema
Dates are interpreted from a String or directly from a DateTime instance.

final schema = vine.object({
  'property': date()
    .before(DateTime.now())
    .after(DateTime.now())
    .between(
      DateTime.now().subtract(Duration(days: 1)), 
      DateTime.now().add(Duration(days: 1))
    )
    .beforeField('otherProperty')
    .afterField('otherProperty')
    .betweenFields('firstProperty', 'secondProperty')
    .requiredIfExist(['otherProperty'])
    .requiredIfAnyExist(['otherProperty'])
    .requiredIfMissing(['otherProperty'])
    .requiredIfAnyMissing(['otherProperty'])
    .nullable()
    .optional()
    .transform((ctx, field) => DateTime.now());
});

Union schema
Union allows you to assume several input value types without losing control over the expected final type.

final schema = vine.object({
  'property': vine.union([
      vine.string(),
      vine.number()
    ])
    .requiredIfExist(['otherProperty'])
    .requiredIfAnyExist(['otherProperty'])
    .requiredIfMissing(['otherProperty'])
    .requiredIfAnyMissing(['otherProperty'])
    .nullable()
    .optional()
    .transform((ctx, field) => 'foo');
});

Array schema
This type of schema is used to control the contents of an array of elements by forcing a specific type of data.

final schema = vine.object({
  'property': vine.array(vine.string())
    .requiredIfExist(['otherProperty'])
    .requiredIfAnyExist(['otherProperty'])
    .requiredIfMissing(['otherProperty'])
    .requiredIfAnyMissing(['otherProperty'])
    .nullable()
    .optional()
    .transform((ctx, field) => []);
});

Object schema
The basis of any validation scheme, the object is used to validate complex structures, even nested ones.

final schema = vine.object({
  'property': object({...})
    .requiredIfExist(['otherProperty'])
    .requiredIfAnyExist(['otherProperty'])
    .requiredIfMissing(['otherProperty'])
    .requiredIfAnyMissing(['otherProperty'])
    .nullable()
    .optional()
    .transform((ctx, field) => 'foo');
});

Enum schema
This scheme enables stricter validation of values by making them contractual to a strict definition.

enum MyEnum implements VineEnumerable<String> {
  first('first'),
  second('second');

  final String value;
  const MyEnum(this.value);
}

final schema = vine.object({
  'property': vine.enumerate(MyEnum.values)
    .requiredIfExist(['otherProperty'])
    .requiredIfAnyExist(['otherProperty'])
    .requiredIfMissing(['otherProperty'])
    .requiredIfAnyMissing(['otherProperty'])
    .nullable()
    .optional()
    .transform((ctx, field) => MyEnum.first.value);
});

Any schema
This type of scheme allows any input value, but we recommend that you use vine.union([...]).

final schema = vine.object({
  'property': vine.any()
    .requiredIfExist(['otherProperty'])
    .requiredIfAnyExist(['otherProperty'])
    .requiredIfMissing(['otherProperty'])
    .requiredIfAnyMissing(['otherProperty'])
    .nullable()
    .optional()
    .transform((ctx, field) => 'foo');
});

OpenApi

To make your validation schemas universal, the package implements a reporter which you can use to export to the OpenApi specification.

final schema = vine.object({
  'stringField': vine.string().minLength(3).maxLength(20),
  'emailField': vine.string().email(),
  'numberField': vine.number().min(18).max(100),
  'booleanField': vine.boolean(),
  'enumField': vine.enumerate(MyEnum.values),
  'arrayField': vine.array(vine.string()).minLength(1),
  'unionField': vine.union([
    vine.string().minLength(3).maxLength(20),
    vine.number().min(10).max(20),
  ])
});

final report = vine.openApi.report(schemas: {'MySchema': schema}); 
print(reporter);

Acknowledgements

I would like to thank Harminder Virk for all his work, which has enabled me to produce a clean, efficient and user friendly package, but above all to use the name and identity of the creation.

HMR, the revolution in development experience

Baptiste Parmantier — Tue, 04 Feb 2025 10:18:56 +0000

Introduction

Hot Module Replacement (HMR) is an essential mechanism for modern developers, as it allows code to be modified in real time without interrupting the workflow.

Its impact can be measured in two major ways:

Faster iterations: changes are instantly visible, reducing validation cycles.
Improved developer experience: no more interruptions due to manual reloads.

In the Dart ecosystem, outside Flutter, traditional HMR is rare. Our implementation offers a pragmatic alternative, designed to meet specific needs while maintaining radical simplicity.

Philosophy

Our solution is based on three founding principles:

Voluntary State Reset

Unlike traditional HMR systems, which preserve state in memory, our approach completely restarts the execution context whenever a change is made.

This choice, although counter-intuitive, offers key advantages:

Predictable behaviour: the application always starts with a clean initial state, eliminating the risks associated with corrupted variables or accumulated side effects.
Compatibility with stateless architectures Ideal for HTTP servers, CLI tools or ephemeral scripts where the external state (database, files) takes precedence.

Total isolation

Each reload relies on Dart isolates, independent units of execution:

Greater stability: a failing isolate does not affect the main process.
No dependency on the Dart VM: The code runs identically in JIT, AOT or as a native executable.
Optimum memory management: isolates are destroyed cleanly after use, avoiding memory leaks.

Minimalism and Performance

With just 6 KB and no external dependencies, the solution stands out for :

Virtually instantaneous reload time (<10 ms): thanks to optimised incremental compilation and lightweight isolate management.
Seamless integration: No complex configuration is required beyond adding the package via pub add.
Universal compatibility: Works with any Dart project, whether it's a backend, a CLI tool or a data processing script.

Backend first

Let's imagine an HTTP server (or any other application). During the development phase, you have to implement features (services, controllers, routes, etc.) for which you have to restart your application several times in order to obtain a rendering that can be tested by Postman or any other similar tool.

It is also important to note that when your application is restarted, it loses all its states in order to re-establish an initial state. This process is in line with the notion of statelessness found in cloud natives.

The time saved is significant, while the development experience is impeccable thanks to a turnkey approach that offers a preset that can be used without any specific configuration.

However, it is important to note that if the default behaviour of the HMR when used in a ‘global’ way is not appropriate, you can design the configuration of your HMR yourself by defining your own behaviour.

Installation

The hmr package can be used in two different ways:

Installable CLI command
As a dependency

Command line

In the first case, you will need to install the executable using the following command :

$ dart pub global activate hmr

All that's left is to run the command in your terminal

$ cd /path/to/your/project
$ hmr

Although ready to use, you can also alter the basic configuration by adding specifications to the pubspec.yaml file

hmr:
  # Change the location of the input file
  entrypoint: bin/main.dart

  # Delay before recompilation in milliseconds
  # If not specified, the recompilation 
  # will be immediate and you don't have any debounce
  debounce: 5

  # Only include files that meet the following criteria
  includes:
    - '**/*.txt'
    - '**/*.dart'

  # Exclude files meeting the following criteria
  excludes:
    - '.dart_tool/**'

Build your own

You can create your own hmr by exploiting the two main components of the package Watcher and Runner.

Watcher: manages file monitoring and all the business rules applied before events are sent (create, update, delete).
Runner: manages isolates and project compilation.

import 'package:hmr/hmr.dart';

void main() {  
  final runner = Runner(
    tempDirectory: Directory.systemTemp,
    entrypoint: File(
      path.join([
        Directory.current.path,
        'bin',
        'main.dart'
      ])
    ));

  final watcher = Watcher(
    onStart: () => print('Watching for changes...'),
    middlewares: [
      IgnoreMiddleware(['~', '.dart_tool', '.git', '.idea', '.vscode']),
      IncludeMiddleware([Glob("**.dart")]),
      DebounceMiddleware(Duration(milliseconds: 5), dateTime),
    ],
    onFileChange: (int eventType, File file) async {
      final action = switch (eventType) {
        FileSystemEvent.create => 'created',
        FileSystemEvent.modify => 'modified',
        FileSystemEvent.delete => 'deleted',
        FileSystemEvent.move => 'moved',
        _ => 'changed'
      };

      print('File $action ${file.path}');
      await runner.reload();
    });

  watcher.watch();
  runner.run();
}

How it works

From change to reload
It all starts with a change to the source code by the developer. This action triggers a cascade of automated steps:

The Watcher, constantly monitoring the file system, instantly detects the change (creation, modification or deletion).

This detection is a non-blocking event that allows the watcher to continue monitoring other files while the reload process is underway.

Example:

A fix is made to a calculation function.
The Watcher identifies the file backup and transmits the event to the Runner in less than 2 ms.

Incremental compilation
Once notified, the Runner orchestrates the targeted compilation:

The modified code is compiled into a .dill file, Dart's intermediate binary format, which will be deposited in a temporary folder so as not to pollute the source project.

Only the modules affected by the change are recompiled, a process optimized to take less than 5 ms even on large projects.

Isolate Management
The heart of the approach lies in Dart isolates, lightweight, isolated execution environments:

The previous instance is killed immediately, freeing up memory and preventing leaks.
The new code (via .dill) is loaded into a blank context, guaranteeing a predictable initial state.

If the new code contains a critical error (e.g. infinite loop), only the isolate is affected. The main process remains stable, allowing subsequent reloading.

Continuous loop
The cycle repeats itself with each modification, creating an immediate feedback loop:

The developer saves the file.
The Watcher detects the change.
The Runner recompiles and relaunches the isolate.
the result is visible without any noticeable delay.

This loop is non-blocking, so that when an isolate is active, we can trigger a new reload if necessary.

Key architectural benefits

Risk Isolation
Isolates function like sandboxes, preventing propagation errors.
No shared dependencies: each instance is autonomous.

Universal compatibility
The .dill format is compatible with all Dart compilation modes (JIT, AOT, exe).

No modifications are required for production deployment.

This diagram illustrates the elegant simplicity of your implementation:

Watcher → Responsive vigil.
Runner → Efficient conductor.
Isolates → Disposable and reliable workers.

By combining these elements, you get a reloading system that's fast, stable and tailored to the demands of modern Dart applications, while avoiding the complexity of traditional solutions.

"A mechanic doesn't have to be complicated to be powerful." 🔄

Conclusion

Our solution is not a simple imitation of existing tools, but a reinterpretation of the real needs of Dart developers. It shines through :

⚡️ Its speed : almost instantaneous reloads, even on large projects.
🐥 Its lightness : A minimalist package, with no superfluous dependencies.
🚀 Its production-first philosophy : identical behaviour in development and deployment.

By deliberately targeting backend and CLI use cases, it fills a gap in the Dart ecosystem while remaining true to the spirit of the language: simple, high-performance, and predictable.

Links

Github : https://github.com/LeadcodeDev/hmr
Dart pub : https://pub.dev/packages/hmr

Discover the new data structure adopted for the next framework release

Baptiste Parmantier — Sat, 18 Jan 2025 09:37:56 +0000

Data structure in Mineral

Baptiste Parmantier for Mineral Foundation ・ Jan 18

#dart #discord #opensource

Data structure in Mineral

Baptiste Parmantier — Sat, 18 Jan 2025 09:07:02 +0000

Introduction

The data structure is at the heart of every IT project, playing a decisive role from the earliest design stages.

It defines the way in which information is organised, stored and manipulated, directly influencing the application's performance, maintainability and scalability.

So, in accordance with the Discord.js library, we chose to load all the data available for a single Discord server when accessed from any model.

Context

Basically, we wanted Mineral to be able to offer data models ready for immediate use by the end developer.

So, in accordance with the Discord.js library, we chose to load all the data available for a single Discord server from any model.

This choice implies that each time we receive an event from the Discord websocket client, we have to serialise the data that is the subject of our event, as well as everything that is accessible to it, whatever the level of depth.

As each structure had to be able to be traced back to its parent (and vice versa), we soon found ourselves having to serialise a huge amount of data for a simple structure.

An example may seem more concrete to you.

Issues

Let's say we receive the MessageCreate event from Discord's websocket client, we need to write our handler in this way.

client.events.server.messageCreate((ServerMessage message) {
  // Our code
});

Serialisation is costly

Following this premise, we need to allow access to our Discord server from our message model, to do this we chain together something like message.channel.server.

Behind the simplicity of this chain, there are a number of implications

The message must contain a complete structure of the Channel into which it was sent.
The Channel must be able to make accessible the complete structure of the Server in which it evolves.
The Server model embeds a whole set of properties including its own channels, members, roles... which, for each of them, must also make it possible to trace back to itself...

And so on...

All these structures, nested in a multitude of levels, generate enormous computational complexity and have a significant impact on the cost of serialising each data structure.

Our main concern is the difficulty of serialisation with which even the smallest data structure will be confronted.

So we increase the computational complexity, again and again.

It is important to note that a complex data structure is transmitted to the client even if it does not require any level of depth.

client.events.server.messageCreate((ServerMessage message) {
  await message.reply(content: 'Hello World');
});

In this case, even though we only need the instance of our message, at no point do we use a message.channel.server chain; and yet the server is indeed serialized and supplied to the client.

Cache dependency

Our philosophy is not to impose anything on developers.
One of our promises is not to impose the use of any caching solution.

Behind this promise, we want the framework to be able to be offered as a stable and viable solution for any project.

In this section, we will use the Discord.js library as a reference for comparison.

This library imposes the use of an in-memory caching solution, which allows it to offer complex data structures such as those described above, allowing developers to access almost any property or action available on the Discord server.

The choice of imposing a caching solution actually hides a deeper problem that introduces excessive consumption of the memory used by the application.
Conversely, not using a memory cache in the Discord.js library makes it impossible to deliver these complex data structures.

It is possible to envisage a system that introduces a notion of lifespan or life cycle of the data, but it is up to the developer to work this out.

Solution adopted

In order to overcome all the problems explained above, we decided to drastically reduce the size of our data structures to make them as minimalist and atomic as possible.

This choice has enabled us to...

reduce the dependency of each data structure on the others
reduce the size of our data structures
reduce serialisation time
reduce the overall complexity of using a resource
eliminate potential errors linked to the absence of properties required for complete serialisation of the complex data structure
eliminate dependency on a caching solution

Impact on DX

It is important to note that these profound changes will result in a modification of the development experience for the end user.

As each delivered structure is smaller, the developer will have to explicitly request the retrieval of certain resources required for his business context.

An example of this is the retrieval of a server from a Channel using a newly introduced function resolveServer().

We selected the term resolve in a very specific case where our data model contains every piece of information needed to construct an HTTP request (usually to retrieve a resource) without any additional parameters being requested from the developer.

We can now see a new way of accessing our data.

client.events.server.messageCreate((ServerMessage message) async {
  if (message.authorIsBot) {
    return;
  }

  final (channel, author) = await (
    message.resolveChannel(),
    message.resolveMember()
  ).wait;

  final str = 'Hello from ${author.username} in ${channel.name}';
  await message.reply(content: str);
});

Caching

We have already stated our desire to allow the end developer to dispense with any caching solution, but make no mistake, it is still extremely interesting and important within your applications.

Using it will drastically reduce the execution time of your code while reducing the number of requests to the Discord API, thereby reducing the risk of a rate-limit.

The current procedure is as follows.

There are 2 possible uses for it:

Without cache

When no caching solution is used, the Datastore will make a direct HTTP request to the Discord API to obtain the result, then serialise it and send it back to the consumer.

With Cache

In this case, there are two possible scenarios.

The cache has the data
The datastore contacts the caching solution, then retrieves the result, serialises it and sends it to the consumer.

The cache does not have the data
In this case, the Datastore will make an HTTP request to the Discord API to retrieve the result. The result is then pushed into the cache, serialized and sent back to the consumer.

Credits
We would like to thank Lexedia and Abitofevrything from the Nyxx team for the discussions and advice that led to this result.

See more in the documentation