Forem: Artur Ampilogov

How to Judge Solutions Like an Engineer

Artur Ampilogov — Mon, 26 Jan 2026 23:45:53 +0000

When we face a problem, often there are several solutions. Which one is just good enough, and which one is ideal?

Ideal solution criteria 🎯

As a solution indicator test, I often use a powerful idea from the Theory of Inventive Problem Solving (TRIZ).

Ideal final result is achieved when the contradiction disappears through a change in approach, or when the function works without the initial system, or when the object (system) automatically removes the contradiction. No system deterioration. No compromises.

Example: Airbus 330 sidestick function ✈️

The drama happened in 2009, when Air France Flight 447 was en route from Brazil to France.

During mid-flight, ice crystals began to accumulate in the pitot tubes, affecting the speed measurement. The autopilot turned itself off. The pilots took manual control.

Historically, to maneuver an airplane, US and Russian manufacturers use control yokes (left picture), while the European concern Airbus uses sidesticks (right picture).

During the pilot's manual control, the copilot abruptly pulled back on his side stick without notice, raising the plane's nose. After some time, the wings lost lift, and the aircraft began to stall, loosing the hight. To overcome the stall and gain speed, the pilot followed the routine steps: pushed forward the side stick to temporarily lower the nose.

The dual sidesticks on Airbus planes work as follows. They are physically independent, and the program calculates the result. There are basically three conditions of a sidestick related to horizon tilt: neutral gives full control to the other side stick, forward, and backward. When both sidesticks are not in the neutral position, their result is some up. This can be presented as a simple function.

function getStickLevel(left: number, right: number) {
   // when left is neutral
   if (left == 0) { 
     return right; // full control to the right
   }
   // when right is neutral
   else if (right == 0) {
     return left; // full control to the left
   }
   // otherwise, average the levels
   else {
     return (left + right) / 2;
   }

Problem 🤔

The copilot accidentally pulled back their sidestick, cancelling the pilot's sidestick's forward level. There were system warnings about both sticks being in use at the same time, but during a stressful situation, the pilots' attention was elsewhere. The recording showed the copilot speaking, "I don't have control of the aircraft anymore now," and "But I've been at maximum nose-up for a while!" when the copilot realized the issue. Ultimately, the airplane crashed into the ocean.

What is the contradiction from the program perspective? The function must sum any given non-zero numbers and not return 0. The mathematical module cannot be used. This is impractical when you have input values, say, -1 and +1.

Solution 💡

Let's move the problem on the physical level out of code. The copilot’s control input must exist so they can instantly assist, and must not exist so it does not interfere invisibly.

One solution is widely used in US and Russian airplanes. The sticks are physically connected and are in sync. You move the left stick, and the right stick immediately mirrors those movements.

That upper-level solution essentially eliminated the problem. Moreover, the best part is zero code. There is no need for a computer controller.

Example: Abu Dhabi speed cameras 📸

Placing many speed limit cameras on a long highway requires a large budget.

Problem 🤔

The contradiction can be stated as follows. A highway is long and requires many expensive cameras. To measure a car's speed at any point, there should be a camera, and at the same time, there should not be a camera.

Solution 1 💡

Abu Dhabi's government experiments with average car speed. The first camera records the time of highway entrance, and the second records the time of exit. The distance is known, so the average car speed can be calculated. If it exceeds the limit, the vehicle definitely sped up too much at least once along the route. You can also drive very fast, half the route, but then you will have to drive the second part slowly. The solution drastically reduces the number of cameras.

Solution 2 💡

In clear weather, from a height of 30 meters (100 feet), the horizon is visible up to 20 km. To locate a phone, cell towers use triangulation. Knowing three radii to the phone, the circle intersections will reveal the exact position.

Today, almost every highway works with transponders in cars. A transponder can be extended with a minimal phone SIM card functionality, so that cell providers can track car speeds. Highways already have cell towers around. Transponders are widely used.

This solution completely eliminates cameras, and the speed limit system continues to function.

Example: ML learning (AI) 🧠

Today, Machine Learning approximates a function that maps millions of given inputs to their expected outputs. Input can be an image, and output can be text describing the image object. This process is called "labelling" and requires an enormous number of examples for the model to create a very good approximation for almost any unseen image. Historically, developers created huge databases with input -> correct answer examples created by hand. Each image is manually highlighted with polygons showing the exact object positions. Just elephant images could require tens of thousands of examples. This is a trunk, this is a leg, this is an ear, and as a whole, this is an elephant.

Such learning, with carefully curated examples of inputs and expected outputs, is called supervised learning.

Problem 🤔

An AI model should learn patterns for object relationships. If we see an elephant trunk, there is likely an elephant head and a body with legs. It is not related to a bird or a table.

From one point of view, there should be a polygon for each image object; from another, there should not be a polygon. Contradiction.

Solution 💡

The following modern approach, called self-supervised learning, does not completely eliminate human labelling but drastically reduces it.

At the first stage, each image is randomly divided into blocks, for example, 3x3, and some blocks are hidden. An AI model trains to guess the hidden parts. By doing so, the model learns patterns between the objects. Like an elephant should have a trunk. It learns relationships on its own without a human manually highlighting all the objects in the image.

The model does not know how elephant objects are called in different languages. The second part is to show prepared, labeled images. This time, we need much less image labelling, as the AI model has already built object relationship patterns.

Note about TRIZ 🧬

Altshuller, author of TRIZ, was a famous inventor who reviewed 40 000 engineering patents and developed algorithms to resolve technical contradictions. The algorithms have been used by research centers worldwide, including companies such as NASA, Boeing, Siemens, GE, Ford, Intel, Samsung, and Apple.

The genius of Altshuller is that he built the invention machine in the physical area.

The given examples do not comply with TRIZ, and there was no intent to do so.

First, TRIZ was created to resolve physical problems. To state the right contradiction, you should have a very good experience with it, and also specify what system quality you want to improve, and what the constraints are.

Second, there are strict algorithms and a list of known physical and chemical approaches to apply to each specific type of contradiction and system quality improvement. The solution is not known, but you likely be able to find the optimal one. Instead, I picked examples of already-known solutions, which is totally counter to TRIZ.

There are many attempts to apply TRIZ in business, the arts, public relations, and even IT. None of them has been endorsed by the TRIZ community without dissent. One reason is that such work requires enormous intellectual resources and perseverance, which is hard to find nowadays. For example, Vikentyev, also a former Alshuller student, analyzes 4-8 kg of scientific books daily (!) for the research database on creativity. Also, there is no guarantee that a similar algorithm exists in a non-physical area.

Conclusion

The concept, when the initial problem disappears while the system continues to function, is very powerful. Also, during programming, I often found that the ideal solution had no code and that systems were improved by other means, such as redefining business tasks or switching to a different service platform.

Resources

Air France Flight 447, Wikipedia
Average car speed in Abu Dhabi
TRIZ site, supported by Altshuller's family and friends. As a bonus, fiction fans can find the list of all fiction ideas of all time.
Introduction to TRIZ. Basic concepts and approaches, v 3.0, the original e-book.
Vikent.ru by Vikentyev I.L., contains the largest world research database about creativity.
Effective TypeScript: 83 Specific Ways to Improve Your TypeScript, 2nd edition, 2024, by Dan Vanderkam, where I initially met the Air France Flight 447 example.
Artificial Intelligence: A Modern Approach, 4th Global edition, 2022, by Stuart Russell and Peter Norvig. The classic book on AI.
European software patent examples by Bardehle Pagenberg.

Concurrency and API Protection - Part 3

Artur Ampilogov — Thu, 16 Oct 2025 06:41:41 +0000

This article is part of a series that explores practical patterns for protecting your systems.

Concurrency and Row Versioning - Part 1
Concurrency and Transactions - Part 2
Concurrency and API Protection - Part 3 ⭐

❓ Concurrent requests and external services

Occasionally, applications encounter broken state errors that are difficult to debug, especially when the issue affects only a small subset of resources. Investigation often reveals that the root cause is a UI that fails to disable the submit button, allowing users to double-click and send multiple requests to the same REST API.

In the previous series, we explored how to protect against such cases using our internal services. Today, modern applications almost always rely on third-party services, with in-house teams focusing primarily on core functionality. That approach enhances both security and development efficiency, but it also introduces a new challenge.

For example, Perci Health uses a famous service for Fast Healthcare Interoperability Resources (FHIR). Transactions are very limited, and conditional row versions are not supported. During testing, we found that a nurse can double-click on a submit button to update an appointment. This could lead to a broken state: while only one appointment encounter should exist, two or three resources appear with incorrect statuses.

A similar case was found during the contract work at Checkatrade. Multi-clicking on the Salesforce button led to two or three modification requests with a slight sub-second difference.

To eliminate or at least reduce such issues, we can move resource protection from the DB to the upper level—the REST API. This idea was initially implemented at Checkatrade with database transactions. This article shows the improved transactionless version.

🚫 No out-of-the-box solutions

At the time of writing, none of the major cloud providers offer a built-in solution to protect REST APIs from concurrent modification requests. Services like Amazon API Gateway, Google Cloud API Gateway, and Azure API Management Gateway provide only basic rate-limiting capabilities.

💡 REST API protection idea

Here is the idea of API protection against concurrent requests.

Consider only modification methods: POST, PUT, PATCH, and DELETE. Do not protect GET requests.
Protect APIs requiring authentication, so we do not touch /auth/sign-in and other public methods.
If the request path has a parameter, then it is the exact resource modification, and we must protect it, for example, DELETE "/appointments/:appointmentId". Then, if there is no parameter and a user is authenticated, we also need to protect the API, and we do it by adding the user ID prefix, so PUT /me becomes PUT /:userId/me, POST /appointments becomes POST /:userId/appointmetns. Otherwise, it is a public API without a parameter, such as POST /auth/sign-in, and there is no need for protection.
Group API parametrized paths per single parent resource, for example, all the following modification requests
- PUT /appointments/100
- POST /appointments/100/end-call
- DELETE /appointments/100
have one parent resource: "/appointments/100".
Mark in a database that the request for a special resource is in process.
If another resource tries to own the modification, return the HTTP 409 Conflict status code (or HTTP 423 Locked). A UI client can fetch fresh data and notify a user what to do with a conflict.
The owner completes the request and unlocks the modification resource.

🎯 REST API protection with Firestore DB

Here, we consider a NodeJS ExpressJS example with Firestore DB. A similar one can be built with almost any REST API framework, like NodeJS Fastify, ASP.NET, Python Jango/FastAPI, Java Vert.x/Spring Boot, Go Gin/Fiber, etc. Any database that supports conditional updates with row versioning, transactions, or a lock mechanism can be used: key-value RedisDB or KeyDB, document-oriented Fistore, Supabase, or MongoDB, a modern SQL database, such as Postgres, MS SQL, Oracle, MySQL, etc.

Now, let us write our Express and Firestore implementation step by step in TypeScript.

Introduce the acquire lock function returning a type result and an optional release lock function. The type result can be skipped - lock is not applicable, locked - lock was successfully applied, conflict - not used due to the resource conflict.
```
  const acquireLock = async (req: Request, res: Response): 
    Promise<{ type: 'skipped' } | 
            { type: 'locked'; release: () => Promise<void> } |
            { type: 'conflict' } > => {

    // implementation
  };
```

Create an ExpressJS middleware that can be reused with multiple APIs

import { Request, Response, NextFunction } from 'express';

export const lockMiddleware = async (
    req: Request,
    res: Response,
    next: NextFunction,
  ) => {        
  const lockResult = await acquireLock(req, res);

  if (lockResult.type === 'conflict') {
      // send HTTP 409 Conflict or 423 Locked code
      res.sendStatus(409); 
      return;
  }

  try {
    next(); // API request handler        
  } finally {
    if (lockResult.type === 'locked') {
      await lockResult.release();
    }
  }
};

Protect only for modification methods.

const PROTECTED_METHODS = new Set(['POST', 'PUT', 'PATCH', 'DELETE']);

if (!PROTECTED_METHODS.has(req.method)) {
  // Not a modification, no need to lock
  return { type: 'skipped' };
}

Assume there is an auth middleware. It is out of the scope of this article to fully implement it, for example, by inspecting the Authorization: Bearer <token> header. One practice is to store the authentication results in the res.locals object. We assume that if res.locals.userId is not empty, the user is authenticated.

Ignore public endpoints, such as /auth/sign-in.

if (!res.locals.userId) {
  // Public endpoint, no need to lock
  return { type: "skipped" };
}

Create two helper functions. First, canonicalize the path so that the action endpoints /resource/:id/action and lock on /resource/:id and /me on /:userId/me.

const canonicalizePath = (
  path: string,
  params: Record<string, string>,
  userId: string,
): string => {
  // clean up the request path from query parameters,
  // so /appointments/1/cancel?abc=1&def=some_value becomes /appointments/1/cancel
  path = path.split('?')[0];
  path = path.length === 0 ? '/' : path;

  // remove double slashes and trim trailing slash (except root)
  path = path.replace(/\/+/g, '/');
  if (path.length > 1 && path.endsWith('/')) {
    path = path.slice(0, -1);
  }

  // If any segment matches a param, keep only the collection path + id
  // e.g. /resource/:id/subresource/:subId/action becomes /resource/:id
  // e.g. /prefix/resource/:id/action becomes /prefix/resource/:id
  const segments = path.split('/').filter(Boolean);
  if (segments.length >= 2) {
    // Iterate through segments; stop when a param value appears
    for (let i = 0; i < segments.length; i++) {
      const segment = segments[i];
      const isParamSegment = Object.values(params).includes(segment);

      if (isParamSegment && i > 0) {
        // Keep prefix up to this param
        const normalized = `/${segments.slice(0, i + 1).join('/')}`;
        return normalized;
      }
    }
  }

  // No params matched in path.
  // Prefix with userId to avoid clashes between different users
  // e.g. /learn/articles/saved becomes /:userId/learn/articles/saved
  // e.g. POST /appointments becomes /:userId/appointments
  return `/${userId}${path}`;
};

The path can be too long for a Firebase collection or have unsupported symbols, so we can hash it.

import crypto from 'crypto';

const buildLockId = (canonicalPath: string) => {
  // path may contain special symbols or be too long, hash it
  return crypto.createHash('sha256').update(canonicalPath).digest('hex');
};

Our acquireLock function becomes

const acquireLock = async (
  req: Request,
  res: Response
): Promise<
  | { type: "skipped" }
  | { type: "locked"; release: () => Promise<void> }
  | { type: "conflict" }
> => {
  if (!PROTECTED_METHODS.has(req.method)) {
    // Not a modification, no need to lock
    return { type: "skipped" };
  }

  if (!res.locals.userId) {
    // Public endpoint, no need to lock
    return { type: "skipped" };
  }

  const canonicalPath = canonicalizePath(
    req.path,
    res.params,
    res.locals.userId
  );
  const lockId = buildLockId(canonicalPath);

  // FOLLOWING IMPLEMENTATION
}

For brevity, the following code snippets will be part of the // FOLLOWING IMPLEMENTATION body section.

A cloud provider can unexpectedly destroy a running application instance for many reasons. Our lock cannot be released and gets stuck forever. The first thing to do is check whether the request is stale and, if so, delete the stuck lock.

// How long a single request may hold the lock before it’s considered stale
const LOCK_TTL_SECONDS = 5;

const db = getFirestore();
const lockRef = db.collection('request-locks').doc(lockId);

const staleLockDoc = await lockRef.get();
if (
    staleLockDoc.updateTime &&
    staleLockDoc.updateTime.toMillis() + LOCK_TTL_SECONDS * 1000 <
      Timestamp.now().toMillis()
) {
  try {
    await lockRef.delete({
      lastUpdateTime: staleLockDoc.updateTime,
    });
  } catch {
    // ignore delete error
  }
}

The lastUpdateTime precondition prevents the accidental removal of a concurrently updated version, which is covered in Concurrency and Row Versioning—Part 1.

The next step is to check for ongoing processing and mark the conflict.

const existingLockDoc = await lockRef.get();
if (existingLockDoc.exists) {
  // Someone else holds a fresh lock
  return { type: 'conflict' };
}

Now it's time to insert a row lock indicator. If another processor did it before us, the row insert will fail, indicating the conflict.

let createdLockDoc: FirebaseFirestore.DocumentSnapshot<
  FirebaseFirestore.DocumentData,
  FirebaseFirestore.DocumentData
>;
try {
  await lockRef.create({
    method: req.method,
    path: req.path,
    canonicalPath,
  });

  createdLockDoc = await lockRef.get();
} catch (error) {
  const firebaseError = error as { code?: string | number; message?: string };
  // error code `6` - the document already exists
  if (firebaseError.code === 6) {
    // Someone else created a lock
    return { type: 'conflict' };
  } else {
    throw error; // rethrow unexpected error
  }
}

The last function part is to return the release lock function.

const release = async () => {
  const logger = new Logger(`${appName}:lockMiddleware`);
  try {
    // Only delete if we still own it (avoid nuking a refreshed lock)
    await createdLockDoc.ref.delete({
      lastUpdateTime: createdLockDoc.updateTime,
    });
  } catch {
    // skip release error
  }
};

return { type: 'locked', release };

To use the lock mechanism in multiple requests, apply the lockMiddleware.

  import { default as express } from 'express';
  import { lockMiddleware } from '../path-to/lockMiddleware';
  import { yourRouterOne } from '../path-to/yourRouterOne';

  const api = express();
  api.use(lockMiddleware); // applied to all routers below
  api.use(yourRouterOne);
  // rest of the routers

Full middleware version

The complete middleware example can be found at lockMiddleware.ts.

Testing

To test concurrent requests, run curl commands almost simultaneously, for example, a POST request.

seq 20 | xargs -n1 -P5 curl -X POST -s -o /dev/null -w "%{http_code} %{errormsg}\n" \
--data-raw '{ "id": "abcde" }' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <TOKEN>' \
https://your-api-path

🚀 Result

With this protection in place, concurrent API requests no longer cause invalid resource states. The best part is that it achieves this without Firestore transactions, keeping it "blazingly" fast.

Concurrency and Transactions - Part 2

Artur Ampilogov — Mon, 13 Oct 2025 21:22:29 +0000

This is a part of a series that explores practical patterns to protect your system:

💭 Why is a transaction needed?

In some cases, you need to modify several resources within a single operation. A transaction ensures that these updates are treated as one atomic unit — either all changes succeed together, or none are applied (they are rolled back).

SQL transaction

Most modern SQL servers support transactions.

BEGIN;

# Step 1 - read
SELECT * FROM users where id = 42
SELECT * FROM subscriptions where user_id = 42

# Step 2 - your logic

# Step 3 - modify resources
UPDATE users
  SET name = 'Albert', version = version + 1
  WHERE id = 42 AND version = 7;

UPDATE subscriptions
  SET plan = 'pro', version = version + 1
  WHERE id = $SUBSCRIPTION_ID AND version = 3;

COMMIT;

The user row modification will be automatically rolled back if the subscription update fails, and vice versa.

🪶 Is SQL transaction heavy?

A decade ago, SQL transactions would immediately downgrade the server's performance. The highest isolation level, SERIALIZABLE, provides consistent checks for concurrent reads and writes but adds locks on rows and tables. Table locks are cumbersome, especially for READ locks, when other concurrent fetch queries should wait until the table is unlocked.

Today, SQL servers represent the art of engineering and have gained huge performance in all aspects.

First, databases improved their algorithms to use a snapshot of data per transaction based on the "Multiversion Concurrency Control (MVCC)" idea introduced in 1981 by Philip A. Bernstein and Nathan Goodman. All queries and modifications run against the database snapshot inside the transaction (requires DB snapshot isolation setting). Concurrent requests do not introduce changed records, and no locks are required.

Second, in 2008, Cahill, Röhm, and Fekete published a revolutionary idea in the article "Serializable isolation for snapshot databases.". It is possible to build an execution graph of all concurrent transactions. If there is a cycle in the graph, there is a resource conflict, and transactions should be cancelled except for one. The beauty is that there is no need for heavy locks.

Moreover, the algorithm provides complete Serializable Snapshot Isolation (SSI). For example, one transaction calculates the user's credit balance and then wants to write a fact of a 50 USD item purchase. Meanwhile, the second transaction adds a new ledger record of -1000 USD in the middle, making the balance and new payment unavailable. For decades, this kind of scenario was almost impossible to track in databases. A new record was added to or deleted from the table you recently queried in a transaction.

BEGIN;

# Step 1
SELECT SUM(value) as balance FROM ledger_entries
 where user_id = 42;

# Concurrent Step 2 by another transaction
# INSERT INTO ledger_entries VALUES(-1000, 'USD');

# Step 3 - add purchase 
# Does `balance` have the right value?
IF balance >= 50 THEN
  INSERT INTO ledger_entries VALUES(-50, 'USD');
  # INSERT INTO purchased ...
END IF;

COMMIT;

This is no longer the issue with modern SQL servers implementing Cahill, Röhm, and Fekete's algorithm. With special settings, the database tracks whether the table query result SELECT SUM(value) as balance FROM ledger_entries where user_id = 42 has been changed by another transaction during the execution. It rolls it back if needed without any lock mechanism.

⚡ Firestore DB transaction

The SQL algorithm performs efficiently on a single server. However, it does not scale effectively in highly distributed databases like Firestore.

Firestore supports transaction isolation, but only at some level. It cannot handle the new row anomaly example. This is still a fascinating fact because document databases were created without transactions in mind: store unstructured data per document and support distributed storage.

Let us review a Firestore transaction example.

await db.runTransaction(async t => {
    // Step 1 - read
    const userRef = db.collection('users').doc('42');
    const userDoc = await t.get(userRef);

    const subscriptionRef = db.collection('subscriptions').doc(userDoc.data.subscriptionId);
    const subscriptionDoc = await t.get(subscriptionRef);

    const ledgerEntries = await db.collection('ledger').where('status', '==', 'success').get();

    // Step 2
    // your logic

    // Step 3
    await t.update(userRef, {
      name: 'Albert'
    });
    await t.update(subscriptionRef, {
      plan: 'pro'
    });   
});

First, Firestore works against a snapshot per transaction, implementing the MVCC principle.

Second, it automatically tracks all fetched document timestamps (versions) against concurrent changes. Similar to what we did in Concurrency and Row Versioning - Part 1 with lastUpdateTime, it just works out of the box, so no need to specify it explicitly. In this example, we initially fetched userDoc, subscriptionRef, and ledgerEntries. Firestore will check if another update has changed them concurrently during the transaction application. This is also true for all fetched ledgerEntries; even though we do not modify them in the transaction. Firetore, on the other hand, as mentioned, cannot track transactionally new inserts or deletes.

There is a main limitation for a Firestore transaction code: all fetch operations should go before modifications. The following is incorrect:

await db.runTransaction(async t => {
    // ❌ wrong, modification should go at the end of Firestore transactions
    await t.update(subscriptionRef, {
      plan: 'pro'
    }

    // ❌ wrong, read operations should go before modifications
    const doc = await db.collection('users').doc('42').get();
});

🏋️‍♂️ Does Firestore use locks in transactions?

Firebase has two common scenarios in mind.

The first occurs when users update the Firestore DB directly from devices via mobile and web applications. Users are expected to likely update only their data and rarely have concurrent conflicts. For mobile clients, Firestore uses a lockless optimistic concurrency approach. All fetched row versions are tracked for changes. Locking documents in a high-latency environment would cause too many data contention failures.

Another scenario is when a backend updates the database. In this case, transaction conflicts can occur very often. It is unwise to make a lot of retries with row versioning. The standard mode (default) uses a pessimistic concurrency approach with read document locks.

When a transaction locks a document, it prevents other transactions, batched writes, and regular (non-transactional) writes from modifying that document. Any concurrent write operations must wait in a queue until the transaction releases its lock. If we do not update the same document too frequently, this approach drastically eliminates the number of transaction retries compared to optimistic concurrency.

A transaction is a powerful concept that should be used carefully.
In the following article, we consider a solution for REST APIs to protect third-party service states against concurrent requests.

Concurrency and Row Versioning - Part 1

Artur Ampilogov — Sun, 12 Oct 2025 20:17:02 +0000

Throughout my career, I’ve occasionally encountered a particularly tricky issue: debugging a broken application state for only a few resources. It often occurs when a user double-clicks a button, unintentionally sending multiple modification requests to the same resource. In other cases, two users might update the same record almost simultaneously.

Modern APIs are designed for scale, but when multiple clients modify the same data concurrently, you inevitably face the classic problem of concurrent updates.

These series will explore practical patterns to protect your system:

Concurrency and Row Versioning - Part 1 ⭐
Concurrency and Transactions - Part 2
Concurrency and API Protection - Part 3

Database row versioning (optimistic concurrency)

Each row in a database table that needs protection carries a version value. When you update a database row, you check that the version you read is current and that no one has modified it.

All modern databases include a timestamp or a rowversion column type out of the box: Postgres, MS SQL, Oracle DB, MySQL. Moreover, timestamps are also supported natively in document databases, such as MongoDB, GCP Firestore, and Supabase.

💡 Row versioning in steps

Here are the steps with an SQL example to protect a row against concurrent updates.

Read record data, including the row version

SELECT version as OLD_VERSION FROM users WHERE ID=1

Implement the required logic to prepare the updated data.

Update the record only if the version has not changed.

UPDATE users
SET firstname = 'Edgar', lastname = 'Codd', version = $NEW_VERSION 
WHERE id = 1 AND version = $OLD_VERSION;

If another concurrent request updated the same row during step 2, then step 3 will fail. The database will reject it due to the condition conflict WHERE version = $OLD_VERSION.

This technique, also known as Optimistic Concurrency, has the main benefit of not requiring a database resource lock and being very light. I remember developers actively using this principle in SQL databases 15 years ago.

⚡ Firestore DB row versioning

Firestore DB, an example of a NoSQL database, supports row versioning using the special metadata.

// Step 1 - read
const userRef = db.collection('users').doc('1');
const doc = await userRef.get();
const oldVersion = doc.updateTime; // <-- get version metadata

// Step 2 - your logic

// Step 3 - conditional update
await userRef.update({
  firstname: 'Edgar',
  lastname: 'Codd'
},
{ 
  lastUpdateTime: oldVersion // <-- conditional update
});

The important things here are to recognize:

Firestore automatically sets unique updateTime metadata during a document update, so we do not set it manually.
Firestore allows conditional update via the special lastUpdateTime precondition check, which can be read as: WHERE lastUpdateTime = $oldVersion.

🌍 Broad Compatibility

Custom row versions can be used even if a database does not natively support timestamp or rowversion types. The only requirement is a conditional update.

Timestamp can be stored as an ISO string. That is how Amazon DynamoDB manages to handle it.

For optimization and fewer bits storage, it is also common to use a small integer version, incrementing it by one during a row update: version = version + 1.

Is DeepSeek’s Influence Overblown?

Artur Ampilogov — Fri, 31 Jan 2025 13:02:17 +0000

At the beginning of this week, the tech stock market crashed after the appearance of a new AI model called "DeepSeek." Nvidia's stock (NVDA) fell by 17%.

According to the official paper, DeepSeek took only $5.6 mln to train with impressive results. This is a remarkable achievement for a large language model (LLM). In comparison, OpenAI's CEO Sam Altman admitted that training OpenAI GPT-4 took over $100 mln, not saying how much more.
Some AI specialists assume that the estimation of the DeepSeek training expense is underreported. Nevertheless, the hidden gem is not how much it cost to train but how drastically it improved runtime requirements.

Modern LLMs present a complex structure, but the number of parameters is an essential metric for all deep-learning models.

What are AI model parameters?

Consider the video of a snow leopard wearing a red coat on a fashion show I generated using Pika 2.1

How does AI know what "red" is?

Imagine creating a small neural network to say whether a given pixel is red. For programs, it is usual to present an arbitrary color with three mixed values: red, green, and blue, also known as RGB, where each value ranges from 0 to 255. For example, (0,0,0) is black, (0,204,102) is green, (204, 0, 0) is red.

For output, we will use the formula:

Input 1 (red) * Weight 1 + 
Input 2 (green) * Weight 2 + 
Input 3 (blue) * Weight 3 = Output

The weights are unknown. The output should be 0 to 100, where 100 means it is definitely red, 80 is likely red, and 0 is not red at all. During training, the neural network will be passed a million examples with the expected result. The model should figure out the constant weight values that return successful results for at least 95% of the training data.

Then, the output neuron can be connected further with the following neurons, becoming an input with the pixel "redness" notion.

Imagine having millions of input values for such a network, for example, representing text characters in a big PDF file, pixels of a large image, or a user prompt. A trained neural network with numerous connections between neurons and the determined weights, representing the strength of a signal, can calculate various results: understand the prompt requirements, generate a photo or text, derive a logical response, or modify a video.

The actual formula is slightly more sophisticated. In addition to weights, bias variables are added to tilt the output, and the output is finally normalized to a fraction between 0 and 1.

The derived constant weights and bias values are the model parameters. They represent actual AI "brain" patterns.

How big are modern LLMs?

Open AI GPT-3, created in 2020, has 175 billion parameters. GPT-4 was never disclosed but is estimated to have 1.8 trillion parameters, with multiple models inside, about 220 billion each.
Google PaLM 2, released in 2023, allegedly has 340 billion parameters, while the modern Google Flex models might be much more significant.
Meta Llama 3.2 is an open-sourced model with 90 billion parameters, but its results are not as good as those of contemporary Open AI and Google models.

Most LLMs store each parameter as a floating-point number of 2 or 4 Bytes. The full GPT-4 model with an estimated 1.8 trillion parameters requires at least 3.6TB RAM to be able to run! It is the minimum theoretical requirement; additional memory is also needed for input parameters, output storage at each neuron, and some technical space.

Just think of it. The model requires a cluster of expensive GPUs with lots of RAM not only to train it but also just to produce a response at runtime.

Calculating massive amounts of neuron outcomes is very slow, often with many steps running repeatedly over and over. GPT-4 output speed is about 30 words per second, and the GPT-o1 reasoning model is even slower. So, when you send a request to ChatGPT, it uses a massive GPU cluster behind the scenes to process only one input. Imagine how many clusters Open AI, Google, Microsoft, xAI, Anthropic, and other AI leaders have to own or lease to process requests globally.

What is about DeepSeek?

DeepSeek made three noteworthy transformations in the AI world:

A parameter of 1 Byte reduces the main memory part requirement by half while still producing exceptional results.
The model has 671 billion parameters, but only 37 billion, 1/18th of the whole network, are used during processing. The network decides which path and neuron groups to take, resulting in faster responses. Think of a real-life analog: whether a doctor, pilot, or physical scientist is better to answer the question, and not all of them at once.
Being fully open-sourced.

With 671 billion parameters, DeepSeek still requires a lot of memory, more than 700 GB of RAM, to run, but it is the first open-source model to produce outcomes almost as good as the most expensive, sophisticated, and private LLMs. Some people started experimenting with local DeepSeek hosting, for example, by grouping Mac Minis into a home cluster.

// Detect dark theme var iframe = document.getElementById('tweet-1872444906851229814-195'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1872444906851229814&theme=dark" }

Conclusion

Hearing about the claimed lower model training cost, did people correctly produce the selling panic in the stock market, thinking there would be no need for so many GPU chips and clusters? Not at all.

First, now that enthusiasts and businesses know running a local and private ChatGPT analog is possible, they will likely order new and small Nvidia clusters created specifically for these needs.

Second, more sophisticated patterns require more model parameters, which, as shown above, demand enormous computer resources. A human brain has about 100 trillion connections between neurons, while the most significant AI models now operate with only a couple of trillion. Future models will require much more RAM to operate. Also, individuals and businesses are increasingly embracing AI benefits.

It is not going to change, and the AI chip market is only going to grow.

React is not about performance

Artur Ampilogov — Wed, 31 Jan 2024 20:57:35 +0000

Key Takeaways

React is the most popular JavaScript rendering framework.
React full-stack frameworks, like Next.JS and Remix, are recommended for usage by the React team.
React full-stack frameworks cannot work without JavaScript which violates REST principles and affects devices performance.
HTML duplicated data appears when React Server Components are in use.
By default React adds extra loading to user devices with unnecessary re-rendering.
Optimizing React code is a manual procedure and that is error prone.

Facebook presented React to the public in 2013.

The project immediately gained popularity, simplifying web application development with the next core ideas:

UI components are written in the JSX syntax, which is similar to the conventional HTML.
Usual JavaScript statements (if, switch, &&, etc) and expressions can be used inside UI components declaration. A developer does not need to learn new attributes, like "ngIf" in Angular, "v-if" in Vue.JS, "{#if} {:else if …}" in HandleBars and Svelte.
The data flow implements the Flux architecture and is unidirectional: changes are populated from top components to their children.

After more than 10 years, React still has the highest number of downloads among JavaScript web libraries. It is the most discussed web framework in StackOverflow and is used in 3.5% of all websites.

There is, though, a negative point. All this time, React websites produce overhead on client devices.

This is still true, even after introducing such optimizations as improving the React engine speed and page interactions with the Fiber algorithm, reducing JavaScript file sizes, code splitting, and implementation of islands using component streaming and selective hydration.

Moreover, the recent optimizations were done with high competition pressure from other projects (Angular, Svelte, VueJS, Marko), and especially from those either supporting JSX syntax or optimizing the work with the React library (Preact, Qwik, Remix, Astro).

Let me describe React performance issues from an end-user perspective.

JavaScript dependency

Representational State Transfer, or REST for short, states that Code-on-Demand (JavaScript) should be optional in web pages and applications. This concept is important for many devices working without JavaScript or having low processing power, as well as for search engine website crawlers.

React always supported rendering JSX components to HTML via the renderToString method. So, it is possible to render a React page on the server and return a response to the devices with a lack of JavaScript. Then, it is feasible to add interactivity to the client side supporting JavaScript with the “render” method (now separated into “renderRoot” and “hydrateRoot”). The problem with that approach is that the whole flow implementation was solely put on developers. This led to the point where many programmers skipped the backend HTML generation and built pure Client JavaScript React applications using the recommended crate-react-app package.

When JavaScript was disabled in a browser, users usually saw a blank page.The reason is that the React NPM package has always been just a library, not a full framework. Only 10 years after the React release, the team added the recommendation to the official documentation to use one of the frameworks with React Server Side Rendering support, like Next.JS or Remix.

Though it might look like the JavaScript dependency problem is finally resolved in the development community, this is not the case at the time of writing. React recently introduced the ability to stream the page content from the server to a client’s browser in parts, and it requires JavaScript to work.

The added streaming optimization resolves the next problem. Some components require data to be fetched from external resources like a database, and it delays rendering. The solution, introduced in MarkoJS in 2014, is to separate “immediately” rendered components, with static data, from “suspended” components waiting for dynamic data from external resources. The page layout with “immediate” components will be streamed to the browser, while placeholders (usually with spinners or skeletons) will be preserved for “suspended” components.

<main>
  <aside>
    <!--Sidebar -->
   </aside>

  <!-- Suspense PLACEHOLDER-->
  <div id="post_placeholder">
    <!-- Spinner -->
  </div>
</main>

Once a “suspended” component is ready, it is streamed back to the client in the same page request (long HTTP connection).

How is it possible to post-stream and update HTML in the middle of the page without JavaScript? Currently, it is not supported. React sends HTML code for the suspended component with a small script code to find the placeholder and replace it:

<!-- New component -->
<article hidden id="post">
  <h1>Post header</h1>
  <p>Post text</p>
</article>

<!-- Script to replace the placeholder with the new component -->
<script>
  const laterStreamedComponent = document.getElementById('post');
  document.getElementById('post_placeholder').replaceChildren(laterStreamedComponent);
</script>

Once all suspended components are sent, the connection with the client can be terminated.

The idea looks good, but what happens when a browser without JavaScript support requests such a page? The infinite spinner or a placeholder skeleton will be displayed for the suspended components. This is a reality for the latest Next.JS version at the time of writing (v14.0.3). Here is the Suspense demo deployed to Vercel (source code).

This is the default behavior of the current Next.JS implementation, where all components are server components by default.

It will be better to care about devices with a lack of JavaScript support. For example, during almost every big presentation, Rich Harris, the creator of Svelte, shows how Svelte will work without JavaScript.

Suspense also cannot improve slow server responses. The Vercel site, which is built on its main product - Next.JS, often shows bad user experience with slow data fetching.

// Detect dark theme var iframe = document.getElementById('tweet-1734615967630536746-429'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1734615967630536746&theme=dark" }

Web page size

For many years React code by default was bundled into a huge delivery package resulting in a thick web client application. When a user opened any page, the whole JavaScript application was downloaded, and then based on the URL React rendered a specific branch (page). Often these applications took several megabytes to download before rendering could start.

In 2018, the React team added in-box support for code splitting with the lazy function, which developers should manually call.

This approach was also used in routing frameworks like react-router-dom, created by Remix authors, and allowed an application to be split into smaller bundles based on the visited URL.
Currently, Remix and Next.JS create per-page bundles automatically at compile time.

You might think that Server Side Rendering should totally eliminate client bundle problems, but that is not the case with React. The architecture with Virtual DOM and unidirectional rendering flow requires React to recalculate the changes from the top changed component down to all its leaves. To render a component, all the related source code should be presented either as a JavaScript function or with the latest addition of Server Components, in a special JSON format.

It is not possible to reuse any HTML code as the source of truth, which leads to data duplication. First, the rendered HTML page is delivered to a browser, and second, the same duplicated content is delivered as part of the JavaScript block (a function or a JSON). The second step, called hydration, will allow the framework to react to user interaction events and render changes in response.

In the second demo, the Poem component consists purely of static data. There is a client component that wraps and toggles the poem visibility:

<ClientComponentWrapper>
    <Poem />
</ClientComponentWrapper>

Please notice that the poem appears twice in the delivered content to a browser: HTML and JavaScript.

At the same time, Next.JS was smart enough not to include static parent HTML in the JSON script. This React limitation led to the official Next.JS recommendation To move Client Components Down the Tree.

The suggestion does not work once you have the common scenario with the React context client component wrapping all other page elements, for example, with a top-level user sign-in state:

<UserSignInContext>  <!-- Top level client component -->
   <Navigation /> <!-- depends on the user sign in status -->
   <MainContent /> <!-- depends on the user sign in status -->
</UserSignInContext>

There are better-optimized solutions.

Miško Hevery, creator of Angular and Qwik, re-uses HTML data in Qwik as much as possible during client components rendering. This is achieved by a light JSON storage with references to the existing HTML elements. No data duplication. The same concept is used in the Fresh framework based on Preact and Deno.

<!--qv q:id=0 -->
    <p>Hello Qwik</p>
<!--/qv-->

Also, Qwik automatically divides components into the smallest possible chunks, and no JavaScript client component is executed until a user interacts with some part that requires it.

Default unoptimized behaviors

Once React receives a change in a top component, it will re-execute all child components, even those with unchanged properties. All local variables of the components will be recreated, including functions. The Tree Rerender demo shows what happens with the table function components once an unrelated input field is changed (source code).

The reality is that by default React burns users’ CPUs.

The proposed solution by React is to manually wrap all related components into the memo function, local functions into “useCallback” function-hook, and local state data in “useMemo” function-hook. This procedure is tedious and error-prone.

The alternative to React hooks is the Signals technology, which precisely executes only the changed dependent components and provides automatic memoization. SolidJS, Preact, Qwik, Svelte, and Angular use Signals. While the React team has reasons not to switch to it, the problem of the unnecessary CPU load of user devices is still here. The issue might be resolved this year with the React Forget compiler, 11 years later after the React release.

The current situation is that React forces developers to do the next manual optimizations:

Split code manually by the usage of lazy function and Suspense wrappers.
Accurately operate with “memo”, “useMemo”, “useCallback” functions.
Manually specify which component is the client one by adding use client directive.
Use “startTransitiom” and “useDeferredValue” to help React faster update the UI.

In comparison, Qwik does the best possible automatic optimization for all previously mentioned cases out of the box.

There are additional optimizations that reduce JavaScript execution. Qwik eliminates the Hydration step with the Resumability. Svelte makes direct DOM updates and does not use Virtual DOM structure, drastically reducing CPU and Memory usage. Astro does not load JavaScript until it is specified by a developer.

Conclusion

Today, React is the most popular JavaScript web framework. At the same time, its recommended usage produces many performance drawbacks on user devices. It will be great to see from React the CPU and memory usage reduction of hundreds of millions, or even billions, user devices worldwide as a priority.

Programming language, don't make me think

Artur Ampilogov — Fri, 15 Sep 2023 18:47:45 +0000

Steve Krug published the famous book "Don't Make Me Think" in 2000 and gathered well-known ideas about Web usability. Authors of programming languages often lower the priority of simplicity or have to implement far-from-ideal solutions due to historical restrictions.

How to measure complexity?

Maintaining a product requires more reading code than writing. Hence, the code legibility is more important than its writing speed.

One way to measure code complexity is to count how many terms a developer should hold in memory to get a result.

Another one is how much time it requires to understand the notion or to revise it.

Data structures and operations

In short, programming is all about operations on data. To allocate memory for a 32-bit integer in statically typed languages, we can declare its type and assign a name to the memory block.

// C, C++, Java, C#
int x;

// Ada
X : Integer;

// TypeScript
let x: number;

// Rust
let x: i32;

// Go
var x int32

There is a way to infer a type by a compiler or even calculate the memory size and data structure at runtime. This approach is used in JavaScript, Python, and Ruby.

// JavaScript
let x;

// Python
x

The untyped version requires additional time for a developer to figure out the argument structures. Consider the function declared in Python:

def display(person, age, payment)

Is person an object, is age a number, is payment represented by a boolean, number, or money?

Compare the same function written in C# that immediately gives answers about the structures:

void Display(string person, int age, decimal payment)

A statically typed language usage also has a vital benefit over a dynamic language - more error checks at compilation time. It is better to find errors at the earliest stage than make them to be discovered by users.

Assignment operator

Most modern programming languages implement the assignment operator as the equal = sign.

int x;
x = 1;

A fresh programming student trying to understand these statements may ask: "Is it possible to write y = a*x^2 + b*x +c in code similar to the Math notation?" Although students have been taught for many years that = sign represents equation in Math, the answer is "no" for most of the programming languages.

In C, C++, Java, C#, PHP, Python, Ruby, Go, and Rust, the equal sign = means an assignment. Pascal and Ada languages use the := sign, which at least is not a common Math symbol in schools (in Math, it is used as the equal by definition sign, in addition to ≡).

But what does the assignment actually mean? The assignment operator in modern languages has two ideas behind it.

Value assignment

The first one is called value assignment and usually uses deep copy of memory blocks.

// C, C++, Java, C#
int x = 1; 
int y = x; // Deep copy of x to y
x = 2; // Change x
y; // y has the old value 1

Reference assignment

The second one is reference assignment, which always uses the shared block of memory between variables.

// C#
int[] array1 = { 1, 2, 3 }; // Array of 3 integers

int[] array2 = array1; // Reference assignment

array1[0] = 9; // Change the first element in array1
array1; // { 9, 2, 3 }
array2; // { 9, 2, 3 }

Changes in array1 immediately affects array2 as both variables point to the same memory location.

Java and C# oblige developers to remember what assignment behavior is associated with a structure. Moreover, C++ and C# allow to override the = operator per object, and Java can achieve the same result by overriding equals method. It is only possible to be sure about the result of the equality statement in these languages once the type implementation is reviewed.

As we just saw, C# uses a memory reference approach for array assignment to reduce memory allocation. Go language, instead, uses deep copy for arrays.

// Go
var array1 [3]int // Array of length 3
array1 = [3]int{ 1, 2, 3 } // Fill the array with numbers

var array2 [3]int // Array of length 3
array2 = array1 // Deep copy from array1 to array2

array1[0] = 9 // Change the first element in array1
array1; // { 9, 2, 3 }
array2; // { 1, 2, 3 }

A tiny declaration change by removing array size in brackets (var array1 []int and var array2 []int) leads to array slicing in Go. The result will be the opposite and the same as with C# reference array assignment.
Go language forces programmers to pay attention to a number in square brackets.

Ruby authors consider all data structures as objects. Even primitive types, like integers, are objects, so everything is linked via a memory reference by default. The result for arrays will be similar to C# and Go slices. Some might expect the same behavior for Ruby integers, but instead, it will be a usual deep copy:

// Ruby
a = 1
b = a # Deep copy instead of referencing
a = 3
puts a # Holds 3
puts b # Holds the original value 1

Passing an argument

Language authors made three options to path an argument to a function.

The first one is a pass by reference so that you can reassign a new value to the outer argument inside the function.
The second is called pass by value, and there are two options. One is to copy a reference to the memory address, you will not be able to reassign the value for an outer argument but will be able to modify the internals of the shared object. Another one is to make a deep copy of an argument so the changes inside the function will not affect the changes of a passed argument.

Passing an argument does not fully correlate with the variable assignment = behavior. Languages presented new symbols to differentiate the operation: ref keyword. & and * signs. Many programmers find the latest two very confusing, for example:

if(*(struct foo**)deque_get(deq, i) != NULL &&
    (*(struct foo**)deque_get(deq, i))->foo >= 1) {
}

Even for experienced engineers, it isn't easy to read.

One may argue that large statements with many terms are relevant only to old languages, like C. Consider the number of terms for a property declaration in modern C#:

[ReadOnly(true)]
internal required override string Prop { get; init; }

Method calls

Modern frameworks widely use function arguments. Here is a common approach to start a Web server in Go:

http.ListenAndServe(":3000", nil)

By reading this code you may guess that the first argument is a port number, but what about the second? Only after inspecting the function you can find out that it is a special controller, and if you pass nil a default one will be used.

A common JavaScript example:

let obj = {a: { b: { c: 1 }}}
JSON.stringify(obj, null, 2)

Can a programmer know what null represents there and 2 without looking at the specification? (Answer: null is passed as an empty replacer, 2 is indentation size)

You might create a function similar to the next one, which also raises questions about the arguments:

func("John", true, null, 1)

Some IDEs, like JetBrains family, immediately show the names of the arguments, but programmers like to use other IDEs as well.
Certain languages allow to rewrite the code with named arguments and even rearrange them:

func(id: 1, name:"John", approved: true, email: null)

That approach is optional and many developers do not use it. The better solution for readability would be to oblige structural usage in all function calls, for example:

func({
  id: 1,
  name: "John", 
  approved: true,
  email: null
})

File search and jumps

Modern software development and modern frameworks compel to create complex project structures.

Try to count how many times you need to jump between files to understand the logic implementation. Are those files far away from each other and in different packages? Do you need to jump to a file to figure out some data structure, or can it be read immediately? Is it possible to declare types in the same file at the top location to simplify the reading?

If you do not write much code it is possible to view live programming streams and check how many context switches happens during file jumps.

Complexity

I can provide many more examples, like breaking the reading flow in Go with = and := operators, hidden importance of = sign and hashCode overrides in Java and C#, JavaScript call, apply and bind function calls, etc. The idea stays the same - how many terms a person needs to find, hold in memory to get the result, and how much time it requires to revise a term.

Conclusion

Popular programming languages made or applied revolutionary changes at some time. C language authors drastically simplified code writing and application support in comparison to programming in assembler. The default compilation result in C is still called a.out or assembly output. Java simplified working with memory via garbage collection. C# simplified Java statements and introduced Language Integrated Query. Go appeared as a simple alternative to C++ for network and console applications. Rust raised the bar of application stability and security. JavaScript, Python, and Ruby have low learning barriers.
Language authors are restricted by previous versions and styles. Nevertheless, it will be great to see new language versions or new programming languages, reducing reading complexity as a major factor.

The feature missed in C++, Java, C#, PHP, Python, Ruby and JavaScript

Artur Ampilogov — Sun, 10 Sep 2023 19:32:35 +0000

Real objects and notions in our lives have some constraints. The height of a person cannot be a negative number, the same as age. An infant in international airlines is a baby under 2 years old. A TCP/IP port on any device cannot be a number larger than 65535.

Programming languages, like C, C++, Java, C#, PHP, Python, Ruby, and JavaScript, typically would imply the usage of a wide range of built-in types for such purposes: integer, float, and decimal. In this article, I will review an old language feature that can improve type declaration for the mentioned cases.

Range constraints

Temperature

The minimum temperature in Celsius degrees has a constant value equal to -273.15.

In 1972, in the book "Structured Programming", three well-known computer scientists mentioned an interesting approach to working with such constraints:

All structured data must in the last analysis be built up from unstructured components, belonging to a primitive or unstructured types. (...)
Although these primitive types are theoretically adequate for all purposes, here are strong practical reasons for encouraging a programmer to define his own unstructured types, both to clarify his intentions about the potential range of values of a variable, and the interpretation of each such value; and to permit subsequent design of an efficient representation.
(...) Such a type is said to be an enumeration, and we suggest a standard notation for the name of the type and associating a name with each of its alternative values.
type suit = (club, diamond, heart, spade);
(...)
type year = 1900 .. 1960;
type coordinate = 0 .. 1023;
(...) We therefore introduce the convention that a .. b stands for the inline range of values between a and b inclusive. This is known as subrange of the type to which a and b belong, (...)

Ole-Johan Dahl, Edsger W. Dijkstra, C.A.R. Hoare, Structured Programming, A.P.I.C. Studies in Data Processing, No. 8, 1972, p.97

Pascal, Modula-2, Delphi, and Ada support custom unstructured (primitive) types. For example, in Ada the temperature type can be declared as:

subtype TemperatureCelsius is Float range -273.15 .. Float'Last;

where Float'Last is the maximum Float number.

PassengersNumber

Airline businesses may allow to purchase of tickets for a traveling group having a maximum of 10 persons in one transaction. Instead of using an integer type, it is more suitable to specify a custom type in Ada with compile and runtime checks:

type PassengersNumber is range 1 .. 10;

Email address

An email address string input may have a constraint to be required and less or equal to 320 characters in length. In Ada, it is possible to define a new type for that purpose as:

type EmailAddress is String (1 .. 320);

It also makes code more readable when variables do not have full names:

String cc;
String body;
// vs
EmailAddress cc;
EmailBody body;

Type predicate

For more complex logic requiring a function check, for example, declaring a type for the standard HTTP client errors codes (4XX) it is possible to use a static predicate:

subtype HTTPClientErrorCode is Integer with
  Static_Predicate => 
    HTTPClientErrorCode in 400 .. 418 | 421 .. 429 | 431 | 451;

Strong typing

One more important concept is lack of implicit conversion between types, also known as strong typing. For example, adding a Float value to a TemperatureCelsius value in Ada will result in an error.

The missed feature

Many modern languages after decades of improvements still do not support such an important feature, it is possible only to mimic the behavour: C++, Python, Java. It will be great for many developers to be able to create custom primitive types in an easy way.

Evolution of C# (videos)

Artur Ampilogov — Wed, 06 Sep 2023 19:11:46 +0000

#1 Intro

#2 Types

#3 Instances

#4 Strings

#5 Propeties

#5 Methods

#6 LINQ and ranges

#8 Pattern matching

#9 Nullability

#10 More

The state of modern Web development and perspectives on improvements

Artur Ampilogov — Thu, 24 Aug 2023 15:19:39 +0000

Key Takeaways

WWW is built on top of REST principles.
Modern Web development violates REST principles.
Pure HTML is not suitable for modern web development.
HTML can be improved with declarative fetch requests, DOM block replacements, and loading code on demand.
WASM can replace JavaScript in browsers when it provides more functionality to the runtime.

Website development has seen multiple approaches during the 23 years of the World Wide Web's (WWW) existence. In this article, I review the principles of WWW, the current state, and the problems of modern Web development. Ultimately, I will present multiple proposals to improve Web development considerably.

Resource representation

Standardizing World Wide Web

In 1988 a remarkable event happened in the World. Engineers connected European networks with the North American continent via Internet Protocol (IP). The Internet immediately received growth in independent interconnected networks, mostly between scientific institutes.

Tim Berners-Lee, a British scientist at CERN, saw high demand for communication standardization and, in 1989, invented the World Wide Web (WWW). The work included the standard for HTML as markup language to present the information, HTTP protocol to send and receive data, and the Web client to communicate with servers and process the markup language.

Roy Thomas Fielding was working on the HTTP protocol improvements and, in 2000, gathered WWW and HTTP ideas in his famous dissertation Architectural Styles and the Design of Network-based Software Architectures. Chapter 5 contains the essential principles under the “Representational State Transfer” name, or REST for short, which includes the following points:

It is the client-server architecture where web clients (browsers, robots) communicate with servers.
The communication must be stateless in nature. Every request from a client should contain all the required information to process the request. This constraint allow to build reliable and scalable systems.
To improve network efficiency, any request can be cached.
Components should have a uniform interface. For example, to use HTTP for communication, even though other transfer protocols might work faster for a specific representation in some cases.
The system should work with the layered architecture. A layer can be a proxy.
Code-On-Demand. REST allows client functionality to be extended by downloading and executing code in the form of applets or scripts… However, it also reduces visibility, and thus is only an optional constraint within REST.
A resource is located in a URI and is represented to a client via a standard media type. A web client only knows how to process the media type. For example, for a browser to display a PNG media type, there is no need for a JavaScript code. To display a PDF file a browser does not need to run some special JavaScript code. The same is true for processing and displaying the HTML media type.

The code-on-demand, with JavaScript, Java Applets, Flash ActionScript, or WASM, is optional in REST. This is a crucial fact in web clients.

The ultimate representation form of a resource should be understandable by a robot. An AI agent should be able to find the information and edit a resource without human interaction and prior knowledge except how to process standard media types.

Modern REST violations

Violation of the optional code-on-demand principle

Today, the idea of optional code-on-demand execution in Web sites, in most cases, is violated.
W3C introduced Web Components to extend HTML tags but made it entirely dependable on JavaScript. All modern Client-Side libraries, like React.JS, Angular, Vue.JS, are built with JavaScript. Sun Microsystems introduced Java Applets based on JVM. Adobe presented Macromedia Flash with a browser extension. Microsft made it possible to run reduced .NET applications in a browser with the Silverlight extension and recently introduced Blazor, which compiles C# code to WebAssembly and executes it on the client side.

The frameworks mentioned do not allow a page interaction without a code being executed in a browser. The exception is Svelte.Kit, which tries hard to make a web page interactive even with disabled JavaScript.

Violation of the resource representation principle

Modern websites pursue the mobile application development style. A web client downloads pages as mobile screens and then makes Remote Procedure Calls (RCP) to a server, usually provided as JSON API endpoints. The application knows how to modify a resource by calling a specific method and what structure of JSON should be sent and returned. This implies a tough coupling of a screen to a backend.

In contrast, with the REST approach, a browser displays a fresh representation of the resource, including dynamically retrieved actions for editing. For example, the HTML version <form action="/post-api"><input name="lastname"><button type="submit">Submit<button><form> is a page representation that itself contains the required structure for an action. A web client also does not know what the response will be and in what format. A server will decide about the subsequent representation, possibly redirecting a web client to a new page.

Reasons for violation

Developers step aside from the optional code-on-demand principle with a straightforward requirement - users want to see rich interfaces. Pure HTML does not allow building applications such as Google Mail, Google Sheets, or Google Docs, not to mention the code necessity for 3d web applications. Even if some HTML extensions will be added to HTML v.6, developers need to add custom client-side validation behavior, for example, to a phone input format (mask) per country:

Developers violate the resource representations principle because it is easier to think in the Remote Procedure Call paradigm. Also, coupling a webpage to a JSON API does not appear as bad in practice as it might sound. Modern browsers often refresh the pages with inactive tabs.
The problem with JavaScript, or EcmaScrpit, to be precise, is its speed. It is a text code that should be parsed and interpreted by a browser. In the next section, I will show how web development may be accomplished pleasingly for users and developers.

Better Future for the World Wide Web

Best performance with advanced HTML requests

The best speed a web client can achieve is by transferring small data and not executing code-on-demand at all.

The next proposals from Hypermedia Systems and HTMX can be implemented in the future HTML version:

Allow PUT, PATCH, DELETE to be declared in HTML actions in addition to the current GET and POST methods. Example:
```
<form action="/contacts/1" method="DELETE">Delete</form>
```
Allow actions to be triggered by any interactive element in addition to <a> and <form> tags. Example:
```
<div action="/contacts" type="GET" trigger="mouseenter">
  Get The Contacts
</div>
```
Allow to replace partial screens or single elements with the result from a declarative request. This will allow to create the genuine REST API with HTML responses as described by Fielding in REST API must be hypertext driven. Example:
```
<button hx-get="/contacts" action-target="#main">Get The Contacts</button>
<div id="main">
  <p>This content will be replaced with the requested HTML result<p>
<div>
```
Allow declarative polling and web socket communications. Example:
```
<div action="/messages" type="GET" trigger="every 3s"></div>
```

Allow to encapsulate code close to the related component by following the Locality of Behavior principle. Example:

 <button>
    <script type="text/javascript">
      <!-- This code is relative only to the nearest button tag -->
    <script>
    <script type="wasm">
    <!-- Similar with binary WASM -->
    <script>
<button>

Better performance with optional HTML requests

The Astro project makes it possible to download JavaScript code for a component only when it is visible on the screen or near scrolled. This feature can be included in the future version of HTML. Example:

<input id="my_input" type="text" placeholder="Name">
  <script type="text/javascript" 
  href="/input-behavior.js" 
  download="when-visible" 
  target="#my_input"></script>
<input>

Declarative HTML element state

Declaring a simple state in div, span, and button elements similar to form inputs will be helpful. State management is often used for block visibility. Example:

//HTML
<div id="popup" value="visible"></div>
<button 
  target="#popup" 
  actionType="valueChange" 
  actionValue="hidden">
  Toggle popup
</button>

//CSS
div[value="hidden"] {
  display: none;
}

Though modern browsers will soon support declarative popups and dialogs even without JavaScript with Popopover and Dialog, simple declarative state management will allow for providing more custom behavior.

Extendable native controls

Modern websites use custom UI elements for checkboxes, radio buttons, input elements, alerts, and dropdowns. Almost all of them are made with some CSS and JavaScript tricks, for example, in Bootstrap, Chakra UI, Material UI, ANT Design. Loading CSS and JavaScript for such components without special reduction customization often takes several hundreds of KB in size, plus time to parse and execute JavaScript code.

The better approach would be to allow customization of native browser components. Open UI introduced SelectList, which is already included in Chrome. The code for select customization without JavaScript can look like:

<selectlist>
  <button id=custombutton type=selectlist>
    <selectedvalue></selectedvalue>
  </button>
  <option>one</option>
  <option>two</option>
</selectlist>
<style>
selectlist:open #custombutton {
  background-color: green;
}
selectlist:closed #custombutton {
  background-color: red;
}
</style>

WebAssebmly with rich runtime

The WWW history shows that corporations cannot agree on one proprietary language for the Web. Sun Microsystems issued a lawsuit against Microsoft in 1997, complaining that JVM was not correctly implemented in Internet Explorer on purpose. Adobe pursued the Flash project with ActionScript until Apple, in 2008, decided not to include Flash in the iPhone Safari browser due to pure performance. Microsoft used its ActiveX extension in browsers that adapt Component Object Model (COM) with C++ and Visual Basic programming. Microsft Silverlight browser extension allowed to run code with a reduced .NET runtime. Google attempted to rescue browsers from JavaScript with Dart language.

Browser vendors make successful agreements contributing to the World Wide Web Consortium (W3C).

The recent contribution is WebAssembly (WASM), a binary stack machine supported in modern browsers. It is natively fast, and deliveries take less space than JavaScript code. There are some disadvantages to the current WASM state.

First is the size. Writing a server-side and client-side program is possible with Rust, and the resulting WASM package will be small enough. At the same time, Microsoft Blazor converts C# code to WASM, but the client delivery has to include the reduced .NET runtime, taking several megabytes for a script. The same is true for GoLang, even with an attempt to reduce the runtime delivery in TinyGo WASM. Developers want to work with their favorite languages, whether it is Java, Kotlin, Dart, C#, F#, Swift, Ruby, Python, C, C++, GoLang, or Rust. These languages produce groups of runtimes. For example, JVM and .NET have many common parts, Ruby and Python are dynamically interpreted at runtime, and all mentioned depend on automatic garbage collection. For smaller WASM packages, browser vendors can include extended runtime implementations, for example, by delivering a general garbage collector as part of WASM. Garbage collection support by WASM is currently in progress: WASM GC, .NET WASM Notes.

Second, WASM still manipulates DOM in a browser via JavaScript interop. Updating the DOM tree natively without a JavaScript proxy will be faster and require fewer CPU resources.

Conclusion

Web development stepped aside from the REST principles to provide interactivity for rich UI interfaces. A new HTML version with a declarative style for fetch requests can lead to ultra-fast websites. New WebAssebly runtimes may allow developers to program Client- and Back-end side code with a favorite language.