Forem: Tao Wen

Technology Minimalist

Tao Wen — Fri, 20 May 2022 06:21:38 +0000

Instead of keep adding new layers, there is a trend to go the opposite direction

Library over Framework

We used to think reuse is a good thing. Now more and more people realise reuse cause more damage than good. There are good reasons

UI design is highly volatile. More and more things become "headless" because we just want the stuff without the "style"
Big framework controls the control flow, making the debugging process mysterious.
Implement your requirement using options/callbacks provided by other guys is not fun. It takes time to memorize the options/callbacks and sometimes there isn't any.

Reuse is still going to happen, just in a more granular way. The UI will be ripped out, the framework will be teared apart, left us with a lot of highly focused libraries which just solve one problem one time.

Boring over Fancy

We used to think adding a new fancy technology will always be a good thing. Now more and more people realise they just need to get the job done with the least amount of moving parts.

Do we always need nosql or hadoop? Can we get the job done with postgresql with some extensions?
Do we always need SPA? Can we get the job done with server generated HTML?
Do we always need another microservice? Can we get the job done by extracting out a library?

Choosing boring technology is not just about stability, it is also about better developoer experience. With less compilation unit, less runtime service, and less co-workers, the job will be a lot easier.

Copy-paste is underrated

We used to think there could be another SaaS, another product, another library to build another abstraction, to make the reuse happen. Well, because we all hate to start from scratch again and again.

Between starting from scratch and reusing a library, there is a middle ground called "copy-paste". We all have used this approach, but shame to admit it. With the aid of github co-pilot, this kind of reuse will be more and more common.

Encapsulation can be cheap

We spent a lot of effort to combat with increasing complexity of legacy code. We want to entangle the codebase by enforced encapsulation. So there was a fashion to start a new project with 100 and more micro-services.

It does not have to be that expensive to enforce encapsulation. With the help of TypeScript and other static type checker, we can enforce encapsulation at compile time without paying runtime cost. Hide some internal implementation details through package.json dependency relationship is not a rocket science, it is called "dependency management".

Technology Minimalist

Minimalist seeks the balance point, minimal but still effective.

Minimalist does not care there is no shiny new keyword appear in their resume.

Minimalist enjoys solving the problem and deliver result, and having the fun of writing useful code during the journey.

Use jsx as server side html template

Tao Wen — Wed, 18 May 2022 07:33:10 +0000

source code: https://github.com/taowen/incremental-html/tree/main/packages/jsx-to-html

// filename: newsletter.tsx
import { jsxToHtml } from '@incremental-html/jsx-to-html'

// server is an express router
server.get('/newsletter', async (req, resp) => {
    const html: string = await jsxToHtml(<div>hello</div>);
    resp.status(200).set({ 'Content-Type': 'text/html' }).end(html);
})

The tsconfig.json should configure like this to translate *.tsx using jsxToHtml

{
    "compilerOptions": {
//...
        "jsx": "react",
        "jsxFactory": "jsxToHtml.createElement",
        "jsxFragmentFactory": "jsxToHtml.Fragment",
//...
}

async context

We can use jsxToHtml as an alterantive to node.js async_hooks.
There is no runtime trick, works in any environment (including deno, cloudflare workers, etc)

test('component with context', async () => {
    const C1 = async (props: {}, ctx: { msg: string }) => {
        await new Promise<void>(resolve => resolve());
        return <div>{ctx.msg}</div>
    }
    const result = <context msg="hello"><C1 /></context>
    expect(await jsxToHtml(result, { msg: 'original msg' })).toBe('<div>\nhello\n</div>');
})

The context will be automatically passed down the tree.
<context> is a built-in component to alter the context in the middle.

How to auto reload node.js server

Tao Wen — Thu, 12 May 2022 00:13:53 +0000

Source code: https://github.com/taowen/vite-howto/tree/main/packages/SSR/auto-reload-node-server

Code Structure & Motivation

It is a node.js application, using express to listen at http://localhost:3000

server/server-entry.ts is the entry point, which listens the http port
server/server.ts is the main logic

DX Problems

dev server should auto reload the node.js server when we have changed the source. nodemon can monitor soure code change and restart node process, but it takes time to restart. It would be nice to make the change without process restart.

UX Problems

vite build server should package every server-entry.ts dependency (except node itself), so we do not need to npm install again when deploy.

Solution Walkthrough

build node.js application to a bundle

server/vite.config.ts

import { defineConfig } from 'vite'

export default defineConfig({
    build: {
        ssr: './server-entry.ts',
        outDir: '../dist'
    },
})

will bundle the server/server-entry.ts to dist/server-entry.js with everything it referenced (except node.js standard library). It is in commonjs format, ready to be executed in node.js environment. build.ssr is provided by vite to build node.js server.

development server

During development, http://localhost:3000/ we want vite to transform server.ts on the fly, so we can skip compilation process after making changes

import express from 'express';
import { createServer as createViteServer } from 'vite';

async function main() {
    const app = express()
    // auto reload in dev mode
    const vite = await createViteServer({
        server: {
            middlewareMode: 'ssr',
            watch: {
                // During tests we edit the files too fast and sometimes chokidar
                // misses change events, so enforce polling for consistency
                usePolling: true,
                interval: 100
            }
        }
    });
    app.all('/(.*)', async (req, resp) => {
        req.url = req.originalUrl;
        console.log(req.method, req.url);
        const { default: handle } = await vite.ssrLoadModule('./server/server.ts');
        handle(req, resp, (e) => {
            if (e) {
                vite.ssrFixStacktrace(e)
                console.error(e.stack)
                resp.status(500).end(e.stack);
            } else {
                resp.status(404).end();
            }
        });
    })
    app.listen(3000, () => {
        console.log('http://localhost:3000')
    });
}

main();

We use await vite.ssrLoadModule('./server/server.ts') to transform the code and run it. Because the ssrLoadModule invoked per request, and server.watch is configured

 const vite = await createViteServer({
    server: {
        middlewareMode: 'ssr',
        watch: {
            // During tests we edit the files too fast and sometimes chokidar
            // misses change events, so enforce polling for consistency
            usePolling: true,
            interval: 100
        }
    }
});

If we changed the server code, we can see the effect just by refreshing browser to send another request to dev server. vite.ssrFixStacktrace(e) will fix the exception stack trace, to report the correct original line number, instead of the line number in transformed file.

How to get more people fund your opensource project

Tao Wen — Sat, 07 May 2022 12:01:36 +0000

Access to all Pro examples; Remove the React Flow attribution; Prioritized Bug Reports and Feature Requests

TypeScript source code is only accesssible to sponsors

The pattern is:

anyone can npm install for free
make examples exclusive for sponsors
make source code exclusive for sponsors
has better chance to get your feature request implemented

Checklist for UI frameworks evaluation

Tao Wen — Tue, 19 Apr 2022 23:03:10 +0000

load server generated page without waiting in blank state for too long, load as much as possible within a time budget, defer unfinished loading to second roundtrip
show loading indicator to allow other areas to show before everything ready, but not too many indicators
show error in area, instead of make whole page unusable
show result of my action without whole page refresh, keep uncommited edit state in other area
show feedback while typing, save extra click
show search results while typing, save extra click
if button click takes some time, show a processing indicator on the button to prevent user clicking twice
if server processing takes time, client may optimistically update before server confirm
show error next to the input
avoid multi page form, prefer minimal data entry initially, grow the form gradually as user provided more information
use infinite scroll to load more
use pull down to refresh
use swipe to show/hide more actions
use drag and drop to re-order items
use drag and drop to connect relationship
use half screen dialog to replace page jumping, use inline editing to replace modal dialog, avoid jumping around if possible
use mansonry to layout double columns, use screen space more efficiently
use FLIP layout animation to avoid content suddenly appear/disappear
preload next page, show progress, save the waiting time after switching
show current and next page side by side with transition animation, if no loading required
go back to previous page without reload waiting
perserve unsaved form in browser
render big page with many dom nodes, showing only the porition in viewport
multiple concurrent actions, end up with a consistent final state

Go through the checklist, and find out how to implement each feature in the framework you are evaluating (for example https://remix.run).

Will Low-Code development tools commercially profitable?

Tao Wen — Wed, 06 Jan 2021 22:03:32 +0000

There is a ideal story: If Low-Code development tools can be 10x more productive, then we gain competitive advantage over the cost of any kind of software project. Outsource vendor will want to purchase such kind of tool to compete with others.

Why this ideology does not work?

Unlike SaaS model, user will find the vendor through market search. Outsource vendor seek customers actively, it does not have the luxury to filter out "good quality" customer through funnel. To keep the utilization rate, sometimes some less ideal project must be taken. Those newly found customer will have high business development cost. Also some revenue will have high cost to collect after the project finished a long time. Among all the cost of outsource project, high development labor cost is just one, sometimes the insignificant one.
Tool can not be fully automated. We should need human to drive the tool. Human need certain period of learning, and need to make design decisions wisely in daily use. Even if the tool is bug-free, and well-documented, learning still takes time. Also to archive engineering property such as "low coupling, high cohesion" is still a black art largely depending on the human designer, no matter what the development tools market itself.
Software development tools is naturally monopoly due to network effect. You do not want a private company to run water/electricity, otherwise the monopoly will make you pay more. You do not want javascript/npm to be privately owned by a company, so that every npm publish will incur a charge to your credit card. Software development tools as public infrastructure should be open sourced, and remain open sourced.
Development tools users are developers, SaaS users are normal corporate worker. Developer will be more concerned with vendor lock-in. Why? To SaaS user, it is just a matter of choice, they will be lock-in by you or others. But to developer, they will think open source developer tools is truly in their hands.
Developer will have career concerns. However, this is a double edged sword. Like salesforce admin will try their best to maintain salesforce in the market, given their investment in the technology.
I am not saying open source === no profitable business opportunity. There are many successful commercial company behind open source projects. But in the area of development tools, it is more likely a consulting model business. It might be profitable, but due to human intensive nature, it will be unscalable. Investing millions of dollars on Low-Code to sell copies of the tool will likely lead to commercial failure. It has a much better chance of success running as not for profit open source project, like Ruby on Rails. Next DHH could be you.
The tool will be easier to sell to have immediate effect on existing codebase. I have written a record & replay tool that could re-produce production error without code modification. The tool took the heart of developers over-night. Low-Code does not work for legacy code, it targets green field projects. It is much much harder to sell if the tool can not solve any existing problem just now. It takes courage to trust something might work in the far future.
10x more productive does not mean 10x drop in latency (time to market). Unlike outsource vendors, SaaS companies, Internet business does not care the total cost that much. However, extreme time to market is desired, in hope of market domination. Modifying PHP file in production is cutting corner, but maybe exactly what is needed to be extremely fast.
Is 10x more productive Low-Code a lie? I still believe it. But it will not a good business commercially given the points above.

There are many startups in this area recently. If not selling development tools directly to developers, what market should they target?

SaaS + lowcode: Besides "truly useful to the user", stability, speed of page loading, the 4th should be "on premise deployment and customization". SaaS can sell to more customer if have Low-Code capability built-in.
lowcode + a family of SaaS: Instead of selling as development tools, lowcode can be wrapped as a family of SaaS. It will not be that feature rich (or feature bloated should we say) to be competitive by itself. But it will be appealing to those who want customization.
Data Centric base: Microsoft launched a product Dataverse recently. It might address the problem described by the book "Software Wasteland". I would describe the solution as "Backend as a database" for the age of frontend-backend separation.

You can also find me at twitter https://twitter.com/nctaowen

Connect react svg components

Tao Wen — Sat, 12 Oct 2019 07:31:49 +0000

Connect two rectangles via straight line

The from and to position is hard coded. Can we make the shapes collaboratively figure out these parameters?

Add from and to property

This time, Rect saved its center to its datum. The Line can figure out x1, y1, x2, y2 by querying its from and to node datum.

Make it draggable

Copied the code from https://dev.to/taowen/make-react-svg-component-draggable-2kc . We can see, the line is not following the rectangles. Let's fix it.

Make the connector follow dragging

We added a custom event called moved. When Rect being dragged, the moved event will be handled by both Rect itself and the connected lines. D3 require multiple listener to be registered in its own namespace, so the event name has different suffix.

Add circle

Circle is easier than Rect, as cx and cy is its center. However, due to we have two lines now, the event namespace need to be unique, so assignId is introduced.

Draw line before drawing the rectangles

We can see the line disappeared, because the connected rect is not drawn yet. We need to fix this.

Order should not matter

Introduced another custom event 'nodeAdded'. If the line can not find the node, it will monitor the nodeAdded event to find out if the collaborators all ready.

Make react svg component draggable

Tao Wen — Fri, 11 Oct 2019 13:11:23 +0000

Create a Rect component

It does nothing special. Just a dummy wrapper of <rect>

Add d3-drag

We use ReactDOM to find out the dom node. And use d3-drag to respond to mouse enter and mouse move. Although svg does not have drag drop event, d3-drag simulated it by intercepting window wide mouse events.

Setting x y attribute works. However, dragging will cause the rect "jump" to the position of mouse cursor. We would want it to stay there.

Make dragging starts without "jump"

The trick is to set the subject to have x y properties from the target rect. The relative position will be kept

How about circle?

For circle, we need to change x to cx, y to cy. And it seems working. The relative position is still kept.

We noticed the code is nearly the same as Rect. Can we make it generic?

Support both circle and rect

Just like change from rect to circle, we changed from x to cx, y to cy. This time, we use translate(x, y) to set the x y coordinate transformation. This transform attribute is supported by both rect and circle, so we only need to make one makeDraggable.

How about group?

It also works on <g>. However, we have to make the individual rect and circle undraggable, otherwise the dragged element will be rect or circle instead of the group.

Out of the Tar Pit, another approach

Tao Wen — Tue, 23 Jul 2019 14:20:53 +0000

What is the problem to solve?

A problem well-stated is Half-solved

The challenge was laid out by “No Silver Bullet — Essence and Accident in Software Engineering”

And “Out of the Tar Pit” elaborated the problem statement, and contributed a possible solution.

I don’t think debating what is essential state, what is accidental state is helpful. But it is clear, the problem is about STATE

The problem to solve is always the same. It is the big elephant in the room all the time. Imperative programming to manage state is too hard.

What is the problem of imperative programming?

We can update those states, quite easily actually. Imperative programming is straightforward, it gets things done. Just put together a bunch of CPU instructions to update state, and CPU will do it. The software execution can be drawn as this naive diagram

The external behavior of a software is essentially a ordered series of state update (yellow circles in the diagram). However, the problem is:

Describe this sequence of state update literally with one to one mapping, will lead to code that is "lengthy", "tedious", "fragmented".

The most straightforward way, the easist way, is not the best way. Even if, we extracted out lots of nice functions (those blue circles), the code might looks tidier, but still “lengthy”, “tedious”, “fragmented”. Readability is a subjective. “Simple Made Easy” suggests “Simple” is objective. I will define simple as following measurable properties:

Fewer states : the number of states, smaller is better
Sequential : if we have to deal with time ordered state update, let it be sequential
Continuous : this line and next line, they are put so close together because they are causally related. On the other thand, if two state update are causally related, they should be put as close as possible.
Isolated : state update is already hard enough to reason about. If they are not isolated, we have to pull all of them in my working memory to think about them all in once.

The goal is not to have zero states, but fewer the better. Then we keep the code sequential/continous/isolated to keep the complexity of state updating code on check. The opposite of those four properties are:

Excessively stateful : many many states, even unnecessary (or accidental state as described in out of tar pit paper)
Concurrent or Parallel : threads or coroutines, they are complex.
Long range causality : the application logic is fragmented, connected via a global state called database
Entangled : we have to ignore the fact they are components, to reason about them or to optimize them as a big entangled mess.

The goal is not to eliminate all states, but to keep them as few as possible. There will always be essential states that we have to manage and update with logic in order, that is what business process is all about after all. Even TLA+ evolved as PlusCal looks a lot like a imperative program. Imperative style is still the best known form to express temporal logic for human comprehension. But we have to keep it simple, by keeping it sequential, continuous and isolated.

What about OOP/DDD?

Using object to encapsulate state is a big part of OOP or DDD in particular. DDD in practices boils down to three steps:

Application service load back the domain model from database
Use aggregate root to encapsulate all the changes
The side-effect is in the form of modified domain model or issued new domain events. Application service save the models and publishes the events

The core idea is “Aggregate Root” can be a black box, nothing can by pass it, that will make our state update code encapsulated in one place. But there are two issues:

The method on aggregate root, is not very different from a free function taking the aggregate root as first argument. OOP is just ordinal imperative programming.
The interaction between objects, especially for the business process involving multiple domain concepts. It is hard to decide which aggregate root is the real aggregate root, which should take care of the process. Or put in another way, business process as a function in essence, itself should be the aggregate root.

We can re-examine the four properties above

Fewer states : as long as the state has to be updated manually in the temporal manner, they are still there
Sequential : still need thread or coroutine to exploit new hardware.
Continuous : the business logic is scatter around. It tends to be less continuous, not more continuous, compared to raw imperative programming.
Isolated : domain model isolates the states, but to optimize the data loading, we have to load them in bigger batch to avoid 1+N too many SQL problem.

What about pure functional?

In “Out of the Tar Pit”, author proposed a style of programming called Functional Relation Programming. It is known as event sourcing in DDD community. I agree, the event (user input) is the only true essential state, everything else can be derived from it, so accidental. But projecting every state from event is not as “easy” as temporal logic expressed by imperative programming. As Rich Hickey suggested, “easy” is subjective. But we can not ignore the fact most people is not familiar with this style, because it is detached from our primary familiarity source, the physical real world.

The physical world do not store the state as a ledger of events. The god does not derivative your height and weight from the food you have eaten since you have born. We make decision according to the situation. It is just the matter we load the situation from a mutable state, or we derivative the situation from the history on demand in memory. Reasoning based on current state for future, coupled with calculating the current state, will complicate the logic.

The approach is still not popular after all these years. We are presenting an alternative approach to manage state, which might be more familiar and practical.

Simpler State Management

The goal is to make it simple

Fewer states: if one state is not necessary eliminate it. If one state can be derived from other state, make it declarative.
Make imperative programming sequential / isolated / continuous

The code is written in TypeScript, but need a special runtime to make it working. You can think it is a new language, with identical syntax with TypeScript.

Fewer states by UI binding

UI binding is mainstream now. This demo looks like this:

The UI is powered by Web DOM, it is a separate state. Using binding like this we can make it a derived state:

<Button @onClick="onMinusClick">-</Button>
<span margin="8px">{{ value }}</span>
<Button @onClick="onPlusClick">+</Button>

The data bind to:

export class CounterDemo extends RootSectionModel {
 value = 0;
 onMinusClick() {
 this.value -= 1;
 }
 onPlusClick() {
 this.value += 1;
 }
}

But why binding is eliminating state? Compare code written in these two styles. First, we update these two states in order

// update state in temporal order
this.value -= 1;
this.updateView({msg: this.value})

Second, we bind the two state, then update the model

// setup the binding in advance
<span margin="8px">{{ value }}</span>

// then update the state
this.value -= 1

The difference is, binding or any declarative style programming, setup the relationship which is true, out of context. This can remove the derived state from temporal reasoning and description. The fewer states need to manually maintained within the time arrow, the better.

Fewer states by even more binding

If binding can remove state from temporal logic, we need more. Let’s layout all the states within the system:

Binding UI state to Frontend state only solved a tiny bit of the problem. Who will manage the Frontend state then? We need to question every single state, can we eliminate it? Or, can we derive it from another state? There is obvious quick win here. When presenting the UI, it loads the data from database, all the way up to UI. There might be some transformation, but can be conveniently described as data binding as well. Given this simple application:

It is just a simple list view, to show some data loaded from database. We can bind the UI directly to some database query:

When "from" or "to" changed, the "filteredReservations" will be updated, and the "totalCount" will also be updated, and the UI will be re-rendered with latest value. Binding to database, eliminate a lot of intermediate state between the UI and DB.

Fewer states by tight coupling

Frontend and backend work together to provide a user interface for the human to interact with. They should be tightly coupled.

If the RPC interface is dedicated for the service of the functionality provided by the form, why not merge the two states? Then it become a shared whiteboard between user/frontend/backend:

Take this application for example:

We can share the form state, with both frontend and backend. Here is the code:

The method saveReservation is marked with "runAt: server". The form state will be sent to backend for calculation, and then the form state updated in the backend will be rendered back to the UI. Although this is unconventional, but with fewer states, the code should be easier to reason about in theory.

This concludes the first part, we have seen following ways to reduce states count:

Derived state: such as computed property, autorun state subscription, database view, materialized view
Bind the database query, use it as if it is local state
Share form between frontend and backend

Simpler Temporal Logic

Declarative style data binding can remove some non-essential state from the system. Now, we need to focus on the temporal logic itself. The first property is sequential.

Sequential

Sequential single thread programming is simple. Yet, CPU tried very hard to parallelize the execution of the single threaded instruction stream. It will look ahead and speculate, as long as there is no data dependency.

Most temporal logic in business application can be single threaded, as long as the I/O can run concurrently, this is where co-routine comes in handy. However concurrent programming with co-routine, need to deal with “async” and “await”, it is still more complex than sequential programming. Can we write the program sequentially, and then speculative execution will try run it concurrently?

This is what Facebook Haxl project has demonstrated to us. The idea is simple, given we have these two tables:

Post has author, and author has inviter. We can map the relation as computed property in code:

If we express the UI rendering logic sequentially:

const author = somePost.author
const editor = somePost.editor
return new UI({ author, editor })

Even the code is sequential, loading author and editor can be executed concurrently, as they are independent from each other. However

const author = somePost.author
const authorInviter = author.inviter
return new UI({ author, authorInviter })

Because inviter depends on author, this two lines can not be executed concurrently. We have implemented this kind of “optimization” in TypeScript, with modification of the compiler and a runtime scheduler.

Isolated

The second property to make temporal logic simple is “isolated”. Here is a frequent case where isolation is broken:

If we render this table from the “Post” object defined above, it will lead to so call “1+N” problem. For each row, there will need extra SQL to fetch author name and inviter of the author. This is where we normally ignore isolation, and load them in batch. What we want is automatically batching up the load operations.

First, all the query in browser need to be batched in one big http query, with all the additional data to be fetched.

2019-07-19T11:25:04.136927Z 27 Query START TRANSACTION
2019-07-19T11:25:04.137426Z 27 Query SELECT id, title, authorId FROM Post
2019-07-19T11:25:04.138444Z 27 Query COMMIT
2019-07-19T11:25:04.772221Z 27 Query START TRANSACTION
2019-07-19T11:25:04.773019Z 27 Query SELECT id, name, inviterId FROM User WHERE id IN (10, 9, 11)
2019-07-19T11:25:04.774173Z 27 Query COMMIT
2019-07-19T11:25:04.928393Z 27 Query START TRANSACTION
2019-07-19T11:25:04.936851Z 27 Query SELECT id, name, inviterId FROM User WHERE id IN (8, 7, 9)
2019-07-19T11:25:04.937918Z 27 Query COMMIT

Then in the backend, the SQL queries should be rewritten as IN query. It is implemented by controlled coroutine call stack evaluation. Instead of doing depth-first evaluation, we do breadth-first. So all I/O operation can be collected then to be rewritten as a batch query.

This way, I/O batching as a cross cutting non-functional concern, no longer complect the essential temporal logic we want to express.

Continuous

The non-continuous version interleaves the temporal logic like this:

Although, billing process is more causally related, it can not be written close to each other. Although, contract singed and assignment dispatched has fewer common business, but because they are temporally related, they have to be written one after another in the order of time.

It is essentially several timelines squashed into one mainline. The POV constantly changes, in this single threaded story telling. The sequential nature is no longer a blessing, but a curse. We need to separate those independent timelines, and describes them in its own encapsulation block. It is called "Process", as business process.

Let's look at this final example

The account life-cycle can be described as a simple state machine. But in normal imperative programming style, it can be written as a continuous temporal logic. Here we have contributed a alternative style:

The process() is a function with infinite loop. It is never meant to be executed forever. Every time, this.recv(), the execution breaks, and the control yield to other process.

locked : this.commit()

normal : this.commit()

statement will label is a savepoint, just like in the game. The process can be loaded back from that savepoint, and resume. But where is the process saved to? Just ordinary Mysql table:

The label will be saved in the "status" column. Other local variable, will not be saved in database, unless there is a object property with the same name (password, retryCount).

Business transaction, or moment interval in color uml, is a good fit for this kind of programming style. It also settled the problem of where to put inter-object interaction in DDD, we can put it in Process. Because the code written out this way is not very different from the box/arrow in whiteboard, and it is not very different from original business requirement, it will facilitate the communication between different groups with shared language.

Unlike BPM or workflow solutions. Process is part of the application, it can be rendered as UI, and sum/avg by the report. We do not need to maintain a extra copy of data (extra states) to feed the "engine".

The UI binding of this application looks like this:

Conclusion

We are building a aPaaS product called MulCloud to enable this kiund of programming style. It has identical syntax with TypeScript, and inherited a lot of familiar ideas. We believe our approach, not only simplified programming, and because we stand-upon giant shoulders, people will also find it to be easy.

There is one more caution to leave behind for future innovators:

MulCloud has to put a lot of effort to build observability solution to allow the developer debug in production. Old-school imperative programming despite the odds, it has one shiny virtue: it has one-to-one mapping with the instruction stream executed by the CPU. By building the program in sequential / isolated / continuous way, the source code is detached from the reality, which is concurrent / entangled / with lots of long range causality. When something goes wrong in the wild, it is harder, much harder to debug. Bret Victor has demonstrated a lot of cool ideas on providing more runtime feedback. We borrowed a lot of them, but that deserves another post.

Simplicity is hard, when simplicity is not the reality.

What represents the past, present, and future: the future(2)

Tao Wen — Mon, 29 Oct 2018 15:09:29 +0000

Active process

In the previous simple process, the coroutine is passive. It waits for future instruction and just does what was told. This contradicts with the requirement document, which normally describes a task as if it is alive. It knows what it is doing, and it knows what it should do next.
In this chapter, we are going to explore the ways to make the future representation more active.

Time scheduler

The first example is about sleep.

lua version: https://tio.run/

es2017 version: https://tio.run/

Call with continuation

coroutine.yield('sleep', {seconds=2}) is awkward. Why sub1_task can not put itself into sleep mode directly? Because it does not have reference to itself. Some language provides “call with continuation” which essentially allow the coroutine to get its remaining calculation as a continuation, and pass that to another function as callback. We can simulate that in lua

lua version: https://tio.run/

“Future” concept is introduced, it wraps the task continuation along with its requirement. When the requirement fulfilled, the task execution will be resumed.

Async and await

Notice that we did not give es2017 version in above example. Because es2017 has its own version of “future” builtin.

es2017 version: https://tio.run/

Although the code looks very different. The underlying mechanism is actually very similar. The compiler/interpreter of es2017 will turn “await” into a continuation passed to the promised given.

We can change the lua code to work like the javascript promise. It just returns a function called “resolve” to capture the task continuation and register it to somewhere, then yield execution.

Network client

Time scheduler is simple. It does not need to return value back into the task. If the coroutine needs to play the role of a network client. It needs a way to get the result back.

lua version: https://tio.run/

Returning value is not a hard job as it turns out. The caller and callee share a “future” object so that they can send request and response via this object. With es2017 await syntax, return value back into coroutine is much simpler.

es2017 version: https://repl.it/@taowen/es2017-client

Network server

At this point, we have already seen how coroutine can compute, sleep and call external service. Now, we need to make the coroutine a server. We only present es2017 version here, the lua code will be pretty much the same as “network client example”.

es2017 version: https://repl.it/@taowen/es2017-server

The difference between client and server is, for the server, it needs two await, but for the client, it only needs one await. One for retrieving request from the scheduler, one for sending back a response. For TCP socket, writing can be blocked by the kernel when the buffer is full. So sending back response is also an async operation.

Passive Process V.S. Active Process

We have checked out several examples of the active process. It can do the following things:

sleep
as a client to call another service
as a server to be called by another service

Compared to the passive process from the first chapter, the active process is indeed very active. It has control of its own destiny. For a direct comparison, let’s say we have two steps, and one sleep between them.

Passive version: https://tio.run/

Active version: https://tio.run/

It is really no different from “passive” v.s. “active” sense. From sub1_task point of view, the passive version use “coroutine.yield” directly to wait for the scheduler to wake it up, the active version use “sleep” which in turn uses “coroutine.yield” to wait for the scheduler. Both versions essentially do the same thing, they call "yield" to park itself.

The difference is about predefined protocol. The active version has a protocol between scheduler and task. If the task park itself to “sleep_future”, the scheduler has the responsibility to wake it up in the specified time. Just like you use os.execute(“sleep 1”), the operating system has the responsibility to wake your os thread up 1 second later.

The benefit of the active process is the scheduler become infrastructure, all unstable business logic is isolated in the task. For the passive process, both sub1_task and schedule will need to be modified if the process needs to be updated.

The scheduler becomes an extensible platform, the tasks are plugins to reuse platform capability.

Coroutine UI

The task parked in the scheduler is anonymous. This is a problem. If we do not know what are you waiting for, how can we know what to give? We know the sub1_task want to know if step1_add or step2_add. This is a “User Interface Problem” essentially.

Here is an example process we want to describe

When the task is launched, it will present the user with two options, either to add or sub. Then the calculation result will be displayed back. One second later, the UI will change back to the two options to let the user do the second calculation. This forms an infinite loop.

This process can be represented by this code

es2017 version: https://jsfiddle.net/taowen/L0p516xv/56/

Just like the scheduler can expose API “sleep”, “recv”, “reply”, it can also provide “user_input” as a reusable infrastructure. Of course, the scheduler can not know what the user interface will look alike, so it takes the argument to define the UI component to use and the model to render the view. Here is the source code of scheduler

The user_input the render the view by setting three variables:

step_ui: which component to render the view
step_ui_input: the model of the view
step_callback: what to do when the user submitted the form

As we have already learned, how “await” “promise” and “resolve” works, it is obvious when the user submitted the form, resolve callback will resume the execution of sub1_task. The actually view rendering and form submission implementation is irrelevant to the topic here. You can play with code at https://jsfiddle.net/taowen/L0p516xv/56/

The scheduler.user_input is provided as a generic tool. It can be used regardless of the business logic we need to describe. The vocabulary of “active process” is expanded. There will be more “future” can be represented by coroutine.

Coroutine hibernation

We can not put the coroutine UI into production, because there is a serious flaw. The coroutine needs to stay alive in computer memory for the whole time. If the user decided to click the button tomorrow, we can not keep the process alive that long. We need to click the “hibernate” button so that the state can be dumped to disk so that we can save resources only resuming the process when needed.

This example requires a special version of lua interpreter: https://github.com/fnuecke/eris

We have expanded what the scheduler can do further. This time, the “hibernate” function allow the task to dump itself into the disk. The representation of the future is totally encapsulated into one coroutine “fib”.

Coroutine database

The hibernation implementation is also seriously flawed. The “continuation.data” is written in an alien binary format. Represent the future using an object such as “order” does not have this problem. We normally would create a table called “order” to save the unfinished order processes.

To represent the order process using coroutine, it would look like:

The order_status column of the order table is a cursor tracking the position of execution. It is essentially the position of “coroutine.yield” in the coroutine. So we can represent the order_status with lua label. When persisting the coroutine, the “program counter” can be translated to “order_status”, and mapped to a database column.

Summary

Now, we can represent calculation logic, UI and database by the active process. The coroutine yield control to its scheduler for reusing predefined and stable atomic operation. All those unstable business logic can be encapsulated in a single coroutine.

Using the active process to describe “long-term future” is unconventional. Looking a coroutine to “invoke” a UI or database is mind-blowing. But it is a plausible representation, and sometimes come in handy.

What represents the past, present, and future

Tao Wen — Tue, 23 Oct 2018 01:18:07 +0000

What represents the past, present, and future: the future(1)

Our software builds a cyberspace for intelligent agents to interact with each other. Using machine learning algorithms to build intelligent agents is a hot topic. But most of the code written is still about building the virtual environment itself. The code and runtime describe what could and would happen in this environment, which is about “the future”. To solve ambiguity, code and runtime must resolve what is really happening at “the present”. The code and runtime also need to faithfully record what has happened in “the past”. To maintain the integrity and lay a solid foundation, all three things should be done properly, otherwise, we perceive the anomaly as “a bug”. Most prominent cyberspace is the online multiplayer game as the article cover suggests.

As the business interest is rapidly shifting to build more intelligent agent, the cyberspace building technique became a commodity, some good old wisdom might be lost and forgotten. The current mainstream approach is mature and working but is far from elegant. This series of three articles attempt to summarize my observation on the possible representations of “the past”, “the present” and “the future”. Some of them deserve more attention.

What represents “the future”? This seems to be a quite obvious question. The so-called business logic has many many ways to express. We are going to go through some very junior concepts from “future representation” point of view. Then we can see it is not actually as simple as we thought.

Source code

“the future” lives in the document, in the source code, and in the runtime. The language we are going to use is lua 5.3 and es2017 (ECMAScript 2017). All code can be tried online at https://tio.run/.

Simple process

We are going to describe a very trivial process: result = (a+b)*3. The “some_important_business_process” describes the process of taking a, b and computing the result.

Function

The simplest way to describe “what could and would happen” is using a function.

lua version: https://tio.run/

es2017 version: https://tio.run/

Partial function

In the previous function, we take both a and b as the input. However, we might not know all the things from the beginning. To tell the story gradually, we take “a” first and then take “b”.

lua version: https://tio.run/

es2017 version: https://tio.run/

The variable “proc” captured “a” into itself. It is a continuation which execution can continue from it.

Object

Having one initial input and one final output is not enough to describe what is to come. The complete representation of the future must describe what would happen given more input in the future. To represent this, the normal choice is to use an object to store the process, and a couple of methods to be called when the story unfolds.

lua version: https://tio.run/

es2017 version: https://tio.run/

The variable “proc” here is same as previous “proc”. It is a continuation which execution can continue from it.

State machine

However, using object does not exactly represent the process. As step1 and step2 are equal. There is no place defining step1 should happen before step2. A more accurate representation is a state machine:

lua version: https://tio.run/

es2017 version: https://tio.run/

The “next_step” is a cursor, that keeps track of execution status. In x86–64 CPU, there is an instruction pointer register called “RIP” which is similar to “next_step” concept here.

Coroutine

Coroutine works like a state machine but looks like a function. It takes much less code to express the same idea. There are two styles of coroutine, one with “yield” (also known as “generator”), one with “async/await”. We are going to look at how “yield” works first.

lua version: https://tio.run/

es2017 version: https://tio.run/

Yield kill two birds with one stone. It returns value to the caller and gets input back. Visually, it looks like this:

Coroutine object I

The coroutine is always resumed by the same function name, such as “next” or “resume”. It is less expressive than “step1” or “step2”. We can wrap the coroutine as an object to make the code looks better.

lua version: https://tio.run/

es2017 version: https://tio.run/

Coroutine object II

Although we have added method “step1” and “step2”, but they are not forced to be invoked in that order. Also, what if we want to have two options for “step1”? Say, it could be “step1_add” or “step1_sub”. Let’s upgrade the coroutine object to version 2.

lua version: https://tio.run/

es2017 version: https://tio.run/

We can see coroutine and object are very similar, and coroutine is more powerful. But without direct syntax support, writing method for coroutine is cumbersome.

This concludes the first chapter. We have used these ways to implement a very simple process

function
object => state machine
coroutine => coroutine object

Function is inadequate in real-world usage. So we either choose object or coroutine to represent the future. Object is explicit about its expectation but implicit about its state. Coroutine is explicit about its state but implicit about its expectation. We have tried to merge object and coroutine into one with limited success.

Also, we have learned many forms of continuation

partial function
object instance
coroutine instance

Continuation will form the foundation of complex scheduling in following chapters.

What makes code slow to execute

Tao Wen — Sun, 30 Sep 2018 08:08:55 +0000

“Within the light cone is destiny” -- Cixin Liu

Performance is primarily determined by the algorithm of the code. There is no question of that. Some algorithm written in python is a lot slower than written in c. To me, this is also an algorithm problem, if you think the code and its language as a whole. It takes a lot more machine instruction to do the same thing in python than c. The sequence of machine instructions is an algorithm.

x86 instruction can be seen as a high-level language for micro-code used internally in the CPU. When CPU decode the x86 instruction to micro-code with speculative execution will cause significant performance difference. This can also be categorized as an algorithm (of increasing concurrency) problem, and it is not what I want to talk about here.

Besides the algorithm, what else can makes code slow to execute on the machine? The answer is "data locality", it takes time to read and write the data for computation. There are multiple examples of it

CPU Cache
GPU Architecture
Distributed Map/Reduce
Heap Management

Walking through these examples, we can see why Heterogeneous Computing is inevitable, and programming model in mainstream programming languages is overly simplified.

CPU Cache

Consider this example https://stackoverflow.com/questions/9936132/why-does-the-order-of-the-loops-affect-performance-when-iterating-over-a-2d-arra

Iterate matrix column-wise

#include <stdio.h>
#include <stdlib.h>

main () {
  int i,j;
  static int x[4000][4000];
  for (i = 0; i < 4000; i++) {
    for (j = 0; j < 4000; j++) {
      x[j][i] = i + j; }
  }
}

Iterate matrix row-wise

#include <stdio.h>
#include <stdlib.h>

main () {
  int i,j;
  static int x[4000][4000];
  for (j = 0; j < 4000; j++) {
     for (i = 0; i < 4000; i++) {
       x[j][i] = i + j; }
   }
}

Row-wise version is 3x to 10x times faster than column-wise version. When the memory address is written continuously, the write can be combined into a single request to save the time on the round-trip.

According to https://en.wikipedia.org/wiki/CPU_cache, as the x86 microprocessors reached clock rates of 20 MHz and above in the 386, small amounts of fast cache memory began to be featured in systems to improve performance. The biggest problem of processor design is that the memory cannot keep up with the speed of the processor.

We can see the chart above, modern CPU can crunch a lot of numbers per second. The theoretical data transfer rate to keep the cpu busy is way higher than the bandwidth currently main memory can provide.

The latency between CPU and main memory is ultimately governed by the speed of light. We have to bring the memory closer to CPU to lower the latency. But on-chip memory (L1/L2/L3 cache) cannot be too large, otherwise, the circuit will be long, and latency will be high.

GPU Architecture

In the design of CPU, the cache is invisible, the memory is coherent. You do not need to take care of the synchronization between the cache and memory of all sockets. Although you do need to use a memory barrier to declare the actions sequence on one memory address as if they have contended directly on the memory itself directly. To make this happen, CPU need high-speed ring/mesh to synchronize the cache between sockets.

This programming model prevents the cores from growing. It is too costly to maintain the illusion of direct memory sharing between cores because the latency of data transfer is inevitable. The memory change needs to be broadcasted to all cache.

GPU architecture is different. It solves the memory latency issue by making the "cache" explicit. Instead of calling it "cache", it is just another layer of memory you can use.

In this programming model, there is not just one "heap" of memory, instead the variable need to be explicit about its scope. For example

__global__ void parallel_shared_reduce_kernel(float *d_out, float* d_in){
    int myID = threadIdx.x + blockIdx.x * blockDim.x;
    int tid = threadIdx.x;
    extern __shared__ float sdata[];
    sdata[tid] = d_in[myID];
    __syncthreads();

    //divide threads into two parts according to threadID, and add the right part to the left one, 
    //lead to reducing half elements, called an iteration; iterate until left only one element
    for(unsigned int s = blockDim.x / 2 ; s>0; s>>=1){
        if(tid<s){
            sdata[tid] += sdata[tid + s];
        }
        __syncthreads(); //ensure all adds at one iteration are done
    }
    if (tid == 0){
        d_out[blockIdx.x] = sdata[myId];
    }
}

Variable either has no prefix, or __shared__ or __global__. In CPU programming, we need to guess how the cache works. Instead, we can just ask in GPU programming.

Distributed Map/Reduce

Map/Reduce can handle big data distributed because it avoids data transfer to do computation on the node where data originally is

The data sent from "map" nodes to "reduce" nodes is substantially smaller than original input because partial computation has already been done.

Heap Management

Big heap is hard to manage. In modern c++ or rust, we use ownership to manage the memory. Essentially, instead of managing the whole heap as a connected object graph using the garbage collector, the heap is divided into smaller scopes. For example

class widget {  
private:  
  gadget g;   // lifetime automatically tied to enclosing object  
public:  
  void draw();  
};  

void functionUsingWidget () {  
  widget w;   // lifetime automatically tied to enclosing scope  
              // constructs w, including the w.g gadget member  
  …  
  w.draw();  
  …  
} // automatic destruction and deallocation for w and w.g  
  // automatic exception safety,   
  // as if "finally { w.dispose(); w.g.dispose(); }"

There are a lot of cases, the lifetime of one memory address aligned well with function enter/exit and object construction/destruction. There are cases, explicit annotation of the lifetime will be needed

fn substr<'a>(s: &'a str, until: u32) -> &'a str;

Lifetime declaration like 'a is similar with __shared__ in CUDA. Explicit lifetime management like c++ and rust is one extreme. Multi-thread safe garbage collected language like Java and Go is another extreme. Language like Nim partition the memory by threads, which lies in the middle of the spectrum. To ensure the memory in the different partition does not accidentally shared, memory needs to be copied when crossing the boundary. Again, minimizing cost of memory copy is a data locality problem.

JVM has garbage collection. Instead of annotating the source code statically, it tries to figure out what goes with what in the runtime by dividing the big heap into rough generations:

It will a lot easier to deal with heap management problem if we do not share big heap in the first place. There will be cases having multiprocessor sharding memory be beneficial (https://en.wikipedia.org/wiki/Symmetric_multiprocessing). But for a lot of cases, co-locating data and computation (actor programming model) will be easier.

Co-locate data and computation

Instruction per clock is not growing very fast.

Main memory access latency is nearly flat

Having to deal with more than one processors, each with its own memory (Heterogeneous Computing) is likely to be pervasive. It is a clear trend that we need to ensure the data and computation co-located with each other in this paradigm. The problem is how to properly partition the data and computation, in the way data copy or message passing is minimized.