Forem: Kasey Speakman

Explaining the LMAX Disruptor

Kasey Speakman — Wed, 10 Sep 2025 06:05:29 +0000

More than a decade ago, the LMAX Disruptor pattern entered the scene. I found it interesting. But frankly had trouble wrapping my head around it. It wasn't until years later when I started working on a JVM project that I finally understood. There is a lot of good performance engineering baked into it. But it basically only exists because of JVM limitations. It wouldn't rise to the level of a pattern in system level languages like Go or Rust, or even a high level language with more performance knobs like C#.

Context

Let's back up a bit. LMAX was building a high frequency trading platform. Such a system optimizes latency and throughput. For these constraints, there's a big problem with their choice of platform. The JVM was designed for Java, which ostensibly is meant for high-level -- especially object-oriented -- programming. It just doesn't give the user many performance knobs to turn.

One of the reasons I had such a hard time understanding the purpose for many of the details in the Disruptor was because I came from .NET. Even though Microsoft basically copied Sun's Java homework to make C# on .NET, in the process they also managed to bake in some improvements including performance tune-ability. Many of the pattern details were workarounds that I didn't recognize. I'll give you some examples:

Java Limitations

Structs

Java does not allow the user to create primitives aka structs aka stack-allocated types. It only allows you to make garbage-collected (heap-allocated) types. C# allows you to create structs. System level languages tend to take the inverse approach. Types are assumed stack-allocated until you explicitly allocate them on the heap.

Here's the oversimplified explanation of how that works. Each code scope (e.g. function execution) runs in its own stack frame. (Note: this is where a stack trace comes from.) When you return from that function, the stack frame goes away along with whatever local variables were allocated in it. So stack-allocated variables do not need any special memory management. When you pass struct data into a function or return struct data from it, this data is copied between stack frames. This is efficient until the data gets too large. That's when you might reach for a separate memory allocation on the heap. Then the stack frame only copies the pointer (8 bytes on a 64-bit system) for the heap data to the next stack frame instead of the full data.

What constitutes "too large depends" on the system, but on .NET I recall the threshold being 24-32 bytes (3-4 longs or reference objects). Above that, object references could be better performance, depending on garbage collector behavior. As with all performance optimizations, you don't know for sure which is better until you test it.

High level languages default to heap allocated data so that it's copying 8 byte pointers between stack frames instead of the full data. This is probably better performance in common cases, but optimized code might do better with structs. Java doesn't give you the option to tune performance in this way.

Memory Layout

Alignment to CPU cache line is another common performance optimization. In short, CPUs have multiple levels of caches. Especially for collections, there are potential performance advantages if you design your data types to be exactly 8, 16, 32, or 64 bytes. This is because data is read from memory into CPU cache lines, which are generally 64 bytes. If your data fits exactly in a cache line, the CPU can use optimizations / pipelining which improves performance when processing collections.

Java has no way to specify memory layout, so Disruptor makes use of padding fields in the classes used by ring buffers. Named like p1, p2, etc. These fields are not used by logic, but are only there to align the data structure. Whereas .NET has StructLayout and related attributes to allow you to add padding explicitly.

Boxing

Even though Java does not allow you to create primitive data types, it has some built in, like int. However, when you use primitives in generics Java "boxes" them up as heap-allocated objects. Especially in generic collections, this uses more memory and costs CPU instructions to box and unbox.

Disruptor doesn't explicitly mention this, but it is a well-known drag on JVM generic performance. One which I wasn't familiar with from .NET as it doesn't box primitives for generics.

Performance optimizations

Overall, the Disruptor is a marriage of qualities from actor and SEDA patterns with some distinct performance optimizations like a ring buffer that is also an object pool. From Actor / Agent pattern, Disruptor took the message inbox and independent processing. However, the disruptor is also step-wise coordinated with previous steps like in SEDA.

The ring buffer plays a prominent role in Disruptor for a few reasons. First, it is a common way to implement a fixed-size queue. This is necessary in high-throughput systems because unbounded queues will eventually reach memory usage limits. The bounded queue forces you to think about your memory usage and full queue behavior. (E.g. load shedding vs blocking)

The ring buffer simultaneously serves as an object pool. Object pools are pretty common in game development. Business devs will be more familiar with them from database or HTTP connection pooling. The general idea being that creating a new object is expensive in some way, so instead keep a pool of them and soft reset and reuse an old instance. For db or http pools, the expensive part is actually making the network connection. But for very high-throughput systems, the churn of allocating new objects and then garbage collecting them in a tight loop is the problem. By pre-allocating blank objects in the ring buffer, these objects will never be garbage collected nor need new instantiations later. So it essentially circumvents garbage collection.

There are other optimizations around the ring buffer such as only using sizes which are powers of 2. The intrinsic way a ring buffer works is that it has an ever-increasing queue position which can go past the actual size of the buffer. So if the ring buffer size is 4 (indexes 0 to 3), but we're processing our 15th item out of the queue, we can calculate the ring buffer position with modulus by the buffer size. 15 modulus 4 is 3. So the item we want is at index 3. The problem is the modulus operation, which can be expensive to calculate in a tight loop. But not for powers of 2. Since the computer operates in binary, powers of 2 are especially simple to divide by. So using a power of 2 as the ring buffer size increases performance.

I didn't cover all the optimizations, but you get the idea.

Fixing the Elm Architecture

Kasey Speakman — Mon, 28 Jul 2025 21:23:09 +0000

Elm architecture aka MVU was a breath of fresh air when I encountered it. It remains the best UI dev experience I know of. As I've done larger and larger projects with it, used it on the backend, and seen its usage in other contexts, it has some friction points. Here are my current thoughts on how to resolve those sore spots.

Code below is in F# unless otherwise noted.

Centralizing logic into `update`

//                          sus
//                           |
//                           v
let init initArg : (Model * Cmd) = ...

One thing that has always bothered me about init is that it comes out swinging by returning a Cmd. This can be a problem in some UI technology (e.g. WPF) that is sensitive to the timing of initializing elements vs side effects. I recall at least one issue about this in the Elmish repo.

It's also annoying that all decisions are made in update... except for the 1 made in init, as indicated by returning Cmd. This means that to test an MVU program both functions have to be exercised. I played around with my own backend-focused lib and determined that this doesn't have to be the case.

I had to write my own library for backend workflows because they have different run characteristics from UIs, but my lib used identical MVU functions, minus view.

All logical decisions could be done in update without changing it in any way. Just move init's Cmd-producing decision into its own Msg case. Then get rid of init. That works, but the caller has to know how to construct an initial Model and Msg to send to update. So then I thought, why not use init for this purpose instead... basically an adapter function to translate the data you have (initArg) into the form needed to kick off the update function.

let init initArg : (Model * Msg) = ...
//                  \---------/
// tells how to          |
// start update          |
//             -----------
//             |
//             v
//         /-------\
let update msg model : (Model * Cmd) = ...

For UI platforms that need it, this would also provide an opportunity for UI elements to be initialized from the Model, and the Msg queued for later processing, without kicking off any side effects.

At the time I made this change, I already implemented a similar tactic for resumable workflows. My code had the model from the previous completed workflow. It just needed to know the correct Msg to resume the workflow. I created a function called resume for that purpose. So it seemed natural for init to do the nearly same thing to start the workflow.

Parameter order

Another thing that always bothered me about Elm is the parameter order of update. You can look at Cmd as a future Msg, so the update function returns nearly the same types as its inputs, but with their order swapped.

//     should be swapped
//          ------
//          |    |
//          v    v
let update msg model = model, Cmd.none

This is definitely a minor point, practically speaking. I just have to bake in an argument swap when I'm testing update. But it's logically incorrect for how update is used. And I can't unsee that.

It is a winding road to explain how Elm arrived at that. But it has to do with its Haskell heritage; Haskell's focus on laziness; and laziness being represented by the right fold operation. Which in this case puts model as the 2nd argument. Whereas left fold -- which would have model as the 1st argument -- is considered eagerly computed. It's an inherited philosophical choice, not a practical one, since Elm doesn't focus on laziness like Haskell. You can even observe this cognitive dissonance in Elm's List.foldl which processes the list left to right, but combines elements right to left as a right fold would, so it behaves unexpectedly when order matters.

// F#, Haskell, and every other language's left fold result
"abc" = List.fold (fun s1 s2 -> s1 + s2) "" ["a"; "b"; "c"]

-- Elm's left fold result
"cba" == List.foldl (\s1 s2 -> s1 ++ s2) "" ["a", "b", "c"]
-- ^
--  \
--   huh?

It seems like update arguments are meant to match Elm's incorrect left fold behavior. This is easily fixable by swapping input argument order.

// before
let update msg model : (Model * Cmd) = ...
//           \  /
//            \/  swap
//            /\
// after     /  \
let update model msg : (Model * Cmd) = ...

Later in this article, you'll see an example of testing update logic by providing a list of Msgs, then checking that the correct side effects were produced. Testing made the most sense with left fold. Which made the most sense with the corrected version of update.

Untestable `Cmd`

Elm's representation of side effects is with Cmd. The Elm philosophy is that users are not allowed to do side effects in Elm, so Cmd is locked down. You can use a Cmd-generating platform function provided by Elm. Or when you inevitably need a side effect that Elm doesn't cover, you must send data out of an asynchronous port to let JS run your side effect and send the result back in thru another async port.

In Elmish, the F# library, Cmd is an alias for a list of user-declared functions. Each of which is provided with a dispatcher to send Msgs back. (Unlike Elm, you actually get to write your side effects in F#! And JS <-> F# interop is easy.) So your update code returns functions to run when it's time to call side effects.

In both cases, Cmd is untestable. In Elm it's intentionally opaque, even the fact that it's a collection under the covers. There is no way to test it at all. In Elmish you could potentially test generated Cmds if your update fn also accepts side effect dependencies as parameters and you pass in mocks instead of the real ones. But I consider this to be the hard way to test things, on top of polluting update with side effect related concerns.

I'll be honest, I never did a lot of unit testing on frontend code. It was just not needed. But backend is a different story. It is all about making business decisions and executing the chosen side effects. It needs to be tested thoroughly for correctness.

Declarative side effects

For backend, the Model is less important to test. Rather, I needed to be sure that the correct side effects were decided. Cmd being hard to test was just not acceptable. So I introduced another step in the MVU process. This involves adding another type similar in style to Msg but for the purpose of declaring a side effect to run. I called it Effect. Then I introduced a function to perform those side effects.

type Msg = ...

// define possible side effects, a DU like Msg
type Effect = ...

type Model = { ... }

let init initArg : (Model * Msg) = ...

//                            test with equality
//                                     v
let update model msg : (Model * List<Effect>) = ...

//    could be Task and/or synchronous too
//                     v
let perform effect : Async<Msg> = ...

Like Msg, Effect is a user-declared type.

Above, update is living its best functional life. Its inputs and outputs are value-equatable immutable data. That means side effects can be tested for correctness with simple equality checks. Like so:

//  expected    model      effects
//                |           |
//                v           v
let expected = { ... }, [LoadWidget 123]

//   basically left fold            model   simulated scenario
//            |                       |             |
//            v                       v     /----------------\
let actual = sim MyWorkflow.update { ... } [Go 123; LoadFailed]

// test workflow with simple equality
assertEqual expected actual

Notice that I don't need to involve init in this test, since it no longer returns a side-effect... only update does.

Above, LoadFailed is an actual Msg that would return from perform when there was a database failure. The only Effect produced is the attempt to load a widget. After that, we're expecting that upon receiving LoadFailed the update fn will stop and not try to produce more side effects. If it does produce more Effects, assertEqual will fail the test. So this not only tests that it does what it should but also that it doesn't do more than it should.

This is what the relevant section of perform might look like.

let perform (dbConfig, logger: ILogger) effect =
    match effect with
    | LoadWidget widgetId ->
        async {
            try
                let! widget = Db.findWidget dbConfig widgetId
                return WidgetLoaded widget
            with ex ->
                logger.LogError(ex, "Widget load failed {widgetId}", widgetId)
                return LoadFailed
        }

We take advantage of partial application to inject dependencies as the first argument.

With this arrangement, side effects no longer need to be mocked -- with all the overhead involved in that -- to test logical decisions. In fact, side effects play no part at all in unit testing. They only have to be integration tested.

We used this same adaptation on frontend. There, having a place for side effects prevented them from being introduced accidentally in update. We also found it made update code easy to reason about. When we did have a UI bug, it was usually quick to spot and fix. At least it was once we got away from sharing state across pages.

Nested page fatigue

Everything I've mentioned so far is easily usable inside Elmish, maybe even Elm, with tiny adapter functions. That's what I did for our Elmish frontend. This one could need more than that, not sure yet.

What is nested page fatigue? It's the vaunted fractal nature of The Elm Architecture. It is great to a degree, but it requires wiring together too dang many things between a parent and child. Every time you create a child page, you have to wire it into the parent page in every possible way.

Msg case for child msgs
Model property or case for child model
init or update case to initialize child
- depends on whether it uses navigation
update case to call child update
view code to call child view

After hundred thousands of lines of UI code and many dozens of pages, the friction is real. And this is the most common complaint about MVU. The backend library taught me that it was possible to avoid this fatigue while still using MVU all the way down.

You see, when I tried to compose F# backend MVU workflows in the traditional way, similar to frontend, it made things very awkward and mixed a lot of concerns between workflows.

If this were OO, that way of wiring things would be a dark-but-humorously-named anti-pattern called Inappropriate Intimacy.

Then I noticed when non-MVU code called MVU workflows, it first used an adapter function to convert it to a normal Async-returning function. Then I realized that this is the best way to compose MVU workflows... treat each one as a single overall side effect. This makes composing them easy:

async {
    // workflow1/2 are records containing init, update, etc
    let! result1 = runWorkflow workflow1 initArg
    let! result2 = runWorkflow workflow2 result1
    ...
}

// or thru perform

let perform deps effect =
    async {
        match effect with
        | MyEffect arg ->
            let! value = runWorkflow workflow1 arg
            return EffectHappens value
    }

Right okay, but how does that help frontend?

The purpose of these backend workflows is to make correct decisions about side effects and run them. So composing them as side effects made sense. The purpose of a frontend program is decide about and display UI.

Composition through `view`

This gave me the idea that a whole Elmish-React child program could be packaged up as a React component, then displayed along side the other HTML elements (also React components) in the parent's view.

let view model dispatch =
    div [] [
        match model.Route with
        | CourseList ->
            let initArg = ()
            Elmish.page initArg CourseList.init CourseList.update CourseList.view
        | CourseDetail courseId ->
            Elmish.page courseId CourseDetail.init CourseDetail.update CourseDetail.view
    ]

Since Elmish has termination support, the self-contained page would be coded to run clean up when it is switched out from view. There's no longer a need to wire it thru all the parent's functions and types as before.

What if I want communication back to the parent? For example, the parent needs to know when a cancel button is clicked on the child page to no longer show it. Easy. Make it an extra argument to the child's view function. Partially apply it.
let onClose () = dispatch Closed
Elmish.page ... (CourseDetail.view onClose)

When I looked around, I realized this was not a particularly original idea. This is the common way Fable.React.useElmish is employed, but being hosted directly from React JS components instead of parent MVU views. (Which also avoids all that parent-child wiring.) I also found common React libraries that tied the navigation route directly to the displayed React components, which only makes sense.

Composition through view would also naturally fit building as dynamically loadable ESM modules instead of a fully bundled app.

This one, I haven't tried yet. The Elmish.page function doesn't exist. I need to write that and figure out how to adapt it to Elmish.mkProgram. It looks to be technically possible, and I am excited to try it the next time I build another MVU app.

Summary

MVU is an amazing pattern that helps me write highly maintainable and correct UIs. I love it. That's probably why the core pattern remains essentially the same even after my fixes.

Yet, these changes have materially improved the experience for me. Maybe they could for you too.

Event Sourcing - Stream Metadata

Kasey Speakman — Sat, 29 Jan 2022 22:16:17 +0000

Discovering the need for stream metadata.

One of the best uses of Event Sourcing is to manage behaviors between entities, aka relationships. This is where all interesting things happen. For example, a product catalog is just a collection of data. It doesn't do anyone any good until those products are sold, shipped, restocked, etc. All of these processes occur between people or companies. They occur in relationships.

Relationships with Event Sourcing

These relationships need to be reflected somewhere in the event stream. In our domain of safety training, the main activity we track is training. Which involves a student and a course. We use a Training ID (a UUID) to identify each training process. If we were creating relational tables, we would include foreign keys to Student ID and Course ID to track the entities involved. How do we express this in an event store?

In starter event

In our first effort, we simply included reference IDs (Student ID and Course ID) in the starter event of our training process stream. For example, the starter event for a training process might be RegistrationCreated. Storing the reference IDs in the starter event supports executing workflows and running queries. (The command and query APIs in CQRS parlance.) Both of these use a "replay" process that can keep the reference IDs from the first event for later use. This worked out great for our first event sourced application. But then we built an app for a more complex domain.

In key events

We rearchitected one of our internal apps for the cloud. This time the domain is auditing. More specifically, verifying a vendor (contractor, supplier, etc) meets requirements before they are allowed to do work for a company.

We started out placing the reference IDs only in the starter event. But this strategy was problematic for some kinds of event listeners -- like process managers -- and for building more complicated data models. In these cases, the "starting" event for the listener / data model is different from the starting event for the stream. We ended up solving this problem by just repeating the same reference IDs on these key events. The minor downside is storing duplicate data.

As an alternative, we considered loading the reference IDs from an existing data model. But we decided this was a net loss. It would have created multiple degrees of coupling (server, code) between subsystems. And add complication to the infrastructure that would benefit relatively few cases.

Stream metadata

We ran into this again recently. Only this time I realized that these reference IDs (so far) are always static for a given stream. So they should be attached to the stream itself rather than specific events. They should be stream metadata. When the stream is "started", reference IDs are saved in stream metadata instead of on the initial event. When an event is loaded from the event store, the stream metadata can optionally be loaded too. So the reference IDs are available to listeners, regardless of which events they care about. No more repeating the same IDs in multiple events.

Prior art

I had seen this concept of stream metadata in my research on Event Store (Greg Young's open source product). But at the time I wasn't sure why I needed it. I have been burned before because I used tools incorrectly when I didn't understand them. So I did not include it in my Postgres event store implementation. In many ways I feel like I am just rediscovering things that Greg Young figured out years ago. Then documenting them in a way that my past self might understand. 😄

Shareholder Value: the Other Pandemic

Kasey Speakman — Sun, 09 Jan 2022 12:50:50 +0000

This post was inspired by Nvidia's recent announcement of the RTX 3090 TI. With its expected MSRP of $2000+, this graphics card comes at a time when many PC builders cannot find graphics cards at all. Most of them go into the hands of crypto miners or are flipped by scalpers at twice the MSRP. This is the latest example of Nvidia getting in on the scalping action too. By repackaging a lower-priced product line as a new SKU to inflate the price. Justifying it with a negligible performance increase. Then reducing or stopping production of the cheaper version. During these global hard times, Nvidia is shipping 12% more GPUs while making 84% more net income year-over-year. By squeezing it out of customers.

A Boring Dystopia

But none of this is new or surprising. (Or even among the worst examples.) It's one of the most cliche things of our time: the corporate zeitgeist of maximizing shareholder value. This philosophy seems to be accepted at every level as the way business is supposed to work.

"Jack CEO, if you can increase profitability 2% next year, the board will give you a 2 million dollar bonus." The CEO slashes employee benefits, gouges customers, and eliminates "waste" such as R&D spending. Profits and stock prices go up, making the investors hundreds of millions. The financial sector calls Jack CEO a hero. He is in higher demand because of this "success". So he'll probably be long gone when the company's market position enters the nose dive he setup. Employee hiring/retention problems and customer dissatisfaction mount as toxicity increases. The company fails to stay relevant without innovation from R&D. If the board keeps trying to find a repeat of Jack CEO's "success" the company might even get its own documentary. Like Enron or USA banks.

The story may play out in different ways, but this one illustrates the logical conclusion of focusing on shareholder value. Investors and executives get richer by sacrificing the company's future. But the giving end of the sacrifice is done by employees and customers. It's no wonder the wealth gap keeps getting wider.

Allow me to Translate

"Shareholder value" is just a dressed up word for greed. And it is enshrined in our global economic systems. Profit (or the potential thereof) is the measure of company health in financial markets. CEOs are required to prioritize it above all. Failing to do so opens the door to lawsuits. For example...

Having chosen a for-profit corporate form, the craigslist directors are bound by the fiduciary duties and standards that accompany that form. Those standards include acting to promote the value of the corporation for the benefit of its stockholders.
...
Thus, I cannot accept as valid ... a corporate policy that specifically, clearly, and admittedly seeks not to maximize the economic value of a for-profit Delaware corporation for the benefit of its stockholders...

eBay vs Craigslist

This lawsuit was about a few different things. The excerpt is about Craigslist creating policies designed to prevent eBay from destroying the company's culture to chase profit. Here the judge's interpretation of the law considers this a breach of duty. As a non-lizard-person I find this horrifying.

What Now?

I don't have any easy answers. My first reaction to hearing the Nvidia announcement was: I wish more product companies were non-profits. Only this would not be a direct solution. Investor money allows companies to expand and innovate at a more rapid pace. (Initially. Before it's ultimately enslaved to profit-seeking.) Whereas non-profits have to grow very carefully. There's no extra profit to reinvest. And no investor help for expansion shortfalls. A non-profit Amazon would have failed immediately instead of "almost" being a long-term Ponzi scheme.

Many jurisdictions have legal frameworks for Benefit or Social Purpose corporations. These enable other goals (in addition to profit) to be recognized by civil law. So the company could take investors but (maybe) still win a lawsuit for refusing to treat employees like test subjects.

The main challenge with these alternatives is that they are likely to struggle to be successful in the market. It is way more effort to achieve responsibility in multiple dimensions at once. Effort that increases the bigger the company gets. Most consumers will appreciate your company's approach but actually buy the cheaper or sooner-to-market option. Or in today's world, the in-stock option.

I waited in line 5 hours for my local Best Buy to open. Unfortunately most people waited at least 9 in their tents and RVs. By the time it was my turn to pick a graphics card, the only choices left were:

an EVGA-price-gouged 3060 TI
- only about 60% faster than my 7-year old 980 TI
a lobotomy-required EVGA 3080 TI for almost $1500
a white hot fury-inducing Nvidia 3080 TI at $1200

I decided to leave without buying anything. I began a lifetime boycott of Nvidia products. I waited 3 years for shortages to subside. Then bought a 9800 XT direct from AMD, because I had to boycott all their partner brands for price-gouging too.

Except that's not what happened. I spent money I didn't have to get the Nvidia 3080 TI.

I am part of the problem. But trying to find the path to change.

Desktop apps in HTML and F#

Kasey Speakman — Mon, 30 Aug 2021 22:21:20 +0000

I created a starter desktop app using HTML/CSS/F# for the front end and F# .NET for the back end. All hosted in WPF.

This post mostly covers the "why". Hit the repo if you just care about "how".

Developers at my company are mostly coding for the web. But once in a while we need to create a Windows desktop app. Because some situations require direct access to hardware and running software. But a big challenge with desktop apps is their UI tech.

Why web UI?

Our devs have a lot of skill investment in web tech. And I don't mean React and Webpack -- although we know a little about those. I'm talking about the open standards of HTML, CSS, and Javascript. Because of standardization I can use web tech to make a consistent-looking UI, no matter what device is consuming it.

Meanwhile desktop UI techs tend to be opinionated and proprietary. We develop for desktop infrequently, so non-transferrable learning investments into desktop UI tech is a hard sell.

Which web UI?

It has been possible to use web tech in Windows desktop apps for a while. Problem was, the solutions were full of compromises.

UWP, for example, allows you to build apps using web tech. But it runs sandboxed, making it hard to do the one job it needs to do: direct access. WebBrowser and WebView have also existed for a while. They are embeddable browsers for WinForms/WPF apps. But they use IE or EdgeHTML for rendering. Need I say more?

Why not Electron?

The elephant in the room is Electron. The main downside for us is the Node.js back-end. Some of our vendors do not provide libraries for it. And even if it can be awkwardly made to work using unsupported libraries, Node requires us to work in Javascript.

Electron's cross-platform support is probably its biggest benefit... but we don't need that.

Yes, earlier I did list Javascript as a positive thing for web tech. I do like the standardized browser APIs available thru JS -- Media, Storage, Intl, etc. But let's be honest, JS also has footguns baked into the standard. And I strongly prefer to use something else: F# on .NET.

WebView2

WebView2 is an embeddable browser for WinForms/WPF like WebView before it. WebView2 uses Chromium-based Edge for rendering. So unlike its predecessor, WebView2 keeps evolving with the web standards. And I get to use F# on .NET.

We have actually used WebView2 since shortly after its release. We used it to show HTML5 video content in a computer lab. It was a rush job and WebView2 was only a month old at the time. I wasn't ready to trust it, much less consider it for other uses. And I forgot about it.

Then earlier this year, Microsoft announced Blazor Desktop based on WebView2. And then I found out Microsoft Teams is ditching Electron for WebView2. This got me to take another look at it. One thing led to another and over the weekend I created a working demo: a WPF app with an F# web UI and F# .NET backend API.

How does it work?

The UI is a webpack-built JS app just like you would expect. It uses F# and Elmish for a functional (as in functional programming) UI. The F# code gets transpiled to JS and uses React for DOM rendering.

The "backend" API is an F# library. In this starter app, the backend is very simple. So there is nothing interesting to demonstrate yet.

A messaging library is shared between the front and back. It defines all the requests and responses that can pass between them.

The host application is WPF. WinForms would work too. Either way there is not much to it. The only control is the WebView2 control. And about 40 lines of wiring code.

For communication between the JS front and .NET back, I used WebView2's string-based messaging. Each side gets some functions to send/receive messages. Types from the messaging library are converted to JSON strings for transport.

The repo

kspeakman / FSharpHtmlDesktopApp

A starter desktop app which uses HTML and F# for the UI and F# .NET for the API.

Closing thoughts

This was a fun weekend project. It has a lot of promise to make desktop apps more manageable for our dev team. I'll try it out this coming month and let you know how it goes. For now, let me know what you think about making desktop apps with web tech and F#.

Readable vs Workable Code

Kasey Speakman — Thu, 05 Aug 2021 22:32:54 +0000

Balancing some code to be readable yet workable.

Today I was coding and decided to refactor some older code. In particular, this code is handling navigation updates in a Fable Elmish app in F#.

let routeToPage routeOpt =
    match routeOpt with
    | None ->
        let page, effects = OpsUser.Search.init None
        OpsUserSearch page, effects
    | Some route ->
        match route with
        | ToOpsUserSearch searchOpt ->
            let page, effects = OpsUser.Search.init searchOpt
            OpsUserSearch page, effects
        | ToOpsUserAdd ->
            let page, effects = OpsUser.Add.init ()
            OpsUserAdd page, effects

This function has many route cases but only two are shown here.

Now to refactor

First I started to remove duplication. Starting at the innermost duplication, notice how page, effect is repeated twice. We can remove this duplication with a helper function.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let routeToPage routeOpt =
    match routeOpt with
    | None ->
        OpsUser.Search.init None
        |> Tuple.mapFirst OpsUserSearch
    | Some route ->
        match route with
        | ToOpsUserSearch searchOpt ->
            OpsUser.Search.init searchOpt
            |> Tuple.mapFirst OpsUserSearch
        | ToOpsUserAdd ->
            OpsUser.Add.init ()
            |> Tuple.mapFirst OpsUserAdd

The mapFirst helper function is not specific to this code. It could be used on any tuple. So I made a small Tuple module that I can extract later if needed.

Next, all of these cases do the same thing, but using a different page tag, init function, and init argument. So we can make a helper to parameterize those.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let routeToPage routeOpt =
    let inline initPage pageTag init initArg =
        init initArg
        |> Tuple.mapFirst pageTag
    match routeOpt with
    | None ->
        initPage OpsUserSearch OpsUser.Search.init None
    | Some route ->
        match route with
        | ToOpsUserSearch searchOpt ->
            initPage OpsUserSearch OpsUser.Search.init searchOpt
        | ToOpsUserAdd ->
            initPage OpsUserAdd OpsUser.Add.init ()

I used inline here as micro optimization. You can omit/ignore that as it won't make any logical difference to the code. Generally it is best to avoid using it in other places unless you know how it will affect execution.

So the cases are down to one-liners. We can further split the main cases from the default one, into a separate function. And reuse that function to also initialize the default case.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let initRoute route =
    let inline initPage pageTag init initArg =
        init initArg
        |> Tuple.mapFirst pageTag
    match route with
    | ToOpsUserSearch searchOpt ->
        initPage OpsUserSearch OpsUser.Search.init searchOpt
    | ToOpsUserAdd ->
        initPage OpsUserAdd OpsUser.Add.init ()

let routeToPage routeOpt =
    match routeOpt with
    | None ->
        initRoute (ToOpsUserSearch None)
    | Some route ->
        initRoute route

Here initRoute will be the primary area that devs will modify going forward. We can also look at Option functions to further reduce duplication in routeToPage.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let initRoute route =
    let inline initPage pageTag init initArg =
        init initArg
        |> Tuple.mapFirst pageTag
    match route with
    | ToOpsUserSearch searchOpt ->
        initPage OpsUserSearch OpsUser.Search.init searchOpt
    | ToOpsUserAdd ->
        initPage OpsUserAdd OpsUser.Add.init ()

let routeToPage routeOpt =
    routeOpt
    |> Option.defaultValue (ToOpsUserSearch None)
    |> initRoute

To remove every ounce of fat we can use point-free style. This necessitates extracting initPage outside of initRoute. Consequently, I'll remove inline so if anyone later tries to reuse it they don't get unexpected performance issues. I will also separate out the default route as its own value. This extracts out the only thing in routeToPage that is likely to change over time.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let initPage pageTag init initArg =
    init initArg
    |> Tuple.mapFirst pageTag

let initRoute = function
    | ToOpsUserSearch searchOpt ->
        initPage OpsUserSearch OpsUser.Search.init searchOpt
    | ToOpsUserAdd ->
        initPage OpsUserAdd OpsUser.Add.init ()

let defaultRoute =
    ToOpsUserSearch None

let routeToPage =
    Option.defaultValue defaultRoute
    >> initRoute

Is this better?

So at this point I have removed all the duplication I can find. Now it's time to look at the bigger picture. How clear is this code? The experienced among you might say it is quite clear. But what if you were new to coding? How many different types of syntax must be mastered? How many layers must a dev trace through to understand routeToPage? Are there any syntaxes that are especially difficult for new devs to understand?

I like to consider readability from a new dev's perspective. This makes the code base more understandable to all readers including future-me. And pragmatically it allows more flexibility. I don't just have to hire experienced devs or devs with a specific background. Devs of all experience levels and backgrounds can transition between projects in less time.

From this perspective the code has a number of aspects that I think harms general readability. Here is what I will likely commit.

let initRoute route =
    let inline initPage pageTag init initArg =
        let page, effects = init initArg
        pageTag page, effects
    match route with
    | ToOpsUserSearch searchOpt ->
        initPage OpsUserSearch OpsUser.Search.init searchOpt
    | ToOpsUserAdd ->
        initPage OpsUserAdd OpsUser.Add.init ()

let defaultRoute =
    ToOpsUserSearch None

let routeToPage routeOpt =
    let route = routeOpt |> Option.defaultValue defaultRoute
    initRoute route

One challenge to readability is point-free notation. A lot of functional programmers like it. But it is not obvious to read at first glance. The point-free functions look like value assignments. You have to look inside the code block for specific signs that is point-free, like function or >>. Then the implicit parameter can leave a bit of doubt in the mind of a new programmer as to what is actually happening. In all I consider point-free to be more of an optimization than an abstraction. And it does not add sufficient value here for the duplication it optimizes away.

Next I considered Tuple.mapFirst. It was only used in one place. It results in more code than not having it. Worst of all, it makes initPage less readable. Because when you dig into mapFirst, you have to context-switch into the mechanics of a tuple, away from the problem you were originally trying to understand. For this reason, even if the function was included in the base F# libs I would avoid using it here for clarity.

I'm not entirely convinced of the value of initPage, but I will keep it for now. It seems readable. It saves a little code in the place where devs will most commonly be working. And understanding it is only a one-level deep journey.

I could have written routeToPage with all pipes.

let routeToPage routeOpt =
    routeOpt
    |> Option.defaultValue defaultRoute
    |> initRoute

But I think the way I coded it is more clear. Specifically, the first line is more obviously converting an optional route to a non-optional one. So even if the dev is new enough that they haven't encountered Option.defaultValue, the function is still understandable at a surface level.

Quite frankly, the end result of this process does not seem any more readable than the original. This speaks volumes about the value of explicitness even if it means duplication. But the result filters down the likely-to-change code into a specific function. And it provides a tool for reuse there.

Closing thoughts

Good code is like good literature: it forms a tangible picture in your mind. But it also is like a good workbench: it is easy to work on. It would be nice if there was a single tactic -- such as the Don't Repeat Yourself principle -- that could accomplish these goals. But I haven't found it yet. Instead good code requires trying different things and evaluating what works.

I shared my thoughts on balancing this code. How do you balance your code?

Organizing MVU Projects

Kasey Speakman — Tue, 29 Dec 2020 17:28:00 +0000

Strategies for organizing large Model-View-Update projects that have worked for us.

We have been working on functional UI projects using the MVU pattern for 4 years now. We first started using this pattern with Elm. Then we switched to full stack F#, using the Elmish library for MVU. We evolved our approach to MVU several times along the way. This post details some practices we learned.

This post assumes some experience with MVU. I do not mention the view function very much since we do not usually find it a challenge to organize. A view function for a particular page usually resides near that page's update function.

Modified TEA for UI state

The Elm Architecture (TEA) is one of the early ideas behind Elm. It defined a nested hierarchy of Msg, Model, Update, and View. But it was disavowed by Elm because it has a fatal flaw. TEA required a lot of laborious boilerplate to convert Msg between parents and children. It also made simultaneous active pages very difficult. We went away from it. But as our programs got larger, we found ourselves with gigantic Msg and update files, requiring us to use Ctrl-F to find things inside them. Basically everything was in a couple of huge piles of code instead of being organized for quick human access.

Quick stats
Latest project's UI code (F#)

Largest app - 35k lines of code

All apps - 65k lines of code

It became clear that we needed some organization. TEA is really just grouping all the things for a single page together in a module. This idea makes a lot of sense, so we went back to that for everything except Msg. All Msgs are declared first in their entirety, centrally and separately from the page parts. This means that every page module can directly send any messages they want, even to other pages. This got rid of all that Msg mapping and OutMsg patterns plaguing TEA.

Update 6 Jan 2021: I just checked out Elm's current documentation and it now mentions only Model, init, view, and update in page modules. So they must have come to a similar conclusion. When we were in the Elm community TEA included Msg but became disavowed because of its shortcomings, and the documentation about it was removed. Then there was a propaganda blitz for flat Msg. We switched away from Elm before this latest iteration on Elm app structure.

In our modified TEA, Msg can still have a hierarchical organization pattern as needed. So we can place a particular set of messages in their own file.

Remember that organization works for you, not the other way around. Every time you add a level of organization to your project, you also add more overhead (structure). It is better to keep things flat as much as possible. And only increase the degree of organization when you discover it is needed.

for UI state

An important subtlety in the section title is "Modified TEA for UI state". Often an app will have data that it is tracking independent of whatever the UI elements are doing. This kind of data should be placed outside the UI state. Everything in the UI state should be viewed as just what is currently being displayed, available to throw away at any time.

It is necessary to point this out because our industry is so accustomed to object-oriented UIs. There the ownership of state is distributed among many different objects. This can lead MVU users to try to make UI pages "own" specific kinds of data on behalf of other pages. But with MVU, pages only represent what is displayed on the screen. This is the way.

This has a couple of major advantages. It eliminates a common source of bugs: two different pages using shared state inconsistently. It also eliminates the need to manage the UI state's lifecycle, since it is part of the page lifecycle.

// changing pages
let pageData, cmds = Home.init ()
{ model with Page = Home pageData }, cmds

In the above example, the important thing here is what is missing. There is no code to check the previous page. Or clean up shared state. It is not needed. Any data that the page needs it will request or create on init.

Independent data

What if pages need to display or update that shared, independent data? The best answer really depends on the situation, but here are some approaches. If the independent data is read-only reference data, you can pass it as an argument to the page's init.

For a more robust approach, treat it as an external integration. This works for cases where the state isn't static or pages can request changes to it. For this, provide Cmds to make these requests. Also Msg cases to process changes in an MVU style. This is a headless MVU service pattern, where the independent data is the service state.

What does "independent data" look like?

I keep using this term independent data like it is something special. It is not. To demonstrate how "not a big thing" it is, here is an example Model.

type Model =
  { ApiKey: string
    Page: Page }

Here, ApiKey is what I am calling independent data. And Page is a piece of UI state. The app model could contain multiples of either kind. The Msg for this app might contain a set of messages to manipulate either or both parts of Model. It depends entirely on what the app needs.

`update` refinements

UI states are just data that represents what is on the screen. So in them we make extensive use of both records and (discriminated) unions. Records are what most people are most accustomed to, similar to structs or DTOs. Unions like you will see below are used when the state can only be one of multiple possibilities. Here is a typical example of a UI state:

type Page =
  | Home of Home.Model
  | CourseSearch of Course.Search.Model
  | CourseDetail of Course.Detail.Model
  // and so forth

This is basically spelling out that Page can be in one of these states. And then each page also has its own inner state (for example Home.Model), which is also some combination of records and unions.

That is well and good, but there are a couple of practical issues when we go to use this in update. With this structure we have to dig the page data out of the union type. That is Problem #1. Separately, since Msg is an app-wide type -- where any page can send any message -- now it is possible to send a Msg to a page which is not active. That is Problem #2.

Technically #2 was always possible. A delayed API operation and an impatient user can lead to receiving a response for page which is no longer active.

To resolve both of these problems, we went to an update style that looks like this:

let update msg model =
  match model.Page, msg with
  | Home pageData, HomeMsg pageMsg ->
    let pageData, cmds = Pages.Home.update pageMsg pageData
    { model with Page = Home pageData }, cmds

This serves to unwrap the page and message data from their respective union types. While also verifying we are handling a message appropriate for the UI state.

You might also notice this has a strong similarity to something else -- a state machine. At its core, a state machine is just a set of states and the valid transitions between them. This match is spelling out the valid transitions (messages) for each UI state.

We also do something that many of you FPers will not like. We use a catch-all match for any unmatched state transitions. This allows us to specify only the scenarios that we want to handle and ignore the rest.

let update msg model =
  match model.Page, msg with
  ...
  | _ ->
    // no changes
    model, []

This is a classic expression problem trade-off of extensibility vs completeness. With the underscore match we are choosing extensibility. By choosing this we lose the advantage of the compiler telling us of intended matches that we forgot. But we also do not get the multitude more unintended matches that we want to ignore. We believe our choice makes sense under the circumstances. Logging the catch-all case provides enough of a hint when we forget to add an intended case.

Routing and Navigation

A "route" is simply enough data to load a specific page. In MVU, pages are constructed by their init function. Therefore a route is just the arguments to the page's init function, along with a tag to indicate the page. It is so much simpler to think about it this way, that I eventually started naming this InitArgs in new projects instead of Route. And this can be used practically to package up all the arguments to the init function. Here is an example.

// PageTypes\Course\Detail.fs
module Course.Detail =
  type InitArgs =
    { CourseId: int }

// AppTypes.fs
type InitArgs =
  | HomeInit of ()
  | CourseDetailInit of Course.Detail.InitArgs

// PageFns\Course\Detail.fs
module Course.Detail
  let init initArgs =
    { CourseId = initArgs.CourseId
      Data = Loading }
    , [ LoadCourseDetail initArgs.CourseId ]

I'll cover the folder organization of the project later.

Switching pages then is a matter of adding a Msg case.

// AppTypes.fs
type Msg =
  | SwitchPage of InitArgs

And handling page changes looks like this.

let switchPage initArgs =
  match initArgs with
  | HomeInit pageArgs ->
    let pageData, cmds = Home.init pageArgs
    { model with Page = Home pageData }, cmds
  | CourseDetailInit pageArgs ->
    let pageData, cmds = Course.Detail.init pageArgs
    { model with Page = CourseDetail pageData }, cmds

let update msg model =
  match model.Page, msg with
  | _, SwitchPage initArgs ->
    switchPage initArgs

Like Msg, InitArgs must be centralized and declared ahead of time so that any page has the ability switch to any other page. Or to calculate a URL so the user can switch pages with a link. Speaking of...

adding URL navigation

The main code you have to fill in for this involves converting InitArgs toUrl and fromUrl. Here is an example.

module InitArgs =

  let toUrl initArgs =
    match initArgs with
    | HomeInit () ->
      "#/home"
    | CourseDetailInit { CourseId = courseId } ->
      sprintf "#/course/%i" courseId

  // using Elmish.UrlParser
  let parseInitArgs state =
    oneOf [
      map HomeInit (s "home" </> unitArg)
      map (fun courseId ->
               CourseDetailInit { CourseId = courseId })
          (s "course" </> int)
    ] state

  let fromUrl location =
    parseHash parseInitArgs location

toUrl is mainly for your pages to generate navigable links to other pages. It is optimal to change pages with links when possible, so that the browser's back/forward buttons work as the user expects. Sometimes it is also desirable to manually update the URL from your app when the page options change significantly so that this URL gets added to history.

fromUrl is used in navigation events to pick the page to display. The function will also get used on app init, in case the user came to your app from a bookmark at a specific page. All that remains is to wire up those cases. MVU libraries have built-in integration for this. It can be done manually as well. Pass the location into app init to handle it there. And add an event listener for location changes, tag it in a Msg case like LocationChanged and dispatch it. Then handle it as another case of update where you call fromUrl and switchPage.

Side effects

A main emphasis of functional programming is pure functions. The complement of this emphasis is that side effects become an explicit concept. In MVU, explicit side effects are represented by Cmd.

However Cmd implementations have some less-than-ideal characteristics. In Elm they were very painful to extend with your own functionality (ports). In F# Elmish, Cmd simply defines a function signature which enables you to do whatever you want and dispatch Msgs to report the results back. This is about the most extensible you can get.

However I saw some opportunities for improvement. First it seemed like a mix of concerns to have update potentially creating a side effect function. Secondly it left update less testable than it could be. You can test the model easily, but it is quite invasive to test whether the correct side effects were requested. The ideal would be for Cmd to be just data representing the side effect and its necessary arguments.

`Effect`

I called the type Effect. It is entirely user-defined. I did not want to call it Cmd since Elmish already uses this name.

type Effect =
  | GetCurrentDateTime
  | SendApiRequest of ApiRequest

The update function returns an Effect list instead of Cmd.

let update msg model : Model * Effect list =
  match model.Page, msg with
  | Home pageData, HomeMsg pageMsg ->
    let pageData, effects = Home.update pageMsg pageData
    { model with Page = Home pageData }, effects

This completely decouples side effects from the update function. Effect is just data and is not tied to any implementation. It also makes it possible for Effect to be value equatable. Which means you can simply setup some expected data and use = to test that actual output matches, just like with Model. No mocks, stubs, or anything invasive at all. So now when you test update you can not only test model changes, but also whether the correct Effects are triggered.

It is sometimes necessary to work with unpredictable reference data like JS Files inside Effect. A match statement can be used to specially check those cases and use value equality for the rest.

The last piece is to code up the side effect implementation. I call the function for this perform.

let perform config effect dispatch =
  match effect with
  | GetCurrentDateTime ->
    dispatch (CurrentDateTimeIs System.DateTime.Now)
  | SendApiRequest request ->
    async {
      // http request details elided
      dispatch (ApiResponseIs result)
    } |> Async.StartImmediate

For Elmish, this new Effect/perform pattern is pretty easy to integrate. It just requires a couple of helper functions.

let init arg =
  let (model, effects) = App.init arg
  (model, effects |> List.map (App.perform model.EffectConfig))

let update msg model =
  let (model, effects) = App.update msg model
  (model, effects |> List.map (App.perform model.EffectConfig))

Program.mkProgram init update App.view
|> Program.withReactBatched "elmish-app"
#if DEBUG
|> Program.withConsoleTrace
#endif
|> Program.run

Here, EffectConfig is where I keep data needed by side effects. For example: API URLs, local storage keys, auth endpoints, etc. These are usually settings found in a JS file deployed with the app. They are grabbed just before the program starts and passed in as the init argument.

Practical issues

My web apps so far have a small number of highly reusable effects, so I tend to make Effect just a flat universal type that any page can use. In larger apps, it can be annoying to get the return Msg back to the specific page that requested it. What I presented above has perform sending responses back with non-page-specific Msgs. So you'd have to add a match for this message to each page that might receive one of these responses.

An alternative approach that I later found my devs using is to provide a return tag with the Effect. This tag is a function which takes the return value and wraps it in a Msg. The Msg would target the specific page that requested it. This hinders the testability of Effects, since you now have to specially treat most of the Effect cases to ignore the tag function and only test the equatable data. But it works since we only have a dozen or less effects, and a handful that use return tags.

You could also avoid this by customizing the available Effects per page. But we haven't found this worth the extra code.

To be perfectly honest, we do not test update currently. In our years of working with MVU there has not been a strong need. Yes we have broken things, but with the other mentioned principles they have been easy to spot and fix quickly. Yet we still wanted the ability to easily test update in case we need to introduce dynamically generated UIs.

The way we handle side effects is my favorite adaptation to the original MVU pattern. Because over the years I have observed that Separation of Concerns is the most important principle to maintainable software. And effect-as-data provides this via loose coupling between logic and side effect implementations.

Project organization tricks

Note on Model

Originally TEA had Msg, Model, init, and update inside the page module. We had to move Msg and the newly invented InitArgs out of the page module and fully define these types for all the pages before the page function definitions.

We can technically keep a page's Model with its functions. But then Model would be the only type that we defined there. So we ended up just moving it up as well. In essence, we split the page's related parts into PageTypes (defined first) and PageFns.

A typical project file structure would look like this.

App project

PageTypes
- Home.fs - Home's Msg, InitArgs, Model
- Course
  - Search.fs
  - Detail.fs
AppTypes.fs - App's Msg, InitArgs, Model
InitArgs.fs - has toUrl, fromUrl
PageFns
- Home.fs - Home's init, update, view
- Course
  - Search.fs
  - Detail.fs
App.fs - App's init, update, view, perform
Main.fs - Elmish Program wireup

F# has a single-pass compiler. And file order is compile order. So this organization structure is almost like we are creating our own two-pass compiler where types are compiled first, then the functions that use them. If F# had a 2-pass compiler -- types then functions -- it might be possible to keep all the page parts organized in a single module.

VS 2019 bug

The project structure would look like above. Except Visual Studio has a long-standing F#-specific bug. When you have two file paths that have the same subfolder as part of the path, intellisense glitches out. In the example above PageTypes and PageFns both contain a Course subfolder. Any file in the PageFns\Course subfolder will have completely broken intellisense as well as any other file defined after it. It also stops displaying the correct file order in the Solution Explorer window.

We simply name the PageFns subfolders with an underscore on the end (PageFns\Course_\Search.fs) to make the path different. The module names are kept the same (no underscore).

Another way around this is to not use folders but instead use multi-dotted files. Example: Course.Search.fs. Of course, this sacrifices the ability to roll-up/hide all the Course files when I am not using them.

I hope this gets priority to fix soon, because it is embarrassing to have to mention it in a post like this one.

Trick: Merging modules

One really nice property of F# is that opening two different namespaces will effectively merge all the types and functions across same-named modules within them. For example.

open PageTypes
open PageFns

type MyMsg = Home.Msg
let myInit = Home.init

// "Home" is a combination of all the stuff from:
// PageTypes\Home.fs
// PageFns\Home.fs

This still allows you to use all the page parts as though they were under one module. This is also how I extend existing types with new functions.

Pain point

The one major pain point with all the tactics I describe above is structural duplication. What I mean by that is Msg, InitArgs, and Model all have the same basic tree structure, but with different types as leaf nodes. To avoid conflicts, I must name them slightly differently.

// AppTypes.fs
type Msg =
  | HomeMsg of Home.Msg
  | CourseSearchMsg of Course.Search.Msg
  | CourseDetailMsg of Course.Detail.Msg

type InitArgs =
  | HomeInit of Home.InitArgs
  | CourseSearchInit of Course.Search.InitArgs
  | CourseDetailInit of Course.Detail.InitArgs

type Page =
  | Home of Home.Model
  | CourseSearch of Course.Search.Model
  | CourseDetail of Course.Detail.Model

Anyone familiar with Haskell is probably jumping up and down, screaming "Higher-Kinded Types". F# doesn't have those.

However I can think of 3 ways to reduce these 3 tree structures to 1 in F#. I will list them (quite subjectively) from least desirable to most.

These are thought experiments. I have not tried them in a real app yet.

3. Generics as fake HKTs

The setup for this is worse than the original solution.

type Area<'home, 'courseSearch, 'courseDetail> =
  | Home of 'home
  | CourseSearch of 'courseSearch
  | CourseDetail of 'courseDetail

type Msg =      Area<Home.Msg,
                     Course.Search.Msg,
                     Course.Detail.Msg>

type InitArgs = Area<Home.InitArgs,
                     Course.Search.InitArgs,
                     Course.Detail.InitArgs>

type Page =     Area<Home.Model,
                     Course.Search.Model,
                     Course.Detail.Model>

Adding a new page is pretty egregious. You have to touch Area in 2 places, then add a line to each of Msg, InitArgs, and Page.

The only improvement with this approach is that the update function is marginally nicer than the original solution.

let update msg model =
  match model.Page, msg with
  | Home pageData, Home msgData ->
    ...
  | CourseSearch pageData, CourseSearch msgData ->
    ...

Some people like this approach, but overall I think this has more losses than gains.

2. Union leaf data

This one is also more work to setup initially.

type Leaf<'msg, 'init, 'page> =
  | Msg of 'msg
  | Init of 'init
  | Page of 'page

type Area =
  | Home of         Leaf<Home.Msg,
                         Home.InitArgs,
                         Home.Model>
  | CourseSearch of Leaf<Course.Search.Msg,
                         Course.Search.InitArgs,
                         Course.Search.Model>
  | CourseDetail of Leaf<Course.Detail.Msg,
                         Course.Detail.InitArgs,
                         Course.Detail.Model>

Adding a new page is a slight improvement over the original solution. It is typing a few lines in one place instead of a new line in a few places.

The update function is slightly more verbose than the original solution.

let update msg model =
  match model.Page, msg with
  | Home (Page pageData), Home (Msg pageMsg) ->
    ...
  | CourseDetail (Page pageData), CourseDetail (Msg pageMsg) ->
    ...

The weird part of this approach is that model.Page and msg have exactly the same type. That means you can accidentally swap them and not get a compiler error. Then the UI won't work and there is no obvious reason why. Although, it should be possible to add some match cases to detect this and log a warning at runtime.

Overall I'd prefer this over #3, but it is still not ideal.

1. Only Msg

The last way I can think of to get one tree structure involves just eliminating InitArgs and Page, leaving us with only Msg.

Bye InitArgs

Getting rid of InitArgs is pretty far-reaching, but also has other benefits. Essentially the init function is removed and its behavior is put in a new case of update function. Then the InitArgs for that page becomes a Msg case. We will have to add one thing, a "zero" or empty model.

// PageTypes\Course\Detail.fs
namespace PageTypes.Course

module Detail =

  type Msg =
    | Init of courseId: Guid
    | CourseLoaded of Result<Course, QueryError>

  type Model =
    { CourseId: Guid
      Data: Remote<Course> }

  let empty =
    { CourseId = Guid.empty
      Data = Loading }

// PageFns\Course\Detail.fs
namespace PageFns.Course

module Detail =

  let update msg model =
    match msg with
    | Init courseId ->
      { model with CourseId = courseId }
      , [ ApiRequest (GetCourse courseId) ]
    | CourseLoaded (Ok course) ->
      { model with Data = Loaded course }, []
    // and so on

  let view model dispatch =
    // stuff

We have to do special handling of the Init message in the main update. We are using this kind of code instead of the switchPage function from before.

// App.fs
let update msg model =
  match model.Page, msg with
  | _, CourseDetail ((Course.Detail.Init _) as pageMsg) ->
    let pageData, effects = Course.Detail.update pageMsg Course.Detail.empty
    { model with Page = CourseDetail pageData }, effects

I used a similar approach when I built a Clojure MVU library.

Moving Page

UI state like Page is a bit different from Msg or InitArgs. The details of it are only used by the page functions. Outside parties do not need to know the page Model's contents. Previously we set it up as a public tree type for consistency with how we are handling other types. But there is another way. We can use a marker interface.

A marker interface is just an empty interface. Any type can just say it implements a marker interface, since there is nothing to implement. Marker interfaces can be looked at as a slightly different union type. You do not have to define all cases up front... a type locally chooses to opt in to the marker interface. But you also do not get compiler guarantees of complete matches. It is another expression problem trade-off toward extensibility rather than completeness.

// Types.fs
type IPage = interface end

// PageTypes\Course\Detail.fs
namespace PageTypes.Course

module Detail =

  type Model =
    { CourseId: Guid
      Data: Remote<Course> }
    interface IPage

// AppTypes.fs
type Model =
  { EffectConfig: EffectConfig
    Page: IPage }

// App.fs
let update msg model =
  match model.Page, msg with
  | :? Home.Model as pageData, Home pageMsg ->
    ...

We can essentially "tag" each page model as IPage. And any IPage can be stored in the Model.Page property. Unwrapping it to a specific page's model is a few more keystrokes. But there is no central Page union type to maintain anymore.

I have not tried this beyond checking that it would compile in a Fable Elmish project.

End result

At this point, we have eliminated all the duplicate tree structured types and are left with only Msg. We have also eliminated the init function. So the page logic is centralized in update.

Conclusion

Over the last 4 years I have made MVU my playground and learned quite a bit. It is a surprisingly resilient and flexible pattern for organizing UI applications. Because of its state-machine-like qualities, we even use it server-side. Hopefully some of the adaptations I discovered have been interesting to you.

This post has a lot of code snippets, but it could use a companion repo with more complete examples. With my ADHD, an effort like that will never go anywhere if left up to me. So if something like that is of interest to you, let me know. I would be more than happy to contribute.

cover image from unDraw

Addendum

I tried the #1 Only Msg approach and faced some challenges.

Moving InitArgs into Msg and removing `init`

Moving InitArgs into Msg

One thing I didn't cover was how this affects navigation. Instead of converting InitArgs to/from URLs, this now means converting Msg to/from URLs. The data to create a URL needs to be dug out of the specific page messages. And there will need to be a catch-all match because only a small amount of messages correspond to URLs.

match msg with
| CourseDetail (Course.Detail.Init initArgs) ->
  "#/course/" + string initArgs.CourseId
| _ ->
  defaultUrl

Alternative InitArgs tactic

It has become our standard practice to create an InitArgs record for each page. This record is a container for all the necessary parameters to the init function. And these page InitArgs need to be app-wide like page Msgs. So any page may use them to construct links to other pages.

We could instead use tupled values or anonymous records for a page's init args.

type InitArgs =
  //| CourseSearchInit of Course.Search.InitArgs
  | CourseSearchInit of search: string * page: int * pageSize: int

We then avoid having to define and organize those page InitArgs records. But some duplication of the parameter definitions may be necessary. For example in init if the type would be ambiguous in its usage.

let init (search, page, pageSize) =
  // ambiguous type error, requires annotating search as string
  let normalizedSearch = search.Trim()
  ...

And of course there are the same tradeoffs of using tuples vs records. When creating a new value, records are more verbose but easier to understand because the values are labeled.

Using tuples versus organizing all the page's InitArgs types to be app-wide, I do not think there is a clear winner. So you just have to weigh the tradeoffs on a case by case basis. It is probably easier to start with tuples if unsure.

Removing `init`

When removing init, a disadvantage is sometimes empty values can be a pain to construct. Like arbitrarily nested records. (I mainly encountered this when trying to remove the overall app init.) So using an explicit init function can feel more natural in those cases.

Overall

This does not feel like a worthwhile change. InitArgs is an additional app-wide type, but it has its own specific usefulness -- converting to/from URLs. And although init usually feels like a special case of update, it can sometimes be useful.

Moving Page

I mentioned using the IPage marker interface to represent pages instead of a central DU. But because F# does not auto-upcast, the main update has to explicitly upcast every page model to IPage. That's arguably worse than just tagging the page-specific model to go in a central DU.

One of the big benefits of using a marker interface is keeping the page Model with its init and update functions. And this is still possible when using a DU. Simply define the central Page DU in the main app file with the app's init and update (versus declaring it app-wide like Msgs). We used to organize it this way, but when I moved the other types (Msg, InitArgs) out of the pages I dragged Model with them. So I over-organized that.

Closing thoughts

Turns out that some of the annoying duplication I mentioned in the main article adds more value than it costs. And there are still minor tweaks available to tone down the annoyance. Without fundamentally deviating from the standard MVU structures.

Entity Component System - an old new thing

Kasey Speakman — Sun, 13 Dec 2020 00:58:31 +0000

Structured Programming rediscovered and honed for game development.

I've been looking into game dev lately. Always been a gamer. Always wanted to make games. But getting into it seems completely overwhelming. I found a podcast on DEV with Jason Weimann talking about game dev, and then his youtube channel with videos like this one.

His videos are incredibly helpful. But I noticed that many of them are centered around dealing with Object-Oriented ceremony and footguns. This is because Unity uses the OO model pervasively, and requires your code to use it as well. Being an old dev interested in system design, I went looking for other game dev organization schemes. I discovered the Entity-Component-System or ECS architecture.

What is ECS?

The three parts of ECS are:

Entity - a game object with a unique ID, contains components
Component - data about a single aspect of a game object
System - logic for a single aspect of game objects

The obvious interaction here is to pass the Entity to a System behavior. Such as "add to inventory" in the Inventory system or "take damage" in the Health system. The behavior can check if the Entity has a component that it knows how to work with. And if so perform the requested behavior on that component.

ECS cleanly separates the systems of the game into maintainable slices. The Unity folks seemed to think so as well since in 2018 they made an announcement that they were adding ECS to Unity. Not only for maintainability reasons but for performance.

At the end of 2020 this still remains to be seen. They have now hidden the beta ECS packages. (You can still install if you know the URL.) Presumably because there is so much churn that it is guaranteed to break game code in future updates.

This looks familiar...

On the whole the system ends up breaking down into data structures with no behavior, and functions or procedures that operate on those data structures (and the implicit side effects those procedures cause) to produce a behavior. This is the same arrangement used by Structured Programming.

Structured Programming came about in the late 1950s, at first primarily being about block structured syntax. And later officially branded in a 1966 paper by Corrado Böhm and Giuseppe Jacopini. But probably more well known as championed by Edsger W. Dijkstra, where we got our "... Considered Harmful" memes.

Like most things that are created and released into the world, the creators and proponents of this paradigm probably assumed it would be used in a certain beneficial way. To tame the spaghetti of Assembly code. But instead of using the tools to organize our code we just moved the spaghetti into procedures with Procedural Programming.

Most languages have procedural programming -- a descendant of structured programming -- as the substrate underneath their paradigm. Meaning once you get past the declarations the code is written as a statement-oriented and block-structured procedure.

What is different?

ECS is an architecture, not a program structure

The primary difference is purpose. Structured Programming is meant to put your program on good footing for maintenance. A video game is just a program, right?

Look at it this way. If a business has a universe of applications and services working together to deliver business value, a video game is a microverse (or sometimes an even larger universe) of all the same kinds of things working together to deliver user value (game experience). The breakdown of a triple-A game into its many systems, when translated into the business world, would be the domain of a software architect. So in essence ECS is not simply a program structure, but a base architecture.

Entities have runtime composition

This significant difference in packaging compared to business systems leads to another important adaptation. Namely, you can add or remove arbitrary Components (data structures) from an entity at runtime. This is quite different from structured programming where data structures were completely pre-defined. Additions and removals of this kind would need a recompile and redeploy.

Since ECS is an architecture, this property of entities makes perfect sense. Put on your Business Dev hat and consider a Customer. Accounting does not need to know about every sales lead... only the ones that buy something. And then they care about their own specific aspects of that customer which sales may not, e.g. billing address. In this analogy the billing address is part of the Accounting ECS component, but not the Sales component. And Accounting ECS system requires that component it be present to perform its function.

Remember, an ECS Component is only data. State would have been a better name for it, but nobody asked me back in 2007 when they named it.

Other connections

I have previously made a connection between Structured Programming and Functional Programming. Basic FP (without all that category theory) takes Structured Programming and draws out one more key concept -- the side effect -- and makes it first class and explicit instead of implicit. This greatly increases the testability and maintainability of code. And so I think ECS could create a strong synergy between game dev and FP. Wait: FP and game dev -- am I crazy? I don't think so... check out John Carmack's thoughts on the subject.

Conclusion

Like most new ideas, ECS stands upon the shoulders of giants that came before. It is not new at its core, but how it is applied is novel. Unity seems to be a popular and "easy" entry into the world of game dev. I look forward to trying their ECS system. Especially since F# is also usable with Unity. ECS and F# would make the ultimate combo for me.

Clojure validation in 30 lines

Kasey Speakman — Tue, 20 Oct 2020 09:46:55 +0000

A function composition approach to validation.

In the previous post, I offered some critique of the validation approaches I found. In his post I bash together an alternative solution.

Goals

Here are the things I was interested in.

validation definitions are functions
composable from smaller pieces
no macros
simple errors

5 functions

There are 5 core functions to compose validations.

`fn`

This creates a basic validation.

(require '[validation.core :as v])

(def validate
  (v/fn :required not-empty))

(validate "asdf")
=> "asdf"

(validate "")
=> {:v/error :required}

(v/error (validate ""))
=> :required

Any function can be used for validation provided it returns nil if anything goes wrong. When that happens, :v/error is returned with the provided trait (:required here).

You provide a trait so you know exactly the errors to expect.

`and`

This combinator can be used to send a value through a series of transformations, each of which might fail. The end value will be returned if all steps are successful. Otherwise the failing trait will be returned as an error.

(def validate
  (v/and (v/fn :required not-empty)
         (v/fn :keyword keyword)))

(validate "asdf")
=> :asdf

(validate "")
=> {:v/error :required}

Note: This is not a strong validation because it does not deal well with non-string inputs or whitespace.

`seq`, `hmap`

Validate a sequence.

(def validate
  (v/seq
   (v/and (v/fn :required not-empty)
          (v/fn :keyword keyword))))

(validate ["a" "b"])
=> (:a :b)

(validate ["a" ""])
=> (:a {:v/error :required})

Validate a hash-map.

(def validate
  (v/hmap
   {:a (v/fn :required not-empty)
    :b (v/and (v/fn :required not-empty)
              (v/fn :keyword keyword))}))

(validate {:a "a" :b "b"})
=> {:a "a", :b :b}

(validate {:a "a" :b ""})
=> {:a "a", :b {:v/error :required}}

(map :b [{:a "a" :b ""}
         (validate {:a "a" :b ""})])
=> ("" {:v/error :required})

`or`

This combinator returns the first passing validation.

(def validate
  (v/or :date-or-int
        (v/fn :date str->date)
        (v/fn :int str->int)))

It takes an additional trait since returning every failing trait would not would not be entirely useful.

Validation functions

The 30 lines of code are just the definition of the 5 core functions and 2 error checks. Validation functions are still needed to plug into them. Clojure has some built-in functions that can be used but you are probably going to want more. Fortunately they are easy to create. Here are some examples.

(defn non-blank [s]
  (when (string? s)
    (not-empty (trim s))))

(defn in-range [min-val max-val]
  (fn [v]
    (when (and (>= v min-val)
               (<= v max-val))
      v)))

When a validation takes multiple parameters as in-range does above, it should return a function to take the last parameter. This bakes in the necessary partial application.

(def validate
  (v/fn :range (in-range 0 4)))

(validate 3)
=> 3

(validate 5)
=> {:v/error :range}

Conclusion

I really like the function composition approach to validation. It is composable and extensible. Its implementation is very small (see below). Which means I can easily change or add more functions. I find it works quite well for user input validation.

The 30 lines

(ns validation.core
  (:refer-clojure :exclude [and or fn seq]))

(defn fn [trait f]
  #(if-some [value (f %)]
    value
    {::error trait}))

(defn and [f & fs]
  #(loop [v (f %) fs fs]
     (if (contains? v ::error)
       v
       (let [f (first fs)]
         (if (nil? f)
           v
           (recur (f v) (next fs)))))))

(defn or [trait f & fs]
  #(loop [possible (f %) fs fs]
     (if-not (contains? possible ::error)
       possible
       (let [f (first fs)]
         (if (nil? f)
           {::error trait}
           (recur (f %) (next fs)))))))

(defn hmap [m]
  #(reduce-kv update % m))

(defn seq [f]
  #(map f %))

(defn error? [v]
  (contains? v ::error))

(defn error [v]
  (::error v))

Validation approaches in Clojure

Kasey Speakman — Fri, 16 Oct 2020 01:51:59 +0000

Classifying the different approaches I've seen and some of their trade-offs.

Lately I have been struggling to come up with a good validation library in Clojure. And it led me to discover and evaluate the different approaches.

Why not use Spec?

Spec is great at type specification, but is not so great for validation. Because spec intentionally avoids data transformations.

Especially on the web, most input values are strings. And frequently they need to be sanitized and transformed before they are ready to have logic performed on them. Since Spec does not allow you to transform data during validation, it is not sufficient for this use case.

Well, sortof
Using spec for transformation is discouraged by the authors as mentioned in the link and alluded to in the documentation. However, Sean Corfield rightly pointed out in Clojure Slack that Spec is extensible to allow data transformations. And his company has a library which does so.

Approaches

Here are some approaches I have seen.

Reverse-engineer Specs

This is a common theme for Clojure validation libraries. You first make a spec, and then the library will attempt to make the data you provide fit into that spec. So if the field is int? and it finds a string, it will automatically convert using a known string-to-int conversion.

The big problem with this approach is the lack of control over the validation process. It is very common for me to have transformations and validations both before and after the user input is coerced.

For example when the user enters a monetary value, I trim and verify the field is not empty first (as a separate error) before bothering to convert to a number. After converting, I transform the floating point number entered by the user into a whole number of cents. Neither the blank-ness nor the cent transformation is representable by Spec and cannot be reverse engineered from an int? spec.

Some of the libraries provide extra APIs to plug in those things. But then it becomes learning the custom API of that library, undercutting the point of using Spec. And spreading out the validation steps in different places.

Declarative validations

These kinds of libraries create a DSL for validations so that you can specify :required for example. And it has code that it invokes when it sees that keyword.

The main trade-offs with this approach is learning the library's DSL (non-portable knowledge). And then figuring out how to work around it when it lacks enough expressiveness for your exact problem. These are often developed based on the problems the author encountered, and are not general enough to a wide range of uses. Once the DSL becomes general enough, then it has become a new language!

This is also explained in the Spec overview section Code is data (not vice versa).

Composition and Partial application

This is what I used in F# to great effect. Validations are expressed in code and value transformations chain forward into the next one. Since it is code you can do most anything. It mainly works as a concise option because F# is curried by default and partial application is automatic.

You see, in F# functions always expect a precise number of arguments. So if you specify less arguments, F# will automatically partially apply those arguments and return a new function which only takes the remaining arguments. That allows you to tersely compose functions to do everything you need for the validation. Then apply data values later.

Clojure functions can take a variable number of arguments, so automatic currying and partial application are not available. Instead, the partial function is used to do partial application and/or comp for function composition.

I haven't found Clojure validation libraries which take this approach. Although I believe it to be possible.

It is also fair to say that composed and partial functions are not particularly easy to read at first. It takes practice to get accustomed to them. But the knowledge is generally portable.

Macros

This path adds new semantics to the language so that you can use code but also have a desired expressiveness. This is the path that Spec chose.

The main trade-off with macros is the same one that always exists with macros. Unlike regular functions, the interaction of different macros can become unpredictable. Since macros literally transform the code. (See go-block limitations, for example.)

To put it another way, macros specialize the language toward one purpose. And that specialization may conflict with another. So this approach has to be undertaken very thoughtfully.

Summary

So that is a sampling of the different types of validation approaches that I found in and around Clojure.

Personally I prefer the composition approach without macros. I am still trying to find a way to represent this well in Clojure. If I can find that, the implementation should be small and user extensible. Let me know if I missed a library that is already taking this approach.

Life after Elm: What we did

Kasey Speakman — Mon, 12 Oct 2020 00:32:21 +0000

It's been just over 2 years since I wrote Elm 0.19 Broke Us 💔. Where did we go from there and how did it compare?

Elmish and F #

For our latest project, we chose Elmish and F#. Elmish provides Elm-like abstractions for F#. Fable translates F# to JS. And for view rendering it uses React. Elmish is general enough that it also works in other UI contexts. There are libraries that adapt it to desktop and mobile applications.

F# was mostly a smooth transition for us. The language is in the same ML family as Elm so the syntax differences are quite minor. The biggest hangup was probably the capitalization differences. We were already using F# on the back-end, so there was at least a little familiarity with it from front-end devs. (Who are now back-end devs too.)

Unnecessary Worries

Some of the big worries that we had about transitioning turned out more theoretical than actual. Here are the ones we maybe focused too much on before switching.

Auto-formatting

Our normal coding workflow depended pretty heavily on the official Elm auto-format. We would often just vomit (especially UI) code into the editor wherever and hit save to watch it snap into its right place. So we thought this would be a big downside since there was no official or obvious path to do this in F#.

Developing a product in F# Elmish for almost 2 years now, I can say that it didn't turn out to be a big deal. One of the major factors is that newlines count as whitespace in F#. Getting all those separators rightly placed is one of the main benefits of Elm auto-format and with F# that requirement is just gone. Secondly, we had the freedom to express code in ways which are more convenient for our usage. We experimented and eventually chose this form.

let view model dispatch =
  div [] [
    div [] [
      str "blah"
    ]
  ]

Without leading symbols (list bracket, comma/semi-colon) in child content, this makes code at any level more friendly to being moved around. And instead of hitting Ctrl-S to auto-format, we just hit the indent or outdent keys a few times to line it up.

And lastly, Elm auto-format created tons of extra vertical whitespace in some common cases. Especially let and case blocks, which are excruciatingly over-spaced sometimes. We would catch ourselves avoiding let to avoid a space explosion. (case was unavoidable so you just accept it.) So this isn't a thing anymore. And in fact F# has an abbreviated let syntax, so we use it a lot more.

This doesn't quite equate with Elm's auto-format, but overall I would say it is equivalent or better quality of life.

Dispatch function

In Elm, when you want to trigger a Msg from a DOM event, you just tell it what message you want to send. Example: onClick Clicked. But Elmish gives you a dispatch function, and in the view's DOM event you have to invoke dispatch with your messages in order to send it, which is a side effect. In F# this looks more like this: OnClick (fun _ -> dispatch Clicked).

I suppose the main concern with this approach was devs doing weird side-effecty things and making views less maintainable. But that concern has not materialized so far. Another minor issue is that DOM event handlers are longer to type since you pretty much always use a lambda function and call dispatch in it.

I asked the maintainer of Elmish about this, who said they intended to write a layer on top of Elmish.React to match what Elm does. But it just never seemed necessary. And by keeping the dispatch function approach, it made Elmish portable to more UI targets without modification.

Likewise, we could have written our own library to translate Elmish's way into the Elm way, but it just did not seem worth it. So ultimately we just accept that DOM event handlers are slightly longer to write. We write helper functions for any especially tedious ones. And although it is less elegant to read, there is a minor upside of less separation from our source material (the DOM). In Elm, you have to write DOM decoders for custom event behavior. But not in F#.

I never found much benefit in decoding DOM values for DOM events anyway. If there was an error, it was silently ignored (I assume in order to have no runtime exceptions) and I usually wanted to see it to know I wired up something wrong.

Side effects everywhere

F# allows side effects anywhere as a pragmatic language design choice. That means you can run side effects in update and F# will not complain. Doing so undercuts the value of the MVU pattern, and obviously things are going to slip through, right? This didn't turn out to be a problem for us. But only because I invested the time to design and designate a specific area where side effects happen.

To do this I added a function and data type to the MVU pattern. The function is perform and the data type is Effect. Looking at it from an Elm perspective, Effect is roughly equivalent to Cmd. And perform is roughly equivalent to JS code you invoked through ports. But with some major quality of life differences.

Adding in perform and Effect was done purely in user space. It only took a couple of tiny adapter functions to make it work with Elmish.

First, Effect is defined by you and can be value equatable. That means not only can you test that the model was updated correctly. But you can also test that the correct side effects were invoked with the correct options. By just using an equality check -- no mocking. 🤯 You can't even do that in Elm. Since Effect is yours to control, unlike Cmd, it is extensible to do whatever side effects you need.

Slight downside, that also means you need to create effects for common scenarios that would already be built into Elm like HTTP requests. On the plus side, there are plenty of libraries that provide F# idioms on top of common JS APIs. For example, we use Fable.SimpleHttp to make HTTP requests.

Secondly, the perform function is written in F#. The reason that is significant is because Elm forces you to write side effects in JS (if you need something other than what is built-in). And the data has to go through an encoding/decoding process to pass between JS and Elm. Not anymore with perform. And if you need to call a JS function as part of your side effect, JS interop in F# is trivial in most cases. Open a namespace and directly call a JS function as though it were in F#. Synchronous, asynchronous, try/catch -- it is all available to you.

You can also export F# code so that it can be called as a JS module. Or import JS modules into F#.

All tolled, side effects everywhere was the largest fear with switching from Elm to F#. But the flexibility provided by being able to call side effects from F# brought the single biggest quality of life improvement.

Downsides

Beginner Experience

Elm is not just a language. It is also a platform and a set of tools. Its documentation therefore can take an end-to-end approach and is quite good for beginners. And our company hires beginners and teaches them to program.

On the other hand our F# solution is a mix of different techs (Fable, Elmish, React, F#), each with their own varying levels of documentation. And mostly they are not aimed at complete beginners, nor are they always cohesive with each other. Even F# introductions usually assume you are coming from OO languages.

I setup some things for them, but invariably they have to be exposed to a lot more of the JS ecosystem horrors earlier than with Elm. (Like most front-end tech.) Which can be overwhelming and imposter-syndrome-triggering as a new dev. It happens in Elm too, but to a lesser degree.

Q & A

I pretend you asked a question and then answer it.

Are you rewriting Elm apps into F#?

Not as a goal unto itself. We still have Angular 1.x apps here and there, so if we didn't rewrite those yet... As long as the 0.18 toolchain keeps working, we can keep maintaining it.

Do you regret switching to F#?

No. On balance it is an improvement from my perspective.

Do you regret posting the complaint about Elm?

On the whole, no. There was the expected vitriol which was no fun, but it also seemed to put a voice to the way others were feeling. And they commented to say so.

Who was at fault according to you?

Ok, I see where this is going... But I will answer your question. Nobody was at fault. I see it as just an unfortunate combination of circumstances.

Don't you think you will just find fault in any community you are a member of?

I refuse to join any club that would have me as a member.

Groucho Marx

Jokes aside, the whole reason you take part in a community is to help advance whatever that community is doing. As part of "helping", there are always some members who disagree with current direction or want "bad" features. But most communities I've been in handled those situations... differently.

Has Elm "improved" according to you since then?

I hope so! I have not kept up with it at all. Are we done with this line of questioning yet?

Is there a template you recommend for Elmish?

I pieced ours together so there isn't one that I have direct experience with. However the SAFE Stack 2.0 templates seem like good starting points. I was glad to see the addition of the minimal template.

You are being manipulated

Kasey Speakman — Tue, 22 Sep 2020 03:42:42 +0000

I wish this were a click-bait title. Sadly not. It is also not trying to push you into a political party or a religion or a business interest. It is an invitation to look at what is being done with our modern media.

Anything I probably want to say in this post is well covered by a documentary called The Social Dilemma. Most of the interviews are with people who made the tech we all use. From management to the developers themselves.

I guess I won't rehearse all the things they presented there. But I will say this about the perceived dividedness that we face today. Media makes the most money by selling you the weapons and motivation to fight your neighbor. Social media does it at scale.

Please watch the documentary so you understand how it is happening. Insert borrowed Netflix password meme here. And as software developers consider how your company is using your creations. And I do not mean ICE using github... I mean something much bigger. Is the business model -- the engine that drives the company -- humane? Or does it data-mine and sell its users to advertisers? That is the fastest way to make money, but it also brings out the worst in humanity. Because what sells best are the things that appeal to our basest instincts.

I heard a song once... went something like this: "Is this the world you want? You're making it." Especially true for developers these days.

Forem: Kasey Speakman

Explaining the LMAX Disruptor

Context

Java Limitations

Structs

Memory Layout

Boxing

Performance optimizations

Fixing the Elm Architecture

Centralizing logic into update

Parameter order

Untestable Cmd

Declarative side effects

Nested page fatigue

Composition through view

Summary

Event Sourcing - Stream Metadata

Relationships with Event Sourcing

In starter event

In key events

Stream metadata

Prior art

Shareholder Value: the Other Pandemic

A Boring Dystopia

Allow me to Translate

What Now?

Desktop apps in HTML and F#

Why web UI?

Which web UI?

Why not Electron?

WebView2

How does it work?

The repo

kspeakman / FSharpHtmlDesktopApp

A starter desktop app which uses HTML and F# for the UI and F# .NET for the API.

Closing thoughts

Readable vs Workable Code

Now to refactor

Is this better?

Closing thoughts

Organizing MVU Projects

Modified TEA for UI state

for UI state

Independent data

What does "independent data" look like?

update refinements

Routing and Navigation

adding URL navigation

Side effects

Effect

Practical issues

Project organization tricks

Note on Model

VS 2019 bug

Trick: Merging modules

Pain point

3. Generics as fake HKTs

2. Union leaf data

1. Only Msg

Bye InitArgs

Moving Page

End result

Conclusion

Addendum

Moving InitArgs into Msg and removing init

Moving InitArgs into Msg

Alternative InitArgs tactic

Removing init

Overall

Moving Page

Closing thoughts

Entity Component System - an old new thing

What is ECS?

This looks familiar...

What is different?

ECS is an architecture, not a program structure

Entities have runtime composition

Other connections

Conclusion

Clojure validation in 30 lines

Centralizing logic into `update`

Untestable `Cmd`

Composition through `view`

`update` refinements

`Effect`

Moving InitArgs into Msg and removing `init`

Removing `init`

`fn`

`and`

`seq`, `hmap`

`or`