Forem: Kasey Speakman

Explaining the LMAX Disruptor

Kasey Speakman — Wed, 10 Sep 2025 06:05:29 +0000

More than a decade ago, the LMAX Disruptor pattern entered the scene. I found it interesting. But frankly had trouble wrapping my head around it. It wasn't until years later when I started working on a JVM project that I finally understood. There is a lot of good performance engineering baked into it. But it basically only exists because of JVM limitations. It wouldn't rise to the level of a pattern in system level languages like Go or Rust, or even a high level language with more performance knobs like C#.

Context

Let's back up a bit. LMAX was building a high frequency trading platform. Such a system optimizes latency and throughput. For these constraints, there's a big problem with their choice of platform. The JVM was designed for Java, which ostensibly is meant for high-level -- especially object-oriented -- programming. It just doesn't give the user many performance knobs to turn.

One of the reasons I had such a hard time understanding the purpose for many of the details in the Disruptor was because I came from .NET. Even though Microsoft basically copied Sun's Java homework to make C# on .NET, in the process they also managed to bake in some improvements including performance tune-ability. Many of the pattern details were workarounds that I didn't recognize. I'll give you some examples:

Java Limitations

Structs

Java does not allow the user to create primitives aka structs aka stack-allocated types. It only allows you to make garbage-collected (heap-allocated) types. C# allows you to create structs. System level languages tend to take the inverse approach. Types are assumed stack-allocated until you explicitly allocate them on the heap.

Here's the oversimplified explanation of how that works. Each code scope (e.g. function execution) runs in its own stack frame. (Note: this is where a stack trace comes from.) When you return from that function, the stack frame goes away along with whatever local variables were allocated in it. So stack-allocated variables do not need any special memory management. When you pass struct data into a function or return struct data from it, this data is copied between stack frames. This is efficient until the data gets too large. That's when you might reach for a separate memory allocation on the heap. Then the stack frame only copies the pointer (8 bytes on a 64-bit system) for the heap data to the next stack frame instead of the full data.

What constitutes "too large depends" on the system, but on .NET I recall the threshold being 24-32 bytes (3-4 longs or reference objects). Above that, object references could be better performance, depending on garbage collector behavior. As with all performance optimizations, you don't know for sure which is better until you test it.

High level languages default to heap allocated data so that it's copying 8 byte pointers between stack frames instead of the full data. This is probably better performance in common cases, but optimized code might do better with structs. Java doesn't give you the option to tune performance in this way.

Memory Layout

Alignment to CPU cache line is another common performance optimization. In short, CPUs have multiple levels of caches. Especially for collections, there are potential performance advantages if you design your data types to be exactly 8, 16, 32, or 64 bytes. This is because data is read from memory into CPU cache lines, which are generally 64 bytes. If your data fits exactly in a cache line, the CPU can use optimizations / pipelining which improves performance when processing collections.

Java has no way to specify memory layout, so Disruptor makes use of padding fields in the classes used by ring buffers. Named like p1, p2, etc. These fields are not used by logic, but are only there to align the data structure. Whereas .NET has StructLayout and related attributes to allow you to add padding explicitly.

Boxing

Even though Java does not allow you to create primitive data types, it has some built in, like int. However, when you use primitives in generics Java "boxes" them up as heap-allocated objects. Especially in generic collections, this uses more memory and costs CPU instructions to box and unbox.

Disruptor doesn't explicitly mention this, but it is a well-known drag on JVM generic performance. One which I wasn't familiar with from .NET as it doesn't box primitives for generics.

Performance optimizations

Overall, the Disruptor is a marriage of qualities from actor and SEDA patterns with some distinct performance optimizations like a ring buffer that is also an object pool. From Actor / Agent pattern, Disruptor took the message inbox and independent processing. However, the disruptor is also step-wise coordinated with previous steps like in SEDA.

The ring buffer plays a prominent role in Disruptor for a few reasons. First, it is a common way to implement a fixed-size queue. This is necessary in high-throughput systems because unbounded queues will eventually reach memory usage limits. The bounded queue forces you to think about your memory usage and full queue behavior. (E.g. load shedding vs blocking)

The ring buffer simultaneously serves as an object pool. Object pools are pretty common in game development. Business devs will be more familiar with them from database or HTTP connection pooling. The general idea being that creating a new object is expensive in some way, so instead keep a pool of them and soft reset and reuse an old instance. For db or http pools, the expensive part is actually making the network connection. But for very high-throughput systems, the churn of allocating new objects and then garbage collecting them in a tight loop is the problem. By pre-allocating blank objects in the ring buffer, these objects will never be garbage collected nor need new instantiations later. So it essentially circumvents garbage collection.

There are other optimizations around the ring buffer such as only using sizes which are powers of 2. The intrinsic way a ring buffer works is that it has an ever-increasing queue position which can go past the actual size of the buffer. So if the ring buffer size is 4 (indexes 0 to 3), but we're processing our 15th item out of the queue, we can calculate the ring buffer position with modulus by the buffer size. 15 modulus 4 is 3. So the item we want is at index 3. The problem is the modulus operation, which can be expensive to calculate in a tight loop. But not for powers of 2. Since the computer operates in binary, powers of 2 are especially simple to divide by. So using a power of 2 as the ring buffer size increases performance.

I didn't cover all the optimizations, but you get the idea.

Fixing the Elm Architecture

Kasey Speakman — Mon, 28 Jul 2025 21:23:09 +0000

Elm architecture aka MVU was a breath of fresh air when I encountered it. It remains the best UI dev experience I know of. As I've done larger and larger projects with it, used it on the backend, and seen its usage in other contexts, it has some friction points. Here are my current thoughts on how to resolve those sore spots.

Code below is in F# unless otherwise noted.

Centralizing logic into `update`

//                          sus
//                           |
//                           v
let init initArg : (Model * Cmd) = ...

One thing that has always bothered me about init is that it comes out swinging by returning a Cmd. This can be a problem in some UI technology (e.g. WPF) that is sensitive to the timing of initializing elements vs side effects. I recall at least one issue about this in the Elmish repo.

It's also annoying that all decisions are made in update... except for the 1 made in init, as indicated by returning Cmd. This means that to test an MVU program both functions have to be exercised. I played around with my own backend-focused lib and determined that this doesn't have to be the case.

I had to write my own library for backend workflows because they have different run characteristics from UIs, but my lib used identical MVU functions, minus view.

All logical decisions could be done in update without changing it in any way. Just move init's Cmd-producing decision into its own Msg case. Then get rid of init. That works, but the caller has to know how to construct an initial Model and Msg to send to update. So then I thought, why not use init for this purpose instead... basically an adapter function to translate the data you have (initArg) into the form needed to kick off the update function.

let init initArg : (Model * Msg) = ...
//                  \---------/
// tells how to          |
// start update          |
//             -----------
//             |
//             v
//         /-------\
let update msg model : (Model * Cmd) = ...

For UI platforms that need it, this would also provide an opportunity for UI elements to be initialized from the Model, and the Msg queued for later processing, without kicking off any side effects.

At the time I made this change, I already implemented a similar tactic for resumable workflows. My code had the model from the previous completed workflow. It just needed to know the correct Msg to resume the workflow. I created a function called resume for that purpose. So it seemed natural for init to do the nearly same thing to start the workflow.

Parameter order

Another thing that always bothered me about Elm is the parameter order of update. You can look at Cmd as a future Msg, so the update function returns nearly the same types as its inputs, but with their order swapped.

//     should be swapped
//          ------
//          |    |
//          v    v
let update msg model = model, Cmd.none

This is definitely a minor point, practically speaking. I just have to bake in an argument swap when I'm testing update. But it's logically incorrect for how update is used. And I can't unsee that.

It is a winding road to explain how Elm arrived at that. But it has to do with its Haskell heritage; Haskell's focus on laziness; and laziness being represented by the right fold operation. Which in this case puts model as the 2nd argument. Whereas left fold -- which would have model as the 1st argument -- is considered eagerly computed. It's an inherited philosophical choice, not a practical one, since Elm doesn't focus on laziness like Haskell. You can even observe this cognitive dissonance in Elm's List.foldl which processes the list left to right, but combines elements right to left as a right fold would, so it behaves unexpectedly when order matters.

// F#, Haskell, and every other language's left fold result
"abc" = List.fold (fun s1 s2 -> s1 + s2) "" ["a"; "b"; "c"]

-- Elm's left fold result
"cba" == List.foldl (\s1 s2 -> s1 ++ s2) "" ["a", "b", "c"]
-- ^
--  \
--   huh?

It seems like update arguments are meant to match Elm's incorrect left fold behavior. This is easily fixable by swapping input argument order.

// before
let update msg model : (Model * Cmd) = ...
//           \  /
//            \/  swap
//            /\
// after     /  \
let update model msg : (Model * Cmd) = ...

Later in this article, you'll see an example of testing update logic by providing a list of Msgs, then checking that the correct side effects were produced. Testing made the most sense with left fold. Which made the most sense with the corrected version of update.

Untestable `Cmd`

Elm's representation of side effects is with Cmd. The Elm philosophy is that users are not allowed to do side effects in Elm, so Cmd is locked down. You can use a Cmd-generating platform function provided by Elm. Or when you inevitably need a side effect that Elm doesn't cover, you must send data out of an asynchronous port to let JS run your side effect and send the result back in thru another async port.

In Elmish, the F# library, Cmd is an alias for a list of user-declared functions. Each of which is provided with a dispatcher to send Msgs back. (Unlike Elm, you actually get to write your side effects in F#! And JS <-> F# interop is easy.) So your update code returns functions to run when it's time to call side effects.

In both cases, Cmd is untestable. In Elm it's intentionally opaque, even the fact that it's a collection under the covers. There is no way to test it at all. In Elmish you could potentially test generated Cmds if your update fn also accepts side effect dependencies as parameters and you pass in mocks instead of the real ones. But I consider this to be the hard way to test things, on top of polluting update with side effect related concerns.

I'll be honest, I never did a lot of unit testing on frontend code. It was just not needed. But backend is a different story. It is all about making business decisions and executing the chosen side effects. It needs to be tested thoroughly for correctness.

Declarative side effects

For backend, the Model is less important to test. Rather, I needed to be sure that the correct side effects were decided. Cmd being hard to test was just not acceptable. So I introduced another step in the MVU process. This involves adding another type similar in style to Msg but for the purpose of declaring a side effect to run. I called it Effect. Then I introduced a function to perform those side effects.

type Msg = ...

// define possible side effects, a DU like Msg
type Effect = ...

type Model = { ... }

let init initArg : (Model * Msg) = ...

//                            test with equality
//                                     v
let update model msg : (Model * List<Effect>) = ...

//    could be Task and/or synchronous too
//                     v
let perform effect : Async<Msg> = ...

Like Msg, Effect is a user-declared type.

Above, update is living its best functional life. Its inputs and outputs are value-equatable immutable data. That means side effects can be tested for correctness with simple equality checks. Like so:

//  expected    model      effects
//                |           |
//                v           v
let expected = { ... }, [LoadWidget 123]

//   basically left fold            model   simulated scenario
//            |                       |             |
//            v                       v     /----------------\
let actual = sim MyWorkflow.update { ... } [Go 123; LoadFailed]

// test workflow with simple equality
assertEqual expected actual

Notice that I don't need to involve init in this test, since it no longer returns a side-effect... only update does.

Above, LoadFailed is an actual Msg that would return from perform when there was a database failure. The only Effect produced is the attempt to load a widget. After that, we're expecting that upon receiving LoadFailed the update fn will stop and not try to produce more side effects. If it does produce more Effects, assertEqual will fail the test. So this not only tests that it does what it should but also that it doesn't do more than it should.

This is what the relevant section of perform might look like.

let perform (dbConfig, logger: ILogger) effect =
    match effect with
    | LoadWidget widgetId ->
        async {
            try
                let! widget = Db.findWidget dbConfig widgetId
                return WidgetLoaded widget
            with ex ->
                logger.LogError(ex, "Widget load failed {widgetId}", widgetId)
                return LoadFailed
        }

We take advantage of partial application to inject dependencies as the first argument.

With this arrangement, side effects no longer need to be mocked -- with all the overhead involved in that -- to test logical decisions. In fact, side effects play no part at all in unit testing. They only have to be integration tested.

We used this same adaptation on frontend. There, having a place for side effects prevented them from being introduced accidentally in update. We also found it made update code easy to reason about. When we did have a UI bug, it was usually quick to spot and fix. At least it was once we got away from sharing state across pages.

Nested page fatigue

Everything I've mentioned so far is easily usable inside Elmish, maybe even Elm, with tiny adapter functions. That's what I did for our Elmish frontend. This one could need more than that, not sure yet.

What is nested page fatigue? It's the vaunted fractal nature of The Elm Architecture. It is great to a degree, but it requires wiring together too dang many things between a parent and child. Every time you create a child page, you have to wire it into the parent page in every possible way.

Msg case for child msgs
Model property or case for child model
init or update case to initialize child
- depends on whether it uses navigation
update case to call child update
view code to call child view

After hundred thousands of lines of UI code and many dozens of pages, the friction is real. And this is the most common complaint about MVU. The backend library taught me that it was possible to avoid this fatigue while still using MVU all the way down.

You see, when I tried to compose F# backend MVU workflows in the traditional way, similar to frontend, it made things very awkward and mixed a lot of concerns between workflows.

If this were OO, that way of wiring things would be a dark-but-humorously-named anti-pattern called Inappropriate Intimacy.

Then I noticed when non-MVU code called MVU workflows, it first used an adapter function to convert it to a normal Async-returning function. Then I realized that this is the best way to compose MVU workflows... treat each one as a single overall side effect. This makes composing them easy:

async {
    // workflow1/2 are records containing init, update, etc
    let! result1 = runWorkflow workflow1 initArg
    let! result2 = runWorkflow workflow2 result1
    ...
}

// or thru perform

let perform deps effect =
    async {
        match effect with
        | MyEffect arg ->
            let! value = runWorkflow workflow1 arg
            return EffectHappens value
    }

Right okay, but how does that help frontend?

The purpose of these backend workflows is to make correct decisions about side effects and run them. So composing them as side effects made sense. The purpose of a frontend program is decide about and display UI.

Composition through `view`

This gave me the idea that a whole Elmish-React child program could be packaged up as a React component, then displayed along side the other HTML elements (also React components) in the parent's view.

let view model dispatch =
    div [] [
        match model.Route with
        | CourseList ->
            let initArg = ()
            Elmish.page initArg CourseList.init CourseList.update CourseList.view
        | CourseDetail courseId ->
            Elmish.page courseId CourseDetail.init CourseDetail.update CourseDetail.view
    ]

Since Elmish has termination support, the self-contained page would be coded to run clean up when it is switched out from view. There's no longer a need to wire it thru all the parent's functions and types as before.

What if I want communication back to the parent? For example, the parent needs to know when a cancel button is clicked on the child page to no longer show it. Easy. Make it an extra argument to the child's view function. Partially apply it.
let onClose () = dispatch Closed
Elmish.page ... (CourseDetail.view onClose)

When I looked around, I realized this was not a particularly original idea. This is the common way Fable.React.useElmish is employed, but being hosted directly from React JS components instead of parent MVU views. (Which also avoids all that parent-child wiring.) I also found common React libraries that tied the navigation route directly to the displayed React components, which only makes sense.

Composition through view would also naturally fit building as dynamically loadable ESM modules instead of a fully bundled app.

This one, I haven't tried yet. The Elmish.page function doesn't exist. I need to write that and figure out how to adapt it to Elmish.mkProgram. It looks to be technically possible, and I am excited to try it the next time I build another MVU app.

Summary

MVU is an amazing pattern that helps me write highly maintainable and correct UIs. I love it. That's probably why the core pattern remains essentially the same even after my fixes.

Yet, these changes have materially improved the experience for me. Maybe they could for you too.

Event Sourcing - Stream Metadata

Kasey Speakman — Sat, 29 Jan 2022 22:16:17 +0000

Discovering the need for stream metadata.

One of the best uses of Event Sourcing is to manage behaviors between entities, aka relationships. This is where all interesting things happen. For example, a product catalog is just a collection of data. It doesn't do anyone any good until those products are sold, shipped, restocked, etc. All of these processes occur between people or companies. They occur in relationships.

Relationships with Event Sourcing

These relationships need to be reflected somewhere in the event stream. In our domain of safety training, the main activity we track is training. Which involves a student and a course. We use a Training ID (a UUID) to identify each training process. If we were creating relational tables, we would include foreign keys to Student ID and Course ID to track the entities involved. How do we express this in an event store?

In starter event

In our first effort, we simply included reference IDs (Student ID and Course ID) in the starter event of our training process stream. For example, the starter event for a training process might be RegistrationCreated. Storing the reference IDs in the starter event supports executing workflows and running queries. (The command and query APIs in CQRS parlance.) Both of these use a "replay" process that can keep the reference IDs from the first event for later use. This worked out great for our first event sourced application. But then we built an app for a more complex domain.

In key events

We rearchitected one of our internal apps for the cloud. This time the domain is auditing. More specifically, verifying a vendor (contractor, supplier, etc) meets requirements before they are allowed to do work for a company.

We started out placing the reference IDs only in the starter event. But this strategy was problematic for some kinds of event listeners -- like process managers -- and for building more complicated data models. In these cases, the "starting" event for the listener / data model is different from the starting event for the stream. We ended up solving this problem by just repeating the same reference IDs on these key events. The minor downside is storing duplicate data.

As an alternative, we considered loading the reference IDs from an existing data model. But we decided this was a net loss. It would have created multiple degrees of coupling (server, code) between subsystems. And add complication to the infrastructure that would benefit relatively few cases.

Stream metadata

We ran into this again recently. Only this time I realized that these reference IDs (so far) are always static for a given stream. So they should be attached to the stream itself rather than specific events. They should be stream metadata. When the stream is "started", reference IDs are saved in stream metadata instead of on the initial event. When an event is loaded from the event store, the stream metadata can optionally be loaded too. So the reference IDs are available to listeners, regardless of which events they care about. No more repeating the same IDs in multiple events.

Prior art

I had seen this concept of stream metadata in my research on Event Store (Greg Young's open source product). But at the time I wasn't sure why I needed it. I have been burned before because I used tools incorrectly when I didn't understand them. So I did not include it in my Postgres event store implementation. In many ways I feel like I am just rediscovering things that Greg Young figured out years ago. Then documenting them in a way that my past self might understand. 😄

Shareholder Value: the Other Pandemic

Kasey Speakman — Sun, 09 Jan 2022 12:50:50 +0000

This post was inspired by Nvidia's recent announcement of the RTX 3090 TI. With its expected MSRP of $2000+, this graphics card comes at a time when many PC builders cannot find graphics cards at all. Most of them go into the hands of crypto miners or are flipped by scalpers at twice the MSRP. This is the latest example of Nvidia getting in on the scalping action too. By repackaging a lower-priced product line as a new SKU to inflate the price. Justifying it with a negligible performance increase. Then reducing or stopping production of the cheaper version. During these global hard times, Nvidia is shipping 12% more GPUs while making 84% more net income year-over-year. By squeezing it out of customers.

A Boring Dystopia

But none of this is new or surprising. (Or even among the worst examples.) It's one of the most cliche things of our time: the corporate zeitgeist of maximizing shareholder value. This philosophy seems to be accepted at every level as the way business is supposed to work.

"Jack CEO, if you can increase profitability 2% next year, the board will give you a 2 million dollar bonus." The CEO slashes employee benefits, gouges customers, and eliminates "waste" such as R&D spending. Profits and stock prices go up, making the investors hundreds of millions. The financial sector calls Jack CEO a hero. He is in higher demand because of this "success". So he'll probably be long gone when the company's market position enters the nose dive he setup. Employee hiring/retention problems and customer dissatisfaction mount as toxicity increases. The company fails to stay relevant without innovation from R&D. If the board keeps trying to find a repeat of Jack CEO's "success" the company might even get its own documentary. Like Enron or USA banks.

The story may play out in different ways, but this one illustrates the logical conclusion of focusing on shareholder value. Investors and executives get richer by sacrificing the company's future. But the giving end of the sacrifice is done by employees and customers. It's no wonder the wealth gap keeps getting wider.

Allow me to Translate

"Shareholder value" is just a dressed up word for greed. And it is enshrined in our global economic systems. Profit (or the potential thereof) is the measure of company health in financial markets. CEOs are required to prioritize it above all. Failing to do so opens the door to lawsuits. For example...

Having chosen a for-profit corporate form, the craigslist directors are bound by the fiduciary duties and standards that accompany that form. Those standards include acting to promote the value of the corporation for the benefit of its stockholders.
...
Thus, I cannot accept as valid ... a corporate policy that specifically, clearly, and admittedly seeks not to maximize the economic value of a for-profit Delaware corporation for the benefit of its stockholders...

eBay vs Craigslist

This lawsuit was about a few different things. The excerpt is about Craigslist creating policies designed to prevent eBay from destroying the company's culture to chase profit. Here the judge's interpretation of the law considers this a breach of duty. As a non-lizard-person I find this horrifying.

What Now?

I don't have any easy answers. My first reaction to hearing the Nvidia announcement was: I wish more product companies were non-profits. Only this would not be a direct solution. Investor money allows companies to expand and innovate at a more rapid pace. (Initially. Before it's ultimately enslaved to profit-seeking.) Whereas non-profits have to grow very carefully. There's no extra profit to reinvest. And no investor help for expansion shortfalls. A non-profit Amazon would have failed immediately instead of "almost" being a long-term Ponzi scheme.

Many jurisdictions have legal frameworks for Benefit or Social Purpose corporations. These enable other goals (in addition to profit) to be recognized by civil law. So the company could take investors but (maybe) still win a lawsuit for refusing to treat employees like test subjects.

The main challenge with these alternatives is that they are likely to struggle to be successful in the market. It is way more effort to achieve responsibility in multiple dimensions at once. Effort that increases the bigger the company gets. Most consumers will appreciate your company's approach but actually buy the cheaper or sooner-to-market option. Or in today's world, the in-stock option.

I waited in line 5 hours for my local Best Buy to open. Unfortunately most people waited at least 9 in their tents and RVs. By the time it was my turn to pick a graphics card, the only choices left were:

an EVGA-price-gouged 3060 TI
- only about 60% faster than my 7-year old 980 TI
a lobotomy-required EVGA 3080 TI for almost $1500
a white hot fury-inducing Nvidia 3080 TI at $1200

I decided to leave without buying anything. I began a lifetime boycott of Nvidia products. I waited 3 years for shortages to subside. Then bought a 9800 XT direct from AMD, because I had to boycott all their partner brands for price-gouging too.

Except that's not what happened. I spent money I didn't have to get the Nvidia 3080 TI.

I am part of the problem. But trying to find the path to change.

Desktop apps in HTML and F#

Kasey Speakman — Mon, 30 Aug 2021 22:21:20 +0000

I created a starter desktop app using HTML/CSS/F# for the front end and F# .NET for the back end. All hosted in WPF.

This post mostly covers the "why". Hit the repo if you just care about "how".

Developers at my company are mostly coding for the web. But once in a while we need to create a Windows desktop app. Because some situations require direct access to hardware and running software. But a big challenge with desktop apps is their UI tech.

Why web UI?

Our devs have a lot of skill investment in web tech. And I don't mean React and Webpack -- although we know a little about those. I'm talking about the open standards of HTML, CSS, and Javascript. Because of standardization I can use web tech to make a consistent-looking UI, no matter what device is consuming it.

Meanwhile desktop UI techs tend to be opinionated and proprietary. We develop for desktop infrequently, so non-transferrable learning investments into desktop UI tech is a hard sell.

Which web UI?

It has been possible to use web tech in Windows desktop apps for a while. Problem was, the solutions were full of compromises.

UWP, for example, allows you to build apps using web tech. But it runs sandboxed, making it hard to do the one job it needs to do: direct access. WebBrowser and WebView have also existed for a while. They are embeddable browsers for WinForms/WPF apps. But they use IE or EdgeHTML for rendering. Need I say more?

Why not Electron?

The elephant in the room is Electron. The main downside for us is the Node.js back-end. Some of our vendors do not provide libraries for it. And even if it can be awkwardly made to work using unsupported libraries, Node requires us to work in Javascript.

Electron's cross-platform support is probably its biggest benefit... but we don't need that.

Yes, earlier I did list Javascript as a positive thing for web tech. I do like the standardized browser APIs available thru JS -- Media, Storage, Intl, etc. But let's be honest, JS also has footguns baked into the standard. And I strongly prefer to use something else: F# on .NET.

WebView2

WebView2 is an embeddable browser for WinForms/WPF like WebView before it. WebView2 uses Chromium-based Edge for rendering. So unlike its predecessor, WebView2 keeps evolving with the web standards. And I get to use F# on .NET.

We have actually used WebView2 since shortly after its release. We used it to show HTML5 video content in a computer lab. It was a rush job and WebView2 was only a month old at the time. I wasn't ready to trust it, much less consider it for other uses. And I forgot about it.

Then earlier this year, Microsoft announced Blazor Desktop based on WebView2. And then I found out Microsoft Teams is ditching Electron for WebView2. This got me to take another look at it. One thing led to another and over the weekend I created a working demo: a WPF app with an F# web UI and F# .NET backend API.

How does it work?

The UI is a webpack-built JS app just like you would expect. It uses F# and Elmish for a functional (as in functional programming) UI. The F# code gets transpiled to JS and uses React for DOM rendering.

The "backend" API is an F# library. In this starter app, the backend is very simple. So there is nothing interesting to demonstrate yet.

A messaging library is shared between the front and back. It defines all the requests and responses that can pass between them.

The host application is WPF. WinForms would work too. Either way there is not much to it. The only control is the WebView2 control. And about 40 lines of wiring code.

For communication between the JS front and .NET back, I used WebView2's string-based messaging. Each side gets some functions to send/receive messages. Types from the messaging library are converted to JSON strings for transport.

The repo

kspeakman / FSharpHtmlDesktopApp

A starter desktop app which uses HTML and F# for the UI and F# .NET for the API.

Closing thoughts

This was a fun weekend project. It has a lot of promise to make desktop apps more manageable for our dev team. I'll try it out this coming month and let you know how it goes. For now, let me know what you think about making desktop apps with web tech and F#.

Readable vs Workable Code

Kasey Speakman — Thu, 05 Aug 2021 22:32:54 +0000

Balancing some code to be readable yet workable.

Today I was coding and decided to refactor some older code. In particular, this code is handling navigation updates in a Fable Elmish app in F#.

let routeToPage routeOpt =
    match routeOpt with
    | None ->
        let page, effects = OpsUser.Search.init None
        OpsUserSearch page, effects
    | Some route ->
        match route with
        | ToOpsUserSearch searchOpt ->
            let page, effects = OpsUser.Search.init searchOpt
            OpsUserSearch page, effects
        | ToOpsUserAdd ->
            let page, effects = OpsUser.Add.init ()
            OpsUserAdd page, effects

This function has many route cases but only two are shown here.

Now to refactor

First I started to remove duplication. Starting at the innermost duplication, notice how page, effect is repeated twice. We can remove this duplication with a helper function.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let routeToPage routeOpt =
    match routeOpt with
    | None ->
        OpsUser.Search.init None
        |> Tuple.mapFirst OpsUserSearch
    | Some route ->
        match route with
        | ToOpsUserSearch searchOpt ->
            OpsUser.Search.init searchOpt
            |> Tuple.mapFirst OpsUserSearch
        | ToOpsUserAdd ->
            OpsUser.Add.init ()
            |> Tuple.mapFirst OpsUserAdd

The mapFirst helper function is not specific to this code. It could be used on any tuple. So I made a small Tuple module that I can extract later if needed.

Next, all of these cases do the same thing, but using a different page tag, init function, and init argument. So we can make a helper to parameterize those.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let routeToPage routeOpt =
    let inline initPage pageTag init initArg =
        init initArg
        |> Tuple.mapFirst pageTag
    match routeOpt with
    | None ->
        initPage OpsUserSearch OpsUser.Search.init None
    | Some route ->
        match route with
        | ToOpsUserSearch searchOpt ->
            initPage OpsUserSearch OpsUser.Search.init searchOpt
        | ToOpsUserAdd ->
            initPage OpsUserAdd OpsUser.Add.init ()

I used inline here as micro optimization. You can omit/ignore that as it won't make any logical difference to the code. Generally it is best to avoid using it in other places unless you know how it will affect execution.

So the cases are down to one-liners. We can further split the main cases from the default one, into a separate function. And reuse that function to also initialize the default case.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let initRoute route =
    let inline initPage pageTag init initArg =
        init initArg
        |> Tuple.mapFirst pageTag
    match route with
    | ToOpsUserSearch searchOpt ->
        initPage OpsUserSearch OpsUser.Search.init searchOpt
    | ToOpsUserAdd ->
        initPage OpsUserAdd OpsUser.Add.init ()

let routeToPage routeOpt =
    match routeOpt with
    | None ->
        initRoute (ToOpsUserSearch None)
    | Some route ->
        initRoute route

Here initRoute will be the primary area that devs will modify going forward. We can also look at Option functions to further reduce duplication in routeToPage.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let initRoute route =
    let inline initPage pageTag init initArg =
        init initArg
        |> Tuple.mapFirst pageTag
    match route with
    | ToOpsUserSearch searchOpt ->
        initPage OpsUserSearch OpsUser.Search.init searchOpt
    | ToOpsUserAdd ->
        initPage OpsUserAdd OpsUser.Add.init ()

let routeToPage routeOpt =
    routeOpt
    |> Option.defaultValue (ToOpsUserSearch None)
    |> initRoute

To remove every ounce of fat we can use point-free style. This necessitates extracting initPage outside of initRoute. Consequently, I'll remove inline so if anyone later tries to reuse it they don't get unexpected performance issues. I will also separate out the default route as its own value. This extracts out the only thing in routeToPage that is likely to change over time.

module Tuple =
    let mapFirst f (first, second) =
        f first, second

let initPage pageTag init initArg =
    init initArg
    |> Tuple.mapFirst pageTag

let initRoute = function
    | ToOpsUserSearch searchOpt ->
        initPage OpsUserSearch OpsUser.Search.init searchOpt
    | ToOpsUserAdd ->
        initPage OpsUserAdd OpsUser.Add.init ()

let defaultRoute =
    ToOpsUserSearch None

let routeToPage =
    Option.defaultValue defaultRoute
    >> initRoute

Is this better?

So at this point I have removed all the duplication I can find. Now it's time to look at the bigger picture. How clear is this code? The experienced among you might say it is quite clear. But what if you were new to coding? How many different types of syntax must be mastered? How many layers must a dev trace through to understand routeToPage? Are there any syntaxes that are especially difficult for new devs to understand?

I like to consider readability from a new dev's perspective. This makes the code base more understandable to all readers including future-me. And pragmatically it allows more flexibility. I don't just have to hire experienced devs or devs with a specific background. Devs of all experience levels and backgrounds can transition between projects in less time.

From this perspective the code has a number of aspects that I think harms general readability. Here is what I will likely commit.

let initRoute route =
    let inline initPage pageTag init initArg =
        let page, effects = init initArg
        pageTag page, effects
    match route with
    | ToOpsUserSearch searchOpt ->
        initPage OpsUserSearch OpsUser.Search.init searchOpt
    | ToOpsUserAdd ->
        initPage OpsUserAdd OpsUser.Add.init ()

let defaultRoute =
    ToOpsUserSearch None

let routeToPage routeOpt =
    let route = routeOpt |> Option.defaultValue defaultRoute
    initRoute route

One challenge to readability is point-free notation. A lot of functional programmers like it. But it is not obvious to read at first glance. The point-free functions look like value assignments. You have to look inside the code block for specific signs that is point-free, like function or >>. Then the implicit parameter can leave a bit of doubt in the mind of a new programmer as to what is actually happening. In all I consider point-free to be more of an optimization than an abstraction. And it does not add sufficient value here for the duplication it optimizes away.

Next I considered Tuple.mapFirst. It was only used in one place. It results in more code than not having it. Worst of all, it makes initPage less readable. Because when you dig into mapFirst, you have to context-switch into the mechanics of a tuple, away from the problem you were originally trying to understand. For this reason, even if the function was included in the base F# libs I would avoid using it here for clarity.

I'm not entirely convinced of the value of initPage, but I will keep it for now. It seems readable. It saves a little code in the place where devs will most commonly be working. And understanding it is only a one-level deep journey.

I could have written routeToPage with all pipes.

let routeToPage routeOpt =
    routeOpt
    |> Option.defaultValue defaultRoute
    |> initRoute

But I think the way I coded it is more clear. Specifically, the first line is more obviously converting an optional route to a non-optional one. So even if the dev is new enough that they haven't encountered Option.defaultValue, the function is still understandable at a surface level.

Quite frankly, the end result of this process does not seem any more readable than the original. This speaks volumes about the value of explicitness even if it means duplication. But the result filters down the likely-to-change code into a specific function. And it provides a tool for reuse there.

Closing thoughts

Good code is like good literature: it forms a tangible picture in your mind. But it also is like a good workbench: it is easy to work on. It would be nice if there was a single tactic -- such as the Don't Repeat Yourself principle -- that could accomplish these goals. But I haven't found it yet. Instead good code requires trying different things and evaluating what works.

I shared my thoughts on balancing this code. How do you balance your code?

Organizing MVU Projects

Kasey Speakman — Tue, 29 Dec 2020 17:28:00 +0000

Strategies for organizing large Model-View-Update projects that have worked for us.

We have been working on functional UI projects using the MVU pattern for 4 years now. We first started using this pattern with Elm. Then we switched to full stack F#, using the Elmish library for MVU. We evolved our approach to MVU several times along the way. This post details some practices we learned.

This post assumes some experience with MVU. I do not mention the view function very much since we do not usually find it a challenge to organize. A view function for a particular page usually resides near that page's update function.

Modified TEA for UI state

The Elm Architecture (TEA) is one of the early ideas behind Elm. It defined a nested hierarchy of Msg, Model, Update, and View. But it was disavowed by Elm because it has a fatal flaw. TEA required a lot of laborious boilerplate to convert Msg between parents and children. It also made simultaneous active pages very difficult. We went away from it. But as our programs got larger, we found ourselves with gigantic Msg and update files, requiring us to use Ctrl-F to find things inside them. Basically everything was in a couple of huge piles of code instead of being organized for quick human access.

Quick stats
Latest project's UI code (F#)

Largest app - 35k lines of code

All apps - 65k lines of code

It became clear that we needed some organization. TEA is really just grouping all the things for a single page together in a module. This idea makes a lot of sense, so we went back to that for everything except Msg. All Msgs are declared first in their entirety, centrally and separately from the page parts. This means that every page module can directly send any messages they want, even to other pages. This got rid of all that Msg mapping and OutMsg patterns plaguing TEA.

Update 6 Jan 2021: I just checked out Elm's current documentation and it now mentions only Model, init, view, and update in page modules. So they must have come to a similar conclusion. When we were in the Elm community TEA included Msg but became disavowed because of its shortcomings, and the documentation about it was removed. Then there was a propaganda blitz for flat Msg. We switched away from Elm before this latest iteration on Elm app structure.

In our modified TEA, Msg can still have a hierarchical organization pattern as needed. So we can place a particular set of messages in their own file.

Remember that organization works for you, not the other way around. Every time you add a level of organization to your project, you also add more overhead (structure). It is better to keep things flat as much as possible. And only increase the degree of organization when you discover it is needed.

for UI state

An important subtlety in the section title is "Modified TEA for UI state". Often an app will have data that it is tracking independent of whatever the UI elements are doing. This kind of data should be placed outside the UI state. Everything in the UI state should be viewed as just what is currently being displayed, available to throw away at any time.

It is necessary to point this out because our industry is so accustomed to object-oriented UIs. There the ownership of state is distributed among many different objects. This can lead MVU users to try to make UI pages "own" specific kinds of data on behalf of other pages. But with MVU, pages only represent what is displayed on the screen. This is the way.

This has a couple of major advantages. It eliminates a common source of bugs: two different pages using shared state inconsistently. It also eliminates the need to manage the UI state's lifecycle, since it is part of the page lifecycle.

// changing pages
let pageData, cmds = Home.init ()
{ model with Page = Home pageData }, cmds

In the above example, the important thing here is what is missing. There is no code to check the previous page. Or clean up shared state. It is not needed. Any data that the page needs it will request or create on init.

Independent data

What if pages need to display or update that shared, independent data? The best answer really depends on the situation, but here are some approaches. If the independent data is read-only reference data, you can pass it as an argument to the page's init.

For a more robust approach, treat it as an external integration. This works for cases where the state isn't static or pages can request changes to it. For this, provide Cmds to make these requests. Also Msg cases to process changes in an MVU style. This is a headless MVU service pattern, where the independent data is the service state.

What does "independent data" look like?

I keep using this term independent data like it is something special. It is not. To demonstrate how "not a big thing" it is, here is an example Model.

type Model =
  { ApiKey: string
    Page: Page }

Here, ApiKey is what I am calling independent data. And Page is a piece of UI state. The app model could contain multiples of either kind. The Msg for this app might contain a set of messages to manipulate either or both parts of Model. It depends entirely on what the app needs.

`update` refinements

UI states are just data that represents what is on the screen. So in them we make extensive use of both records and (discriminated) unions. Records are what most people are most accustomed to, similar to structs or DTOs. Unions like you will see below are used when the state can only be one of multiple possibilities. Here is a typical example of a UI state:

type Page =
  | Home of Home.Model
  | CourseSearch of Course.Search.Model
  | CourseDetail of Course.Detail.Model
  // and so forth

This is basically spelling out that Page can be in one of these states. And then each page also has its own inner state (for example Home.Model), which is also some combination of records and unions.

That is well and good, but there are a couple of practical issues when we go to use this in update. With this structure we have to dig the page data out of the union type. That is Problem #1. Separately, since Msg is an app-wide type -- where any page can send any message -- now it is possible to send a Msg to a page which is not active. That is Problem #2.

Technically #2 was always possible. A delayed API operation and an impatient user can lead to receiving a response for page which is no longer active.

To resolve both of these problems, we went to an update style that looks like this:

let update msg model =
  match model.Page, msg with
  | Home pageData, HomeMsg pageMsg ->
    let pageData, cmds = Pages.Home.update pageMsg pageData
    { model with Page = Home pageData }, cmds

This serves to unwrap the page and message data from their respective union types. While also verifying we are handling a message appropriate for the UI state.

You might also notice this has a strong similarity to something else -- a state machine. At its core, a state machine is just a set of states and the valid transitions between them. This match is spelling out the valid transitions (messages) for each UI state.

We also do something that many of you FPers will not like. We use a catch-all match for any unmatched state transitions. This allows us to specify only the scenarios that we want to handle and ignore the rest.

let update msg model =
  match model.Page, msg with
  ...
  | _ ->
    // no changes
    model, []

This is a classic expression problem trade-off of extensibility vs completeness. With the underscore match we are choosing extensibility. By choosing this we lose the advantage of the compiler telling us of intended matches that we forgot. But we also do not get the multitude more unintended matches that we want to ignore. We believe our choice makes sense under the circumstances. Logging the catch-all case provides enough of a hint when we forget to add an intended case.

Routing and Navigation

A "route" is simply enough data to load a specific page. In MVU, pages are constructed by their init function. Therefore a route is just the arguments to the page's init function, along with a tag to indicate the page. It is so much simpler to think about it this way, that I eventually started naming this InitArgs in new projects instead of Route. And this can be used practically to package up all the arguments to the init function. Here is an example.

// PageTypes\Course\Detail.fs
module Course.Detail =
  type InitArgs =
    { CourseId: int }

// AppTypes.fs
type InitArgs =
  | HomeInit of ()
  | CourseDetailInit of Course.Detail.InitArgs

// PageFns\Course\Detail.fs
module Course.Detail
  let init initArgs =
    { CourseId = initArgs.CourseId
      Data = Loading }
    , [ LoadCourseDetail initArgs.CourseId ]

I'll cover the folder organization of the project later.

Switching pages then is a matter of adding a Msg case.

// AppTypes.fs
type Msg =
  | SwitchPage of InitArgs

And handling page changes looks like this.

let switchPage initArgs =
  match initArgs with
  | HomeInit pageArgs ->
    let pageData, cmds = Home.init pageArgs
    { model with Page = Home pageData }, cmds
  | CourseDetailInit pageArgs ->
    let pageData, cmds = Course.Detail.init pageArgs
    { model with Page = CourseDetail pageData }, cmds

let update msg model =
  match model.Page, msg with
  | _, SwitchPage initArgs ->
    switchPage initArgs

Like Msg, InitArgs must be centralized and declared ahead of time so that any page has the ability switch to any other page. Or to calculate a URL so the user can switch pages with a link. Speaking of...

adding URL navigation

The main code you have to fill in for this involves converting InitArgs toUrl and fromUrl. Here is an example.

module InitArgs =

  let toUrl initArgs =
    match initArgs with
    | HomeInit () ->
      "#/home"
    | CourseDetailInit { CourseId = courseId } ->
      sprintf "#/course/%i" courseId

  // using Elmish.UrlParser
  let parseInitArgs state =
    oneOf [
      map HomeInit (s "home" </> unitArg)
      map (fun courseId ->
               CourseDetailInit { CourseId = courseId })
          (s "course" </> int)
    ] state

  let fromUrl location =
    parseHash parseInitArgs location

toUrl is mainly for your pages to generate navigable links to other pages. It is optimal to change pages with links when possible, so that the browser's back/forward buttons work as the user expects. Sometimes it is also desirable to manually update the URL from your app when the page options change significantly so that this URL gets added to history.

fromUrl is used in navigation events to pick the page to display. The function will also get used on app init, in case the user came to your app from a bookmark at a specific page. All that remains is to wire up those cases. MVU libraries have built-in integration for this. It can be done manually as well. Pass the location into app init to handle it there. And add an event listener for location changes, tag it in a Msg case like LocationChanged and dispatch it. Then handle it as another case of update where you call fromUrl and switchPage.

Side effects

A main emphasis of functional programming is pure functions. The complement of this emphasis is that side effects become an explicit concept. In MVU, explicit side effects are represented by Cmd.

However Cmd implementations have some less-than-ideal characteristics. In Elm they were very painful to extend with your own functionality (ports). In F# Elmish, Cmd simply defines a function signature which enables you to do whatever you want and dispatch Msgs to report the results back. This is about the most extensible you can get.

However I saw some opportunities for improvement. First it seemed like a mix of concerns to have update potentially creating a side effect function. Secondly it left update less testable than it could be. You can test the model easily, but it is quite invasive to test whether the correct side effects were requested. The ideal would be for Cmd to be just data representing the side effect and its necessary arguments.

`Effect`

I called the type Effect. It is entirely user-defined. I did not want to call it Cmd since Elmish already uses this name.

type Effect =
  | GetCurrentDateTime
  | SendApiRequest of ApiRequest

The update function returns an Effect list instead of Cmd.

let update msg model : Model * Effect list =
  match model.Page, msg with
  | Home pageData, HomeMsg pageMsg ->
    let pageData, effects = Home.update pageMsg pageData
    { model with Page = Home pageData }, effects

This completely decouples side effects from the update function. Effect is just data and is not tied to any implementation. It also makes it possible for Effect to be value equatable. Which means you can simply setup some expected data and use = to test that actual output matches, just like with Model. No mocks, stubs, or anything invasive at all. So now when you test update you can not only test model changes, but also whether the correct Effects are triggered.

It is sometimes necessary to work with unpredictable reference data like JS Files inside Effect. A match statement can be used to specially check those cases and use value equality for the rest.

The last piece is to code up the side effect implementation. I call the function for this perform.

let perform config effect dispatch =
  match effect with
  | GetCurrentDateTime ->
    dispatch (CurrentDateTimeIs System.DateTime.Now)
  | SendApiRequest request ->
    async {
      // http request details elided
      dispatch (ApiResponseIs result)
    } |> Async.StartImmediate

For Elmish, this new Effect/perform pattern is pretty easy to integrate. It just requires a couple of helper functions.

let init arg =
  let (model, effects) = App.init arg
  (model, effects |> List.map (App.perform model.EffectConfig))

let update msg model =
  let (model, effects) = App.update msg model
  (model, effects |> List.map (App.perform model.EffectConfig))

Program.mkProgram init update App.view
|> Program.withReactBatched "elmish-app"
#if DEBUG
|> Program.withConsoleTrace
#endif
|> Program.run

Here, EffectConfig is where I keep data needed by side effects. For example: API URLs, local storage keys, auth endpoints, etc. These are usually settings found in a JS file deployed with the app. They are grabbed just before the program starts and passed in as the init argument.

Practical issues

My web apps so far have a small number of highly reusable effects, so I tend to make Effect just a flat universal type that any page can use. In larger apps, it can be annoying to get the return Msg back to the specific page that requested it. What I presented above has perform sending responses back with non-page-specific Msgs. So you'd have to add a match for this message to each page that might receive one of these responses.

An alternative approach that I later found my devs using is to provide a return tag with the Effect. This tag is a function which takes the return value and wraps it in a Msg. The Msg would target the specific page that requested it. This hinders the testability of Effects, since you now have to specially treat most of the Effect cases to ignore the tag function and only test the equatable data. But it works since we only have a dozen or less effects, and a handful that use return tags.

You could also avoid this by customizing the available Effects per page. But we haven't found this worth the extra code.

To be perfectly honest, we do not test update currently. In our years of working with MVU there has not been a strong need. Yes we have broken things, but with the other mentioned principles they have been easy to spot and fix quickly. Yet we still wanted the ability to easily test update in case we need to introduce dynamically generated UIs.

The way we handle side effects is my favorite adaptation to the original MVU pattern. Because over the years I have observed that Separation of Concerns is the most important principle to maintainable software. And effect-as-data provides this via loose coupling between logic and side effect implementations.

Project organization tricks

Note on Model

Originally TEA had Msg, Model, init, and update inside the page module. We had to move Msg and the newly invented InitArgs out of the page module and fully define these types for all the pages before the page function definitions.

We can technically keep a page's Model with its functions. But then Model would be the only type that we defined there. So we ended up just moving it up as well. In essence, we split the page's related parts into PageTypes (defined first) and PageFns.

A typical project file structure would look like this.

App project

PageTypes
- Home.fs - Home's Msg, InitArgs, Model
- Course
  - Search.fs
  - Detail.fs
AppTypes.fs - App's Msg, InitArgs, Model
InitArgs.fs - has toUrl, fromUrl
PageFns
- Home.fs - Home's init, update, view
- Course
  - Search.fs
  - Detail.fs
App.fs - App's init, update, view, perform
Main.fs - Elmish Program wireup

F# has a single-pass compiler. And file order is compile order. So this organization structure is almost like we are creating our own two-pass compiler where types are compiled first, then the functions that use them. If F# had a 2-pass compiler -- types then functions -- it might be possible to keep all the page parts organized in a single module.

VS 2019 bug

The project structure would look like above. Except Visual Studio has a long-standing F#-specific bug. When you have two file paths that have the same subfolder as part of the path, intellisense glitches out. In the example above PageTypes and PageFns both contain a Course subfolder. Any file in the PageFns\Course subfolder will have completely broken intellisense as well as any other file defined after it. It also stops displaying the correct file order in the Solution Explorer window.

We simply name the PageFns subfolders with an underscore on the end (PageFns\Course_\Search.fs) to make the path different. The module names are kept the same (no underscore).

Another way around this is to not use folders but instead use multi-dotted files. Example: Course.Search.fs. Of course, this sacrifices the ability to roll-up/hide all the Course files when I am not using them.

I hope this gets priority to fix soon, because it is embarrassing to have to mention it in a post like this one.

Trick: Merging modules

One really nice property of F# is that opening two different namespaces will effectively merge all the types and functions across same-named modules within them. For example.

open PageTypes
open PageFns

type MyMsg = Home.Msg
let myInit = Home.init

// "Home" is a combination of all the stuff from:
// PageTypes\Home.fs
// PageFns\Home.fs

This still allows you to use all the page parts as though they were under one module. This is also how I extend existing types with new functions.

Pain point

The one major pain point with all the tactics I describe above is structural duplication. What I mean by that is Msg, InitArgs, and Model all have the same basic tree structure, but with different types as leaf nodes. To avoid conflicts, I must name them slightly differently.

// AppTypes.fs
type Msg =
  | HomeMsg of Home.Msg
  | CourseSearchMsg of Course.Search.Msg
  | CourseDetailMsg of Course.Detail.Msg

type InitArgs =
  | HomeInit of Home.InitArgs
  | CourseSearchInit of Course.Search.InitArgs
  | CourseDetailInit of Course.Detail.InitArgs

type Page =
  | Home of Home.Model
  | CourseSearch of Course.Search.Model
  | CourseDetail of Course.Detail.Model

Anyone familiar with Haskell is probably jumping up and down, screaming "Higher-Kinded Types". F# doesn't have those.

However I can think of 3 ways to reduce these 3 tree structures to 1 in F#. I will list them (quite subjectively) from least desirable to most.

These are thought experiments. I have not tried them in a real app yet.

3. Generics as fake HKTs

The setup for this is worse than the original solution.

type Area<'home, 'courseSearch, 'courseDetail> =
  | Home of 'home
  | CourseSearch of 'courseSearch
  | CourseDetail of 'courseDetail

type Msg =      Area<Home.Msg,
                     Course.Search.Msg,
                     Course.Detail.Msg>

type InitArgs = Area<Home.InitArgs,
                     Course.Search.InitArgs,
                     Course.Detail.InitArgs>

type Page =     Area<Home.Model,
                     Course.Search.Model,
                     Course.Detail.Model>

Adding a new page is pretty egregious. You have to touch Area in 2 places, then add a line to each of Msg, InitArgs, and Page.

The only improvement with this approach is that the update function is marginally nicer than the original solution.

let update msg model =
  match model.Page, msg with
  | Home pageData, Home msgData ->
    ...
  | CourseSearch pageData, CourseSearch msgData ->
    ...

Some people like this approach, but overall I think this has more losses than gains.

2. Union leaf data

This one is also more work to setup initially.

type Leaf<'msg, 'init, 'page> =
  | Msg of 'msg
  | Init of 'init
  | Page of 'page

type Area =
  | Home of         Leaf<Home.Msg,
                         Home.InitArgs,
                         Home.Model>
  | CourseSearch of Leaf<Course.Search.Msg,
                         Course.Search.InitArgs,
                         Course.Search.Model>
  | CourseDetail of Leaf<Course.Detail.Msg,
                         Course.Detail.InitArgs,
                         Course.Detail.Model>

Adding a new page is a slight improvement over the original solution. It is typing a few lines in one place instead of a new line in a few places.

The update function is slightly more verbose than the original solution.

let update msg model =
  match model.Page, msg with
  | Home (Page pageData), Home (Msg pageMsg) ->
    ...
  | CourseDetail (Page pageData), CourseDetail (Msg pageMsg) ->
    ...

The weird part of this approach is that model.Page and msg have exactly the same type. That means you can accidentally swap them and not get a compiler error. Then the UI won't work and there is no obvious reason why. Although, it should be possible to add some match cases to detect this and log a warning at runtime.

Overall I'd prefer this over #3, but it is still not ideal.

1. Only Msg

The last way I can think of to get one tree structure involves just eliminating InitArgs and Page, leaving us with only Msg.

Bye InitArgs

Getting rid of InitArgs is pretty far-reaching, but also has other benefits. Essentially the init function is removed and its behavior is put in a new case of update function. Then the InitArgs for that page becomes a Msg case. We will have to add one thing, a "zero" or empty model.

// PageTypes\Course\Detail.fs
namespace PageTypes.Course

module Detail =

  type Msg =
    | Init of courseId: Guid
    | CourseLoaded of Result<Course, QueryError>

  type Model =
    { CourseId: Guid
      Data: Remote<Course> }

  let empty =
    { CourseId = Guid.empty
      Data = Loading }

// PageFns\Course\Detail.fs
namespace PageFns.Course

module Detail =

  let update msg model =
    match msg with
    | Init courseId ->
      { model with CourseId = courseId }
      , [ ApiRequest (GetCourse courseId) ]
    | CourseLoaded (Ok course) ->
      { model with Data = Loaded course }, []
    // and so on

  let view model dispatch =
    // stuff

We have to do special handling of the Init message in the main update. We are using this kind of code instead of the switchPage function from before.

// App.fs
let update msg model =
  match model.Page, msg with
  | _, CourseDetail ((Course.Detail.Init _) as pageMsg) ->
    let pageData, effects = Course.Detail.update pageMsg Course.Detail.empty
    { model with Page = CourseDetail pageData }, effects

I used a similar approach when I built a Clojure MVU library.

Moving Page

UI state like Page is a bit different from Msg or InitArgs. The details of it are only used by the page functions. Outside parties do not need to know the page Model's contents. Previously we set it up as a public tree type for consistency with how we are handling other types. But there is another way. We can use a marker interface.

A marker interface is just an empty interface. Any type can just say it implements a marker interface, since there is nothing to implement. Marker interfaces can be looked at as a slightly different union type. You do not have to define all cases up front... a type locally chooses to opt in to the marker interface. But you also do not get compiler guarantees of complete matches. It is another expression problem trade-off toward extensibility rather than completeness.

// Types.fs
type IPage = interface end

// PageTypes\Course\Detail.fs
namespace PageTypes.Course

module Detail =

  type Model =
    { CourseId: Guid
      Data: Remote<Course> }
    interface IPage

// AppTypes.fs
type Model =
  { EffectConfig: EffectConfig
    Page: IPage }

// App.fs
let update msg model =
  match model.Page, msg with
  | :? Home.Model as pageData, Home pageMsg ->
    ...

We can essentially "tag" each page model as IPage. And any IPage can be stored in the Model.Page property. Unwrapping it to a specific page's model is a few more keystrokes. But there is no central Page union type to maintain anymore.

I have not tried this beyond checking that it would compile in a Fable Elmish project.

End result

At this point, we have eliminated all the duplicate tree structured types and are left with only Msg. We have also eliminated the init function. So the page logic is centralized in update.

Conclusion

Over the last 4 years I have made MVU my playground and learned quite a bit. It is a surprisingly resilient and flexible pattern for organizing UI applications. Because of its state-machine-like qualities, we even use it server-side. Hopefully some of the adaptations I discovered have been interesting to you.

This post has a lot of code snippets, but it could use a companion repo with more complete examples. With my ADHD, an effort like that will never go anywhere if left up to me. So if something like that is of interest to you, let me know. I would be more than happy to contribute.

cover image from unDraw

Addendum

I tried the #1 Only Msg approach and faced some challenges.

Moving InitArgs into Msg and removing `init`

Moving InitArgs into Msg

One thing I didn't cover was how this affects navigation. Instead of converting InitArgs to/from URLs, this now means converting Msg to/from URLs. The data to create a URL needs to be dug out of the specific page messages. And there will need to be a catch-all match because only a small amount of messages correspond to URLs.

match msg with
| CourseDetail (Course.Detail.Init initArgs) ->
  "#/course/" + string initArgs.CourseId
| _ ->
  defaultUrl

Alternative InitArgs tactic

It has become our standard practice to create an InitArgs record for each page. This record is a container for all the necessary parameters to the init function. And these page InitArgs need to be app-wide like page Msgs. So any page may use them to construct links to other pages.

We could instead use tupled values or anonymous records for a page's init args.

type InitArgs =
  //| CourseSearchInit of Course.Search.InitArgs
  | CourseSearchInit of search: string * page: int * pageSize: int

We then avoid having to define and organize those page InitArgs records. But some duplication of the parameter definitions may be necessary. For example in init if the type would be ambiguous in its usage.

let init (search, page, pageSize) =
  // ambiguous type error, requires annotating search as string
  let normalizedSearch = search.Trim()
  ...

And of course there are the same tradeoffs of using tuples vs records. When creating a new value, records are more verbose but easier to understand because the values are labeled.

Using tuples versus organizing all the page's InitArgs types to be app-wide, I do not think there is a clear winner. So you just have to weigh the tradeoffs on a case by case basis. It is probably easier to start with tuples if unsure.

Removing `init`

When removing init, a disadvantage is sometimes empty values can be a pain to construct. Like arbitrarily nested records. (I mainly encountered this when trying to remove the overall app init.) So using an explicit init function can feel more natural in those cases.

Overall

This does not feel like a worthwhile change. InitArgs is an additional app-wide type, but it has its own specific usefulness -- converting to/from URLs. And although init usually feels like a special case of update, it can sometimes be useful.

Moving Page

I mentioned using the IPage marker interface to represent pages instead of a central DU. But because F# does not auto-upcast, the main update has to explicitly upcast every page model to IPage. That's arguably worse than just tagging the page-specific model to go in a central DU.

One of the big benefits of using a marker interface is keeping the page Model with its init and update functions. And this is still possible when using a DU. Simply define the central Page DU in the main app file with the app's init and update (versus declaring it app-wide like Msgs). We used to organize it this way, but when I moved the other types (Msg, InitArgs) out of the pages I dragged Model with them. So I over-organized that.

Closing thoughts

Turns out that some of the annoying duplication I mentioned in the main article adds more value than it costs. And there are still minor tweaks available to tone down the annoyance. Without fundamentally deviating from the standard MVU structures.

Entity Component System - an old new thing

Kasey Speakman — Sun, 13 Dec 2020 00:58:31 +0000

Structured Programming rediscovered and honed for game development.

I've been looking into game dev lately. Always been a gamer. Always wanted to make games. But getting into it seems completely overwhelming. I found a podcast on DEV with Jason Weimann talking about game dev, and then his youtube channel with videos like this one.

His videos are incredibly helpful. But I noticed that many of them are centered around dealing with Object-Oriented ceremony and footguns. This is because Unity uses the OO model pervasively, and requires your code to use it as well. Being an old dev interested in system design, I went looking for other game dev organization schemes. I discovered the Entity-Component-System or ECS architecture.

What is ECS?

The three parts of ECS are:

Entity - a game object with a unique ID, contains components
Component - data about a single aspect of a game object
System - logic for a single aspect of game objects

The obvious interaction here is to pass the Entity to a System behavior. Such as "add to inventory" in the Inventory system or "take damage" in the Health system. The behavior can check if the Entity has a component that it knows how to work with. And if so perform the requested behavior on that component.

ECS cleanly separates the systems of the game into maintainable slices. The Unity folks seemed to think so as well since in 2018 they made an announcement that they were adding ECS to Unity. Not only for maintainability reasons but for performance.

At the end of 2020 this still remains to be seen. They have now hidden the beta ECS packages. (You can still install if you know the URL.) Presumably because there is so much churn that it is guaranteed to break game code in future updates.

This looks familiar...

On the whole the system ends up breaking down into data structures with no behavior, and functions or procedures that operate on those data structures (and the implicit side effects those procedures cause) to produce a behavior. This is the same arrangement used by Structured Programming.

Structured Programming came about in the late 1950s, at first primarily being about block structured syntax. And later officially branded in a 1966 paper by Corrado Böhm and Giuseppe Jacopini. But probably more well known as championed by Edsger W. Dijkstra, where we got our "... Considered Harmful" memes.

Like most things that are created and released into the world, the creators and proponents of this paradigm probably assumed it would be used in a certain beneficial way. To tame the spaghetti of Assembly code. But instead of using the tools to organize our code we just moved the spaghetti into procedures with Procedural Programming.

Most languages have procedural programming -- a descendant of structured programming -- as the substrate underneath their paradigm. Meaning once you get past the declarations the code is written as a statement-oriented and block-structured procedure.

What is different?

ECS is an architecture, not a program structure

The primary difference is purpose. Structured Programming is meant to put your program on good footing for maintenance. A video game is just a program, right?

Look at it this way. If a business has a universe of applications and services working together to deliver business value, a video game is a microverse (or sometimes an even larger universe) of all the same kinds of things working together to deliver user value (game experience). The breakdown of a triple-A game into its many systems, when translated into the business world, would be the domain of a software architect. So in essence ECS is not simply a program structure, but a base architecture.

Entities have runtime composition

This significant difference in packaging compared to business systems leads to another important adaptation. Namely, you can add or remove arbitrary Components (data structures) from an entity at runtime. This is quite different from structured programming where data structures were completely pre-defined. Additions and removals of this kind would need a recompile and redeploy.

Since ECS is an architecture, this property of entities makes perfect sense. Put on your Business Dev hat and consider a Customer. Accounting does not need to know about every sales lead... only the ones that buy something. And then they care about their own specific aspects of that customer which sales may not, e.g. billing address. In this analogy the billing address is part of the Accounting ECS component, but not the Sales component. And Accounting ECS system requires that component it be present to perform its function.

Remember, an ECS Component is only data. State would have been a better name for it, but nobody asked me back in 2007 when they named it.

Other connections

I have previously made a connection between Structured Programming and Functional Programming. Basic FP (without all that category theory) takes Structured Programming and draws out one more key concept -- the side effect -- and makes it first class and explicit instead of implicit. This greatly increases the testability and maintainability of code. And so I think ECS could create a strong synergy between game dev and FP. Wait: FP and game dev -- am I crazy? I don't think so... check out John Carmack's thoughts on the subject.

Conclusion

Like most new ideas, ECS stands upon the shoulders of giants that came before. It is not new at its core, but how it is applied is novel. Unity seems to be a popular and "easy" entry into the world of game dev. I look forward to trying their ECS system. Especially since F# is also usable with Unity. ECS and F# would make the ultimate combo for me.

Static vs Dynamic Typing for ADHD

Kasey Speakman — Mon, 07 Dec 2020 22:15:40 +0000

Static typing has boring overhead but it augments my brain's limited working memory.

The PHP years

My first professional software product was a sign-out board for consultants and company cars. At the time I was employed quite cheaply as IT support not a programmer. I wrote it in PHP, which is dynamic in case you don't know, circa 2000. Later I added an expense reporting feature to replace their existing process of emailing spreadsheets to accounting.

I went on to write many small things in PHP on the side. A service to receive automated responses from an SSN validation provider. (Similar to what web hooks do today.) A limited blog-type website for a charity. (WordPress didn't exist back then.) And a few other things.

After a couple of years the company I worked for closed, and I was without a job. My dad hired me to write an internal ordering system for his small business, which produces lighted outdoor signs. There is a tiny chance that some of those golden arches you see from the interstate in the USA were made to order at his shop.

My equipment betrays me

Pricing a sign and speccing it out for manufacturing is complicated. The wizard-style app I ended up writing had a dozen or 2 possible pages of data collection. It had branching paths depending on selected options and multiple pricing rules. This is where PHP broke me.

Later after I tried other languages, I came to have a strong distaste for PHP. But I realize now that the negative experience I had with PHP was not its fault. It was my brain's tiny working memory -- a trait of ADHD. To keep the small details of a wizard process in mind across a dozen pages was too much for it. Especially when ADHD prevents me from choosing what to focus on. So every day became a frustrating process of continually losing focus and forgetting the information I needed to carry from one page to the next. And having to repeatedly remind myself of the same information. Eventually breaking through and having enough information to make a little hard-fought progress. Some days I was too burned out and simply let the hours pass in defeat. I realized none of this at the time and I just assumed programming for a living was not for me.

The (mostly) typed years

I did eventually finish that app. And as much as I wanted to do hardware work, people kept wanting to pay me for software. I took on some work writing in VB, then C#, Javascript, some others, and finally F#. After C# I realized that I could maybe see my way to coding for a living and did so from then on. Fast forward a decade to the last few months, I starting learning Clojure.

Rediscovering dynamism

I created MVU and validation libraries in Clojure. I absolutely loved how tiny yet understandable they were. I started rewriting my front-end-only budget simulator in it. While doing that, a feeling struck me that I had not felt for a long time. It was frustration with myself for not being able to remember basic information from one page to the next and the fear that I had remembered it wrong. It was notable because it happened repeatedly.

Then I discovered I had ADHD, and these experiences started to make sense.

Dynamism in context

Part of the reason I was interested in Clojure was precisely because it was dynamic. After many years of writing in typed languages and creating larger and larger apps, maintaining types can become tedious. And it definitely adds coding overhead that I was interested in reclaiming. What I did not realize until recently is that my neurological traits make types highly beneficial for me. Types essentially provide a library for me to look up the shape of the data. And editors instrument this with the ability to hover and see the type definition in an instant. So types augment my limited working memory very effectively.

Programming already requires an expanded working memory as you progress toward an overall solution. But with ADHD a lot of the important minutia (like what did I name that field, or what kind of data was in it?) is impossible to keep in working memory because it is uninteresting and therefore evades focus.

For other people, this may be less of an issue and dynamic languages are a net savings. But for me the overhead of maintaining types is well worth the benefit. It is especially detrimental to the self-esteem to get so repeatedly frustrated with one's own limitations.

Same for FP
This is also the reason that the functional programming paradigm works so well for me. FP focuses on using deterministic (aka pure) functions. By definition, this mean I do not have to remember any state from outside the function. I only have to deal with what is right in front of my face. This plays to my strengths.

I have not given up on Clojure. Spec allows me to have types, in effect. The main issue is that Spec is still in alpha and extensive use of specs seems to be considered abnormal in Clojure code. For now I am keeping my eye on it. I have enjoyed writing code in Clojure so I am interested in figuring out a system that works for me there at some point.

Path ahead

Overall, this realization about my neurological traits means I will be preferring types in some form or another for the foreseeable future. And F# is my go-to language.

ADHD: The Grief of Discovery

Kasey Speakman — Fri, 20 Nov 2020 05:27:53 +0000

Mourning the loss of the person I might have been.

This is a followup to I have ADHD 🌧️. There is a strange phenomenon where the discovery of having ADHD late in life is followed by a grieving period.

When I was first told I had ADHD it was surprising to me. I had heard of ADHD and made some assumptions about it. I thought it would be an add-on diagnosis to the problem I really had. But it turns out that ADHD is quite serious by itself. If it is unmanaged, it can ruin lives. And it certainly has caused a lot of problems in mine.

ADHD carries with it significantly higher risk of substance abuse, incarceration, divorce, suicide, bankruptcy, traffic accidents, and many other bad outcomes. More on this in a later post... if I can concentrate long enough.

I was overjoyed to finally know what was wrong and that there are ways to manage it. But the joy was short-lived.

Fundamentally, ADHD leaves you feeling not good enough for the rest of society. Especially for someone like me who was diagnosed as an adult. I spent years trying my hardest and failing at the most basic tenants of modern life: structure, order, consistency... routine. I could not help but feel that I was missing some vital human quality that everyone else has. It not only affected my behavior in the present, but also my outlook on what is possible.

For example, in early childhood I was overly talkative and frequently got in trouble for it at school. Not only that but it tended to annoy other kids, and they unsurprisingly reacted with negative reinforcement. I became very socially reserved and anxious. I knew that I could not get social situations right. So I had to change how I approached it -- now with a lot of caution and fear. I became more isolated and did not have many friends. The ones I did have, I typically did not spend much time with outside of school obligations. Because hanging with me could be exhausting. The high school years were very hard.

I was told I had intelligence. But I did not feel smart. How is it smart to not do your homework or study for tests? I had some classes that interested me so I was easily able to do assignments and excel in those. This was evidence that I could do well "if I tried". But all my other subjects were a cold war, waged internally and externally with my parents and teachers. As much as I wanted to prove I was the intelligent person others believed I could be, eventually I was choosing easier classes to avoid the conflict.

I had dreams of going to MIT, which to me was the pinnacle of technology schools. But I knew there was no way I could get accepted there, much less complete a degree. For all my gifts I could not even manage high school well, graduating with a middling rank in my class. My 4-year college degree took 9 years, off and on. I finished at a local college and really only with the help of my classmates who had formed a study group.

I am thankful for that local college and my classmates or I might never have finished my degree at all! And in the 9 years it took, I ended up with 3 majors.

To top it off I had an addiction to video games. I could not get enough of them since I got my first Nintendo around age 10. Now I see it for what it is, self-medicating that missing dopamine. But at the time I and my parents thought it was the crippling illness that hindered my life goals.

Goals is the wrong word. I never had goals. The act of making goals is just laying a minefield of disappointment for future me. I think the right word here is potential.

The Me that never was

So at this point I go from being joyful at the discovery I have ADHD to morbid grief. The person that I was growing up never had a chance. Their own brain was fighting them tooth and nail. And that traditional future where they realized their potential is gone forever. It wounded me deeply, like the loss of a twin brother. I spent a day crying over it.

It would be easy to reach for blame here. I mean it is always the parent's fault right? Shouldn't they have seen it? Or the teachers? Well not really, no. Considering the knowledge of ADHD was not widespread back then and my brain lives to adapt (and trying to appear "normal"), it would have been hard for them to see it. When I look at how bad outcomes can be for unmanaged ADHD, I think my parents did a great job by giving me boundaries and a compass to keep me away from most of them.

The Me that is

The good (?) thing about ADHD is that I lose interest in things, including grief, quickly. (Plus I didn't actually lose someone.) I'm sure it will come back up now and again (like it did while writing this post). But there is no use playing the What If game. Also taking stock of today, I am doing well. Not sure if y'all heard but experienced programmers make a decent living. And I have managed to form meaningful relationships and keep them, by the grace of God.

So the right question is what to do now.

Well you are reading it. I am sharing my story and raising awareness. A lot of people still think ADHD (and mental health in general) is like having brown hair, not a big deal. Or like the Easter Bunny, not real. Or like the opioid crisis, a pharma conspiracy. Or like spoiled kids, a parental problem. It is none of those things. If unmanaged it can devastate lives, not just of those who have it. It is torturous to watch a loved one struggle and self-destruct!

So my call to action is to educate yourself. Not every problem is going to turn out to be ADHD. But if you've done everything you know to do and still just "don't feel right" or "can't do right" talk to someone who has the training and tools to help you figure it out. And if you have kids who are struggling in that way, get them to the right people. Why waste another moment?

Managed ADHD is a gift

I forgot to add this part initially.

From what I have researched many people with managed ADHD (through medication or developing skills or both) are glad to have it. We tend to have certain characteristics that make us really good at specific and rare things that society and businesses need. Even though mine is not managed to the degree I want, I know that I bring a unique x-factor to our team that helps it succeed.

I'm left to wonder whether thinking differently is intrinsically a disorder or whether modern society just hasn't got around to making space for us yet.

I have ADHD 🌧️

Kasey Speakman — Tue, 17 Nov 2020 23:48:15 +0000

I am pathologically incapable of performing uninteresting tasks. And I hate myself for it.

First I should probably explain what ADHD really is. It is a deficiency of specific neurotransmitters in the brain. Or "malfunctioning" neurons which flush the neurotransmitters too early. Or both. This results in neurons not firing when they should. Disclaimer: this is my understanding, I am not a neuroscientist.

The primary consequence is that people with ADHD have little control over their ability to focus attention on tasks. Two common tricks that we use to focus are pressure (deadlines, something broken in production, etc) and being genuinely interested. Especially strong versions of these stimuli can push the ADHD person into a state of hyperfocus. In this state, the body can completely forget about sleep, food, and bathroom breaks while the person works or plays furiously. The counter-balance to that is when not stimulated, the ADHD brain cannot engage in planning or organizing tasks.

It is not about willpower
Everyone experiences lack of motivation sometimes. A non-ADHD person has to muster the willpower to do uninteresting tasks. For an ADHD person in this state, their brains are chemically preventing them from concentrating. So it is like piloting an airplane through a hurricane... willpower is not enough to get you through.

There are actually 2 other behavior areas affected by ADHD: impulsivity and hyperactivity (edit: and emotional dysregulation, but this is harder to quantify scientifically). And there is a spectrum of severity.

Certain (now rare) occupations or lifestyles are well-suited to the differences in thinking of an ADHD person. So in those cases it may be a benefit rather than a problem. In the more sedentary modern life that most of us live, ADHD makes things quite difficult but is highly treatable with medicine and/or therapy.

My experiences

For me personally, hyperactivity and impulsivity are milder. Those are usually the things that cause frustrated parents to have their child examined. And so I went undiagnosed for the first 41 years of my life. Instead I came to understand myself as a lazy and undependable jerk, whom I loathed.

However, I did not make it through 20 years as a software developer by getting nothing done. With a lot of struggle, I organically adapted to my limitations.

Writing code

Some people can trick themselves into getting things done by setting fake deadlines. This unfortunately has no effect for me. Even when other people give me deadlines, when I can tell they are arbitrary (which they usually are), they provide no stimulation for me. What does work along these lines is when immediate help is required (production issue, mentoring, etc).

Aside from emergent situations, the only thing that keeps me on task is being interested in the subject matter. And I only stay interested while there is a mystery left to solve. Earlier in my career my code looked like it was written by many different people. Because in a larger app, I would switch between many different approaches before I finished it. Trying interesting new approaches was the only way I could focus on writing code.

As time went on, my interest went from tactics in the small to application organization and then whole system design (architecture). It is perhaps fitting that the area that came to interest me the most about software was the area where I struggle the most personally: how to organize things.

Eventually I became valued for my expertise. Because I had tried or researched a lot of things, and I remembered what did and did not work about them and why. And I could synthesize solutions that matched the problem constraints. Although introverted, I learned also to talk with customers to tease out those problem constraints.

I never cease to be interested in designing solutions. But the main problem is that most companies need someone to fill in code primarily and design solutions a relatively small percentage of the time. Even as a lead developer. And after 20 years, it has gotten very hard for me to find interesting things in coding.

Management would be another option, except that I have little tolerance for authoritarian structures (top-down management). The way I approach my teams is by providing information and guidance, but letting them decide. And I prefer to be approached in the same way. I find the prospect of enforcing the arbitrary and unexplained whims of those "above" me onto those "below" me to be an intolerable situation. So that rules out management in most companies.

Changing jobs

I must periodically change jobs or I will probably get fired. Keeping up with the routine of an office job eventually becomes extremely challenging. When I first start a job, it is an exciting time and it is easy to keep up the routine. There is a lot to learn about the position, the company, the team dynamics, the subject matter, etc.

But eventually things start to get comfortable. And with all my heart I wish that meant I settled into a routine. But instead it means that there is no longer enough stimulation to engage me in the routine. I start coming in a few minutes late. Then it gets later and later. I don't miss scheduled meetings. But eventually I might come in afternoons or call in for a mental health day.

Some of this issue is sleep. I do not feel tired at night. If I try to go to bed my mind is still occupied or I am still locked on to an interest (research, video game, personal coding, etc.). I cannot transition to sleeping. Many times I go to bed anyway. Sometimes there is sleep, but often I will just lie in bed awake most of the night with my mind racing. I have typically found it best to just get up and do whatever is stuck in my mind to release it. Then I can sleep. But that means I stay up late.

Getting up in the morning isn't hard when I already have something on my mind or am looking forward to something. But on an average day, I lay in bed letting my mind wander until eventually it hits an interesting thought. And only then can I get up. And usually being late to something (pressure-induced focus) is required to finally get up. Days which I know have particularly uninteresting tasks, I keep thinking about them, but it provides pressure against getting up. On days when I call in, I have usually laid there in bed for a long time, unable to move. But as soon as I call in and no longer have to worry about it, I think of something that gets me out of bed. It could be as simple as a delightful cup of coffee.

When things start to get comfortable at work, the danger begins for me. I reckon this takes about 2 years. I will probably have started coming in late well before that as I got used to the office situation. But by 2 years I will probably have gotten comfortable with the code base. And I cannot focus to work on it anymore. Then I need to find another job to restart the process before I get fired.

I have actually stayed 5 or 6 years at a couple of jobs and never gotten fired. Miraculously in some cases. I think it is a combination of trying to change projects or positions within a company to reignite interest. And also when approached about my inability to follow routine I own it and try to figure out ways to adapt. They don't last but I try.

If I keep soldiering on and manage not to get fired but also not change the situation, I eventually become deeply unhappy. Depressed even.

Personal life

None of this is even to mention the effects of ADHD on my personal life. I can probably count on one hand the things I do as consistent routines. Actually the only things I can think of are making the bed and making coffee when I get up (not necessarily in the morning). Pretty much everything else might stabilize for a week or two, but will eventually be in flux as to whether and when I do it.

I learned long ago that joining any groups that had routine activities was a recipe for disappointment. It is especially bad because I tend to go all-in at first. But then when it becomes routine, I cannot even show up anymore. This hurts everyone's feelings as they think I do not like them anymore or I got offended or whatever. And I didn't even know why myself.

As you might imagine, this has some detrimental effect on relationships. I am amazingly blessed with an angel of a wife who has put up with me for 13 years, not even knowing I had an illness. We have certainly had arguments about me not helping with chores or sleeping during the day or playing video games for 12 straight hours to the exclusion of all else. But we always kept trying with each other. As I write this I realize that I in no way deserve the grace that she or my children have given me.

I also feel an immense guilt for not being able to write code some days (or weeks). I feel that I am hurting my team and my company and my salary would be better spent on others. I feel that I deserve to be fired as a matter of fairness.

How did I not know?

An outside observer might think it should have been obvious that something was wrong. And to a degree I have always known something was wrong, that some "normal" things seemed really hard for me. But I was led to believe from early age I was being lazy and selfish. And that is what I thought was wrong and told others was wrong with me. I didn't exactly behave like that, but sometimes it fit.

And it gets attributed to personality. For example, my wife showed me the Type 5 Enneagram, which is said to have limited energy so they are very careful about how they spend it. It seems to fit some of my behavior quite well, so people assume it is part of my personality.

What actually made me think that my problem might be an illness was a video called Devs and Depression that @ben posted in a comment several weeks ago. So glad Ben posted it and I saw it.

What now?

Right now I am in a waiting game. There seems to be more demand for mental health professionals than supply. Especially during this time where everyone is forced into new social patterns. I had to wait a few weeks for a therapist appointment. And am currently waiting another month to see a psychiatrist to get further testing and probably a prescription. Meanwhile I am languishing in the guilt and shame of what most employers would deem inexcusable negligence. An unfortunate but familiar companion to me.

Normally I would have already sought another job, for all our sakes. Except I love my company and my team, and we are on the verge of some growth. Growth that could provide me an opportunity to change roles to work that fits me better. And I could get medicine soon. Which might make things better. I don't even know what the world might be like if I am able to directly control what I can focus on. Or if I am far too practiced in my current approaches to be able to. I tear up when I think it possible that I could have a daily routine last for more than a couple of weeks. It seems too much to hope.

So here I am with seemingly everything on the line, waiting to see what situation resolves first. It is not the most comfortable state.

There is a strange phenomenon when someone discovers they have ADHD as an adult. After some initial joy there is a period of mourning. I wrote about mine here.

Take away

The reason I wrote this post was to share my experiences and raise awareness. ADHD is not a made-up thing or because parents are being too soft (opinions which I have held in the past). It is not in the person's control. It is a significant problem. It is easy to miss, and people with it are conditioned to hide their difficulties. People who have it end up with self-esteem problems because they are constantly expected to do things which are very hard or impossible for them but 90% of people seem to do with ease.

Consider that you never really know what is going on with a person's behavior. And they may not know either! If you feel safe doing so, lead with compassion and offer your support. Help them find out what's wrong. You could be a life-saver.

Clojure validation in 30 lines

Kasey Speakman — Tue, 20 Oct 2020 09:46:55 +0000

A function composition approach to validation.

In the previous post, I offered some critique of the validation approaches I found. In his post I bash together an alternative solution.

Goals

Here are the things I was interested in.

validation definitions are functions
composable from smaller pieces
no macros
simple errors

5 functions

There are 5 core functions to compose validations.

`fn`

This creates a basic validation.

(require '[validation.core :as v])

(def validate
  (v/fn :required not-empty))

(validate "asdf")
=> "asdf"

(validate "")
=> {:v/error :required}

(v/error (validate ""))
=> :required

Any function can be used for validation provided it returns nil if anything goes wrong. When that happens, :v/error is returned with the provided trait (:required here).

You provide a trait so you know exactly the errors to expect.

`and`

This combinator can be used to send a value through a series of transformations, each of which might fail. The end value will be returned if all steps are successful. Otherwise the failing trait will be returned as an error.

(def validate
  (v/and (v/fn :required not-empty)
         (v/fn :keyword keyword)))

(validate "asdf")
=> :asdf

(validate "")
=> {:v/error :required}

Note: This is not a strong validation because it does not deal well with non-string inputs or whitespace.

`seq`, `hmap`

Validate a sequence.

(def validate
  (v/seq
   (v/and (v/fn :required not-empty)
          (v/fn :keyword keyword))))

(validate ["a" "b"])
=> (:a :b)

(validate ["a" ""])
=> (:a {:v/error :required})

Validate a hash-map.

(def validate
  (v/hmap
   {:a (v/fn :required not-empty)
    :b (v/and (v/fn :required not-empty)
              (v/fn :keyword keyword))}))

(validate {:a "a" :b "b"})
=> {:a "a", :b :b}

(validate {:a "a" :b ""})
=> {:a "a", :b {:v/error :required}}

(map :b [{:a "a" :b ""}
         (validate {:a "a" :b ""})])
=> ("" {:v/error :required})

`or`

This combinator returns the first passing validation.

(def validate
  (v/or :date-or-int
        (v/fn :date str->date)
        (v/fn :int str->int)))

It takes an additional trait since returning every failing trait would not would not be entirely useful.

Validation functions

The 30 lines of code are just the definition of the 5 core functions and 2 error checks. Validation functions are still needed to plug into them. Clojure has some built-in functions that can be used but you are probably going to want more. Fortunately they are easy to create. Here are some examples.

(defn non-blank [s]
  (when (string? s)
    (not-empty (trim s))))

(defn in-range [min-val max-val]
  (fn [v]
    (when (and (>= v min-val)
               (<= v max-val))
      v)))

When a validation takes multiple parameters as in-range does above, it should return a function to take the last parameter. This bakes in the necessary partial application.

(def validate
  (v/fn :range (in-range 0 4)))

(validate 3)
=> 3

(validate 5)
=> {:v/error :range}

Conclusion

I really like the function composition approach to validation. It is composable and extensible. Its implementation is very small (see below). Which means I can easily change or add more functions. I find it works quite well for user input validation.

The 30 lines

(ns validation.core
  (:refer-clojure :exclude [and or fn seq]))

(defn fn [trait f]
  #(if-some [value (f %)]
    value
    {::error trait}))

(defn and [f & fs]
  #(loop [v (f %) fs fs]
     (if (contains? v ::error)
       v
       (let [f (first fs)]
         (if (nil? f)
           v
           (recur (f v) (next fs)))))))

(defn or [trait f & fs]
  #(loop [possible (f %) fs fs]
     (if-not (contains? possible ::error)
       possible
       (let [f (first fs)]
         (if (nil? f)
           {::error trait}
           (recur (f %) (next fs)))))))

(defn hmap [m]
  #(reduce-kv update % m))

(defn seq [f]
  #(map f %))

(defn error? [v]
  (contains? v ::error))

(defn error [v]
  (::error v))

Forem: Kasey Speakman

Explaining the LMAX Disruptor

Context

Java Limitations

Structs

Memory Layout

Boxing

Performance optimizations

Fixing the Elm Architecture

Centralizing logic into update

Parameter order

Untestable Cmd

Declarative side effects

Nested page fatigue

Composition through view

Summary

Event Sourcing - Stream Metadata

Relationships with Event Sourcing

In starter event

In key events

Stream metadata

Prior art

Shareholder Value: the Other Pandemic

A Boring Dystopia

Allow me to Translate

What Now?

Desktop apps in HTML and F#

Why web UI?

Which web UI?

Why not Electron?

WebView2

How does it work?

The repo

kspeakman / FSharpHtmlDesktopApp

A starter desktop app which uses HTML and F# for the UI and F# .NET for the API.

Closing thoughts

Readable vs Workable Code

Now to refactor

Is this better?

Closing thoughts

Organizing MVU Projects

Modified TEA for UI state

for UI state

Independent data

What does "independent data" look like?

update refinements

Routing and Navigation

adding URL navigation

Side effects

Effect

Practical issues

Project organization tricks

Note on Model

VS 2019 bug

Trick: Merging modules

Pain point

3. Generics as fake HKTs

2. Union leaf data

1. Only Msg

Bye InitArgs

Moving Page

End result

Conclusion

Addendum

Moving InitArgs into Msg and removing init

Moving InitArgs into Msg

Alternative InitArgs tactic

Removing init

Overall

Moving Page

Closing thoughts

Entity Component System - an old new thing

What is ECS?

This looks familiar...

What is different?

ECS is an architecture, not a program structure

Entities have runtime composition

Other connections

Conclusion

Static vs Dynamic Typing for ADHD

Centralizing logic into `update`

Untestable `Cmd`

Composition through `view`

`update` refinements

`Effect`

Moving InitArgs into Msg and removing `init`

Removing `init`

`fn`

`and`

`seq`, `hmap`

`or`