Forem: Andrew Meredith

What are micro frontends anyway?

Andrew Meredith — Fri, 29 Oct 2021 17:37:48 +0000

I have a good bit of experience as a developer, and I still honestly don't understand what micro frontends are.

Andrew Meredith

@a4w_m6h

I have seen so many people talking about micro frontends, and I honestly can't tell if there is a single, well-accepted definition of micro-frontend. Can it just be separate SPAs that link to each other?

15:55 PM - 29 Oct 2021

Specifically, it seems like they are either just standalone SPAs with links between them, or they are essentially Web Components that are plugged into some site.

Can you all help me out? What are micro frontends? Do you use them? Do you like them?

How to Grow as a Software Engineer - 5 Tips

Andrew Meredith — Wed, 27 Oct 2021 19:22:44 +0000

So you got a job as a software engineer. What now? There are 1,000 technical skills that you can dig into. There is always a new framework to learn. Your company probably has a large codebase that you can get familiar with. You need to know Git, the command line... the list goes on. In many ways, how you approach career growth is more important than the specific things you learn. In this post, we'll look at 5 tips for helping you grow as a software engineer. We will not be talking about specific technologies since those change all the time. Instead, we'll look at the mindset that will best help you advance in your software engineering career:

Set Goals
Ask Questions
Let Yourself Get Stuck
Follow Trends
Don't Follow Trends

1. Set Goals

If you know what you want to achieve, there is always a way to plan how to get there. Without clear goals, it is all too easy to spend hours on things that, in retrospect, did not serve to help you in your career. Reasonable goals guide us in determining what to do and what to say "no" to. That means that a goal must be specific enough to serve as a compass guiding your journey but general enough that it is not just another task on your to-do list. A good rule of thumb is that a goal should be something that you cannot complete over a weekend, and it should also not be something that will take more than a year.

Goals should also be actionable. Just like a good software feature spec, a goal should have criteria that you can use to determine whether you have met it. For example, if you want to learn Vue, your goal should not be "Learn Vue." Try something like "Learn Vue well enough to rewrite [PROJECT X] in it without referring to documentation" or "Learn Vue so that I can write a blog post comparing it to React." With more concrete goals and completeness criteria, you will find that it is easier to decide what to work on next.

Also, set aside some time - maybe just five or ten minutes per week - to think back over what you have accomplished and to refocus if necessary. Keep a document where you write down the progress you have made, the things you have learned, and then return to it when you feel discouraged. I promise that it will be motivating to remember how far you have come already!

2. Ask Questions

Whether working on a team or as an individual developer, one of the most helpful practices that you can foster is asking questions - even if you already have a good idea of the answer! By asking someone a question (in person or online), you may get a direct answer, and that is great. However, you may learn another way to think about your problem. Even the exercise of formulating a compelling question can help the way you think about it.

Kent Beck

@kentbeck

Sometimes you know the answer & asking the question is still valuable.

16:12 PM - 26 Oct 2021

Another good reason to ask questions is to fight imposter syndrome. Most of us struggle with feeling like we are not adequate to solve the problem in front of us. The best way to fight imposter syndrome is by being honest with yourself and others about what you know and what you don't know (yet). By asking questions, you tell yourself and everyone around you that you want to learn and that you can learn.

3. Let Yourself Get Stuck

Asking questions is great, but sometimes the best thing you can do is struggle through a complex problem. While you should not try to solve every problem from scratch, you should occasionally take the opportunity to use what you already know to figure out something that you don't. As you advance in your career, you will find that you sometimes face problems that nobody on your team or StackOverflow has solved before. Developing the problem-solving skills to overcome these situations can be one of the most valuable tools you will ever put in your toolbox.

The feeling of accomplishment when you are stuck for a while and finally figure out how to move on is incredibly motivating, and it will give you a nice boost of confidence. If you have been working on one problem for a while without making progress, go back to the previous tip and ask some questions. There is no shame in getting help, and some issues really are incredibly tough. Challenge yourself, but never drive yourself to the point where you feel defeated. Remember that when a problem feels too hard, it is not because you are not good enough to solve it. It is because it requires specific knowledge you haven't gained yet, so cut yourself some slack, do some research, ask questions, and move on.

If you are the note-taking type, then it can be helpful to chronicle how you wrestled with the problem. Even if you never return to these notes, the process of writing them will help you remember what worked and what did not the next time you encounter a similar problem.

4. Follow Trends

Career growth follows some sort of progression. You start at some point, and you grow toward some future version of yourself. To best direct the goals you set (remember the first tip!) to get to the version of yourself that you want to be, you need to be strategic. Want to be a kick-ass front-end engineer? Learning React and Tailwind are probably better uses of your time than learning jQuery. Want to get into cloud-native development? Go, Kubernetes, and service mesh should be the things that you focus on.

Try to dedicate a little time every week to keep up with what is new and exciting in the communities that interest you. This can be difficult because there is a lot of noise to filter for every bit of signal. At first, it may be difficult to tell what is hype and what is a legitimate trend. You will develop the skills over time to predict whether some new technology is going to have a lasting impact. Still, at least initially, you may want to be among the early adopters or early majority for a tool or technique rather than try to be among the innovators. It feels good to be on the bleeding edge, but it feels better to look back with confidence in what you have chosen to invest your time in.

Finally, your interests will probably change over time, and that is a good thing! Don't let dedication to something you used to enjoy prevent you from pivoting and pursuing something else. Sometimes it can even be advantageous to shift directions in your career because you will be a more well-rounded individual and bring a unique perspective that not everyone else will have. Personally, I have learned more from engineers who started doing something else - from QA to quantum physics research - than from people who have spent their entire career in a single track. Your career is your own.

5. Don't Follow Trends

Wait, didn't I just say to not follow trends? Paying attention to the trends helps you decide what goals to set for yourself, but (from someone who has been there) chasing every new technology leads to burn-out. Try to have just a couple of goals at any point, and don't spend time on things that don't further those goals. Ultimately, when you force yourself to prioritize what is most important, you have to just wave at some trends as they pass by. Especially in fast-moving ecosystems, there will always be more you want to discover than you will be able to. Remember that even the developers you look up to are not experts in everything, and they likely know surprisingly little about most of the tech headlines we see.

You can measure your skillset by depth and breadth. Now, it may be perfectly legitimate to have a goal like "Improve the breadth of my front-end knowledge by reading 1 book on each of the 4 most popular frameworks," but do not try to become an expert in all four frameworks simultaneously. Make sure that you are clear on when you are working for depth and when you are working for breadth.

Conclusion

I hope that this article helps you focus your time and energy on the things that help you get to where you want to be. Whatever direction you want to take your career, whatever the specific things you decide to invest yourself in, remember to cultivate the habit of being mindful about where you are going and purposeful about how you get there.

Sorting a Dependency Graph in Go

Andrew Meredith — Tue, 26 Oct 2021 18:38:37 +0000

Recently, I was thinking about how many of the nontrivial problems that I run into with software engineering boil down to a few simple problems.

// Detect dark theme var iframe = document.getElementById('tweet-1428817076878299150-887'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1428817076878299150&theme=dark" }

Just look at any book on algorithms, and the majority of them will be some variation on sorting or searching collections. Google exists because the question of "what documents contain these phrases?" is a genuinely hard problem to solve (okay, that is vastly simplifying the enormous scope of Google's product, but the basic idea still holds).

What is Topological Sorting?

One of these common problems that I have run into again and again over the course of my career is topologically sorting the nodes of a dependency graph. In other words, given some directed acyclic graph - think software packages that can depend on other software packages or tasks within a large company project - sort them such that no item in the list depends on anything that comes later in the list. Let's imagine that we are making a cake, and before we can get started, we need some ingredients. Let's simplify things and say we just need eggs and flour. Well, to have eggs, we need chickens (believe me, I'm resisting the urge to make a joke here), and to have flour, we need grain. Chickens also need grain for feed, and grain needs soil and water to grow in. If we consider the graph that expresses all of these dependencies, it looks something like this:

One possible topological order of this graph is:

[]string{"soil", "water", "grain", "chickens", "flour", "eggs", "cake"}

However, there are other possible sort orders that maintain topological ordering:

[]string{"water", "soil", "grain", "flour", "chickens", "eggs", "cake"}

We could also put the flour after the eggs, since the only thing that depends on eggs is cake. Since we can rearrange the items, we could also fulfill some of them in parallel while maintaining that no item appears before anything that depends on it. For example, by adding a level of nesting, we can indicate that everything within an inner slice is independent of anything else in that slice:

[][]string{
    {"soil", "water"},
    {"grain"},
    {"chickens", "flour"},
    {"eggs"},
    {"cake"},
}

From this graph, we get a nice "execution plan" for how to prepare the dependencies for a cake. First, we need to find some soil and water. Next, we grow grain. Then, we simultaneously raise some chickens and make flour. Next, we collect the eggs. Then finally, we can make our cake! That may seem like a lot of work for petit fours, but good things take time.

Building a Dependency Graph

Now that we understand what we are trying to do, let's think about how to write some code that is able to build this sort of dependency list. We will certainly need to keep track of the elements themselves, and we will need to keep track of what depends on what. In order to make both "What depends on X?" and "What does X depend on?" efficient, we will track the dependency relationship in both directions.

We have enough of an idea of what we need to start writing some code:

// A node in this graph is just a string, so a nodeset is a map whose
// keys are the nodes that are present.
type nodeset map[string]struct{}

// depmap tracks the nodes that have some dependency relationship to
// some other node, represented by the key of the map.
type depmap map[string]nodeset

type Graph struct {
    nodes nodeset

    // Maintain dependency relationships in both directions. These
    // data structures are the edges of the graph.

    // `dependencies` tracks child -> parents.
    dependencies depmap
    // `dependents` tracks parent -> children.
    dependents depmap
    // Keep track of the nodes of the graph themselves.
}

func New() *Graph {
    return &Graph{
        dependencies: make(depmap),
        dependents:   make(depmap),
        nodes:        make(nodeset),
    }
}

This data structure should suit our purposes, since it holds all of the information we need: nodes, "depends-on" edges, and "depended-on-by" edges. Let's think now about creating the API for adding new dependency relationships to the graph. All we need is a method for declaring that some node depends on another, like so: graph.DependOn("flour", "grain"). There are a couple of cases that we want to explicitly disallow. First, a node cannot depend on itself, and second, if flour depends on grain, then grain must not depend on flour, otherwise we would create an endless dependency cycle. With that, let's write the Graph.DependOn() method.

func (g *Graph) DependOn(child, parent string) error {
    if child == parent {
        return errors.New("self-referential dependencies not allowed")
    }

    // The Graph.DependsOn() method doesn't exist yet.
    // We'll write it next.
    if g.DependsOn(parent, child) {
        return errors.New("circular dependencies not allowed")
    }

    // Add nodes.
    g.nodes[parent] = struct{}{}
    g.nodes[child] = struct{}{}

    // Add edges.
    addNodeToNodeset(g.dependents, parent, child)
    addNodeToNodeset(g.dependencies, child, parent)

    return nil
}

func addNodeToNodeset(dm depmap, key, node string) {
    nodes, ok := dm[key]
    if !ok {
        nodes = make(nodeset)
        dm[key] = nodes
    }
    nodes[node] = struct{}{}
}

This will effectively add a dependency relationship to our graph once we implement Graph.DependsOn(). We could very easily tell whether a node depends on some other node directly, but we also want to know whether there is a transitive dependency. For example, since flour depends on grain, and grain depends on soil, flour depends on soil as well. This will require us to get the direct dependencies of a node, then for each of those dependencies, get its dependencies and so on until we stop discovering new dependencies. In computer science terms, we are computing a fixpoint to find the transitive closure of the "DependsOn" relation on our graph.

func (g *Graph) DependsOn(child, parent string) bool {
    deps := g.Dependencies(child)
    _, ok := deps[parent]
    return ok
}

func (g *Graph) Dependencies(child string) nodeset {
    if _, ok := g.nodes[root]; !ok {
        return nil
    }

    out := make(nodeset)
    searchNext := []string{root}
    for len(searchNext) > 0 {
        // List of new nodes from this layer of the dependency graph. This is
        // assigned to `searchNext` at the end of the outer "discovery" loop.
        discovered := []string{}
        for _, node := range searchNext {
            // For each node to discover, find the next nodes.
            for nextNode := range nextFn(node) {
                // If we have not seen the node before, add it to the output as well
                // as the list of nodes to traverse in the next iteration.
                if _, ok := out[nextNode]; !ok {
                    out[nextNode] = struct{}{}
                    discovered = append(discovered, nextNode)
                }
            }
        }
        searchNext = discovered
    }

    return out
}

Sorting The Graph

Now that we have a graph data structure, we can think about how to get the nodes out in a topological ordering. If we can discover the leaf nodes - that is, nodes that themselves have no dependencies on other nodes - then we can repeatedly get the leaves and remove them from the graph until the graph is empty. On the first iteration we will find the elements that are independent, then on each subsequent iteration, we will find the nodes that only depended on elements that have already been removed. The end result will be a slice of independent "layers" of nodes that are sorted topologically.

Getting the leaves of the graph is simple. We just need to find that nodes that have no entry in dependencies. This means that they do not depend on any other nodes.

func (g *Graph) Leaves() []string {
    leaves := make([]string, 0)

    for node := range g.nodes {
        if _, ok := g.dependencies[node]; !ok {
            leaves = append(leaves, node)
        }
    }

    return leaves
}

The final piece of the puzzle is actually computing the topologically sorted layers of the graph. This is also the most complex piece. The general strategy that we will follow is to iteratively collect the leaves and remove them from the graph until the graph is empty. Since we will be mutating the graph, we want to make a clone of it so that the original graph is still intact after we perform the sort, so we'll go ahead and implement that clone:

func copyNodeset(s nodeset) nodeset {
    out := make(nodeset, len(s))
    for k, v := range s {
        out[k] = v
    }
    return out
}

func copyDepmap(m depmap) depmap {
    out := make(depmap, len(m))
    for k, v := range m {
        out[k] = copyNodeset(v)
    }
    return out
}

func (g *Graph) clone() *Graph {
    return &Graph{
        dependencies: copyDepmap(g.dependencies),
        dependents:   copyDepmap(g.dependents),
        nodes:        copyNodeset(g.nodes),
    }
}

We also need to be able to remove a node and all edges from a graph. Removing the node is simple, as is removing the outbound edges from each node. However, the fact that we keep track of every edge in both directions means that we have to do a little extra work to remove the inbound records. The strategy that we will use to remove all edges is as follows:

Find node A's entry in dependents. This gives us the set of nodes that depend on A.
For each of these nodes, find the entry in dependencies. Remove A from that nodeset.
Remove node A's entry in dependents.
Perform the inverse operation, looking up node A's entry in dependencies, etc.

With the help of a small utility that allows us to remove an node from a depmap entry, we can write the method that completely removes a node from the graph.

func removeFromDepmap(dm depmap, key, node string) {
    nodes := dm[key]
    if len(nodes) == 1 {
        // The only element in the nodeset must be `node`, so we
        // can delete the entry entirely.
        delete(dm, key)
    } else {
        // Otherwise, remove the single node from the nodeset.
        delete(nodes, node)
    }
}

func (g *Graph) remove(node string) {
    // Remove edges from things that depend on `node`.
    for dependent := range g.dependents[node] {
        removeFromDepmap(g.dependencies, dependent, node)
    }
    delete(g.dependents, node)

    // Remove all edges from node to the things it depends on.
    for dependency := range g.dependencies[node] {
        removeFromDepmap(g.dependents, dependency, node)
    }
    delete(g.dependencies, node)

    // Finally, remove the node itself.
    delete(g.nodes, node)
}

Finally, we can implement Graph.TopoSortedLayers():

func (g *Graph) TopoSortedLayers() [][]string {
    layers := [][]string{}

    // Copy the graph
    shrinkingGraph := g.clone()
    for {
        leaves := shrinkingGraph.Leaves()
        if len(leaves) == 0 {
            break
        }

        layers = append(layers, leaves)
        for _, leafNode := range leaves {
            shrinkingGraph.remove(leafNode)
        }
    }

    return layers
}

This method clearly outlines our strategy for topologically sorting the graph:

Clone the graph so that we can mutate it.
Repeatedly collect the leaves of the graph into a "layer" of output.
Remove each layer once it is collected.
When the graph is empty, return the collected layers.

Now we can go back to our original cake-making problem to make sure that our graph solves it for us:

package main

import (
    "fmt"
    "strings"

    "github.com/kendru/darwin/go/depgraph"
)

func main() {
    g := depgraph.New()
    g.DependOn("cake", "eggs")
    g.DependOn("cake", "flour")
    g.DependOn("eggs", "chickens")
    g.DependOn("flour", "grain")
    g.DependOn("chickens", "grain")
    g.DependOn("grain", "soil")
    g.DependOn("grain", "water")
    g.DependOn("chickens", "water")

    for i, layer := range g.TopoSortedLayers() {
        fmt.Printf("%d: %s\n", i, strings.Join(layer, ", "))
    }
    // Output:
    // 0: soil, water
    // 1: grain
    // 2: flour, chickens
    // 3: eggs
    // 4: cake
}

All of that work was not exactly a piece of cake, but now we have a dependency graph that can be used to topologically sort just about anything. You can find the full code for this post on GitHub. There are some notable limitations to this implementation, and I would like to challenge you to improve it so that it can:

Store nodes that are not simple strings
Allow nodes and edges/dependency information to be added separately
Produce string output for debugging

I hope you enjoyed this tutorial! Please let me know what you would like me to cover next.

So, I wrote a book

Andrew Meredith — Sat, 20 Feb 2021 20:24:09 +0000

I just finished writing my very first book! It is an introduction to the ClojureScript programming language that is primarily aimed at curious JavaScript developers. This has been several years in the making, and I am very proud to have finally produced a finished book!

The entire book is free online, and if you would like to contribute your thanks to the effort I have put in, you can pre-order a physical copy at https://gumroad.com/l/learn-cljs. I would love to know what you think about the book!

Concurrency with core.async

Andrew Meredith — Sat, 08 Feb 2020 23:02:25 +0000

This article is Lesson 25 of Andrew's book, Learn ClojureScript

Asynchronous programming loves at the heart of web development. Almost every app needs to communicate with an API backend, respond to user input, or perform some other IO task without blocking the main thread. While it is entirely possible to use JavaScript's Promise API from ClojureScript, we have another paradigm for asynchronous programming at our disposal - the core.async library. This library implements the same concurrency model as the Go programming language, and it allows us to write code as sequential processes that may need to communicate with each other.

In this lesson:

Learn about CSP, the concurrency model behind ClojureScript (and Go)
Think of concurrent problems in terms of processes
Use channels to communicate between processes

Overview of CSP

ClojureScript's library for concurrency is based on a mathematical process calculus (concurrency model) called Communicating Sequential Processes, which was described by Tony Hoare in 1978. The basic idea behind CSP is that there are a number of independent processes that each execute some ordered sequence of steps. These processes can communicate with each other by sending or receiving messages over channels. When a process wants to read a message from a channel, it blocks until a message is available, then it consumes the message and moves on. A process can also place a message on a channel either synchronously or asynchronously. By using communication over channels, multiple processes can synchronize such that one process waits for a specific input from another before proceeding.

In ClojureScript, the core.async library provides the functionality that we need to create these asynchronous workflows in thr form of the go macro, which creates a new lightweight process, chan, which creates a channel, and the operators, <! (take) and >! (put), and alts! (take from one of many channels). Using only these primitives, we can create very sophisticated asynchronous communication patterns. Before diving in with core.async, let's take a quick step back to talk about CSP.

In CSP, the fundamental object is the process. A process is simply an anonymous (unnamed) piece of code that can execute a number of steps in order, potentially with its own control flow. Code in a process always runs synchronously - that is, the process will not proceed on to the next step until the previous step completes. Each process is independent of every other process, and they all run concurrently (ClojureScript is is charge of scheduling what process should run when). Finally, even though communication is a cornerstone of CSP, processes do not necessarily have to communicate with any other processes.

Concurrent Processes

Moving on from processes, the next key object in CSP is the channel¹. A channel is simply a conduit that can carry values from one process to another. By default, each channel can only convey a single value at once. That is, once a process sends a value on a channel, the next process that tries to send on that channel will be parked until another process takes the value out of that channel. Additionally, trying to take a value from an empty channel with park the receiver until a value is put in. Channels can also be created with buffers that can hold up to some specified number of values that have not been taken out of the channel. Additionally, these buffers can either park producers when they fill up (which is the default behaviour), or they can silently discard any new values (via dropping-buffer) or push out the oldest value in the buffer (via sliding-buffer).

Note:

We mention that processes can park when trying to read from an empty channel or write to a full channel. From the perspective of the process, it is blocked and cannot make any progress until the state of the channel changes. However, from the perspective of the ClojureScript runtime, other processes can continue running, and the parked process can eventually be resumed if the state of the channel changes. We avoid using the language of blocking, since ClojureScript runs in the single-threaded context of JavaScript, and parking a process does not block that thread.

Synchronization with Channels

With this understanding of how processes and channels work, we are ready to dig in to an example. Let's say that we are building an SQL query editor, and whenever the user is focused in the query input and presses Ctrl + Enter, we send off the query to a server and wait for a response. We will have one process that watches keystrokes, and another process that coordinates user input and performing a server request when necessary.

Go Blocks as Lightweight Processes

In ClojureScript, we create a process using the go macro containing the block of code to execute. A simple go block could look like the following:

(go (println "Hello Processes!"))

This will asynchronously print, Hello Processes! to the console, similar to the following JavaScript code:

setTimeout(() => console.log('Hello Processes!'), 0);

We can create as many of these go blocks as we would like, and they will all run independently of each other. The interesting part comes when we introduce channels into a go block. In the next example, we read values from one channel and forward the ones that satisfy a predicate onto another channel. This is essentially a channel filter operation:

(go (loop []
  (let [val (<! in-ch)]                                    ;; <1>
    (when (pred? val)                                      ;; <2>
      (>! out-ch val)))                                    ;; <3>
  (recur)))

A Filtering Process

Read a value from in-ch
Test the value with pred?
Write the value to out-ch

This example illustrates a common paradigm in core.async: we create go blocks that infinitely loop, performing the same task over and over. Just like JavaScript has its global event loop that will execute our code and callbacks, we can create mini event loops that are concerned only with a very small piece of functionality. In fact, this infinitely-looping process pattern is so common that core.async provides a go-loop macro that combines a go block with a loop. Using this macro, our code becomes:

(go-loop []
  (let [val (<! in-ch)]
    (when (pred? val)
      (>! out-ch val)))
  (recur))

To illustrate how each process runs independently, we can make use of the timeout function provided by the core.async library. This function returns a channel that closes after a specified timeout (in milliseconds). Let's create 2 processes that each log to the console on a given interval:

(go-loop []
  (<! (timeout 100))
  (println "Hello from process 1")
  (recur))

(go-loop []
  (<! (timeout 250))
  (println "Hello from process 2")
  (recur))

Quick Review

True or false? Each go block always runs to completion before any other go blocks are run.
What is the more concise way of writing (go (loop [] ... (recur)))?

Communicating Over Channels

Returning to the example of the SQL query editor, we could spawn a process to listen to keyboard input and emit events for all "key chords" consisting of one or more modifier keys (Ctrl, Alt, Shift, etc.) plus another key. For this, we will need to listen for keydown and keyup events and place them on channels. When we detect a chord, we place the results on another channel:

(def keydown-ch (chan))                                    ;; <1>
(gevent/listen js/document "keydown"
  #(put! keydown-ch (.-key %)))

(def keyup-ch (chan))                                      ;; <2>
(gevent/listen js/document "keyup"
  #(put! keyup-ch (.-key %)))

(def is-modifier? #{"Control" "Meta" "Alt" "Shift"})

(def chord-ch (chan))
(go-loop [modifiers []                                     ;; <3>
          pressed nil]
  (when (and (seq modifiers) pressed)                      ;; <4>
    (>! chord-ch (conj modifiers pressed)))
  (let [[key ch] (alts! [keydown-ch keyup-ch])]            ;; <5>
    (condp = ch
      keydown-ch (if (is-modifier? key)                    ;; <6>
                   (recur (conj modifiers key) pressed)
                   (recur modifiers key))
      keyup-ch (if (is-modifier? key)
                 (recur (filterv #(not= % key) modifiers)
                        pressed)
                 (recur modifiers nil)))))

Detecting Key Chords

Put the key for all keydown events on a channel
Put the key for all keyup events on a channel
Keep track of any modifiers keys held down as well as the last other key pressed
If we have any modifiers as well as a pressed key, send the chord on the chord-ch channel
Wait for values of either keydown-ch or keyup-ch channels
Add the key that was pressed or remove the key that was released and recur

Sending Values Asynchronously

In addition to using a go-loop that maintains state on each recursive pass, we encounter a couple of new pieces of core.async here. The first is the put! function. This function puts a value on a channel asynchronously. The normal put and take operators (>! and <! respectively) are only designed to be run inside a go block. One option that we have is to spin up a new go block every time that we want to put a value onto a channel. For instance, the keydown listener could have been written as follows:

(gevent/listen js/document "keydown"
  #(go (>! keydown-ch (.-key %))))

However, this incurs some additional overhead that we may not want every time that a key is pressed. The put! function is a much cheaper option in this case. Remember that using >! will park the process, but when we are not in a go block, we do not want to wait for a channel to be ready before proceeding. We want to be able to send a value off asynchronously, knowing that it will be received by the channel when it is ready. This is exactly what put! does. Like >!, it takes a channel and a value to put on that channel. Additionally, we can supply a callback as a third argument, which will be called when the value is delivered to the channel.

Alternating Between Channels

As we mentioned in out introduction to CSP above, there is one additional function that we often use to consume values from more than one channel: alts!. Like >! and <!, alts! can only be called from within a go block. It takes a vector of channels to "listen to" and parks until it receives a value from any of them. Upon receiving a value, it evaluates to a vector whose first element is the value received and whose second element is the channel from which the value came. By checking the channel that we get from alts!, we can determine where the value came from and decide what to do with it.

One common use case is to implement a timeout by alternating between a channel that we expect to eventually deliver a value and a timeout channel that will close after a certain amount of time:

(go
  (let [[val ch] (alts! long-task-ch (timeout 5000))]
    (if (= ch long-task-ch)
      (println "Task completed!" val)
      (println "Oh oh! Task timed out!"))))

Adding Communication

So far, we have only a single process that is reading from and writing to channels. Let's change that with another process that submits the query to a mocked server and updates the results area:

(defn mock-request [query]                                 ;; <1>
  (let [ch (chan)]
    (js/setTimeout
      #(put! ch (str "Results for: " in))
      (* 2000 (js/Math.random)))                           ;; <2>
    ch))

(go-loop []
  (let [chord (<! chord-ch)]                               ;; <3>
    (when (and (= chord ["Control" "r"])
               (= js/document.activeElement query-input))
      (aset results-display "innerText" "Loading...")
      (aset results-display "innerText"
        (<! (mock-request (.-value query-input)))))        ;; <4>
    (recur)))

Making a Mock Request

Simulate making a request to a server, returning a channel that will eventually get results
Wait for a random interval between 0 and 2 seconds to simulate latency
Wait for key chords
Perform a request and wait for the results before updating results-display

Here we spin up another process that repeatedly takes key chords from the chord-ch channel, and checks to see if we have the correct chord and whether the query input is focused. If both of these conditions are met, then we simulate making a server request, and when the results come back, we update the results area. One thing to note is that (aset results-display "innerText" (<! (mock-request (.-value query-input)))) will halt its evaluation until we can take a value from the channel returned by (mock-request (.-value query-input)). Internally, the go macro rewrites our code into a state machine, but all that we need to know is that whenever we need to park a process until a value is ready, any code that depends on that value will be deferred until after the value is delivered.

Quick Review

Describe the difference between >! and put!
How could we change the previous go block to time out a request after 1500 milliseconds?

Channels as Values

As we saw in the mock-request function, we can create channels anywhere in our code. We can pass them as arguments to functions or return them from functions. Although the serve the special purpose of facilitating communication between processes, they are just regular ClojureScript values.

It is a common idiom to return a channel from a function that produces some result asynchronously. Whereas in JavaScript, we would usually return a Promise (or write the function as async), we often return a channel when we intend for a function to be called from within a go block. Author and Clojure instructor Eric Normand suggests naming functions that return channels with a < prefix.² Following this convention, our mock-request function would become <mock-request. This makes it easy to visually distinguish functions that return channels from other functions. Remember, however, that a function that returns a channel is less general than one that accepts a callback because when we return a channel, we dictate that any value eventually produced by that function must be consumed in a go block. For this reason, we should usually prefer writing functions that take callbacks if we do not know how we will eventually want to call them.

In addition to channels that simply create a channel that they will eventually put a value into, we can create some interesting higher-order channel functions. For instance, we can create a channel that merges values from any number of other channels:

(defn merge-ch [& channels]
  (let [out (chan)]
    (go-loop []
      (>! out (first (alts! channels)))
      (recur))
    out))

At the beginning of the lesson, we already saw another useful higher-order channel function that filters the values in a channel to only ones that satisfy a predicate. Let's look at one more example - a function that synchronizes 2 channels such that it waits for one value in each channel then produces a pair of [chan-1-val chan-2-val]:

(defn synchronize-ch [chan-1 chan-2]
  (let [out (chan)]
    (go-loop []
      (>! out [(<! chan-1) (<! chan-2)])
      (recur))
    out))

With very simple functions, we can easily create very sophisticated asynchronous systems. Most importantly, because channels are values, we can create functions that manipulate channels in such a way that we abstract the communication patterns out of our business logic.

Summary

In this lesson, we learned about the Communicating Sequential Processes concurrency model and core.async, its ClojureScript implementation. After learning about the core concepts of CSP, we explored how go blocks create lightweight processes that run concurrently and can park when waiting to write to or read from a channel that is not ready. We talked about how channels help us communicate between processes and synchronize their state. Finally, we saw that the fact that channels are plain ClojureScript values makes it possible for us to manipulate the communication structure of our processes separately from our business logic. While core.async has the same expressive power as Promises and async/await in JavaScript, it presents a very useful way of thinking about concurrency that lends itself to SPAs, which often have multiple sequential interactions that all need to be handled concurrently.

The original formulation of CSP did not have channels. Instead, each processed had a unique name and passed messages to other processes directly. All notable implementations of CSP today use named channels to communicate between anonymous processes. ↩
Eric Normand's style guide for core.async can be found at https://purelyfunctional.tv/mini-guide/core-async-code-style/ ↩

Handling Exceptions and Errors

Andrew Meredith — Sun, 02 Feb 2020 00:55:37 +0000

This article is Lesson 24 of Andrew's book, Learn ClojureScript

We would like to think of programming as an idealized discipline where we write our programs declaratively and the computer converts them into pure computational models that can churn away until the heat death of the universe. In practice, bad things will happen, and they will happen much sooner than the heat death of the universe. Networks fail, we divide by zero, we miss a corner case that generates input in the "wrong" shape. All of these are exceptional conditions that we need to be aware of and somehow handle when they occur. Welcome to the domain of exceptions!

In this lesson:

Handle exceptional conditions with try/catch
Add metadata to errors
Handle exceptions as values
Use conditions for more flexible error handling

Handling Exceptions with try/catch

Handling exceptions in ClojureScript is more similar to than it is different from handling them in JavaScript. ClojureScript provides a special form called try that compiles down to a try/catch construct in JavaScript. The basic usage should look familiar to any JavaScript programmer:

(try
  (do-stuff 42)
  (call-api {:id 17})                                      ;; <1>
  true                                                     ;; <2>
  (catch js/Error e                                        ;; <3>
    (println "An error occurred:" e)
    false)
  (finally
    (do-cleanup)))                                         ;; <4>

Multiple expressions can occur inside the body of try
try is an expression and returns a value
catch is always used with the class of the value that should be caught
If a finally clause is present, it is called for side effects

While try/catch behaves almost exactly like its JavaScript counterpart, there are a few differences that are worth noting:

First, try is an expression rather than a statement. When all of the expressions inside the body of the try succeed, then the try itself evaluates to the value of its body's final expression. When any expression throws an exception that is caught, the try evaluates to the value of the final expression in the catch block. Just like in JavaScript, a finally clause may optionally be specified. If it is, it will be run purely for side effects after the body and potentially the catch clause are evaluated. Only the value of the body of the try (on success) or the body of the catch (on failure) will be returned - the value of finally is discarded.

Second, the constructor of the exception to catch must be specified, and if the value thrown is not of that constructor's type ,the catch clause is not evaluated, causing the exception to be re-thrown. This requirement arises from ClojureScript's roots in Clojure, which runs on the JVM and whose try/catch form mimics that of Java. In order to maintain more syntactic consistency with Clojure, ClojureScript follows the same syntax. In practice, js/Error is almost always used as the value of the error type. However, since JavaScript lets us throw any value - not just an Error - there are times when we need to catch some other value. In lieu of the constructor of the error, we may use the keyword :default to catch a value of any type - including ClojureScript values:

dev:cljs.user=> (try
           #_=>   (throw {:type :custom-error
           #_=>           :message "Something unpleasant occurred"})
           #_=>   (catch :default e
           #_=>     (println "Caught value:" e)))
Caught value: {:type :custom-error, :message Something unpleasant occurred}
nil

Quick Review

What is the value of each of the following expressions?

;; 1
(try
  :success
  (catch :default _
    :failure))

;; 2
(* 2
  (try
    (throw (js/Error.))
    4
    (catch :default _
      6)
    (finally 8)))

;; 3
(try
  (try
    (throw "a string")
    (catch js/Error e
      "Inner")
  (catch :default e
    "Outer")))

Conveying Information

Sometimes it is desirable to convey extra information when we throw an error. For instance, if we are loading a string from localStorage, parsing it, then using it to construct a domain object, there are at 3 steps that can fail, and we probably want to handle these failures differently - perhaps to determine what type of message to display to the user or whether to log the error to a service for later inspection. In this case, we can use ex-info to create a ExceptionInfo object, which is a subclass of the JavaScript Error type that ClojureScript defines. This function allows us to attach a message, a map of arbitrary metadata, and an optional field describing the cause of the exception. For example:

(ex-info "A parse error occurred"                          ;; <1>
         {:line 17 :char 8 :last-token "for"}              ;; <2>
         :unexpected-end-of-string)                        ;; <3>

Creating an ExceptionInfo error

Error message
Metadata
Cause (optional)

When we catch an ExceptionInfo, we can use the ex-message, ex-data and ex-cause functions to extract the extra information back out. Going back to the example of parsing a string in localStorage and hydrating a domain model from it, we can detect the type of error and handle it differently depending on where the error occurred.

(def required-attrs [:id :email])
(def allowed-attrs [:id :email :first-name :last-name])

(defn make-user [user-data]
  (cond
    (not (every? #(contains? user-data %) required-attrs))
    (throw (ex-info "Missing required attributes"
                    {:required required-attrs
                     :found (keys user-data)}
                    :validation-failed))
    (not (every? #(some (set allowed-attrs) %) (keys user-data)))
    (throw (ex-info "Found disallowed attributes"
                    {:allowed allowed-attrs
                     :found (keys user-data)}
                    :validation-failed))
    :else (assoc user-data :type :user)))

(defn hydrate-user []
  (let [serialized-user (try
                          (.getItem js/localStorage "current-user")
                          (catch js/Error _
                            (throw (ex-info "Could not load data from localStorage"
                                            {}
                                            :local-storage-unsupported))))
        user-data (try
                    (.parse js/JSON serialized-user)
                    (catch js/Error _
                      (throw (ex-info "Could not parse user data"
                                      {:string serialized-user}
                                      :parse-failed))))]
    (-> user-data
        (js->clj :keywordize-keys true)
        make-user)))

(try
  (hydrate-user)
  (catch ExceptionInfo e
    (case (ex-cause e)
      :local-storage-unsupported
      (display-error (str "Local storage not supported: "
                                                     (ex-message e)))
      :parse-failed
      (do (display-error "Could not load user data from browser")
          (log-error {:type :user-parse-failed
                      :source (:string (ex-data e))}))
      :validation-failed
      (do (display-error "There was an error in your submission. Please correct it before continuing.")
          (update-field-errors (ex-data e)))
      ;; Re-throw an unknown error
      (throw e))))

Using this pattern, we can provide more information along with an exception that can be used by the handling code in order to dispatch different business logic. In JavaScript, we can achieve a similar result by subclassing Error and including instanceof checks in our error handling logic. ClojureScript is simply a bit more opinionated, and it provides the tools to do this out of the box.

Functional Alternatives to Exceptions

While handling errors with try/catch should be familiar to every JavaScript developer, it runs against the grain of pure functional programming, of which we have already seen the benefits. When exceptions can be thrown, a function is no longer a mapping of input to output. It additionally becomes a mechanism for signaling control flow: when a function throws an exception, it does not return a value to its caller, and the code that ultimately receives the value thrown is not necessarily the immediate caller. We can always use the pattern that we learned in Lesson 21 to segregate our exception-handling code from our core business logic. This is usually possible, and it is often the simplest option. However, there are more functional ways to handle exceptional conditions, and we will look at two of those: treating errors as values, and using condition systems.

Control Flow for try/catch

Errors as Values

The simplest and most functionally pure option for writing code to deal with exceptional conditions is to simply return any error that may occur. In this case, a function that may fail will return a wrapper value that can contain either a success value or an error. There are many ways to represent a wrapper value, but a simple option is as a 2-element vector that either has :ok in the first position and a normal value in the second position or has :error in the first position and an error value in the second position. It is trivial to create a few functions that define this "error" type and work with its values:

(ns errors.err
    (:refer-clojure :exclude [map]))

(defn ok [val]
  [:ok val])

(defn error [val]
  [:error val])

(defn is-ok? [err]
  (= :ok (first err)))

(def is-error? (complement is-ok?))

(def unwrap second)

(defn unwrap-or [err on-error]
  (if (is-ok? err)
    (unwrap err)
    (on-error (unwrap err))))

(defn map [f err]
  (if (is-ok? err)
    (-> err unwrap f ok)
    err))

(defn flat-map [f err]
  (if (is-ok? err)
    (-> err unwrap f)
    err))

Defining an error type

With these few functions, we can construct and transform values that have this "error type" wrapper.¹

(defn div [x y]                                            ;; <1>
  (if (zero? y)
    (error "Cannot divide by zero")
    (ok (/ x y))))

(unwrap (div 27 9))                                        ;; <2>

(unwrap (div 27 0))                                        ;; <3>

(map #(+ % 12)
     (div 27 9))                                           ;; <4>

(unwrap
  (flat-map                                                ;; <5>
    #(div % 2)
    (div 27 9)))

Define a division function that can fail if asked to divide by 0
A success value, [:ok 3]
An error value, [:error "Cannot divide by zero"]
map will transform a success value inside an error type
flat-map will take a success value inside an error type and return the result of passing this to another error-type producing function.

When we write code in this style, we can end up with functions that all include the same boilerplate. For example, every function that handles an error returned by another function that it calls will look like the following:

(defn get-results-and-handle-error []
  (unwrap-or
    (get-results)                                          ;; <1>
    (fn [err]                                              ;; <2>
      (display-err err)
      [])))

Handling an Error

get-results may fail
The callback will be called when get-results fails, and its value will be returned from get-results-and-handle-error

On the other hand, when we have a function that should propagate an error when one of the functions that it calls fails, our function will look like this:

(defn transform-results-and-propagate-error []
  (map
    #(transform-results %)                                 ;; <1>
    (get-results)))

Propagating an Error

transform-results will be called with results only when get-results succeeds

The downside to this approach is that we need always be mindful of which functions may fail and deal with their results as wrapped results. In the end, we have more boilerplate for handling errors, but our control flow is explicit, and our functions remain free of side effects. Sometimes the trade-off will be worthwhile, but other times we are better served by a judicious use of exception handling code at the boundaries of our application. There are several excellent libraries that help minimize the boilerplate associated with this error-as-value approach, such as Adam Bard's failjure.

Control Flow for Errors as Values

Conditions and Restarts

Given the long history of the Lisp family of programming languages, we should briefly mention the concept of conditions, which were popularized in Common Lisp.² The basic idea behind a condition system is that there are pieces of code that can have different outcomes that are beyond our control. For example, we cannot parse malformed input, a browser may have disabled a certain feature that we wish to use, etc. Additionally, the code that encounters these special conditions is not necessarily the code that we want deciding what to do as a result. However, control should not be passed arbitrarily far up the call stack when some condition is encountered. A popular library for working with conditions in ClojureScript is special.

Condition systems let us signal a condition, which may be handled by a function that was registered to handle that specific type of condition. Finally, this handler can optionally invoke a restart, which returns control to the location where the condition was signaled. Additionally, the lower-level function that signaled the condition may offer multiple restarts, and it is up to the handler to choose which restart to invoke.

In order to see how this works, let's return to the example of parsing a string in localStorage to hydrate a domain model. We end up with low-level code that we call from higher-level code. That low-level code that gets data from localStorage should not be in charge of deciding what is the appropriate course of action - otherwise, it gets coupled to the higher-level logic of the program and is no longer reusable. Using the special library, our code might look like this:

(defn get-localstorage [key]
  (try
    (.getItem js/localStorage key)
    (catch js/Error _
      (condition :localstorage-unsupported nil))))         ;; <1>

(defn get-parsed-data [key]
  (let [serialized (get-localstorage key)]
    (try
      (if-let [parsed (js->clj
                        (.parse js/JSON serialized)
                        :keywordize-keys true)]
        parsed
        (condition :no-data key
          :normally {}))                                   ;; <2>
      (catch js/Error _
        (condition :parse-error {:key key :string serialized}
          :normally {}
          :reparse #(get-parsed-data %))))))               ;; <3>

(defn handle-parse-error [{:keys [key]}]
  (if (= key "current-user")
    (condition :reparse "currUser")                        ;; <4>
    (do (display-error "Cannot parse")
        (initialize-user))))

(defn hydrate-user []
  (let [managed-fn (manage get-parsed-data                 ;; <5>
                     :localstorage-unsupported (fn [_]     ;; <6>
                                                 (display-error "Unsupported")
                                                 "{}")
                     :parse-error handle-parse-error)
                     :no-data (fn [_]
                                (initialize-user)))]
    (managed-fn "current-user")))

Handling conditions

Signal the :localstorage-unsupported condition with nil
Provide a default value if the condition is not managed
Provide a "restart" that allows us to proceed with specific behaviour
Trigger the :reparse restart
Create a managed version of the get-parsed-data function with handlers for each condition type
Declare a handler as the condition keyword followed by a function of the condition value to the desired value

This example is fairly dense, so let's unpack it a bit. First, there are two functions that we use as part of the condition system: condition and manage. condition signals a condition of a specific type along with a value. For example, when there is an error parsing user data, we signal the condition of type :parse-error with the value, {:key key :string serialized}. manage creates a version of the function that provides handlers for each condition that may be signaled, whether in the function that was called directly or in any functions that it eventually calls arbitrarily deep in the call stack. The handlers are given the value of the signaled condition and may either return a value that is used at the location where the condition was signaled, or they can signal a new condition.

This second option is how "restarts" are accomplished: when we signal a condition, we can provide restart handlers as well. The :normally handler is called automatically if no handler is provided higher in the call stack. Any other restart can be selected by raising the corresponding restart condition in uor handler function. In the example above, we provide a :reparse restart that attempts to parse data from another key from localStorage. We signal this restart in the handle-parse-error function: (condition :reparse "currUser"). This ability to provide code in the lower-level functions that can be dispatched based on higher-level logic is what makes conditions so powerful.

Control Flow for Conditions

Quick Review

What is the result of the following code?

(let [f (fn [s]
          (if (= 0 (mod (count s) 2))
            (condition :even-length s
              :normally "EVEN"
              :shout (.toUpperCase s))
            (str "You said: " s)))
      managed (manage f
                :even-length (fn [s]
                               (if (= "loud" s)
                                 (condition :shout)
                                 (str s "!"))))]
  [(managed "test")
   (managed "foo")
   (managed "loud")])

As we can see, conditions let us decouple the code that specifies the recovery strategy from the code that decides what recovery strategy to invoke. Like exceptions, conditions are not purely functional because they introduce control flow outside of the normal return value of a function. However, since a restart returns control to the function where the error occurred, they are closer to the functional end of the spectrum than are exceptions.

Summary

In this lesson, we learned how to use ClojureScript's version of try/catch to deal with exceptions that occur in our code or code that we call. We saw how to use ex-info to create error values that convey extra information that we can use in our error handling code. Finally, we looked at two ways to approach errors in a more functional programming-friendly way. We "wrapped" the return values of functions that could fail in a special error type, and we also used conditions to allow higher-level code to specify the strategy for handling an exceptional case in lower-level code.

Yes, this is just the Error monad. ↩
As much as anything can be said to have been popularized by Common Lisp ↩

Namespaces and Program Structure

Andrew Meredith — Thu, 09 Jan 2020 13:47:28 +0000

This article is Lesson 23 of Andrew's book, Learn ClojureScript

In ClojureScript, the unit of modularity is the namespace. A namespace is simply a logical grouping of functions and data that can be required and used by another namespace. External libraries expose modules as namespaces that we can require in our code, and we can (and should) break our code into multiple modules as well. In this lesson, we'll learn how to declare and require modules as well as how to use them to create clean architectural boundaries in our programs.

In this lesson:

Organize functions and data into namespaces
Architect a logical and intuitive project structure
Work with namespaces at the REPL

Namespace Declarations

In each of the ClojureScript files that we have written so far, Leiningen has kindly generated a namespace declaration at the top of the file for us. We have used the (:require ...) form to pull in third-party libraries, but we have judiciously avoided diving in to what exactly is happening here. That is about to change.

For the purpose of exploring namespaces in this lesson, let's create a new Figwheel project:

$ lein new figwheel import-fns
$ cd import-fns

If we look at the file that was generated in src/import_fns/core.cljs, we see the following declaration at the top of the file:

(ns import-fns.core
    (:require ))

The ClojureScript compiler will turn this declaration into a Google Closure Compiler module declaration, which will ultimately be compiled into a JavaScript object containing all of the functions and vars that we define in this namespace. From this bare-bones declaration, we can note a couple of things:

We declare the name of the namespace as import-fns.core. This name must follow the naming convention discussed in Lesson 5.
There is a skeleton (:require ...) form that we can use to require code from other namespaces into this one.

Using Require

Since declaring a name is fairly self-explanatory, we will focus on the second item - requiring code from other namespaces. The :require form in the ns declaration is by far the most common way to require code from another file - including third party libraries - to make it available in our code¹. Let's look at the various forms of :require, using the project that we created above. We will again build on the example of a text-based adventure game, specifically displaying a player's inventory:

(ns import-fns.core
    (:require import-fns.ui                                ;; <1>
              [import-fns.format :refer [pluralize]]       ;; <2>
              [import-fns.inventory :as inventory]         ;; <3>
              [goog.dom :refer [getElement]                ;; <4>
                        :rename {getElement get-element}]))

(defn item-description [i item]
  (let [qty (inventory/item-qty i item)
        label (if (> qty 1) (pluralize item) item)]
      (str qty " " label)))

(let [i (-> (inventory/make-inventory)
            (inventory/add-items "Laser Catapult" 1)
            (inventory/add-items "Antimatter Scrubber" 5))]
  (import-fns.ui/render-list (get-element "app")
    (map (partial item-description i)
         (inventory/list-items i))))

Require the entire import-fns.ui namespace
Require a single function from the import-fns.format namespace
Require the import-fns.inventory namespace with the alias inventory
Require a single function from the Google Closure Library namespace, renaming that function

This snippet illustrates all of the common ways of requiring code into a namespace. After the :require keyword, we can include any number of libspecs. The libspec may be either a fully-qualified namespace or a vector containing the namespace and some optional modifiers. This syntax is interpreted by the ns special form to pull in vars from other namespaces according to the exact form of the libspec.

Form 1: Simple Namespace

In the first form, we simply use the name of a namespace that we would like to require: import-fns.ui. We can then refer to any public var within this namespace by using the full namespace followed by / and the name of the var, e.g. import-fns.ui/render-list.

Form 2: Refer

In the second case, we follow the namespace by :refer [pluralize]. :refer will make every var that is listed in the vector that follows available without any prefix. Thus, we can write (pluralize item) instead of (import-fns.format/pluralize item). Tny function in the import-fns.format namespace can still be called using the fully-qualified syntax.

Form 3: Aliased Namespace

In the third case, we alias the namespace that we require using :as. This works similar to the first case in that we are able to refer to any public var in the namespace. However, we can use the symbol that we specified after :as instead of the full namespace. Thus, import-fns.inventory/make-inventory becomes inventory/make-inventory. The use of :as can be very useful when requiring from namespaces with very verbose names. In general, we should also prefer :as to :refer, since it makes it clear where a var that we reference comes from without needing to look back at the namespace declaration.

Form 4: Aliased Vars

Finally, in the fourth case, we rename a specific var that we referred. Here, we alias the getElement function from the Google Closure Library's goog.dom module in order to give it a more idiomatic kebab-case name. In practice, renaming is used more often to prevent a name collision. For example, we may want to import functions called mark-inactive from both my-app.users and my-app.customers namespaces. We can use :rename to import them as mark-user-inactive and mark-customer-inactive.

Require Form	Description	Function Usage
`my.namespace`	require entire namespace	`my.namespace/compute`
`my.namespace :refer [compute other-fn]`	require specific functions and refer to them unqualified	`compute`
`my.namespace :as mine`	require namespace with alias	`mine/compute`
`my.namespace :refer [compute] :rename {compute calculate}`	require namespace with specific vars aliased	`calculate`

Forms of Require

Note:

There is a form, :refer-clojure that allows us to rename a var from the ClojureScript standard library (or exclude it altogether) in the event that we want to define a var in our namespace with the same name. (:refer-clojure :rename {str string}) will alias the str function as string, so that we are free to reuse the str name without generating any warnings. Similarly, we can exclude the str function altogether with, (:refer-clojure :exclude [str]). That said, we should usually be giving things names that are descriptive enough that they do not conflict with anything in the standard library.

Importing Google Closure Library Classes

In the case that we are importing a class constructor from the Google Closure Library, there is one form that we should be aware of in addition to :require - that is :import. This form exists only for bringing a class constructor from a Google Closure module into our namespace in the same way that the :refer keyword in a libspec lets us bring in a var from another namespace. Like :require, :import expects to be followed by a number of import specs. The most common form is a vector with the namespace from which to import classes followed by the unqualified constructor names to import from that namespace:

(ns my-ns
  (:import [goog.math Coordinate Rect]))

(.contains (Rect. 10 50 5 5)
           (Coordinate. 12 50))

The thing to remember about :import is that it is only used for requiring classes (including enums) from Google Closure modules - never for including a ClojureScript namespace or a (non-constructor) function from a Google Closure module.

Requiring Macros

One of ClojureScript's most powerful features is its macro support. However, the way in which macros are implemented presents a bit of a quirk in the way that we have to require them. A macro has access to the ClojureScript code that is "inside" its call site as raw data (lists, vectors, symbols, etc.), and it can manipulate this code as data structure before returning the code that actually gets compiled to JavaScript. The tricky thing is that since the ClojureScript compiler is written in Clojure, the macros themselves are Clojure code that generates ClojureScript code. We will learn more about macros in a later lesson, but for now the important thing to remember is that since macros are Clojure code, we need to go about requiring them another way. That way is the :require-macros form:

(:require-macros [macro-ns :as macros])

This form functions very much like :require with the exception that it works with importing macros from Clojure namespaces rather than vars from ClojureScript namespaces.

Macro Compilation

Grouping Related Functions

Now that we have discussed the mechanics of declaring a namespace and requiring other code that we would like to use, let's take a step back and look at how to organize our namespaces. One of the features of ClojureScript that can be both liberating and frustrating is that it is not opinionated about how namespaces are structured. We can put as little or as much into a namespace as we would like, and as long as we do not create circular dependencies, all is well. Some of the insights from object-oriented programming can also help us here: high cohesion and low coupling.

High cohesion means that if two pieces of code are closely related in purpose, they should be close in the architecture of the program - usually in the same namespace. Low coupling is the other side of the same coin: if two pieces of code are unrelated, they should be distant from each other in the architecture of the program. Unlike OOP practitioners, we extend this principle to data as well. Since hiding data is not a goal of functional programming, we tend to group functions and the data that they operate on into the same namespace and expose both freely.

Let's return to the adventure game inventory code by way of example. In the import-fns.core namespace, we require the import-fns.inventory and import-fns.ui namespaces as well as pluralize from import-fns.format. These files contain the business and presentation logic for this little app. First, we will look at import-fns.inventory:

(ns import-fns.inventory)                                  ;; <1>

(defn make-inventory []                                    ;; <2>
  {:items {}})

(defn- add-quantity [inventory-item qty]                   ;; <3>
  (update-in inventory-item [:qty]
    (fn [current-qty] (+ current-qty qty))))

(defn add-items
 ([inventory item] (add-items inventory item 1))
 ([inventory item qty]
  (update-in inventory [:items item]
    (fnil add-quantity
      {:item item :qty 0}
      qty))))

(defn list-items [inventory]
  (keys (:items inventory)))

(defn item-qty [inventory item]
  (get-in inventory [:items item :qty] 0))

src/import_fns/inventory.cljs

All functions in this file will be part of the import-fns.inventory namespace
Declare a public function named make-inventory
Use defn- to declare a private function

As the name suggests, this namespace contains all of the code relating to creating and managing an inventory. It has no formatting or display logic. We expose make-inventory for constructing a new inventory, add-items for adding some quantity of a specific item, list-items for getting the distinct items in the inventory, and item-qty for getting the quantity of a particular item. We make the add-quantity function private by declaring it with defn- because it exists solely as an implementation detail of add-items and serves little value outside this context. Next, let's look at the import-fns.format namespace:

(ns import-fns.format
  (:require [clojure.string :as s]))

(defn ends-with-any? [word suffixes]
  (some (fn [suffix]
          (s/ends-with? word suffix)
         suffixes)))

(defn replace-suffix [word old-suffix new-suffix]
  (let [prefix-len (- (count word)
                      (count old-suffix))
        prefix (.substring word 0 prefix-len)]
    (str prefix new-suffix)))

(defn pluralize [word]
  (cond
    (ends-with-any? word ["s" "sh" "ch" "x" "z"])
    (str word "es")

    (s/ends-with? word "f")
    (replace-suffix word "f" "ves")

    (s/ends-with? word "fe")
    (replace-suffix word "fe" "ves")

    (re-find #"[^aeiou]y$" word)
    (replace-suffix word "y" "ies")

    :else (str word "s")))

src/import_fns/format.cljs

Again, this namespace contains all of the string formatting functions that we need for the app. Even though we only require pluralize in our main namespace, we make all of the functions public since they all contain reusable logic. Additionally, when we expose every function, we make it possible to test every function in isolation. Note that all of these function are general functions that will work with any string. If we had code that was specific to inventories, we would put it in the inventory namespace. So what should we do if we have a set of functions that are formatting functions specific to inventories? One common practice is to create a new namespace called something like import-fns.format-inventory, and this namespace could require functions and data from both formatting and inventory namespaces to perform its specialized work. When we do not have to squeeze the architecture of our code into a class hierarchy, it gives us a great deal of flexibility. Finally, for the sake of completeness, let's look at the import-fns.ui namespace:

(ns import-fns.ui
  (:require [goog.dom :as gdom]))

(defn append-list-item [list text]
  (gdom/appendChild list
    (let [li (gdom/createElement "li")]
      (gdom/setTextContent li text)
      li)))

(defn render-list [elem items]
  (let [ul (gdom/createElement "ul")]
    (doseq [item items]
      (append-list-item ul item))
    (gdom/removeChildren elem)
    (gdom/appendChild elem ul)))

src/import_fns/ui.cljs

There is not much that is novel to note about this namespace, except that its API is formed around rendering text rather than an inventory specifically. The "glue code" that ties inventories and rendering together is all in our core namespace.

Avoiding “Hidden” OOP

One thing that many new ClojureScript developers will realize is that with atoms and namespaces that can contain private variables, we can easily emulate stateful objects. This is true - we can define a private var to hold our object state within an atom, then we can use public functions to define the API and private functions as implementation details. However, as soon as we introduce hidden state, we lose the benefits of functional purity and referential transparency in terms of both testability and reasonability.

Namespaces and the REPL

Before moving on, let's look briefly at how to interact with namespaces from the REPL. When we used :require or :import in the namespace declaration, the ns special form used these directives to determine what code to make available within our namespace. There are macros that act as analogs to these directives - require and import - which we can run from the REPL to expose code to our REPL session:

dev:cljs.user=> (require '[import-fns.format :as fmt] :reload)
nil
dev:cljs.user=> (fmt/pluralize "burrito")
"burritos"

The primary difference is that we must quote the vector that we pass to require - otherwise, the REPL would try to resolve the import-fns.format and fmt symbols to vars that are bound in our current namespace. Quoting the vector (i.e. using '[] instead of []) causes ClojureScript to interpret each symbol in it as a literal symbol rather than attempting to resolve them. An additional difference is that we can add the :reload keyword after the require spec to cause the REPL to reevaluate the namespace in order to pick up on any changes that were made to the file since we last required it.

By default, when we start up a REPL, it operates inside a namespace called cljs.user. Anything that we def (or defn) will be defined inside this namespace for the duration of the REPL session. If we would like to operate inside another namespace, we can use the in-ns special form to instruct the REPL to operate within another namespace instead:

dev:cljs.user=> (in-ns 'import-fns.format)

dev:import-fns.format=> (replace-suffix "programmer" "er" "ing")
"programming"

Using in-ns can simplify our workflow when we are practicing REPL-driven development. We can start writing our code for a namespace in the REPL then copy all of our definitions to the file when we are satisfied with how they work.

Summary

In this lesson, we learned what namespaces are as well as how to declare a namespace with its dependencies using :require and :import. We also looked at how to think about organizing our code into namespaces using the principles of high cohesion and low coupling. Finally, we looked at how to interact with namespaces from the REPL so that we can effectively navigate and test our code.

There is another form, (:use [namespace :only [var1 var2]]) that works just like (:require [namespace :refer [var1 var2]]). It used to be used heavily in Clojure before :refer was available inside (:require), but it has all but fallen out of use. ↩

Managing State

Andrew Meredith — Sun, 05 Jan 2020 05:35:32 +0000

This article is Lesson 22 of Andrew's book, Learn ClojureScript

This lesson has been a long time coming, and it is a critical one. By this point, we have seen that it is possible to write whole applications without using any mutable data, but for most cases, it is inconvenient to say the least. As we learned in the last lesson, ClojureScript encourages writing programs as a purely functional core surrounded by effectful code, and this incudes code that updates state. Being a pragmatic language, ClojureScript gives us several constructs for dealing with values that change over time.

In this lesson:

Use atoms to manage values that change over time
Observe and react to changes in state
Use transients for high-performance mutations

Atoms

As we have seen many times, ClojureScript encourages us to write programs primarily as pure functions that transform immutable values. We have also seen that this can be somewhat cumbersome. Following the philosophy of pragmatism over purity, ClojureScript provides a convenient tool for representing state that changes over time: the atom. Atoms are containers that can hold a single immutable value at any point in time. However, the value that they refer to can be swapped out with another value. Moreover, our code can observe whenever these state swaps occur. This gives us a convenient way to deal with state that changes over time.

Unlike JavaScript, ClojureScript separates the ideas of identity and state¹. An identity is a reference to a logical entity. That entity may change over time, but it still retains its identity in the same way that a river retains its identity even though it has different water flowing through it over the course of time. An identity may be associated with various values over the course of time, and these values are its state. Atoms are the state containers that we use in CloureScript to represent identities.

A River Has Many States Over Time

Updating state with `swap!`

The most trivial example of an identity that we can learn from is the lowly counter. A counter is an identity whose state is a number that increases over time. We can wrap any clojure value in an atom just by calling (atom v) where v is the value that becomes the atom's initial state:

(def counter (atom 0))

Since an atom provides a reference to some value at any point in time, we can dereference it - that is, get the immutable value to which it refers - by using the deref macro or its shorthand form, @.

counter                                                    ;; <1>
;; => #object[cljs.core.Atom {:val 0}]

(deref counter)                                            ;; <2>
;; => 0

@counter                                                   ;; <3>
;; => 0

Dereferencing an Atom

The atom itself is an object that wraps a value
Atoms can be dereferenced with deref
Prefixing an atom's name with @ is sugar for calling deref

Of course to be able to do anything useful with an atom we have to be able to update it's state, and we will use the swap! function to do this. swap! takes an atom and a transformation. That function will be given the atom's current state and should return its new state. swap! itself will return the atom's new state. Any additional arguments to swap! are passed as additional arguments to the transformation function. For our simple counter, we can use inc to increment it and + to add more than 1 at a time.

(swap! counter inc)

@counter
;; => 1

(swap! counter + 9)

@counter
;; => 10

Atoms are ClojureScript's way of providing a very controlled mechanism for updating state. When we dereference an atom, we still get an immutable value, and even if the atom's state is updated, the value that we received does not change:

(def creature
  (atom {:type "water"
         :life 50
         :abilities ["swimming" "speed"]}))

(def base-creature @creature)                              ;; <1>

(swap! creature update :abilities conj "night vision")

@creature                                                  ;; <2>
;; => {:type "water"
;;     :life 50
;;     :abilities ["swimming" "speed" "night vision"]}

base-creature                                              ;; <3>
;; => {:type "water"
;;     :life 50
;;     :abilities ["swimming", "speed"]}

Dereference the atom before we swap! in a new state
After the swap!, the atom's state has changed
The initial state that we got is unchanged

We can also provide a function that acts as a validator that lets us define what sort of values are allowed in the atom using the set-validator! function². The validator function takes what would be the new value of the atom. If it returns false (or throws an error), then our attempted update will fail and throw an error. For instance, to guarantee that we can never set a negative :life value on our creature, we could supply a validator to ensure this property:

(set-validator! creature
  (fn [c] (>= (:life c) 0)))

(swap! creature assoc :life 10) ;; Ok

(swap! creature assoc :life -1) ;; Throws error

(:life @creature) ;; 10

As we just observed, updating the atom's state in a way that makes the validator return false results in an exception being thrown and no update being made. Validators are not commonly used in ClojureScript, but like pre- and post-conditions for functions, they can be a useful tool during development.

Quick Review

What value does swap! return?
How does a validator function indicate whether a state should be allowed or not?

Replacing state with `reset!`

While swap! is useful for transforming the state of an atom, sometimes we just want to update the atom's entire state at once. Using ClojureScript's standard library, this is not a difficult task: (swap! counter (constantly 0)). constantly returns a function that always returns a specific value every time it is called, so in this case, it returns a function that will always return 0, given any argument, which will effectively reset the counter state to 0. However, this code is not as cleat as it could be, which is why ClojureScript also provides the reset! function. This function simply takes the atom and a value, which it sets as the atom's new state. Like swap!, it returns the new state:

(reset! counter 0)

@counter
;; => 0

The reset! function is useful especially when we have some known initial state that we want to revert to, but otherwise, swap! is more commonly used in practice.

Observing Change with Watches

One of the most useful features of atoms is the ability to be notified whenever their state changes. This is accomplished with the add-watch function. This function takes 3 arguments: the atom to watch, a keyword that uniquely identifies the watcher, and a watch function. The watch function itself takes the keyword that was passed to add-watch, the atom itself, the atom's old state, and its new state. In most cases, the old and new state are the only things that we are interested in. To get our feet wet, let's implement a simple counter with buttons that can be used to add or subtract from its value.

Watching a Counter Atom

(defonce app-state (atom 0))                               ;; <1>

(def app-container (gdom/getElement "app"))

(defn render [state]                                       ;; <2>
  (set! (.-innerHTML app-container)
        (hiccups/html
          [:div
            [:p "Counter: " [:strong state]]
            [:button {:id "up"} "+"]
            [:button {:id "down"} "-"]])))

(defonce is-initialized?
  (do
    (gevents/listen (gdom/getElement "app") "click"
      (fn [e]
        (condp = (aget e "target" "id")
          "up"   (swap! app-state inc)
          "down" (swap! app-state dec))))

    (add-watch app-state :counter-observer                 ;; <3>
      (fn [key atom old-val new-val]
        (render new-val)))

    (render @app-state)

    true))

Counter Component

Create an atom to hold the counter state
Render takes the current state
Add a watch function that re-renders the component whenever state changes

In this example, we use add-watch to observe changes to the state of the app-state atom. There is a related function, remove-watch, that can de-register the watch function. It takes the atom that is being observed and the keyword identifying the watcher to remove. If we wanted to remove the watcher in the example above, we could call this function like so:

(remove-watch app-state :counter-observer)

Challenge

Take the Contact Book app from Lesson 20 and refactor it to keep the state in an atom.

Transients

While atoms are the defacto tool for managing state that changes over time, transients come in handy when we need to introduce mutability for the sake of performance. If we need to perform many transformations in a row on a single data structure, ClojureScript's immutable data structures are not the most performant. Every time we perform a transformation of an immutable data structure, we create garbage that JavaScript's garbage collector will need to clean up. In cases like this, transients can be very useful.

A transient version of any vector, set, or map may be created with the transient function:

(transient {})
;; #object[cljs.core.TransientArrayMap]

The API for working with transients is similar to the standard collection API, but the transformation functions all have a ! appended, e.g. assoc!, conj!. The read API, however, is identical to that of immutable collections. A transient collection may be converted back to its persistent counterpart using the persistent! function:

(-> {}
    transient                                              ;; <1>
    (assoc! :speed 12.3)
    (assoc! :position [44, 29])
    persistent!)                                           ;; <2>

Convert map to a transient
Convert transient map back to a persistent (immutable) structure

Transients are not commonly used and should only be considered as a performance optimization when we have proven that a portion of code is too slow.

Using State Wisely

ClojureScript's state management - particularly atoms - give us great power to more naturally and intuitively model things that change over time, but with that power comes the potential of introducing anti-patterns. If we follow a couple of simple guidelines, we can ensure that our code remains clear and maintainable.

Guideline #1: Pass atoms explicitly

In order to keep a function testable and easy to reason about, we should always explicitly pass in any atom(s) on which it operates as arguments rather than operating on a global atom from its scope:

;; Don't do this
(def state (atom {:counter 0}))                            ;; <1>
(defn increment-counter []
  (swap! state update :counter inc))

;; OK
(defn increment-counter [state]                            ;; <2>
  (swap! state update :counter inc))

Increment a global counter atom
Increment a counter atom passed in as a parameter

While neither function is pure (they both have the side effect of mutating state), the second option is more testable and reusable because we can pass in any atom that we wish. We do not need to implicitly depend on the current global state.

Guideline #2: Prefer fewer atoms

In general, an application should have fewer atoms with more data rather than a separate atom for every piece of state. It is simpler to think about transitioning our entire app state one step at a time rather than synchronizing separate pieces of state:

;; Don't do this
(def account-a (atom 100))                                 ;; <1>
(def account-b (atom 100))
(swap! account-a - 25)
(swap! account-b + 25)

;; OK
(def accounts (atom {:a 100                                ;; <2>
                     :b 100}))
(swap! accounts
  (fn [accounts]
    (-> accounts
        (update :a - 25)
        (update :b + 25))))

Represent each piece of state as a separate atom
Represent our "world" as an atom

While the second version is a bit more verbose, it has the advantage of creating cohesion between different steps that are all part of a "transaction", and it allows us to create complex state transitions without relying on many separate inputs. As we will see in the next section, this is also a common pattern when using the Reagent framework.

Summary

In this lesson, we were introduced to the critical feature of managing state that changes over time. As we have seen, we can create complete applications without resorting to mutability, adding just a small amount of controlled mutability can make our code dramatically simpler. We spent most of our time looking at how to use atoms to work with mutable state and how to observe and react to those state changes. We also looked briefly at transients - mutable version of ClojureScripts collections - and learned that while they are good for optimizing performance, they are not good general state containers. Finally, we looked at a couple of guidelines for constraining our use of mutable state that help make our applications maintainable and testable.

See https://clojure.org/about/state for a discussion on identity and state. ↩
Alternatively, a validator may be supplied when the atom is created by passing a map as a second argument to atom where the :validator key points to the validator function: (atom init-val {:validator validator-fn}) ↩

Functional Programming Concepts

Andrew Meredith — Wed, 20 Nov 2019 05:30:00 +0000

This article is Lesson 21 of Andrew's book, Learn ClojureScript

ClojureScript sits at the intersection of functional programming and pragmatism. In this lesson, we will take a deeper look at what it means to be a functional language. As we shall see, functional programming is about much more than simply being able to use functions as values. At its core, the more important concepts are composability, functional purity, and immutability. Composability means that we build larger modules and systems out of small, reusable pieces. The concept of functional purity means that our functions do not have side effects like mutating global state, modifying the web page, etc. Immutability means that instead of modifying variables in-place, we produce new, transformed values. These three concepts together make for effective functional programming, and by the end of this lesson, we will have a better understanding of what it means to write functional code in ClojureScript.

In this lesson:

Apply bottom-up design by composing small functions into larger ones
Make programs easier to reason about with functional purity
Learn the key role that immutability plays in functional programming

Composing Behaviour from Small Pieces

In imperative programming, we often keep some sort of mutable state and write functions that operate on this state. The key insights introduced by object-oriented programing is that programs are easier to reason about when we encapsulate our mutable state with the methods that are allowed to operate on that state into an object. Clean object-oriented code is factored into methods that have only a single responsibility. While this may sound good on the surface, it often ends up being both more and less restrictive than we desire. It is more restrictive than we would like because it is difficult to share similar logic between multiple objects without introducing significant complexity. It is less restrictive than we would like because any of these methods may change the state of the object to which they belong such that the future behaviour of any method on the object may be altered in a way that the caller does not anticipate.

Like object-oriented programming, functional programming encourages writing functions that do one thing, but it does not suffer from the two deficiencies stated above. Instead, functions may operate on any data without having to be encapsulated into an object, which leads to simpler code with less duplication. Additionally, pure functions by definition do not modify any state, and their behaviour cannot be affected by any mutable state, so their behaviour is well-defined in all cases.

When our data is modeled using common data structures (primarily maps and lists), and we do not rely on shared mutable state, something very interesting occurs: we can compose a handful of functions in many, many ways. In fact, we can often create our programs without doing much outside of composing functions from the standard library. At this point, we must include the quote that must make an appearance in every good text on functional programming to further this point:

It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures.
- Alan Perlis

What Perlis means is that when we have 100 functions that operate on the same common data type or abstraction, we can compose them to do many more than 100 things. However, if we go the object-oriented route and tie methods to specific classes of objects, then the ways in which we compose behaviour is much more limited.

As we will be building a group chat application at the end of this section, let's consider a component that displays a user's "badge", which is essentially their nickname and their current online status.

User Status Badge

We can break this component down into a couple of small composable pieces. First, let's write functions that will get a user's nickname as well as a function that wraps any hiccup-like structure that we give it in a strong tag to make it bold.

(def alan-p {:first-name "Alan"                            <1>
             :last-name "Perlis"
             :online? false})

(defn nickname [entity]                                    <2>
  (or (:nickname entity)
      (->> entity
           ((juxt :first-name :last-name))
           (s/join " "))))

(defn bold [child]                                         <3>
  [:strong child])

(bold (nickname alan-p))
;; => [:strong "Alan Perlis"]

Define sample data
Extract a user's nickname
Make some DOM bold

Next, since we know that we will want to add classes to certain elements for styling purposes, we can create a function to add a class to any hiccup-like tag.

(defn concat-strings [s1 s2]
  (s/trim (str s1 " " s2)))

(defn with-class [dom class-name]
  (if (map? (second dom))
    (update-in dom [1 :class] concat-strings class-name)
    (let [[tag & children] dom]
      (vec (concat [tag {:class class-name}]
                   children)))))

Since we are using plain data structures to represent the DOM, this function can be written in terms of data manipulation functions from the standard library. In fact, it does not refer to anything specific to hiccup at all! Now we can write a function that adds an "online" or "offline" class to the user badge based on the value of the user's online? flag:

(defn with-status [dom entity]
  (with-class dom
    (if (:online? entity) "online" "offline")))

Note that even though we are inspecting the online? property of a user, there is nothing preventing us from using this function on entities that we want to add in the future, such as chatbots. Finally, we can define a user-status component almost purely in terms of these small building blocks that we just created:

(defn user-status [user]
  [:div {:class "user-status"}
    ((juxt
      (comp bold nickname)                                 <1>
      (partial with-status                                 <2>
               [:span {:class "status-indicator"}]))
     user)])

comp creates a new function that composes others together
partial creates a function that already has some arguments supplied

We first saw juxt in the last lesson, but there are two more functions in this example that are used very commonly in ClojureScript and are incredibly useful in combining smaller functions into larger applications: comp and partial. comp performs function composition, similar to mathematical function composition:

Mathematical Function Composition

As in mathematics, composing functions f and g creates a function that when applied to some argument, x, evaluates (g x) and then passes that result as the argument to f, as in the following example.

(= ((comp f g) x)
   (f (g x)))

Function Composition in ClojureScript

We can think of comp as similar to the -> macro applied in reverse, except that rather than evaluating the entire pipeline of functions, it produces a new function that will evaluate the pipeline with any input that we give it. In the case of our user status component, we use (comp bold nickname) to create a function that will take a user and return a bolded version of that user's nickname. We can think of the new function as a pipeline that connects each function from right to left.

comp Function Pipeline

The other new function that we used is partial. While partial is not directly related to function composition, it does give us the ability to take a general function and create a more specified version of it by supplying one or more of its parameters. The canonical example of partial application is the addition function: (add x y). We can use partial to supply the x argument, creating a new function that only takes y and adds the x that we already supplied:

(defn add [x y]                                            <1>
  (+ x y))

(def add-5 (partial add 5))                                <2>

(add-5 10)
;; 15

Define our own add function that only has 2 parameters
Create a partially applied version

In this example, partial generates a new function that will call add with 5 plus any other argument that we give it. The call to partial is functionally equivalent to the following definition:

(def add-5
  (fn [y]
    (add 5 y)))

This trivial example of building a user status component out of small, composable functions that operate on simple data should illustrate the power of function composition to create an extensible, reusable codebase.

Writing Pure Functions

Side effects are essential in every useful program. A program with no side effects could not modify the DOM, make API requests, save data to localStorage, or any of the other things that we typically want to do in a web application. Why, then, do we talk about writing code without side effects as a good thing? The purely functional programming model does not allow side effects, but all functional programming languages provide at least some facility for side effects. For instance, Haskell provides the IO monad for performing impure IO operations behind an otherwise pure functional API. ClojureScript is even more practical though, allowing us to write effectful code as needed. If we were not careful, we could end up writing code that has all of the same pitfalls as most JavaScript code in the wild. Since the language itself is not going to prevent us from writing code riddled with side effects, we need to intentionally constrain ourselves to write mostly pure applications. Thankfully, ClojureScript makes that sort of constraint easy.

Keeping a Purely Functional Core

While we need side effects, we should strive to segregate functions that perform side effects from those that can be pure. The pure functions can then be easily tested and reasoned about. For instance, let's take a look at the code that we wrote for the temperature converter app in Lesson 15:

(defn update-output [_]
  (if (= :celsius (get-input-uom))
    (do (set-output-temp (c->f (get-input-temp)))
        (gdom/setTextContent output-unit-target "F"))
    (do (set-output-temp (f->c (get-input-temp)))
        (gdom/setTextContent output-unit-target "C"))))

While this code gets the job done, it is not especially clean or elegant because it both performs a conversion and does I/O. In order to test this code, we need to run it on a page where all of the elements exist, and in order to test it, we would have to manually set the input fields, call the function, then assert on the content of the output element. Instead, we could refactor this into several pure functions: a pure function to get the label, a pure function to perform the conversion, and an impure function that reads from and mutates the DOM.

(defn target-label-for-input-unit [unit]                   ;; <1>
  (case unit
    :fahrenheit "F"
    :celsius "C"))

(defn convert [unit temp]                                  ;; <2>
  (if (= :celsius unit)
    (c->f temp)
    (f->c temp)))

(defn update-output [_]                                    ;; <3>
  (let [unit (get-input-unit)
        input-temp (get-input-temp)
        output-temp (convert unit input-temp)
        output-label (target-label-for-input-unit unit)]
    (set-output-temp output-temp)
    (gdom/setTextContent output-unit-target output-label)))

Extracted code for getting the converted unit label from the input unit
Extracted code for converting a temperature from one unit to the other
The impure code now only orchestrates the UI logic

Not only is the pure functional core of our code easier to test, it is also more resilient to changes that we may want to make to the UI and user experience application. If we wanted to change the way that the UI works, we would only need to replace the update-output function, not any of the conversion logic.

Referential Transparency

A function is said to be referentially transparent if it fits in the pure substitution model of evaluation that we have discussed. That is, a call to a function can always be replaced with the value to which it evaluates without having any other effect. This implies that the function does not rely on any state other than its parameters, and it does not mutate any external state. It also implies that whenever a function is called with the same input values, it always produces the same return value. However, since not every function can be referentially transparent in a real application, we apply the same strategy of keeping our business logic pure and referentially transparent while moving referentially opaque code to the "edges" of the application. A common need in many web apps is to take the current time into consideration for some computation. For instance, this code will generate a greeting appropriate to the time of day:

(defn get-time-of-day-greeting-impure []
  (condp >= (.getHours (Date.))
    11 "Good morning"
    15 "Good day"
    "Good evening"))

The problem with this code is that its output will change when given the same input (in this case, no input) depending on what time of day it is called. Getting the current time of day is intrinsically not referentially transparent, but we can use the technique that we applied earlier to separate effectful functions from a functionally pure core of business logic:

(defn get-current-hour []                                  ;; <1>
  (.getHours (Date.)))

(defn get-time-of-day-greeting [hour]                      ;; <2>
  (condp >= hour
    11 "Good morning"
    15 "Good day"
    "Good evening"))

(get-time-of-day-greeting (get-current-hour))
;; "Good day"

Factor out the function that is not referentially transparent
Ensure that the output of our business logic is solely dependent on its formal parameters

This is a fairly trivial example, but any code that relies on external state - whether it is the time of day, the result of an API call, the (Math.random) random number generator, or anything other than its explicit parameters - breaks the functional paradigm and is more difficult to test and evolve as requirements change.

Immutable Data

We have just walked through the process of refactoring code that mutates the DOM into an impure wrapper around a functional core. However, there is another type of side effect that we must avoid if we want to write functional code: data mutation. In JavaScript, the two built-in collection data structures - objects and arrays - are mutable, meaning they can be modified in-place. All of the object-oriented features in JavaScript rely on mutability. For example, it is very common to see code that makes directly manipulates an object, such as the following:

var blog = {
  title: 'Object-Oriented JavaScript',
  tags: ['JavaScript', 'OOP'],
  rating: 4
};

blog.tags.push('mutability');
blog.rating++;
blog.title += ' considered harmful';
blog.isChanged = true;

In ClojureScript, instead of modifying an object, we create copies of the original object with modifications. While this sounds inefficient, ClojureScript uses data structures that are constructed in such a way that similar objects can often share much of their structure in the same memory locations. As we will see, these immutable data structures actually enable highly-optimized user interfaces and can even speed up an application. Working with immutable data structures takes a mind shift but is every bit as easy as working with mutable data, as we can see below.

(def blog {:title "Functional ClojureScript"
           :tags ["ClojureScript" "FP"]
           :rating 4})

(def new-blog
  (-> blog                                                 ;; <1>
      (update-in [:tags] conj "immutability")
      (update-in [:rating] inc)
      (update-in [:title] #(str % " for fun and profit"))
      (assoc :new? true)))

new-blog                                                   ;; <2>
; {:title "Functional ClojureScript for fun and profit",
; :tags ["ClojureScript" "FP" "immutability"], :rating 5, :new? true}
blog
; {:title "Functional ClojureScript", :tags ["ClojureScript" "FP"], :rating 4}

Build up a series of transformations using the -> macro
Inspect both the original blog and transformed new-blog maps

In this example, we see that the original blog variable is untouched. We stack a series of modifications on top of the blog map and save the modified map as new-blog. The key takeaway here is that none of the update-in or assoc functions touched blog - they returned a new object that was similar to the one passed in but with some modification. Immutable data is key to functional programming because it gives us the assurance that our programs are repeatable and deterministic (at least the parts that need to be). When we allow data to be mutated as it is passed around, complexity skyrockets. When we allow mutable data, we need to keep track of a potentially enormous number of variables in our heads to debug a single computation.

Note:

The author once worked on a team responsible for a very large JavaScript application. That team discovered that unexpected mutation was the cause of so many defects that they started keeping a tally of every hour spent finding and fixing bugs that were due to mutable data. The team switched to the Immutable.js library when their tallies no longer fit on the whiteboard.

Functional Design Patterns

Design patterns have gotten something of a bad reputation over the past decade or so, and in many cases, the criticism has been well-deserved. Most of the "Gang of Four"¹ design patterns were discovered as workarounds for a lack of flexibility in the object-oriented languages at the time. In fact, Peter Norvig, a major player in the Lisp and AI communities, found that 16 out of the 23 design patterns presented in the Gang of Four book are either unnecessary in Lisp or arise through natural use of the language.² Still, the goal of design patterns is to create a common vocabulary around describing problems that occur often in software development and present a template for an often-used solution. Even in a dynamic functional language like ClojureScript, there is some merit to defining problem and solution sets that occur often, so in this section, we will describe several: constructor, closure, strategy, and middleware.

Constructor

This should be a familiar pattern, as we already covered it in Lesson 19 and revisited it in Lesson 20. The idea here is to abstract the creation of some data structure behind a function in order to assign a name to that data structure and to make it easier to change the structure in the future. Since we have already been using this pattern for a couple of lessons, we need not belabor an explanation here.

Closure

Like JavaScript, ClojureScript has the concept of lexical closures. That is, a function can refer to any variables that were visible in the scope in which the function was defined. This allows us to retain (immutable) state in our functions. This allows us to do things like define a DOM element at the top level of a namespace and refer to it within a function defined in that same namespace:

(def user-notes (gdom/getElement "notes"))

(defn get-notes []
  (.-value user-notes))

This is a pretty trivial use of a closure. Their real power shines when used in conjunction with higher-order functions. Recall how at the beginning of this chapter we defined an add function that simple adds 2 numbers together. We then used partial application to generate a function that always adds 5 to its argument. It would be much more flexible, however, if we defined a make-adder function that accepts a number and returns an adding function that always adds that number to its argument. Because of closures, we can do this:

(defn make-adder [x]
  (fn [y]
    (add x y)))

((make-adder 1) 5)  ;; 6
((make-adder 2) 5)  ;; 7
((make-adder 10) 5) ;; 15

Note that the function we return from make-adder can reference any arguments passed into is parent function by name. Closures require us to tweak the mental model of evaluation that we introduced back in Lesson 4 and extended in Lesson 12, since we can no longer assume that we can simply replace a function call by the function's definition with all formal parameters replaced by actual parameters. If we did that in the make-adder case, we would end up with the following:

((make-adder 1) 10)

((fn [y]
  (add x y)) 10)

This would be a big problem because we would "lose" knowledge of the 1 that we passed into make-adder and would leave the symbol x that is not bound to any value. Let's introduce the concept of an environment, which is simply a mapping of symbols to vars that were visible where the function was defined, and update our model of evaluation to say that when we evaluate a function, we replace all symbols within the function definition with the formal parameter of the same name or the value found for the corresponding name in the environment if no matching formal parameter is found.

Using this simple principle, we can use closures to emulate objects from the OOP world. Consider that we have a constructor that takes some initial state for an object, and we return a map of functions that can either update this state, returning a new object, or query the state, returning some value.

Purely Functional Objects

Provide a no-arg constructor for convenience
By convention, methods ending in ! will update the object state and return a new object
Methods not ending in ! will not update the object but will return a value instead
apply calls a function with a collection of arguments

You Try It

Add a few more methods to the mailbox "object":
- :all-messages - returns all messages in the mailbox
- :unread-messages - returns all unread messages in the mailbox
- :mark-all-read! - mutates the state by marking every message as read

Strategy

As in object-oriented programming, the strategy pattern is a way to separate the implementation of an algorithm from higher-level code. This pattern is so natural in functional languages that it barely qualifies as a pattern. Let's take an example from the standard library: filter. This function filters (surprise, surprise!) a sequence, but it does not specify the criteria by which each element should be tested for inclusion. That criteria is dictated by the function passed to filter, which specifies the concrete strategy for determining which elements should be included in the new sequence.

(let [xs [0 1 2 3 4 5 6 7 8 9]]
  (println (filter even? xs))
  (println (filter odd? xs)))
;; (0 2 4 6 8)
;; (1 3 5 7 9)

Middleware

The final pattern that we will consider is the middleware pattern. This pattern allows us to declare "hooks" in a request/response cycle that can transform the request on the way in, the response on the way out, or both. It can even be used to short-circuit a request.

Imagine that we need to make a call to some API, but we want to be able to validate the request before it is sent. We could add the validation logic directly to the function that performs the API request, but this is less than ideal for two reasons: first, it couples validation logic to API logic, and second, it makes our app less testable by combining both pure and impure business logic in the same function. No problem, let's add separate validation function:

(defn handler [req]
  (println "Calling API with" req)
  {:data "is fake"})

(defn validate-request [req]
  (cond
    (nil? (:id req)) {:error "id must be present"}
    (nil? (:count req)) {:error "count must be present"}
    (< (:count req) 1) {:error "count must be positive"}))

;; Client code
(if-let [validation-error (validate-request req)]
  validation-error
  (handler req))

This works well for a while, but now we need to add logging around each request that goes out and its response. We could add more logic to our client code, such as the following:

(do
  (println "Request" req)
  (let [res (if-let [validation-error (validate-request req)]
              validation-error
              (handler req))]
    (println "Response" res)
    res))

Even with 2 hooks that we want to add to the request/response, the code is starting to get a bit unwieldy. With the middleware pattern, we can extract each hook into a function that "wraps" the handler function. A middleware is simply a function that takes a handler and returns a new handler. If we think of a handler in general as any function from Request -> Response, then a middleware is a function from (Request -> Response) -> (Request -> Response). One nice feature of middleware is that since they have the same input and output types, they can be combined in any order using ordinary function composition!

Middleware Pattern

Returning to the wrapped API handler example above, writing the validation and logging as middleware would look something like the following:

(defn with-validation [handler]
  (fn [req]
    (if-let [error (validate-request req)]
      error
      (handler req))))

(defn with-logging [handler]
  (fn [req]
    (println "Request" req)
    (let [res (handler req)]
      (println "Response" res)
      res)))

(let [wrap-handler (comp with-logging with-validation)
      handler (wrap-handler handler)]
  ; Example invalid request
  (handler {})
  ;; Request ()
  ;; Response {:error id must be present}

  ; Example valid request
  (handler {})
  ;; Request {:id 123, :count 12}
  ;; Calling API with {:id 123, :count 12}
  ;; Response {:response is fake}
  )

Quick Review

Why can middleware be composed using ordinary function composition?
Does the order in which middleware are composed matter? What would have happened if we defined wrap-handler as (comp with-validation with-logging)?

Summary

In this lesson, we looked briefly at three cornerstones of functional programming: minimizing and segregating the side effects, using immutable data, and keeping our business logic referentially transparent. When we these ideas, we naturally write programs that take a piece of data, pass it through a number of transformations, and the result is the output of the program. In reality, most programs are made up of many pure data pipelines that are glued together with impure code that interacts with the DOM, but if we follow the functional programming concepts that we have learned in this lesson, we will end up with a clean functional core of business logic that is primarily concerned with data transformation. Over the next few lessons, we will learn more language features and techniques that will enable us to be effective practitioners of functional programming.

Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, John Vlissides, Richard Helm, and Ralph Johnson ↩
Design Patterns in Dynamic Languages ↩

Capstone 3 - Contact Book

Andrew Meredith — Fri, 01 Nov 2019 04:30:19 +0000

This article is Lesson 20 of Andrew's book, Learn ClojureScript

Over the past few lessons, we have learned the core tools for working with data in ClojureScript. First, we learned about the basic collection types - lists, vectors, maps, and sets - as well as the most common functions for working with these collections. Then we took a closer look at the important sequence abstraction that allows us to operate on all sorts of sequential data using a uniform interface. Next, we discovered the reduce function and the many cases in which it can be used to summarize a sequence of data. Last, we walked through the process of modeling a real- world analytics domain. With this knowledge in our possession, we are ready to build another capstone project. This time, we will take the example of a contact book that we mentioned back in Lesson 16, and we will build a complete implementation, ClojureScript Contacts.

In This Lesson:

Create a complete ClojureScript application without any frameworks
Build HTML out of plain ClojureScript data structures

Screenshot of ClojureScript Contacts

Data Modeling

In this lesson, we will use the techniques and patterns from the previous chapter to model the data for our contact book. The goal will be to practice what we have learned rather than to introduce much new material. We will primarily model our data using maps and vectors, and we will implement the constructor function pattern for creating new contacts. We will also implement the operations that the UI will need to update the contact list using simple functions to transform our data. With that, let's dig in to the data model!

Constructing Entities

Since a contact book represents an ordered list of contacts, we will need a data structure to represent that ordered list, and as we have already seen, a vector fits the bill nicely. We can define an empty contact list as an empty vector - no constructor function necessary:

(def contact-list [])

Since an empty vector is not terribly interesting, let's turn our attention to the contacts that it will hold. Each contact will need a first name, last name, email address, and a physical address, including city, state, and country. This can easily be accommodated with a nested map, such as the following:

{:first-name "Phillip"
 :last-name "Jordan"
 :email "phil.j@hotmail.com"
 :address {:street "523 Sunny Hills Cir."
           :city "Springfield"
           :state "MI"
           :postal "11111"
           :country "USA"}}

In order to construct a new contact, we will use a variation on the constructor pattern introduced in the last lesson. Instead of passing in each field individually, we will pass in a map that we expect to contain zero or more of the fields that make up a contact. You will recall from the last lesson that the select-keys function takes a map and a collection of keys that should be selected, and it returns a new map with only the selected keys copied. We can use this function to sanitize the input, ensuring that our contact contains only valid keys.

(defn make-contact [contact]
  (select-keys contact [:first-name :last-name :email :postal :address]))

Since the address itself is a map, let's factor out creation of an address to another function. We can then update the make-contact function to use this address constructor:

(defn make-address [address]
  (select-keys address [:street :city :state :country]))

(defn make-contact [contact]
  (let [clean-contact (select-keys contact [:first-name :last-name :email])]
    (if-let [address (:address contact)]
      (assoc clean-contact :address (make-address address))
      clean-contact)))

This new version of make-contact introduces one expression that we have not seen before: if-let. This macro works just like if except that it binds a name to the value being tested (just like let does). Unlike let, only a single binding may be provides. At compile time, this code will expand to something like the following¹:

(if (:address contact)
  (let [address (:address contact)]
    (assoc clean-contact :address (make-address address)))
  clean-contact)

if-let Transformation

However, we can make the make-contact function a bit more concise and easier to read using one of ClojureScript's threading macros, -> (pronounced "thread first"). This macro allows us to take what would otherwise be a deeply nested expression and write it more sequentially. It takes a value and any number of function calls and injects the value as the first argument to each of these function calls. Seeing this transformation in action should make its functionality more intuitive:

(-> val                                                    ;; <1>
    (fn-1 :foo)                                            ;; <2>
    (fn-2 :bar :baz)                                       ;; <3>
    (fn-3))

(fn-3                                                      ;; <4>
  (fn-2
    (fn-1 val :foo)
    :bar :baz))

Thread-First Transformation

Start with val as the value to thread through the following expressions
fn-1 will be evaluated with the arguments, val and :foo
The result of the evaluation of fn-1 will be threaded as the first argument to fn-2
The macro will rewrite into a nested expression that evaluates fn-1 then fn-2, then fn-3

This macro is extremely common in ClojureScript code because of how it enhances the readability of our code. We can write code that looks sequential but is evaluated "inside-out". There are several additional threading macros in ClojureScript that we will not go into now, but we will explain their usage as we run into them.

Thread First Macro

With this macro, we can make our make-contact function even clearer:

(defn maybe-set-address [contact]                          ;; <1>
  (if (:address contact)
    (update contact :address make-address)
    contact))

(defn make-contact [contact]
  (-> contact                                              ;; <2>
      (select-keys [:first-name :last-name :email])
      (maybe-set-address)))

Refactor the code that conditionally constructs an address
Rewrite make-contact using the -> macro

Quick Review

Does if-let allow multiple bindings? For example, what would this code do?

(if-let [contact (find-by-id 123)
         address (:address contact)]
  (println "Address:" (format-address address)))

How would the -> macro rewrite the following expression?

(let [input {:password "s3cr3t"}]
  (-> input
      (assoc :password-digest (-> input :password digest))
      (dissoc :password)))

Defining State Transitions

In order for our UI to do anything other than display a static list of contacts that we define in code, we need to enable some interactions in the UI. Again, we are building our low-level domain logic before any UI code so that we can take advantage of the bottom-up programming style that ClojureScript encourages - composing small, granular functions into larger and more useful structures.

First, we want the user to be able to add a new contact to the contact list. We can assume that we will receive some sort of form data as input, which we can pass to our make-contact constructor, adding the resulting contact to our list. We will need to pass in the contact list and input data as arguments and produce a new contact list.

(defn add-contact [contact-list input]
  (conj contact-list
        (make-contact input)))

We can paste these function definitions into the REPL and then test them to make sure that they function as expected:

cljs.user=> (-> contact-list                               ;; <1>
                (add-contact {:email "me@example.com"
                              :first-name "Me"
                              :address {:state "RI"}})
                (add-contact {:email "you@example.com"
                              :first-name "You"}))
[{:first-name "Me", :email "me@example.com"}
 {:first-name "You", :email "you@example.com"}]

Testing with the REPL

Once again, the -> macro makes our code easier to read and write

Next, we will need a way to remove a contact from the list. Since we are using a vector to hold our contacts, we can simply remove an element at a specific index:

(defn remove-contact [contact-list idx]
  (vec                                                     ;; <1>
    (concat                                                ;; <2>
      (subvec contact-list 0 idx)                          ;; <3>
      (subvec contact-list (inc idx)))))

Removing a Contact

vec converts a sequence into a vector
concat returns a seq that contains all elements in the sequences passed to it in order
subvec returns a portion of the vector that it is given

Since there are a couple of new functions here that we have not seen yet, let's quickly look at what they do. Starting from the "inside" of this function, we have subvec. This function provides an efficient way to obtain a slice of some vector. It comes in a 2-arity and a 3-arity form: (subvec v start) and (subvec v start end). This function work similarly to JavaScript's Array.prototype.slice() function. It returns a new vector that starts at the start index of the original vector and contains all elements up to but not including the end index. If no end is provided, it will contain everything from start to the end of the original vector.

Next, we have concat. This function takes a number of sequences and creates a new lazy² seq that is the concatenation of all of the elements of its arguments. Because the result is a seq, we use the vec function to coerce the result back into a vector. Since much of ClojureScript's standard library operates on the sequence abstraction, we will find that we often need to convert the result back into a more specific type of collection.

Quick Review

We mentioned that vec converts a sequence into a vector. Given what we learned about the sequence abstraction, what will happen if you pass vec a map? What about a set?

Creating the UI

Now that we have defined all of the functions that we need to work with our data model,let's turn our attention to creating the application UI. In Section 5, we will learn how tocreate high-performance UIs using the Reagent framework, but for now, we will take naive approach ofre-rendering the entire application whenever anything changes. Our application will have twomain sections - a list of contacts that displays summary details about each contact and a largerpane for viewing/editing contact details.

We will use the hiccups library to transform plain ClojureScript data structures into an HTML string. This allows us to represent the interface of our application as a ClojureScript data structure and have a very simple interface to the actual DOM of the page. In order to use this library, we need to add it to our dependencies in project.clj:

:dependencies [;; ...Other dependencies
               [hiccups "0.3.0"]]

project.clj

Then, we need to import this library into our core namespace. Note that since we are using a macro from this library, the syntax is a little different:

(ns contacts.core
  (:require-macros [hiccups.core :as hiccups])
  (:require [hiccups.runtime]))

core.cljs

The translation between ClojureScript data structures and HTML is very simple:

HTML tags are represented by vectors whose first element is the tag name as a keyword
Attributes are represented by maps and should come immediately after the tag name. Omitting attributes is fine.
Any remaining elements in the vector are children of the outer element

For example, the following code renders a div containing a single anchor tag:

(hiccups/html                                              ;; <1>
  [:div                                                    ;; <2>
    [:a {:href "https://www.google.com"                    ;; <3>
         :class "external-link"}
        "Google"]])                                        ;; <4>
;; <div><a class="external-link" href="https://www.google.com">Google</a></div>

Rendering With Hiccups

The html macro renders hiccups data to an HTML string
Create a div. We do not need to specify any attributes
Create an a element with an attributes map as a child of the outer div
The a only contains text

With this knowledge, we can start defining components for our various UI elements that produce a hiccups-compatible data structure. We can compose these functions together to create a data structure that represents the entire UI, and we'll pass this to another function that renders the whole structure to the page.

UI State

Let's take a quick step back and think about the additional state that we need for the UI. First, we need to have a flag indicating whether we are in editing mode. In editing mode, a form with contact details will be displayed in the right-hand pane. Also, we need to keep track of which contact has been selected for editing - in the case of a new contact that has not been saved yet, this property will be nil or omitted. Naturally, we also need the contacts on the application state. This leaves us with a pretty simple state model to support everything that we want in our UI:

(def initial-state
  {:contacts contact-list
   :selected nil
   :editing? false})

We will also define a refresh! function that is responsible for rendering the entire application and attaching event handlers every time our state changes:

(defn attach-event-handlers! [state])                      ;; <1>

(defn set-app-html! [html-str]
  (set! (.-innerHTML app-container) html-str))

(defn render-app! [state]
  (set-app-html!
    (hiccups/html
      [:div])))                                            ;; <2>

(defn refresh! [state]                                     ;; <3>
  (render-app! state)
  (attach-event-handlers! state))

(refresh! initial-state)                                   ;; <4>

All event handlers that we need to attach will be attached in this function
We will replace the empty div with our actual application HTML
refresh! will be called every time that we update the application state
We kick off the application with an initial refresh! to render the page from initial-state

Rendering Contacts

Let's start with the component that displays the contact summary in the list. We will be using the Bulma CSS framework to help with styling, so most of the markup that we are generating is for the purpose of styling the page.

(defn format-name [contact]                                ;; <1>
  (->> contact                                             ;; <2>
       ((juxt :first-name :last-name))                     ;; <3>
       (str/join " ")))

(defn delete-icon [idx]
  [:span {:class "delete-icon"
          :data-idx idx}
    [:span {:class "mu mu-delete"}]])

(defn render-contact-list-item [idx contact selected?]
  [:div {:class (str "card contact-summary" (when selected? " selected"))
         :data-idx idx}                                    ;; <4>
    [:div {:class "card-content"}
      [:div {:class "level"}
        [:div {:class "level-left"}
          [:div {:class "level-item"}
            (delete-icon idx)
            (format-name contact)]]
        [:div {:class "level-right"}
          [:span {:class "mu mu-right"}]]]]])

Rendering a Contact Summary

Extract the logic for a contact's display name into another function
Use the ->> (thread last) macro to pass a value through as the last argument to each subsequent function
The juxt function extracts the first and last name from the contact. It is described below
idx is needed so that we will be able to get the correct contact in our event handlers

Both of the things that we should note about this code occur in the format-name function. First, we use the ->> macro to thread a value through several function calls. This works almost exactly like the -> macro that we used earlier in this lesson with the exception that it feeds threads the value as the last argument to each subsequent function.

Second, the juxt function is a very interesting function that deserves a bit of an explanation. juxt takes a variable number of functions and returns a new function. This resulting function will call each of the original functions passed to juxt in turn using the arguments provided to it. The results are finally placed into a vector in the same order as the functions that were passed to juxt. For example, this code snippet will create a function that will get 2-element vector containing the minimum and maximum values from a collection:

(def minmax
  (juxt #(reduce Math/min %)
        #(reduce Math/max %)))

(minmax [48 393 12 14 -2 207])
;; [-2 393]

The reason for the extra set of parentheses in the call to juxt - ((juxt :first-name :last-name)) - is that we need to call the function returned by juxt rather than threading the contact into the call to juxt itself. Since keywords are functions that can look themselves up in maps, this expression will effectively create a vector with the first name and last name of the contact respectively.

((juxt :first-name :last-name) {:first-name "Bob" :last-name "Jones"})
;; ["Bob" "Jones"]

Adding Interactions

Let's now think about the interactions that we want to enable on the contact summary. When the row is clicked, we want to open up this contact's details for view/editing. We will attach an event handler to each list item that will set an :editing? flag on the app state and will set :selected to the index of the contact that was clicked (this is why we needed to pass idx into the rendering function and set the data-idx attribute on the row).

(defn on-open-contact [e state]
  (refresh!
    (let [idx (int (aget e "currentTarget" "dataset" "idx"))]
      (assoc state :selected idx
                   :editing? true))))

(defn attach-event-handlers! [state]
  ;; ...
  (doseq [el (array-seq (gdom/getElementsByClass "contact-summary"))]
    (gevents/listen el "click"
      (fn [e] (on-open-contact e state)))))

Now that we can render a single contact list item, let's create the list that will display each list item:

(defn render-contact-list [state]
  (let [contacts (:contacts state)
        selected (:selected state)]
    [:div {:class "contact-list column is-4 hero is-fullheight"}
      (map-indexed (fn [idx contact]
                     (render-contact-list-item idx contact (= idx selected)))
                   contacts)]))

This function is pretty straightforward: it renders a wrapper div with each contact list item, then it delegates to the render-contact-list-item function to render the summary for each contact in turn. There is a new function that we have not yet met, however: map-indexed. This function works just like map except that it calls the mapping function with the index of the element in the sequence as well as the element itself.

This pattern of building a UI by composing components is common in the JavaScript world aswell, but in most ClojureScript applications, the method of composing UIs is functioncomposition with pure functions that produce plain data structures.

Finally, we will briefly cover rendering the contact details form and adding/updating contacts without too much additional explanation. First, let's look at the function that renders the entire app HTML:

(defn render-app! [state]
  (set-app-html!
    (hiccups/html
      [:div {:class "app-main"}
        [:div {:class "navbar has-shadow"}
          [:div {:class "container"}
            [:div {:class "navbar-brand"}
              [:span {:class "navbar-item"}
                "ClojureScript Contacts"]]]]
        [:div {:class "columns"}
          (render-contact-list state)
          [:div {:class "contact-details column is-8"}
            (page-header (:editing? state))
            [:div {:class "hero is-fullheight"}
              (if (:editing? state)
                (render-contact-details (get-in state [:contacts (:selected state)] {}))
                [:p {:class "notice"} "No contact selected"])]]]])))

We have already seen the render-contact-list function, but we still need to define page-header, which will display the buttons for adding or saving a contact, and render-contact-details, which will render the edit form. Let's start with render-contact-details:

(defn form-field                                           ;; <1>
  ([id value label] (form-field id value label "text"))
  ([id value label type]
   [:div {:class "field"}
     [:label {:class "label"} label]
     [:div {:class "control"}
       [:input {:id id
                :value value
                :type type
                :class "input"}]]]))

(defn render-contact-details [contact]
  (let [address (get contact :address {})]                 ;; <2>
    [:div {:id "contact-form" :class "contact-form"}
      (form-field "input-first-name" (:first-name contact) "First Name")
      (form-field "input-last-name" (:last-name contact) "Last Name")
      (form-field "input-email" (:email contact) "Email" "email")
      [:fieldset
        [:legend "Address"]
        (form-field "input-street" (:street address) "Street")
        (form-field "input-city" (:city address) "City")
        (form-field "input-state" (:state address) "State")
        (form-field "input-postal" (:postal address) "Postal Code")
        (form-field "input-country" (:country address) "Country")]]))

Rendering Contact Details

Since we will be rendering multiple form fields, we will extract the code for a field into its own function
The address may or may not be present, so we provide an empty map as a default

Now let's look at the code for processing the form and either adding or updating a contact:

(defn get-field-value [id]
  (let [value (.-value (gdom/getElement id))]
    (when (not (empty? value)) value)))

(defn get-contact-form-data []
  {:first-name (get-field-value "input-first-name")
   :last-name (get-field-value "input-last-name")
   :email (get-field-value "input-email")
   :address {:street (get-field-value "input-street")
             :city (get-field-value "input-city")
             :state (get-field-value "input-state")
             :postal (get-field-value "input-postal")
             :country (get-field-value "input-country")}})

(defn on-save-contact [state]
  (refresh!
    (let [contact (get-contact-form-data)
          idx (:selected state)
          state (dissoc state :selected :editing?)]        ;; <1>
      (if idx
        (update state :contacts                            ;; <2>
                replace-contact idx contact)
        (update state :contacts
                add-contact contact)))))

state within the let will refer to this updated state
Use our domain functions to update the contact list within our app state

Before moving on, let's take a look at the use of update to transform our application state. update takes an indexed collection (a map or vector), a key to update, and a transformation function. This function is variadic, and any additional arguments after the transformation function that will be passed to the transformation function following the value to transform. For instance, the call, (update state :contacts replace-contact idx contact), will call replace-contact with the contacts list followed by idx and contact.

Now, we will finally implement the page header with its actions to create and save contacts:

(defn action-button [id text icon-class]
  [:button {:id id
            :class "button is-primary is-light"}
    [:span {:class (str "mu " icon-class)}]
    (str " " text)])

(def save-button (action-button "save-contact" "Save" "mu-file"))
(def cancel-button (action-button "cancel-edit" "Cancel" "mu-cancel"))
(def add-button (action-button "add-contact" "Add" "mu-plus"))

(defn page-header [editing?]
  [:div {:class "page-header"}
    [:div {:class "level"}
      [:div {:class "level-left"}
        [:div {:class "level-item"}
          [:h1 {:class "subtitle"}
            [:span {:class "mu mu-user"}]
            "Edit Contact"]]]
      [:div {:class "level-right"}
        (if editing?
          [:div {:class "buttons"} cancel-button save-button]
          add-button)]]])

(defn on-add-contact [state]
  (refresh! (-> state
                (assoc :editing? true)
                (dissoc :selected))))

(defn on-save-contact [state]
  (refresh!
    (let [contact (get-contact-form-data)
          idx (:selected state)
          state (dissoc state :selected :editing?)]
      (if idx
        (update state :contacts replace-contact idx contact)
        (update state :contacts add-contact contact)))))

(defn on-cancel-edit [state]
  (refresh! (dissoc state :selected :editing?)))

(defn attach-event-handlers! [state]
  ;; ...
  (when-let [add-button (gdom/getElement "add-contact")]
    (gevents/listen add-button "click"
      (fn [_] (on-add-contact state))))

  (when-let [save-button (gdom/getElement "save-contact")]
    (gevents/listen save-button "click"
      (fn [_] (on-save-contact state))))

  (when-let [cancel-button (gdom/getElement "cancel-edit")]
    (gevents/listen cancel-button "click"
      (fn [_] (on-cancel-edit state))))))

By now, this sort of code should be no problem to read and understand. In the interest of space, we will not reprint the entire application code, but it can be found at the book's GitHub project.

You Try It

Allow contacts to have phone numbers. Each phone number will need a :type and :number. This will require updates to the models, components, and event handlers.

Summary

While this application is not a shining example of modern web development (don't worry - we will get to that in Section 5), it showed us how to build a data-driven application using the techniques that we have been learning in this section. While this is not a large application, is is a non-trivial project that should have helped us get more familiar with transforming data using ClojureScript. Still, much of our ClojureScript code still looks a lot like vanilla JavaScript with a funky syntax. In the next section, we will start to dig in to writing more natural, idiomatic ClojureScript.

The actual implementation of the if-let macro is slightly more complex, but the effect is the same. ↩
Lazy evaluation was covered in Lesson 11. ↩

Mastering Data With Maps and Vectors

Andrew Meredith — Thu, 24 Oct 2019 22:25:59 +0000

This article is Lesson 19 of Andrew's book, Learn ClojureScript

In this lesson, we will explore some of the features of ClojureScript that make it simple to work with data. ClojureScript places a string emphasis on relying generic collection types and the standard functions that operate on them rather than creating highly specialized functions that only work on a single type of object. The object-oriented approach, which most mainstream languages encourage, is to create objects that encapsulate both the data and behaviour of a specific type of "thing". The practice that ClojureScript encourages, however, is to separate functions and data. Data is pure information, and functions are pure transformations of data.

In this lesson:

Master the most common map functions: assoc, dissoc, merge, and select-keys
Get and set deeply-nested values
Use the constructor pattern for creating common objects

Functions and Data

Example: Modeling a Domain

Say that we have been tasked with creating an analytics app. Before we get started, we want to model the type of objects that we will be working with. If we were using a statically typed language, we would probably start by writing type definitions. Even if we were working in JavaScript, we would likely define "classes" for the objects that we will be working with. As we define these objects, we would have to think about both the data that they contain and the operations that they support. For example, if we have a User and a ClickEvent, we might need the operation, User.prototype.clickEvent().

Our analytics domain deals users and their actions

With ClojureScript, we will consider data and functions separately. This approach ends up being flexible, as we will see that most of the operations that we want to perform on the data are simple and re-usable. In fact, it is common to find that the exact operation that you need is already part of the standard library. Ultimately, the combination of the concision of code and the richness of the standard library means that we write fewer lines of code than we would in JavaScript, which leads to more robust and maintainable applications.

Domain Modeling with Maps and Vectors

We are now quite familiar with what maps and vectors as well as some of the collection and sequence operations that can be used on them. Now we can put them in practice in a real domain: an analytics dashboard. The main concepts that we need to model are user, session, pageview, and event, and the relationships between these models are as follows:

A user has one or more sessions
A session has one or more pageviews and may belong to a user or be anonymous
A pageview has zero or more events

We now know enough to create some sample data. Let's start at the "bottom" with the simplest models and work our way up to the higher-level models. Since an event does not depend on any other model, it is a good place to start.

Modeling Events

An event is some action that the user performs while interacting with a web page. It could be a click, scroll, field entry, etc. Different events may have different properties associated with them, but they all have at least a type and a timestamp.

(def my-event {:type :click                                ;; <1>
               :timestamp 1464362801602
               :location [1015 433]                        ;; <2>
               :target "#some-elem"})

Modeling an event

Every event will have :type and :timestamp entries
The remaining entries will be specific to the event type

When we think of data types like event in ClojureScript, we usually create at least a mental schema of the data type. There are libraries that we can use to enforce a schema on our data, most notably clojure.spec, but for now we will just enforce the "shape" of our data structures by convention. That is, we will ensure that whenever we create an event, we create it with a timestamp and a type. In fact, it is a common practice to define on or more functions for constructing the new data types that we create. Here is an example for how we might do this with events:

cljs.user=> (defn event [type]
              {:type type
               :timestamp (.now js/Date)})
#'cljs.user/event

cljs.user=> (event :click)
{:type :click, :timestamp 1464610050488}

Using a constructor function

This function simply abstracts the process of creating a new object that follows the convention that we have established for events. We should also create a constructor function for click events specifically:

cljs.user=> (defn click [location target]
              (merge (event :click)
                     {:location location, :target target}))
#'cljs.user/click

cljs.user=> (click [644 831] "#somewhere")
{:type :click,
 :timestamp 1464610282324,
 :location [644 831],
 :target "#somewhere"}

The only thing about this code that might be unfamiliar is the use of the merge function. It takes at least two maps and returns a new map that is the result of adding all properties from each subsequent map to the first one. You can think of it as conj-ing every entry from the second map onto the first.

Quick Review: Merge

In the REPL, define 2 maps and merge them together
Define 3 maps and merge them together, e.g. (merge map-1 map-2 map-3)
Does merge mutate (change) any of the maps that we pass in?
What is the result of the following expression?

(let [orig {:name "Cookie Monster" :food "Cookies!!"}
      overwrite {:profession "puppet" :food "Lasagne"}]
  (merge orig overwrite))

You Try It

We are representing coordinates on a page with a 2-element vector containing, [x-position, y-position]. Define a function, location, that will create a location given two numbers, such that the following expressions will yield the same result:

;; Expression 1 - Define location inline
(click [644 831] ".link")

;; Expression 2 - Construct location with a function
(click (location 644 831) ".link")

A Word on Constructors

We have been talking about the concept of constructors in ClojureScript. Unlike JavaScript, constructors in ClojureScript are just plain functions that return data. There is no special treatment of constructor functions in the language - they are merely a convenience for us developers to easily create new data while consolidating the creation code in one place.

Modeling Pageviews

With events done, we can now model pageviews. We will go ahead and define a constructor for pageviews:

cljs.user=> (defn pageview
              ([url] (pageview url (.now js/Date) []))     ;; <1>
              ([url loaded] (pageview url loaded []))
              ([url loaded events]
                {:url url
                 :loaded loaded
                 :events events}))
3

cljs.user=> (pageview "some.example.com/url")              ;; <2>
{:url "some.example.com/url",
 :loaded 1464612010514,
 :events []}

cljs.user=> (pageview "http://www.example.com"             ;; <3>
                      1464611888074
                      [(click [100 200] ".logo")])
{:url "http://www.example.com",
 :loaded 1464611888074,
 :events [{:type :click,
           :timestamp 1464611951519,
           :location [100 200],
           :target ".logo"}]}

Modeling a Pageview

Define pageview with 3 arities
pageview can be called with just a URL
...or with a URL, loaded timestamp, and vector of events

Just as we did with events, we created a constructor to manage the details of assembling a map that fits our definition of what a Pageview is. One different aspect of this code is that we are using a multiple-arity function as the constructor and providing default values for the loaded and events values when they are not supplied. This is a common pattern in ClojureScript for dealing with default values for arguments.

Modeling Sessions

Moving up the hierarchy of our data model, we now come to the Session. Remember that a session represents one or more consecutive pageviews from the same user. If a user leaves the site and comes back later, we would create a new session. So the session needs to have a collection of pageviews as well as identifying information about the user's browser, location, etc.

cljs.user=> (defn session
              ([start is-active? ip user-agent] (session start is-active? ip user-agent []))
              ([start is-active? ip user-agent pageviews]
                {:start start
                 :is-active? is-active?
                 :ip ip
                 :user-agent user-agent
                 :pageviews pageviews}))
5
cljs.user=> (session 1464613203797 true "192.168.10.4" "Some UA")
{:start 1464613203797, :is-active? true, :ip "192.168.10.4", :user-agent "Some UA", :pageviews []}

Modeling a Session

There is nothing new here. We are simply enriching our domain with more types that we will be able to use in an analytics application. The only piece that remains is the User.

You Try It

Now that we have walked through the definition of events, pageviews, and sessions, you have all of the tools that you need to define a data type for users.

Define the "shape" of a user. It should include at least the following: :id, :name, :sessions.
Create a constructor function that can create a user with or without a collection of sessions
For extra credit, create another function called anonymous-user that creates a user that has no id or name

We now have a fairly complete domain defined for our analytics application. Next, we'll explore how we can interact with it using primarily functions from ClojureScript's standard libary. Below is a sample of what some complete data from our domain looks like at this point. It will be helpful to reference this data as we move on.

;; User
{:id 123
 :name "John Anon"
 :sessions [

   ;; Session
   {:start 1464379781618
    :is-active? true
    :ip 127.0.0.1
    :user-agent "some-user-agent"
    :pageviews [

      ;; Pageview
      {:url "some-url"
       :loaded 1464379918936
       :events [

         ;; Event
         {:type :scroll
          :location [403 812]
          :distance 312
          :timestamp 1464380102036}

         ;; Event
         {:type :click
          :location [644 112]
          :target "a.link.about"
          :timestamp 1464380117760}]}]}]}

Sample data for an analytics domain

Working With Associative Data

Most of our analytics data is in the form of maps, which are simple key-value associations. As we have just seen, there is quite a lot of data that can be modeled using only maps, so it stands to reason that ClojureScript would provide good tools for operating on them. This is indeed the case. We will look at several functions that we will keep coming back to when we work with maps: assoc, dissoc, and select-keys. There are more function in the standard library that can be used on maps, but these are the most commonly used and deserve some explanation. The Clojure Cheetsheet is an excellent reference for the functions that we will not be able to cover.

More or Less: Adding and Removing Elements

ClojureScript has a very helpful pair of functions for adding and removing map entries: assoc and dissoc. Unlike setting and deleting JavaScript object properties, assoc and dissoc do not touch the maps that we supply. Instead, they return new maps. By now, we should be familiar with the idea of working with immutable data, but it still takes some getting used to.

Adding Values With `assoc`

Let's consider the session model that we just created. It has identifying information about user's visit to our website. Our new requirement is to add a duration to every session once the user has logged out or left the site. In this case, we just need to add a new entry to the session map - let's call it :duration.

Associating Data Into a Map

This is exactly the case that the assoc function solves: associating some key with a value inside a map. assoc takes a map and a key and value to associate into the map. It can also accept any additional number of keys and values as arguments, and it will associate all of the keys and values in the map.

cljs.user=> (def trail {:name "Bear Creek Trail"
                        :distance 7.5})
#'cljs.user/trail

cljs.user=> (assoc trail :difficulty :moderate)            ;; <1>
{:name "Bear Creek Trail",
 :distance 7.5,
 :difficulty :moderate}

cljs.user=> (assoc trail                                   ;; <2>
                   :difficulty :moderate
                   :location "Colorado"
                   :max-elevation 12800)
{:name "Bear Creek Trail",
 :distance 7.5,
 :difficulty :moderate,
 :location "Colorado",
 :max-elevation 12800}

Adding Entries to a Map

Adding a single entry
Adding multiple entries

With that, we can write a function that, given an end timestamp, will add a :duration entry with the number of seconds in the session:

cljs.user=> (defn with-duration [session end-time]
              (let [duration-in-ms (- end-time (:start session))
                    duration-in-s (.floor js/Math (/ duration-in-ms 1000))]
                (assoc session :duration duration-in-s)))

cljs.user=> (def my-session (session (.now js/Date) true "127.0.0.1" "Some UA"))
#'cljs.user/my-session

;; Wait a few seconds

cljs.user=> (with-duration my-session (.now js/Date))
{:start 1464641029299,
 :is-active? true,
 :ip "127.0.0.1",
 :user-agent "Some UA",
 :pageviews [],
 :duration 14}

Quick Review: Assoc

Is there a difference between (assoc some-map key val) and (conj some-map [key val])?
Does assoc mutate (change) the map that is passed in?

Removing Values With dissoc

Now imagine that we have added a setting where users can request that we not track their IP or user agent, so we will need to remove this data from the map before we send it off to the server. This is exactly the functionality that dissoc gives us: it takes a map and any number of keys to remove from the map, and it returns a new map without the keys we specified. Let's create a function, untracked, that returns a session without these entries:

cljs.user=> (defn untrack [session]
              (dissoc session :ip :user-agent))
#'cljs.user/untrack

cljs.user=> (untrack my-session)
{:start 1464641029299, :is-active? true, :pageviews []}

Quick Review: dissoc

Use dissoc to remove the :region key from this map: {:landmark "Uncompahgre", :region "San Juan Mountains"}
What happens when the map does not contain one or more of the keys that we pass to dissoc, e.g. (dissoc {:temp 212} :color :material :mass)?
Update the with-duration function that we created earlier to remove the :is-active? key from the session.

Refining a Selection With select-keys

Another handy function to have in our toolbox when working with maps is select-keys. It takes a map and a collection of keys to retain, and it returns a new map with only the keys that were passed in. If we had some portion of the application that was only interested in when a session started, whether it was active, and its pageviews, we could use select-keys to narrow down the data to only what we are interested in:

cljs.user=> (select-keys my-session [:start :is-active? :pageviews])
{:start 1464641029299,
 :is-active? true,
 :pageviews []}

You Try It

It is intuitive that ClojureScript considers maps to be associative. Interestingly, vectors are also associative collections that map an integer index to the element at that index:

cljs.user=> (associative? [])
true

Define a vector with several elements at the REPL with
Use get to retrieve the element at a specific index
Use assoc to update the element at a specific index
Try using the merge and dissoc, functions on the vector. Do the results surprise you?

Working With Nested Data

In any but the simplest of programs, we will need to work with nested data at some point. The analytics application that we are considering in this chapter is a good example, since we have events nested inside pageviews, which are in turn nested inside sessions, which themselves are nested inside users. Using only the function we have seen so far would be intractable at best. We will now turn our attention to several functions that allow us to work with nested data.

Drilling Down With get-in

We have seen the get function a number of times for accessing a specific element in a map or a vector. It has a cousin, get-in, that is used for setting values that are nested deeper inside a data structure. Instead of supplying a single key for the value to get out, we supply a sequence of keys that will be looked up in turn. We can think of this sequence as a path to the data that we are interested in. It is like a road map for the computer to follow to locate the data to retrieve. For instance, to get the first pageview of the first session of some user, we could use something like the following:

(get-in user [:sessons 0 :pageviews 0])

Getting Nested Data

This will first look up the :sessions get on the user that we passed in. Next, it will get the first session (at index 0), then it will get the :pageviews key on this session. Finally, it will get the first of the pageviews. Notice that the get-in is really just a convenience for repeated calls to get:

(get
  (get
    (get
      (get user :sessions)                                 ;; <1>
     0)                                                    ;; <2>
   :pageviews)                                             ;; <3>
  0)                                                       ;; <4>

Get the user's sessions
Get the first
Get the pageviews
Get the first

This concept of a path is used commonly in ClojureScript to describe how to "get to" some specific piece of data. An analogy in the JavaScript world would be chained property access on some specific object:

user.sessions[0].pageviews[0];

Getting Nested Data With JavaScript

At first glance, the JavaScript version looks at least as clear as the ClojureScript version - in fact, perhaps a bit clearer. However, one key feature of get-in is that if at any point in the path the next property does not exist, the evaluation will stop, and the whole thing will evaluate to nil. A more accurate JavaScript translation would be the following:

user &&                                                    // <1>
    user.sessions &&
    user.sessions[0] &&
    user.sessions[0].pageviews &&
    user.sessions[0].pageviews[0];                         // <2>

Check every intermediate step that may be undefined
Only get the nested data if every step in the path to it is defined

Quick Review: get-in

Fill in the blank to make this expression true

(= "second"
   (get-in {:tag "ul"
            :children [{:tag "li"
                        :id "first"}
                       {:tag "li"
                        :id "second"}]}
           ...)

What does the following expression evaluate to?

(get-in {} [:does :not :exist])

Setting With assoc-in

Just as get-in is a variation of get that allows for nested data access, assoc-in is a variation of assoc that allows for the setting of nested data. Calling assoc-in is very similar to calling assoc - the difference is that instead of supplying a simple key, we pass in a path to the data that we want to set.

(assoc-in user
          [:sessions 0 :pageviews]                         ;; <1>
          [(pageview "www.learn-cljs.com" 123456 [])])     ;; <2>

Path to the data to update
Value to associate

Quick Review: assoc-in

What is the result of the following:

(assoc-in {:tag "ul"
           :children [{:tag "li"
                       :id "first"}
                      {:tag "li"
                       :id "second"}]}
          [:children 1 :class]
          "last-item")`

What is the result of the following:

(assoc-in {} [:foo :bar :baz] "quux")

Updating With update-in

Now that we have seen how get-in and assoc-in work, it is time to complete our trio of functions for working with nested data with update-in. Like assoc-in, it takes a data structure and a path, but instead of taking a simple value to put into the data structure, it takes a function to apply to the existing item that it finds at the specified path. The entry at this path is then replaced with whatever the function returns. Let's consider a simple example:

cljs.user=> (update-in {:num 1} [:num] inc)
{:num 2}

In this case, we specified that we wanted to operate on the element located at the path [:num] and increment it. This yielded a new map in which the :num key is the increment of the :num key in the original map. In this simple example, we worked with flat data, but the principle is the same for nested data. Going back to the analytics example, let's say that we wanted to add 10px to the x-coordinate of a click event. We could easily accomplish this with a single update-in:

(defn add-to-click-location [click-event]
  (update-in [:location 0] #(+ 10 %)))

When we start building single-page apps with Reagent, we will constantly be making use of update-in, so it is important to make sure that we are comfortable with how to use it.

Quick Review: update-in

What is the result of (update-in {} [:foo :bar] inc)?
Does update-in work with both maps and vectors? Why or why not?

Summary

We covered a lot of ground in this chapter, and we are now able to do quite a bit of data manipulation, including:

Combining maps with merge
Adding and removing single properties with assoc and dissoc
Working with deeply nested data using get-in, assoc-in, and update-in

Between the sequence operations that we covered in the last lesson and the additional operations that we just learned, we can write quite intricate, data-driven programs. Next up, we'll put together all that we have learned about collections and sequences to build a contact list application that keeps its data in localStorage.

Summarizing Data

Andrew Meredith — Thu, 24 Oct 2019 22:25:05 +0000

This article is Lesson 18 of Andrew's book, Learn ClojureScript

So far we have learned some useful operations that we can perform over sequences:

map for transforming each element
filter for selecting certain elements
into for converting between collection types

There is a lot that we can do with just these few functions. In fact, we rarely need to use something like a for loop with these functions at our disposal. In the last lesson, we used these functions to calculate sales tax for items in a shopping cart and filter to only the taxable items. However, consider the scenario where we would like to get the total price of the cart or the average value of items in the cart. In this case, we cannot use map - we want a single value, and map always evaluates to another sequence the same size as the original. Filter is also obviously not what we are looking for. Is there some way to get "summary" data from out of a sequence without resorting to for loops? There are indeed other options, the most common of which is the reduce function.

In this lesson:

Aggregating values over an entire sequence
Writing reducing functions
Simplifying recursive code with reduce

Exercise: Getting cart value

Let's take the same general shopping cart model from the last lesson and try to find the total value of every item in the cart. We will again consider an example of how we could do this imperatively in JavaScript. This time, we are going on a space journey and need to pick up a couple of supplies that we may need on the way:

const cart = [                                             // <1>
    { name: "Tachyon Emitter Array", price: 1099.45 },
    { name: "Dilithium Matrix", price: 2442.00 },
    { name: "Antimatter Chamber Sealant Rings (4)", price: 19.45 },
    { name: "Toothbrushes (2-pack)", price: 8.50 }
];

let total = 0;                                             // <2>
for (let item of cart) {
    total += item.price;
}

console.log(total.toFixed(2)); // "3569.40"

Imperative Shopping Cart Value

Use the same cart data structure that we used in the previous lesson
Calculate a running total with a for loop

The pattern here is easy to see: we have some value - in this case the variable, total that is updated for every element in the cart array. We see and use code like this on a daily basis, so it is not hard to figure out what is going on, but there is unnecessary complexity in the for loop. In ClojureScript, the pattern that we will use is a reduce.

cljs.user=> (def cart                                      ;; <1>
              [{:name "Tachyon Emitter Array" :price 1099.45}
               {:name "Dilithium Matrix" :price 2442.00}
               {:name "Antimatter Chamber Sealant Rings (4)" :price 19.45}
               {:name "Toothbrushes (2-pack)" :price 8.50}])
#'cljs.user/cart

cljs.user=> (defn add-price [total item]                   ;; <2>
              (+ total (:price item)))
#'cljs.user/add-price

cljs.user=> (def total (reduce add-price 0 cart))          ;; <3>
#'cljs.user/total

cljs.user=> (.toFixed total 2)
"3569.40"

Define the cart data structure
Create a reducing function that takes a running total and a new element and yields a new total
Apply this reducing function to the entire cart to extract the total

Understanding Reduce

While the workings of map and filter were evident just looking at some code that uses them, we need to dig a little deeper with reduce. Like map and filter, takes a function and a sequence and returns something. However, reduce also takes an extra parameter, and the function that it takes looks a little different than the functions that we passed to map and filter. For one thing, this function takes two parameters instead of just one. It also does not always return a sequence. So then, how does reduce work?

Reduce commonly takes 3 parameters: an initial value, a reducing function, and a sequence. It first evaluates the reducing function with the initial value and the first element of the sequence as its arguments and takes the evaluated value and passes it into the reducing function again, along with the second item of the collection. It continues evaluating the reducing function for each element in the sequence, feeding the resulting value back into the next call. Finally, reduce itself evaluates to whatever the final call to the reducing function yields. This it quite a lot to wrap our heads around, so let's use a diagram to help us understand visually what is going on.

We can see that reduce keeps building up a value over each evaluation of the reducing function. On the first call, it added the initial value, 0 and the first element of the sequence, 5, to get 5. It then added this value and the next element in the sequence, 10, to get 15. It proceeded through the entire sequence in this manner and yielded the last value, 25, as the final result. Notice that we were able to take a function that does not know how to operate on a sequence (+) and somehow applied it to get the sum of all of the numbers in a sequence.

Imagine someone packing a snowball for a snowball fight. they hold a little bit of snow in one hand and pack on more snow with their other hand. Every time they pack on more snow, the snowball grows larger and larger, until the snowball is to their liking. Reduce operates in a very similar way, "accumulating" more and more data with each pass. For this reason, the first parameter that is passed to the reducing function is often called an accumulator or a memo.

Quick Review

Given the following code:

;; Create a seq of words
(def words (clojure.string/split
            "it was the best of times it was the worst of times"
            " "))

(defn count-words [counts word]
  (update-in counts [word] #(inc (or % 0))))

(def word-counts (reduce count-words {} words))

What is the data type of word-counts?
What is the value of word-counts?

Before trying this code out at the REPL, use a pencil and paper to write out the calls that will be made to the count-words function. If you need to look up documentation on a specific function that is unfamiliar, you can either call (doc function-name) from the REPL or look up the function on clojuredocs.org.

Reduce Use Cases

Reduce is the go-to tool for any case when:

We need to accumulate state while walking over a sequence, and
There is not an existing function in the standard library that does what we want

Imagine that our app tracks user events on a webpage for analytics purposes. We have a list of events that are modeled as a map with a the :event that the user performed and the :timestamp at which the event took place:

(def events [{:event :click, :timestamp 1463889739}
             {:event :typing, :timestamp 1463889745}
             {:event :click, :timestamp 1463889746}
             {:event :click, :timestamp 1463889753}])

Our task is to figure out the longest interval during which the user was idle. To do this, we need to keep track of the longest time between events that we have seen so far and the last timestamp that we have seen. Since there are multiple items that we want to keep track of, we can use a map as the accumulator:

(defn longest-idle-time [events]
  (:max-idle                                               ;; <1>
    (reduce (fn [{:keys [max-idle last-ts]} event]         ;; <2>
              (let [ts (:timestamp event)
                    idle-time (- ts last-ts)]
                {:max-idle (max max-idle idle-time)        ;; <3>
                 :last-ts ts}))
            {:max-idle 0
             :last-ts (:timestamp (first events))}         ;; <4>
            events))))

Since reduce will return a map, retrieve only a single value
Define the reducing function inline
Return a map from the reducing function
Define the initial value as a map

In this case, we want to know the longest interval between events in seconds, but we also need to keep track of when the previous event occurred so that we can calculate the new interval time. Using a map as the accumulator allows us to keep track of as many pieces of state as we need, and as a final step we get only the specific value that we are interested in, discarding the intermediate results that we no longer need.

Try This

Write a function that returns how many times the user clicked something.
Write a function that takes a sequence of events and tells us how many times the user double-clicked, where a double-click is defined as two clicks within the same timestamp. You may assume that the list of events is ordered by timestamp ascending.

Being More Concise

Reduce is a handy function that helps write clear and expressive code, but there are a couple things that we should keep in mind in order to write even more concise code. First, there is a 2-argument version of reduce that we can use in quite a few circumstances:

(reduce reducing-fn vals)

In this case, we do not supply an initial value to pass to reducing-fn. It will instead be called initially with the first two elements in vals. This works very in cases where we are doing something like summing numbers:

cljs.user=> (reduce + [6 7 8])
21

;; (+ 6 7)  => 13
;; (+ 13 8) => 21

Besides saving a few keystrokes, this 2-argument version of reduce is easier to read.

The second tip for making reduce more concise is to try to map or filter values before reducing them. Going back to the shopping list example, if we wanted to get the subtotal of the price of all taxable items, we could perform everything in a single reduce:

(reduce (fn [total item]
          (if (:taxable? item)
            (+ total (:price item))))
        0
        cart)

However, it is usually clearer to have transformation, filtering, and reducing done as separate steps:

(reduce +                                                  ;; <1>
        (map :price                                        ;; <2>
             (filter :taxable? cart)))                     ;; <3>

Sum all values
Of the prices
Of taxable items

With map, filter, and reduce split into their own separate steps, the intention of this code is crystal clear, and it is simple. Each concern is tightly encapsulated, and no piece of code is trying to do too much. Additionally, each piece is more re-usable than if we had put all of the logic in a single reducing function.

Challenge

Reduce is a much more general function than map or filter in fact, map and filter can both be implemented using reduce. Here is an implementation of map:

(defn my-map [xform vals]
  (reduce (fn [new-vals elem]
            (conj new-vals (xform elem)))
            '()
            (reverse vals)))

Following a similar pattern, implement your own version of filter, and call it my-filter.

Summary

Let's consider what we can now do with reduce:

Keep a running total over a sequence
Re-write much recursive code more succinctly
Implement any sequence operation as a reduce

In this lesson, we covered the fundamental reduce function, and we applied it to the problem of summing the price of every item in a shopping cart. Because of its generality and its ability to simplify recursive code, reduce is quite common in ClojureScript code. Now that we know how to work with sequences, we will turn our attention to modeling a domain with ClojureScript collections.

Forem: Andrew Meredith

What are micro frontends anyway?

How to Grow as a Software Engineer - 5 Tips

1. Set Goals

2. Ask Questions

3. Let Yourself Get Stuck

4. Follow Trends

5. Don't Follow Trends

Conclusion

Sorting a Dependency Graph in Go

What is Topological Sorting?

Building a Dependency Graph

Sorting The Graph

So, I wrote a book

Concurrency with core.async

Overview of CSP

Go Blocks as Lightweight Processes

Quick Review

Communicating Over Channels

Sending Values Asynchronously

Alternating Between Channels

Adding Communication

Quick Review

Channels as Values

Summary

Handling Exceptions and Errors

Handling Exceptions with try/catch

Quick Review

Conveying Information

Functional Alternatives to Exceptions

Errors as Values

Conditions and Restarts

Quick Review

Summary

Namespaces and Program Structure

Namespace Declarations

Using Require

Form 1: Simple Namespace

Form 2: Refer

Form 3: Aliased Namespace

Form 4: Aliased Vars

Importing Google Closure Library Classes

Requiring Macros

Grouping Related Functions

Avoiding “Hidden” OOP

Namespaces and the REPL

Summary

Managing State

Atoms

Updating state with swap!

Quick Review

Replacing state with reset!

Observing Change with Watches

Challenge

Transients

Using State Wisely

Guideline #1: Pass atoms explicitly

Guideline #2: Prefer fewer atoms

Summary

Functional Programming Concepts

Composing Behaviour from Small Pieces

Writing Pure Functions

Referential Transparency

Immutable Data

Functional Design Patterns

Constructor

Closure

You Try It

Strategy

Middleware

Quick Review

Summary

Capstone 3 - Contact Book

Data Modeling

Constructing Entities

Quick Review

Defining State Transitions

Quick Review

Creating the UI

UI State

Updating state with `swap!`

Replacing state with `reset!`

Adding Values With `assoc`