Forem: Howard M. Lewis Ship

Datastar Observations

Howard M. Lewis Ship — Sat, 07 Feb 2026 23:35:22 +0000

I've been very impressed, so far, with Datastar, a tiny JavaScript library for front-end work; I've been switching a personal side-project from using Svelte for it's UI to Datastar, and as amazing as Svelte is, Datastar has impressed me more.

Datastar's essential concept is for the client to shift virtually all logic and all markup rendering back to the server; event handlers can succinctly call server endpoints, which return markup, and the markup is morphed into the running DOM. This makes the server-side is the system of record. Datastar has a nice DSL, based on data-* attributes, allowing you to do nearly anything you need to do in the client, declaratively.

Alternately, the server can start an SSE (server sent event) stream and send down markup to morph into the DOM, or JavaScript to execute, over any period of time. For example, my project has a long running process and it was a snap to create a modal progress dialog and keep it updated as the server-side process looped through its inputs.

The mantra of Datastar is to trust the morph and the browser -- it's surprisingly fast to update even when sending a fair bit of content. It feels wasteful to update a whole page just to change a few small things (say, mark a button as disabled) but it works, and it's fast, and it frees you from a nearly all client-side reactive updates (and all the related edge cases and unforseen consequences).

The server side is not bound to any particular language or framework (they have API implementations for Clojure, Java, Python, Ruby, and many others) ... and you could probably write your own API in an afternoon.

I especially like side-stepping the issue of needing more representations of data; the data lives server-side, all that is ever sent to the client is markup. There's no over-the-wire representation, and no parallel client-side data model. All that's ever exposed as endpoints are intentional ones that do work and deliver markup ... in other words, always use-case based, never schema based.

There's a minimal amount of reactive logic in the client, but the essence of moving the logic to server feels like home; Tapestry (way back in 2005) had some similar ideas, but was far more limited (due to many factors, including JavaScript and browser maturity in that time).

I value simplicity, and Datastar looks to fit my needs without doing so much that is magical or hidden. I consider that a big win!

Moving to deps.edn

Howard M. Lewis Ship — Thu, 09 Dec 2021 01:05:34 +0000

I've been using Clojure for ... some time now; I think I started experimenting with it in 2009, possibly earlier. At both Aviso and Walmart I have used, and often fought with, Leiningen, the standard build tool.

Finally, I'm changing over to using clj and deps.edn. As is often the case, the inertia of using the tool you know can get in the way of learning a tool that may just fit your needs better.

deps is another case of Clojure's emphasis on data and simplicity, and on explicitness as a better goal than conciseness.

At its core, deps does two things:

Build the classpath (resolving dependencies and downloading as needed)
Start a Clojure instance and execute a function using the classpath

There's a few bells and whistles built on top of these two essentials, such as aliases to adjust the classpath and the executed function and function arguments, such as to launch a test runner.

Most importantly, though, deps's more narrow focus makes it easier to understand what's going on as it executes, and to build new and customized functionality that every project seems to need.

For Lacinia (and my other open source projects), I had a goal:

Store the version number of the library in one place
Include that version number in generated documentation

In Leiningen, your basic meta-data (defproject) includes the version number right there, accessible to any plugins. With deps, a version number doesn't really make sense ... remember, we're just putting together the classpath and running some Clojure code.

So, if it comes to building and publishing a JAR, you need to implement some project-level build tools.

You do this by defining a :build alias:

  ;; clj -T:build <command>

  :build {:deps {io.github.clojure/tools.build {:git/tag "v0.7.4" :git/sha "ac442da"}
                 slipset/deps-deploy {:mvn/version "0.2.0"}}
          :ns-default build}}

The -T option alias also includes the current directory as a source, so we can create a build.clj file there:

(ns build
  (:require [clojure.tools.build.api :as b]
            [deps-deploy.deps-deploy :as d]))

(def lib 'com.walmartlabs/lacinia)
(def version "1.1-alpha-7")
(def class-dir "target/classes")
(def jar-file (format "target/%s-%s.jar" (name lib) version))
(def copy-srcs ["src" "resources"])

(defn clean
  [_params]
  (b/delete {:path "target"}))

(defn jar
  [_params]
  (let [basis (b/create-basis)]
    (b/write-pom {:class-dir class-dir
                  :lib lib
                  :version version
                  :basis basis
                  :src-dirs ["src"]
                  :resource-dirs ["resources"]})
    (b/copy-dir {:src-dirs copy-srcs
                 :target-dir class-dir})
    (b/jar {:class-dir class-dir
            :jar-file jar-file}))
  (println "Created:" jar-file))

(defn deploy
  [_params]
  (clean nil)
  (jar nil)
  (d/deploy {:installer :remote
             :artifact jar-file
             :pom-file (b/pom-path {:lib lib :class-dir class-dir})
             :sign-releases? true
             :sign-key-id (or (System/getenv "CLOJARS_GPG_ID")
                              (throw (RuntimeException. "CLOJARS_GPG_ID environment variable not set")))}))

Each function here becomes a runnable command, such as clj -T:build clean. Each of these functions accepts a params map and returns it (these are values provided as command arguments, read as Clojure forms, and assembled into a map of keys and values).

Core to all of this is clojure.tools.build.api/create-basis; a basis is a description of the features of the project, including the classpath with source directories and third party dependencies resolved and expanded. Once you have the basis, other build.api commands are used to generate Jar files and deploy to Clojars.

The above build.clj code is good so far ... a bit of boilerplate, but it is reasonably expressive and not too verbose. It's very easy to see how to customize this for different projects.

But back to my challenge - I want to generate documentation with the right version number. Normally, Codox is invoked via an alias defined in deps.edn:

 ;; clj -Xdev:codox

  :codox
  {:extra-deps {codox/codox {:mvn/version "0.10.7"}}
   :exec-fn codox.main/generate-docs
   :exec-args {:metadata {:doc/format :markdown}
               :name "com.walmartlabs/lacinia"
               :version "1.1-alpha-7"
               :description "Clojure-native implementation of GraphQL"}}}

This adds Codox to the classpath along with the project dependencies, and the dependencies of the :dev alias, and invokes codox.main/generate-docs to do the actual work.

Alas, this approach still requires keeping a second copy of the project version synchronized.

Ideally, there should be a way to inject the true version number, from build.clj, into the :exec-args above, but that goes against he grain of deps; the contents of deps.edn are just data; there's no functions to be called to read the version number from somewhere else.

The next best thing is to have a have a codox build command, defined in build.clj, that calls codox.main/generate-docs.

I did a few iterations on that approach but there are problems: the classpath for -T commands does not include the project's classpath, and Codox needs to be able to load namespaces as part of how it organizes and generates the documentation.

Eventually, I came across the following solution:

Build a classpath that includes the main dependencies, and Codox
Build and execute a Java command to run Codox

It ends up looking like this:

(defn codox
  [_params]
  (let [basis (b/create-basis {:extra '{:deps {codox/codox {:mvn/version "0.10.8"}}}
                               ;; This is needed because some of the namespaces
                               ;; rely on optional dependencies provided by :dev
                               :aliases [:dev]})
        expression `(do
                      ((requiring-resolve 'codox.main/generate-docs)
                       {:metadata {:doc/format :markdown}
                        :name ~(str lib)
                        :version ~version
                        :description "Clojure-native implementation of GraphQL"})
                      nil)
        process-params (b/java-command
                         {:basis basis
                          :main "clojure.main"
                          :main-args ["--eval" (pr-str expression)]})]
    (b/process process-params)))

At the top, we build a new basis that includes the project's base dependencies, the :dev alias dependencies, and Codox (and its dependencies). We're going to start a process to run a Java command to execute a Clojure expression - this expression ultimately is what invokes generate-docs - and build.api provides all the heavy lifting related to setting up the classpath and starting processes.

Even though the above code isn't a macro definition, we can still use the Clojure syntax quote (the backtick) to create a snippet of Clojure code that gets included on the command line to the Java process.

So, running clj -T:build codox will invoke the codox function, which in-turn starts a second process running Java with an entirely different classpath.

At the end of the day, we have a short command to generate documentation, and our library's version number appears in just one place, build.clj.

I'm just scratching the surface of what can be accomplished using these techniques, but I'm excited at the possibilities of using real Clojure code to tackle some of our interesting build- and deploy-time requirements.

Debugging Clojure at the REPL using tap>

Howard M. Lewis Ship — Wed, 15 Sep 2021 23:56:26 +0000

Sometimes, old habits die hard. For a long time, I've done a lot of debugging in my Clojure code at the REPL, using prn.

With Clojure, debugging by writing to the console is not as primitive as it may sound; sure, you can fire up the debugger and set a break point, but by injecting a prn at just the right spot, and with just the right data, you often get just exactly the useful data you need to diagnose a problem.

Other times, with meatier data structures, I will break out clojure.pprint/pprint ... but this requires more work, to import the namespace, and means I also have to cleanup the ns declaration.

Now, I noticed a ways back that Clojure 1.10 added the tap> function, and I vaguely knew it would be helpful for this kind of thing; I finally got around to trying it out to debug some hairy NullPointerException bugs in Lacinia.

What does tap> do? By default ... nothing. It's just a function you can pass a value to, but nothing happens when you do (it also returns true).

tap> is hardly useful until you add a tap with add-tap; a tap is any function that takes a single argument.

Invoking tap> will cause the function (or functions, if add-tap is called multiple times) to be invoked asynchronously.

So if you, for example, (add-tap clojure.pprint/pprint) each subsequent call to tap> will be pretty-printed to the console. And the tap> function is in core, so no require needed.

I've found that I don't want to tap> simple values, because single values do not provide much context into whatever problem I'm trying to solve. Instead I build a map that is tapped as a unit:

(defn fn-i-want-to-debug 
  [simple-arg sometimes-nil-but-too-large-to-print-arg]
  (tap> {:in `fn-i-want-to-debug
         :simple simple-arg
         :nil? (nil? sometimes-nil-but-too-large-to-print-arg})
   ...)

Because of course in Clojure, it's always easy to just make a new map on the fly. Here, the backquote (it's the syntax quote) will ensure that the function name is a fully namespace qualified symbol.

As the above example shows, sometime an argument is too large to effectively print but maybe, for your debugging, you just need to know if it is nil or not ... or maybe you want to dissoc some keys, or otherwise prepare the data that gets output. That's fine, that's why REPL oriented development can be faster and better than firing up the debugger.

A debugger can only show you one thing at a time; and even using Cursive, it can be a lot of clicks and a lot of squinting at the screen to see exactly what's going on.

By comparison, console output (via tap>) can be a little history of what's going on in your program leading up to a problem.

Further, nothing is stopping you from using a debugger and tap> together (but you may find that you don't need that nearly as often as you'd think).

tap> is better than straight-up calls to prn or pprint because you can leave the calls to tap> in your code with little or no cost; you can add-tap when you need to ... and in fact, you can also remove-tap when you want to continue developing without the extra output, all without restarting your REPL.

That's pretty classic Clojure for you ... a minimal and practical tool that becomes significantly more useful because Clojure and its REPL are so well wedded together.

Down the rabbit hole with Clojure, defrecord, and macros

Howard M. Lewis Ship — Mon, 16 Aug 2021 23:54:16 +0000

Lisp Programmers Knows The Value Of Everything And The Cost Of Nothing - Alan Perlis

Our team at Walmart are well known for our use of Clojure and GraphQL.

As I've said recently, if you are doing it right, you're creating an orchestration layer, and that's the bread and butter of our team; we're responsible for customer purchase history - your history of in-store transactions and on-line orders at Walmart. Other teams write the front end (web, iOS, and Android) that allow our customers to view their orders, initiate returns, reorder products, and so forth - our job is to facilitate the efforts of those front end teams by exposing all the necessary data, sliced and diced and organized for their convenience.

That means that we need to interact with many (many) back-end systems, then integrate all the data about the customer, their orders, product information, details about returns ... this list goes on and on.

And we have to do this fast. Part of Walmart's home page is based on our service, so we're on a tight performance budget to meet our service level agreements.

A recent cross-team stress test revealed some problems in our service; partly our cluster was too small for the amount of simulated traffic, but at the same time, we also determined (using JDK Mission Control) that we had a fair amount of CPU bound, not I/O bound, performance issues.

Part of this was due to some inefficiencies in Lacinia, since corrected, but we also found other hot-spots specific to our application's code.

The Challenge

One hot-spot in particular was ... interesting. As I mentioned above, our code does a lot of integrating and organizing data from multiple systems; internally, a major piece of that was the "pre-processed order line", a map that contains a flattening of data from many systems around a single line of a single order; we then use that pre-processed data to organize order lines into categories and groups and generate the final GraphQL structured output.

Roman defined a defrecord for this PreprocessedOrderLine, and just using that had some measurable benefits, but also introduced a new hot-spot: constructing the PreprocessedOrderLine record itself.

We ended up with three bad options:

Create a hash map, then call the map->PreprocessedOrderLine constructor function - We're creating a map to avoid the cost of creating a map?
Create an empty PreprocessedOrderLine record, then use assoc to add in the keys and values - We're creating a record then doing 60 mutations on it?
Use the ->PreprocessedOrderLine constructor function, and pass 60 positional arguments into it - Although nominally just a wrapper around new, if there are more than 20 parameters Clojure creates a much less efficient variadic function.

Which leaves us with:

Invoke the constructor directly

But this approach didn't sit well from a maintainability viewpoint; we frequently add new keys (now, new fields) to this record, and have to be very careful at construction time to pass all the parameters to the PreprocessedOrderLine constructor in exactly the right order.

What to do? Well, the nature of problem solving in Clojure is to look at where you are, look at where you want to be, and find the way to bridge that gap.

Bridging That Gap

My thinking was this: What if we had an additional constructor for the record, kv->PreprocessedOrderLine, that would accept key/value pairs (like the hash-map function), but would instead figure out the correct parameters, in the correct order, to invoke the PreprocessedOrderLine instance constructor?

For this to make sense, we'd need to figure out how to map keyword keys to PreprocessedOrderLine constructor argument positions. And for this to not be a new hot-spot, the ordering calculation would have to happen once, at compile time, not on each execution of kv->PreprocessedOrderLine.

Unavoidably, since we're talking about moving what would normally be execution time behavior to compile time, we're talking about writing a macro.

My goal here was to write a new macro, defkvrecord, whose purpose would be to do everything that defrecord does, but then add a third way to construct a record instance, the kv-> prefixed macro.

Here's where we'll end up:

(defkvrecord Point [x y z])
=> #'user/kv->Point

We can define our record type, as normal.

(kv->Point :y 2 :z 3 :x 1)
=> #user.Point{:x 1, :y 2, :z 3}

Evaluating the kv->Point macro results in a new instance of Point, and the order of the keys doesn't matter.

(macroexpand-1 '(kv->Point :y 2))
=> (new user.Point nil 2 nil)

macroexpand-1 shows shows how kv->Point expands into a new constructor call. Unspecified fields receive a nil.

(macroexpand-1 '(kv->Point :y 2 :color :red))
=> (clojure.core/assoc (new user.Point nil 2 nil) :color :red)

When extra keys, not matching a field, are provided, a call to assoc is used to add those extra keys.

(doc kv->Point)
-------------------------
user/kv->Point
([& key-values])
Macro
  Factory function for class user.Point taking keywords and field values.
=> nil

The macro has proper documentation.

It's worthwhile to reflect here that in this scenario, our final goal is to create a macro that defines another custom macro.

If writing a macro is potentially shooting yourself in the foot, what is this? I guess it's building a machine to shoot yourself in the foot, automatically.

The Solution

It's intimidating to jump from nothing to the final implementation of the defkvrecord macro; instead, let's break our thinking down into smaller steps.

First, imagine just writing kv->Point by hand, but as a function:

(defrecord Point [x y z])

(defn kv->Point 
  [& key-values]
  (let [kv-map (apply hash-map key-values)
        params (map kv-map [:x :y :z])]
    (apply new Point. params)))

This is the general idea; but note that this code won't actually work, - I've used apply with new, which won't compile because new is a special form. Let's ignore that for the moment.

What's important is that we go from keys and values, to a map, to a correctly ordered list of constructor parameters.

Next, let's handle the case where there are keys that don't match fields:

(defn kv->Point
  [& key-values]
  (let [field-kws [:x :y :z]
        kv-map (apply hash-map key-values)
        extras (reduce dissoc kv-map field-kws)
        extra-kvs (seq (reduce into [] extras))
        params (mapv kv-map field-kws)
        point (apply new Point. params)]
    (if extra-kvs
      (apply assoc point extra-kvs)
      point)))

Here, extras is a map of any keys that don't match a field; we convert that to a seq of keys and values as extra-kvs and, if there are any, we use apply assoc to add them to the Point record.

At this point, we're ready to build the kv->Point macro:

(defmacro kv->Point
  [& key-values]
  (let [field-kws [:x :y :z]
        kv-map (apply hash-map key-values)
        extras (reduce dissoc kv-map field-kws)
        extra-kvs (reduce into [] extras)
        params (mapv kv-map field-kws)
        base (concat (list 'new user.Point) params)]
    (if (seq extra-kvs)
      (concat (list 'assoc base) extra-kvs)
      base)))

Remember, a macro returns forms that will be evaluated; this is often done using the syntax quote, but it is also valid to just build up the forms using list and concat, as we've done here.

Now, in terms of generalizing from this specific record type to any record type, we need to adapt for:

The correct kv-> macro name
The actual field names
The actual class name

In addition, we want (doc kv->Point) to work correctly.

Knowing how we built to this point, the final code should not be too daunting to understand:

(defmacro defkvrecord
  "Extension to clojure.core/defrecord, but adds an additional macro, kv->RecordType,
  that accepts keyword keys and values and constructs an instance of the record directly,
  via its constructor.

  This is intended for use with records that have a very large number of fields, and
  where creating a map to invoke the record's map-> constructor is a performance hotspot;
  this code is more manageable than attempting to call the ->RecordType positional
  constructor function, which is untenable when there are more than a few fields.

  The macro is of the form:

    (defmacro kv->RecordType [& key-values] ...)

  The keys are keywords, and the values are expressions that will be plugged into the new record
  instance via its constructor (when the keyword matches the name of a field of the record)
  or via an optional call to assoc (after the record has been constructed).

  Any field whose value is not supplied as a key value pair is passed to the constructor as nil."
  [record-name fields & opts+specs]
  (let [strip-ns (fn [sym] 
                   (with-meta (-> sym name symbol) 
                              (meta sym)))
        record-name' (strip-ns record-name)
        kv-symbol (symbol (str "kv->" record-name'))
        ns-part (namespace-munge *ns*)
        classname (symbol (str ns-part "." record-name'))
        field-names (map strip-ns fields)
        field-kws (mapv #(-> % name keyword) fields)]
    `(do
       (defrecord ~record-name' [~@field-names] ~@opts+specs)
       (defmacro ~kv-symbol
         {:doc ~(str "Factory function for class "
                      classname 
                      " taking keywords and field values.")
          :arglists '([& ~'key-values])}
         [& key-values#]
         (let [kv-map# (apply hash-map key-values#)
               extras# (reduce dissoc kv-map# ~field-kws)
               extra-kws# (seq (reduce into [] extras#))
               params# (map kv-map# ~field-kws)
               base# (concat (list 'new ~classname) params#)]
           (if extra-kws#
             (reduce concat (list 'assoc base#) extras#)
             base#))))))

This is the final version of defkvrecord; it's more complicated since we need to go from a list of field symbols to the list of field keywords and we have do some extra work - duplicating code from defrecord's implementation - to work out the class name and other details.

Still, in 27 lines of code, we've added a significant extension to Clojure's built-in record support, and done so without compromising on performance.

In the long run, going from map->PreprocessedOrderLine to kv->PreprocessedOrderLine changed the time to instantiate a single instance from 11.5 microseconds to 104.0 nanoseconds, a speedup of about 99%. Of course, it would be necessary to instantiate hundreds of PreprocessedOrderLines per request to shave even a single millisecond off of the request processing time, which calls into question the usefulness of the entire exercise.

Wrap Up

This was a fun journey because the end result was a case of having your cake and eating it too - we got the improved performance without any penalty in terms of the maintainability of our application's code.

And finally, a bonus tip: when developing macros, always (set! *print-meta* true). You need to see the meta-data on the forms your macros output!

Joker: Flex Your Scripting Muscles In (Almost) Clojure

Howard M. Lewis Ship — Fri, 07 Jun 2019 20:07:08 +0000

If you read enough dev blogs, and have a touch of imposter syndrome,
it starts to look like our field is full of Uber-coders who can constantly
switch gears between, say, writing machine learning code in Python,
complex user interfaces in JavaScript and React, and then jump down and throw together a quick device driver in C. For fun, I imagine these Uber-coders then go home to write retro-games in 6502 assembly.

These people probably exist. I am not one of them. I've coded professionally
in many languages, everything from C, to PL/1, to Java, to Lua. But for day-to-day work, I specialize. I like Clojure. And I'm terrible at switching gears and languages, unlike my mythical Uber-coder.

Clojure is intensely good at what it does: REPL oriented development,
functional thinking, data transformation, and leveraging existing Java libraries.

But you don't write shell scripts in Clojure, even if you are as hopeless at Bash scripting as me.

Not that you don't want to, it's just that startup time is an obstacle: all the JVM startup time, plus all the Clojure startup time, plus all the time to load whatever libraries you need.

It adds up.

If you are working on an application, you may only fire up your REPL once in an entire day, so this startup time is inconsequential. But if you are writing a script ... either a throwaway, or something to help you automate your environment, you don't want to be twiddling your thumbs waiting for that script to load (even a one second wait can feel like forever).

That's why Joker is such a revelation; Joker is an interpreted dialect of Clojure, written in Google's Go, intended just for scripting: we use it internally on my team for all of our automation.

Nearly everything that makes Clojure amazing exists in Joker, including immutable data types, macros, and dynamic typing, and the vast majority of Clojure's core library.

Joker starts up very quickly:

> time joker -e '(println "There and back again")'
There and back again

real    0m0.050s
user    0m0.038s
sys 0m0.017s

That's virtually instantaneous. By comparison, Clojure has a noticable start up time:

> time clj -e '(println "A price must be paid")'
A price must be paid

real    0m1.011s
user    0m1.695s
sys 0m0.124s

Better yet, Joker is batteries included: it has a selection of built-in libraries, that cover all of your typical tasks, including sending HTTP requests, JSON and YAML encoding, basic cryptography ... all the things you need to, say, work with Amazon AWS.

One feature near and dear to me is joker.tools.cli, a port of Clojure's
clojure.tools.cli library. This is first class support for GNU-style command line options.

Using Joker

Let's say you got yourself a sweet job working for the Department of Defense, and wanted to create a script to control a few things at NORAD:

You can name this file norad, and make it executable (chmod u+x norad).

Because we made this a first class script, with properly parsed options, we can ask norad what it does:

> norad --help
norad OPTIONS

  -d, --defcon VALUE  Set DEFCON level
  -v, --version       Show version information
  -h, --help          Show this summary

That's a relief; a sloppy script that expected the user to remember its non-standard options might have changed the DEFCON to null, or launched some missiles or something. Real options can be a safety net.

And even though this script is non-trivial, it's still hella fast:

> time norad --defcon 3
Setting DEFCON to 3

real    0m0.086s
user    0m0.088s
sys 0m0.020s

And as someone who crusades for good feedback from my tools and libraries, it's great to see this:

> norad -d 0
norad OPTIONS

  -d, --defcon VALUE  Set DEFCON level
  -v, --version       Show version information
  -h, --help          Show this summary

Errors:
Failed to validate "-d 0": level must be 1 to 5

... even in a throwaway script. More safety net.

How it Works

That first line:

#!/usr/bin/env joker

That's standard Posix for "find the joker command and run the rest of the file through it". Any additional arguments to the command will be provided in *command-line-args*, a sequence of strings.

Because this is a script, and not a program, there's no main; there's just Clojure forms, evaluated one after another.

The ns form is convenient for bringing in the other built-in libraries that are needed. We gave this namespace the name script, but that's pretty arbitrary.

The next few forms define constants and utility functions needed to parse command line options and perform actions based on them. A real implementation of set-defcon might, for example, send an HTTP request to update the big display in the war room.

The last form, the let block, is where the real work starts; this is where those command line arguments are parsed into options, and then actions are dispatched. This is all very familiar to any Clojure programmer.

But that's it. No main. No classes. No compilation. No CLASSPATH. No overhead. Just your code, in (very nearly) The One True Language, but running instantly.

GraphQL's Secret Super Power

Howard M. Lewis Ship — Fri, 12 Apr 2019 21:43:29 +0000

GraphQL has a certain, almost hidden feature in it, that is of critical importance to me. It's mentioned in the documentation, but I think many people breeze past that page in the excitement of everything else GraphQL has to offer.

Before I reveal what this feature is, let me explain some of my personal history and how it relates.

Back in the early naughts, I created an open-source, server-side Java web framework, Apache Tapestry. I'm proud of many unique features of Tapestry that were forward thinking at the time, even if such features are now considered quite essential. Even so, I often struggled at the "elevator pitch" for why Tapestry was so valuable.

Eventually, I came up with four essential aspects of Tapestry that drove its design and served to highlight what was uniquely useful about it: Simplicity, Consistency, Efficiency, and Feedback.

Simplicity at the time was essentially conciseness and hiding some of the necessary machinery. Thanks to Rich Hickey, my exact definition of simplicity has shifted subsequently.

Consistency concerned being able to use the same techniques in different related areas: in Tapestry, creating a page, a component that can be reused within the same application, or a component that can be reused across applications were virtually identical.

Efficiency concerned the overhead of using the framework; essentially, the beneficial features of Tapestry had to be deliverable with very good performance. In fact, Tapestry (despite doing far more per-request work than many competing frameworks) was always in the top-tier, performance-wise.

And that leaves Feedback, and its rather vague definition. Feedback, to my mind, covers many things within the general job of making the developer's life bearable. One aspect was that Tapestry supported live changes to code and templates, thus streamlining the developer's feedback loop: being able to see a change as soon as you refreshed a page, rather than in tens of seconds or longer (sometimes much, much longer) was quite liberating, even before the ever-present distraction of Twitter threatened to break your concentration while you waited.

But, at its core, Feedback was a commitment throughout the code to catch common errors and report them with as much detail as possible; as well as a built-in detailed exception report page that provided a wealth of information, including stack traces and highlighted source lines from component templates.

The intent was that when anything went wrong, the details needed to quickly diagnose the problem was right in front of you. I spent a lot of time in Tapestry training and conference sessions triggering that exception report and digging through it, then quickly fixing things and continuing to the next step.

So, how does Feedback (as well as Simplicity, Consistency, and Efficiency) relate to GraphQL?

Simplicity: for all that GraphQL tacks a type system on top of queries and JSON, the overall design of GraphQL is deliberately simple. If simplicity is what allows you to easily understanding the behavior of your system, then GraphQL earns high marks.

Consistency: everything essential in implementing a GraphQL schema is a field (that may have arguments). Queries and mutations? Fields on a root Query or Mutation object. Subscriptions? Still a field. Input objects: still looking like fields (without arguments). A single type system defines input objects and field arguments as well as all output values. It's all very consistent.

Efficiency: A gain and a loss here; certainly it's more expensive computationally to parse and execute a GraphQL query than it is to route a request and spew out JSON; likewise, the fact that all GraphQL queries route through a single endpoint can be an impediment to traditional caching approaches.

However, some of these challenges can be mitigated by leveraging server-stored queries (the client sends up the name of a query, not the query itself) and the fact that the client requests only what's actually needed can make a significant difference in actual throughput, especially in terms of downloading data to mobile devices. Our GraphQL integration layers allow most content from backend systems to be filtered out before it goes over the wire; multi-kilobytes JSON payloads full of extraneous data is filtered down to a handful of output values, numbering in the dozens of characters, not thousands.

That leaves Feedback: GraphQL gets high marks here for its introspection capabilities, which make things like GraphiQL possible. But the specific aspect of feedback that prompted me to write this posting was input validation; GraphQL specifically discusses exactly how and when to capture all manner of problems in the GraphQL query, and exactly how those problems are to be reported back to the client. This includes unparsable query documents, references to unknown operations, fields, or arguments, omitting of required arguments, and even cases where the wrong scalar values (or improperly formatted scalar values) are provided.

In each of these cases, GraphQL can provide an error detailing exactly what went wrong, and pinpoint the exact location in the query document related to the failure.

By contrast, I'm working on integrating a piece of our application with another team's JSON-based API. We have to send a fairly large, deeply nested JSON object as the body of the request. If our code omits a field, includes an extra field, or provides any content that isn't exactly what the API expects, we get back the following:

That's it. No context, no indication of what underlying property of the request JSON is at fault, no clue as what to fix or how to proceed. No feedback.

At a very minimum, an equivalent GraphQL API would have identified precisely which field was executing when an error occurred, or which argument of which field had invalid or missing data. The error would be associated with a specific position within the input GraphQL query document. There would be a place for me to start in figuring out what went wrong.

So, that may itself be the secret sauce in GraphQL; it's not just that it's useful (simple, consistent, efficient) when everything is working well, but it's extra-useful, with feedback, when things go wrong.