Forem: Zev Averbach

How To Get Black Working in PyCharm in 2021

Zev Averbach — Fri, 11 Jun 2021 15:58:58 +0000

tl;dr install this plugin and have PyCharm start blackd via a shell script on launch. Reassign the "reformat code" keyboard shortcut to call the plugin.

Immediate Performance Issues

Whilst making my yearly pilgrimage from vim to PyCharm (I usually go back to vim after a bit), I found it less than straightforward to apply the black formatter instead of using PyCharm's built-in "reformat code" settings.

For those who don't know what black is, it's a way to never think about formatting in Python ever again.

After following the Black docs about setting black up in PyCharm, I found the file watcher for it to be incredibly slow, interrupting the flow of my work while I waited for it to format a single file.

So I instead turned off the file watcher and remapped the "reformat code" keyboard shortcut to instead call black on the currently displayed file. This is great, but it also opens a terminal and displays the executed command as well as

Process finished with exit code 0

every single time the code is reformatted. A little research yielded a --quiet parameter to black, but for whatever reason this didn't silence the output, at least not when executed via PyCharm.

Plugin?

Jetbrains IDEs have a pretty decent plugin marketplace and community. Maybe not on par with VSCode by now, but then again there's a lot more (nice) batteries included in Jetbrains IDEs than VSCode.

A quick search yielded BlackConnect, which glues PyCharm to a blackd daemon. It requires installing and running blackd (pip install black[d]), but it's nice and fast and doesn't bring attention to itself. I remapped the "reformat code" keyboard shortcut to invoke BlackConnect.

A Little Bit of Configuration

Making it work on startup requires an extra step, though: I made an on_pycharm_startup.sh file containing a single command blackd, did chmod +x on it, then added it to Tools>Startup Tasks. Boom! Black works quickly and silently when manually called via keyboard shortcut.

A note: BlackConnect has a "reformat on save" option, but it's just as slow as and less reliable than running black from a file watcher. Maybe this will change at some point.

How Do PostCSS + Tailwind Work? Part 1: Shrinking the Build

Zev Averbach — Wed, 25 Nov 2020 13:54:55 +0000

I love when a library or utility has an intuitive API that you can just hit the ground running with, or when it has a massive audience so you can Stack Overflow your way to success. However, at least in web land things can be complex and counterintuitive enough that I'd like surpass "well-crafted StackOverflow search" familiarity with them.

One such technology is Tailwind CSS. I don't fancy myself a designer, and to be honest I've never gone deep with CSS: To be more honest, my best trick is "float: right;" which usually sends me down an increasingly fiddly path. I love Tailwind UI because everything looks so modern right out of the box, so I don't have to fiddle very much.

However, the class lists in Tailwind UI's components are necessarily verbose, and I'm eager to start using Tailwind CSS the recommended way: By ~~building~~ ~~(compiling?)~~ pre-processing my Tailwind-using projects' CSS. My understanding is that this 1) reduces the build size because it only includes the Tailwind classes you use, and 2) allows you to make your components/HTML readable again by abstracting (@apply-ing) many class names into one.

For this post I'm going to focus on reducing the build size of the CSS.

Foundations

Tailwind is a PostCSS plugin, it turns out. The docs point in every direction but its npm page, which puzzled me at first: The reason for this is that it's a tool — a parser — which

...parses CSS into an abstract syntax tree (AST); passes that AST through any number of “plugin” functions; and then converts that AST back into a string, which you can output to a file.
-- It's Time for Everyone to Learn About PostCSS, David Clark

Thus, the plugins are where the action happens.

Experiment #1: `postcss-cli`

Both the PostCSS and Tailwind docs start immediately giving pointers about how to set yourself up in various bundlers. However, I'd like to hold off on that if possible and see if I can run the Tailwind plugin on its own. A brief, down-the-page section in the PostCSS README mentions postcss-cli which can be used like so:

$ postcss --use <plugin-name> [-c options.json] -o main.css css/*.css

First things first:

$ npm install postcss postcss-cli tailwindcss

I don't quite know what sort of content to feed into the Tailwind plugin, but let's try the bare minimum:

/* hi.css */

@tailwind base;
@tailwind components;
@tailwind utilities;

That's from the 'install' docs of Tailwind. As you might guess, @tailwind is not valid CSS, so I'm thinking this will get pre-processed when I do

$ postcss --use tailwindcss -o main.css hi.css

Things are not working as expected on my Mac — it can't find the CLI to invoke it — but I Googled my way to npx postcss --use ..., which does work. Oh, technology.

What this produces is what I assume is the entire codebase of those three "modules" referred to in hi.css (fun fact, @tailwind <thing> is called an "At-rule").

It's nearly 172,000 lines long! And indeed, from experimenting it looks like, as of today "base" is about 540 lines of code, "components" is 220, and "utilities" is 171,000!

It looks like "components" is dedicated exclusively to the class name "container" on various screen sizes and with different size prefixes. "base" doesn't define any classes, just sets some defaults for every conceivable HTML tag ("Preflight", they call it).

Experiment #2: `postcss.config.js`

I'm thinking that if I create this file (from the Tailwind docs again),

// postcss.config.js

module.exports = {
  plugins: {
    tailwindcss: {},
  },
}

maybe I won't have to do --use when using the CLI?

$ postcss -o main.css hi.css

Yes! This prevents the need for specifying which plugin(s) to invoke.

Shrink the Build

Going back to those 172,000 lines of 'rendered' Tailwind CSS code: Let's get Tailwind's help in whittling them down.

First stop, $ npx tailwindcss-cli@latest init which creates a tailwind.config.js. According to the docs we have to make this alteration to get unused CSS purged:

// tailwind.config.js

module.exports = {
  purge: [
    './src/**/*.html',
    './src/**/*.svelte',
  ],
  theme: {},
  variants: {},
  plugins: [],
}

You can add *.jsx, etc. if you're working with other frameworks.

Experiment #3: How Small Will It Get?

First let's run the purge with no HTML/Svelte files whatsoever:

$ export NODE_ENV=production && npx tailwindcss-cli@latest build src/hi.css -o build/main.css

Woot! This produces a main.css of only about 370 lines, which seems to be a subset of @reset.

Experiment #4: Adding One Line To The Build

Now I'll add an index.html that uses one Tailwind class:

<!--src/index.html-->

<div class="bg-blue-400"></div>

Running build produces another compact main.css, but this time it seems to include all of @base as well as .bg-blue-400. main.css is only 11kb, whereas the non-purged version from the beginning is 3.5mb!

Next Time

In the next post we'll explore how to customize and extract classes in Tailwind, as well as integrating it with a bundler (probably Snowpack!).

Patch Your Idols: Making OSS Work the Way Your Team Does

Zev Averbach — Wed, 07 Oct 2020 11:47:14 +0000

When a critical software package your team uses 1) is open source and 2) isn't working quite the way you want or expect it to, patch it so it does! Your team will be more productive and you'll get more familiarity with how it works, which could pay dividends down the line.

Case Study: Apache Airflow

One of my responsibilities at work is to help my fellow data engineers be more efficient at their work building and maintaining ETL pipelines. The tool we use for this is Apache Airflow, which enables us to define and schedule pipelines. When a pipeline has been created, it appears on a list of "DAGs" (directed acyclic graphs) like this:

In our group this list is significantly longer, long enough to paginate (100 DAGs per page by default in Airflow), and some of them are switched to "off" (that first column).

Challenges + Patches

Challenge: Where Is My Running DAG?

If you see that bright green circle under "DAG Runs" for example_branch_dop_operator_v3, that indicates there are three copies of that DAG running at this moment. Imagine that you're interested in finding and expanding on a running DAG among a list of hundreds: It means either manually perusing multiple pages of DAGs looking for those bright green circles or doing a search. For our team the chances that what you've come to the Airflow GUI looking for is, indeed, a currently running DAG.

Solution: Sort the DAGs by "Running"

This is where we dive into the Airflow code base. I very naughtily did this, as well as the ensuing changes, in the site-packages/airflow directory on our live Airflow webserver. Sorry/not sorry!

Firstly, I right-clicked on one of those bright green "running DAG" circles and did "inspect element". This led me eventually to some jQuery which renders the color and number for each of the three circles in a DAG's "DAG Runs", each based on a Javascript object with keys like "state" and "count". This object is obtained via a fetch to Airflow's back end, namely an endpoint called /dag_stats.

We'll want to sort our DAGs by something in the response to that fetch call. However, the /dag_stats endpoint is called after the DAG list page is loaded, so we'll have to do a little surgery:

First let's find this endpoint in Airflow's code base: $ grep -r dag_stats airflow/ leads us here. payload near the end of that function looks to be a dictionary whose keys are dag_ids and whose values are lists of dictionaries, each dictionary containing (among other things) the "state", "count", and "color" for each of the possible DAG states. We know there are three possible DAG states given the three circles in the GUI.

So, how to get that DAG status dictionary before rendering all the DAGs so we can sort by it? Well, that jQuery I tracked down earlier is in a template file called dags.html. Whatever endpoint renders that template is what we're after. Sure enough, in the same views.py module as /dag_stats we have this (the "/" endpoint). In it there's a complex, parametrized-via-config SQLAlchemy query to get and filter and sort the DAGs that will be passed into the template. In lieu of adding a complex (though surely more efficient) multi-table query in the body of this endpoint, let's just copy all the code from the /dag_stats endpoint and sort the DAGs by it just before rendering:


dags = sorted(dags, key=lambda dag: (-payload[dag.dag_id][1]['count'], dag.dag_id))

That [1] index is the "running" state dictionary, as surmised (speculated!) by the order in which it appears on the GUI.

This code change -- a single line plus some re-purposing of existing code -- causes all running DAGs to be listed first!

Challenge: When Did My DAG Actually Last Run??

This is a weird one: For some reason Airflow shows the last execution_date in the "Last Run" column, and if you want to know the actual last run date/time you have to hover over that little "info" span next to the date/time displayed there. In our team's case this causes many, if not most, DAGs to show a "last run" date that's 24 hours earlier than the truth. This leads to a lot of head-scratching and unnecessary attempts at debugging (DAG code, the Airflow scheduler, etc.) -- a poor user experience, and a waste of time!

Let's swap these values, shall we? Searching through dags.html we see a nicely commented table cell for "Last Run": our target. last_run.start_date and last_run.execution_date look to be the two values we want to swap, in addition to their swapping their labels.

This solves it! The actual last run date displays, as we all expected to happen in the first place.

Challenge: Do You Speak CRON?

The way DAGs are scheduled is by supplying a CRON string to the argument schedule_interval. The way the Airflow GUI displays a DAG's schedule is by simply rendering the CRON string. However, not everyone "speaks CRON" on our team -- I'm one of those people, I have to Google these strings pretty much every time. What's more, I can imagine a scenario where a non-engineer peeks in at the DAGs and would gather exactly nothing from, say, 0 0 * * *.

Surely there's some human-friendlier way to render these? A quick Google search yields Cron Descriptor , "a Python library that converts cron expressions into human readable strings." If we plunk that into our environment and import it (from cron_descriptor import get_description), we've nearly reached a readable solution:


def convert_schedule_to_natural_language(cronstring):
    if dag.schedule_interval and not dag.schedule_interval.startswith('@'):
        try:
            dag.schedule_interval = get_description(dag.schedule_interval)
        except cron_descriptor.Exception.FormatException:
            pass
    return dag

...

dags = [convert_schedule_to_natural_language(dag) for dag in dags]

...
return self.render(...

Since @hourly and such are already human-readable, we leave those alone. We then convert the schedule_interval of every DAG before rendering it. As a result, the first two intervals from the above screenshot are translated into 'At 00:00 AM' and 'Every minute', respectively. A little weird, but more or less readable!

Conclusion: Optimize the Widely Used Tools!

All told, these changes took about 75 minutes, including finding the package for translating CRON strings into natural language. My estimate is that we'll earn those 75 minutes back across the team in the next 48 hours. After that, it's all increased-team-productivity gravy. My advice:

1) Whenever you spot an opportunity like this in a tool that more than a few people use regularly, and

2) it will take an afternoon or less to augment,

It's a golden opportunity; take it!

How Does Svelte Actually Work? part 2

Zev Averbach — Mon, 09 Dec 2019 15:49:28 +0000

Here's part 1:

How Does Svelte Actually Work? part 1

Zev Averbach ・ Dec 5 '19 ・ 11 min read

#svelte #javascript

and here's the repository for what we've done so far:

zevaverbach / how-svelte-works

Contains the markdown and code for the blog series.

(hint: we have barely changed Svelte's sample app).

Episode Recap

On the previous episode of HDSAW we took a bewildering ride through the 400 or so lines of vanilla JavaScript the Svelte compiler produces from the sample app. Here's that vanilla JS for your perusal, and see below for an updated version.

What We've Learned So Far

When we run npm run dev to compile the sample app, Rollup takes the contents of main.js and runs the Svelte compiler on it. This produces a file called bundle.js in the build directory.

bundle.js defines and immediately calls an anonymous function, creating an instance of App called app. The code which runs inside the constructor of App first renders the Svelte app, then binds a small handful of methods to the instance. These methods can be used to get and set the state of the app, including by the user in the browser's JavaScript console!

Outstanding Questions

1) Is it a security problem that the user can directly manipulate the app's state from a browser's JavaScript console?
2) What is the difference between app.$set and app.$inject_state, if any?
3) How does bundle.js change with increasing app complexity? Multiple components, for example, or dynamically re-rendering props/state.
4) What is __svelte_meta for?
5) Where and when does mount actually get called?
6) Can dirty ever contain anything besides a single integer? In other words, are elements updated one after the next, or can update sometimes operate on more than one element at a run?
7) When are components and elements destroyed? Is Svelte as efficient about unnecessary re-renders as billed?
8) What are the setters and getters on App for and why are they implemented the way they are?
9) How does all this fit together? Asked another way, is it possible to have a basic understanding of how a web framework we use actually works?

We'll try to get through a few of these today, and then I'm sure more questions will emerge as we go.

A Skinnier Bundle

This time around we're going to generate the build.js using npm run build, so as to not distract ourselves with the events that get emitted in dev mode. First, though, let's tell Rollup not to minify that file so we can actually read it:

// ./rollup.config.js
...
        // In dev mode, call `npm run start` once
        // the bundle has been generated
        !production && serve(),

        // Watch the `public` directory and refresh the
        // browser on changes when not in production
        !production && livereload('public'),

        // If we're building for production (npm run build
        // instead of npm run dev), minify
        // production && terser() 👈👈👈 comment this out
    ],
    watch: {
...

npm run build produces this really quite svelte code, which is now only about 300 lines!:

// src/bundle.js
var app = (function () {
    'use strict';

    function noop() { }
    function run(fn) {
        return fn();
    }
    function blank_object() {
        return Object.create(null);
    }
    function run_all(fns) {
        fns.forEach(run);
    }
    function is_function(thing) {
        return typeof thing === 'function';
    }
    function safe_not_equal(a, b) {
        return a != a ? b == b : a !== b || ((a && typeof a === 'object') || typeof a === 'function');
    }

    function append(target, node) {
        target.appendChild(node);
    }
    function insert(target, node, anchor) {
        target.insertBefore(node, anchor || null);
    }
    function detach(node) {
        node.parentNode.removeChild(node);
    }
    function element(name) {
        return document.createElement(name);
    }
    function text(data) {
        return document.createTextNode(data);
    }
    function space() {
        return text(' ');
    }
    function attr(node, attribute, value) {
        if (value == null)
            node.removeAttribute(attribute);
        else if (node.getAttribute(attribute) !== value)
            node.setAttribute(attribute, value);
    }
    function children(element) {
        return Array.from(element.childNodes);
    }
    function set_data(text, data) {
        data = '' + data;
        if (text.data !== data)
            text.data = data;
    }

    let current_component;
    function set_current_component(component) {
        current_component = component;
    }

    const dirty_components = [];
    const binding_callbacks = [];
    const render_callbacks = [];
    const flush_callbacks = [];
    const resolved_promise = Promise.resolve();
    let update_scheduled = false;
    function schedule_update() {
        if (!update_scheduled) {
            update_scheduled = true;
            resolved_promise.then(flush);
        }
    }
    function add_render_callback(fn) {
        render_callbacks.push(fn);
    }
    function flush() {
        const seen_callbacks = new Set();
        do {
            // first, call beforeUpdate functions
            // and update components
            while (dirty_components.length) {
                const component = dirty_components.shift();
                set_current_component(component);
                update(component.$$);
            }
            while (binding_callbacks.length)
                binding_callbacks.pop()();
            // then, once components are updated, call
            // afterUpdate functions. This may cause
            // subsequent updates...
            for (let i = 0; i < render_callbacks.length; i += 1) {
                const callback = render_callbacks[i];
                if (!seen_callbacks.has(callback)) {
                    callback();
                    // ...so guard against infinite loops
                    seen_callbacks.add(callback);
                }
            }
            render_callbacks.length = 0;
        } while (dirty_components.length);
        while (flush_callbacks.length) {
            flush_callbacks.pop()();
        }
        update_scheduled = false;
    }
    function update($$) {
        if ($$.fragment !== null) {
            $$.update();
            run_all($$.before_update);
            $$.fragment && $$.fragment.p($$.ctx, $$.dirty);
            $$.dirty = [-1];
            $$.after_update.forEach(add_render_callback);
        }
    }
    const outroing = new Set();
    function transition_in(block, local) {
        if (block && block.i) {
            outroing.delete(block);
            block.i(local);
        }
    }
    function mount_component(component, target, anchor) {
        const { fragment, on_mount, on_destroy, after_update } = component.$$;
        fragment && fragment.m(target, anchor);
        // onMount happens before the initial afterUpdate
        add_render_callback(() => {
            const new_on_destroy = on_mount.map(run).filter(is_function);
            if (on_destroy) {
                on_destroy.push(...new_on_destroy);
            }
            else {
                // Edge case - component was destroyed immediately,
                // most likely as a result of a binding initialising
                run_all(new_on_destroy);
            }
            component.$$.on_mount = [];
        });
        after_update.forEach(add_render_callback);
    }
    function destroy_component(component, detaching) {
        const $$ = component.$$;
        if ($$.fragment !== null) {
            run_all($$.on_destroy);
            $$.fragment && $$.fragment.d(detaching);
            // TODO null out other refs, including component.$$ (but need to
            // preserve final state?)
            $$.on_destroy = $$.fragment = null;
            $$.ctx = [];
        }
    }
    function make_dirty(component, i) {
        if (component.$$.dirty[0] === -1) {
            dirty_components.push(component);
            schedule_update();
            component.$$.dirty.fill(0);
        }
        component.$$.dirty[(i / 31) | 0] |= (1 << (i % 31));
    }
    function init(component, options, instance, create_fragment, not_equal, props, dirty = [-1]) {
        const parent_component = current_component;
        set_current_component(component);
        const prop_values = options.props || {};
        const $$ = component.$$ = {
            fragment: null,
            ctx: null,
            // state
            props,
            update: noop,
            not_equal,
            bound: blank_object(),
            // lifecycle
            on_mount: [],
            on_destroy: [],
            before_update: [],
            after_update: [],
            context: new Map(parent_component ? parent_component.$$.context : []),
            // everything else
            callbacks: blank_object(),
            dirty
        };
        let ready = false;
        $$.ctx = instance
            ? instance(component, prop_values, (i, ret, value = ret) => {
                if ($$.ctx && not_equal($$.ctx[i], $$.ctx[i] = value)) {
                    if ($$.bound[i])
                        $$.bound[i](value);
                    if (ready)
                        make_dirty(component, i);
                }
                return ret;
            })
            : [];
        $$.update();
        ready = true;
        run_all($$.before_update);
        // `false` as a special case of no DOM component
        $$.fragment = create_fragment ? create_fragment($$.ctx) : false;
        if (options.target) {
            if (options.hydrate) {
                // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
                $$.fragment && $$.fragment.l(children(options.target));
            }
            else {
                // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
                $$.fragment && $$.fragment.c();
            }
            if (options.intro)
                transition_in(component.$$.fragment);
            mount_component(component, options.target, options.anchor);
            flush();
        }
        set_current_component(parent_component);
    }
    class SvelteComponent {
        $destroy() {
            destroy_component(this, 1);
            this.$destroy = noop;
        }
        $on(type, callback) {
            const callbacks = (this.$$.callbacks[type] || (this.$$.callbacks[type] = []));
            callbacks.push(callback);
            return () => {
                const index = callbacks.indexOf(callback);
                if (index !== -1)
                    callbacks.splice(index, 1);
            };
        }
        $set() {
            // overridden by instance, if it has props
        }
    }

    /* src/App.svelte generated by Svelte v3.16.0 */

    function create_fragment(ctx) {
        let main;
        let h1;
        let t0;
        let t1;
        let t2;
        let t3;
        let p0;
        let t4;
        let t5;
        let t6;
        let t7;
        let p1;

        return {
            c() {
                main = element("main");
                h1 = element("h1");
                t0 = text("Hello ");
                t1 = text(/*name*/ ctx[0]);
                t2 = text("!");
                t3 = space();
                p0 = element("p");
                t4 = text("Your lucky number is ");
                t5 = text(/*number*/ ctx[1]);
                t6 = text(".");
                t7 = space();
                p1 = element("p");
                p1.innerHTML = `Visit the <a href="https://svelte.dev/tutorial">Svelte tutorial</a> to learn how to build Svelte apps.`;
                attr(h1, "class", "svelte-1tky8bj");
                attr(main, "class", "svelte-1tky8bj");
            },
            m(target, anchor) {
                insert(target, main, anchor);
                append(main, h1);
                append(h1, t0);
                append(h1, t1);
                append(h1, t2);
                append(main, t3);
                append(main, p0);
                append(p0, t4);
                append(p0, t5);
                append(p0, t6);
                append(main, t7);
                append(main, p1);
            },
            p(ctx, [dirty]) {
                if (dirty & /*name*/ 1) set_data(t1, /*name*/ ctx[0]);
                if (dirty & /*number*/ 2) set_data(t5, /*number*/ ctx[1]);
            },
            i: noop,
            o: noop,
            d(detaching) {
                if (detaching) detach(main);
            }
        };
    }

    function instance($$self, $$props, $$invalidate) {
        let { name } = $$props;
        let { number } = $$props;

        $$self.$set = $$props => {
            if ("name" in $$props) $$invalidate(0, name = $$props.name);
            if ("number" in $$props) $$invalidate(1, number = $$props.number);
        };

        return [name, number];
    }

    class App extends SvelteComponent {
        constructor(options) {
            super();
            init(this, options, instance, create_fragment, safe_not_equal, { name: 0, number: 1 });
        }
    }

    const app = new App({
        target: document.body,
        props: {
            name: 'World',
            number: 42
        }
    });

    return app;

}());
//# sourceMappingURL=bundle.js.map

This file, bundle.js, is what all of the following code blocks refer to, unless otherwise noted.

Q: Where are the elements created in `create_fragment` mounted?

Here, in init:

...
if (options.target) {
    ...
    👇👇👇👇
    mount_component(component, options.target, options.anchor);
    👆👆👆👆
    flush();
    ...
}

And mount_component, in the first two lines of its definition, destructures fragment from component.$$ then calls fragment.m:

function mount_component(component, target, anchor) {
    const { fragment, on_mount, on_destroy, after_update } = component.$$;
    fragment && fragment.m(target, anchor);
...

As a reminder from part 1, m stands for "mount" (it was actually called that when compiling in dev mode), and it takes the elements created in fragment.c and adds them to the DOM with eminently readable and grokkable JS:

m(target, anchor) {
    // these 👇👇👇 three parameters are equivalent to `document.body`, <main/>, and `undefined`
    insert(target, main, anchor);
    append(main, h1);
    append(h1, t0);
    append(h1, t1);
    append(h1, t2);
    append(main, t3);
    append(main, p0);
    append(p0, t4);
    append(p0, t5);
    append(p0, t6);
    append(main, t7);
    append(main, p1);
},
...
function append(target, node) {
    target.appendChild(node);
}

function insert(target, node, anchor) {
    target.insertBefore(node, anchor || null);
}

Reminder: This is one of the things that makes Svelte great, and unique among popular frameworks: The abstractions are not so deep or complex; so far, we are never far from document.doSomething operations.

Disappearing Questions

Now that we've left dev mode for our explorations, it seems that a few outstanding questions have evaporated:

What is `__svelte_meta` for?

It doesn't appear in bundle.js in production mode! It's for dev tooling, must be: "Error X on line Y, character Z", and so on.

Mystery solved! 🕵

What is the difference between `app.$set` and `app.$inject_state`, if any?

Well, their implementations are identical,

// version of bundle.js in dev mode
function instance($$self, $$props, $$invalidate) {
    ... 
    $$self.$set = $$props => {
        if ("name" in $$props) $$invalidate(0, name = $$props.name);
        if ("number" in $$props) $$invalidate(1, number = $$props.number);
    };
    $$self.$inject_state = $$props => {
        if ("name" in $$props) $$invalidate(0, name = $$props.name);
        if ("number" in $$props) $$invalidate(1, number = $$props.number);
    };
    ...
}

annnnnddd... $inject_state doesn't even get defined in the production build,

// production build of bundle.js
function instance($$self, $$props, $$invalidate) {
    let { name } = $$props;
    let { number } = $$props;

    $$self.$set = $$props => {
        if ("name" in $$props) $$invalidate(0, name = $$props.name);
        if ("number" in $$props) $$invalidate(1, number = $$props.number);
    };

    return [name, number];
}

so let's assume it's a very careful piece of naming to distinguish what dev tools use to set state, as compared with how state is set internally in the app.

Turns out, yep!:

Svelte

@sveltejs

@zevav That's another dev mode hook, used for hot module reloading

18:53 PM - 06 Dec 2019

Finally, it looks like the setter and getter methods on App don't appear in production mode. We can therefore scratch that question off the list, as we're not concerned with the dev tooling part of the generated code for now.

The `$$invalidate` Callback

In the instance function (👆👆👆 much shorter in prod), the $set method is bound to app. Presumably this is the only way prop updates propagate through the app, so it'll behoove us to understand how it's defined in the calling code:

...
function init(component, options, instance, create_fragment, not_equal, props, dirty = [-1]) {
...
    instance(component, prop_values, (i, ret, value = ret) => {
        if ($$.ctx && not_equal($$.ctx[i], $$.ctx[i] = value)) {
            if ($$.bound[i])
                $$.bound[i](value);
            if (ready)
                make_dirty(component, i);
        }
        return ret;
    })

This is hard to read, so let's write it as a normal, somewhat simplified, non-anonymous function:

function invalidateProp(i, ret, value = ret) {
    const currentVal = $$.ctx[i]
    const newVal = value
    if (currentVal !== newVal) {
        // do some things with `$$.bound` and `ready` that will never trigger in the current app
    }
    $$.ctx[i] = newVal
    return ret
}

So if invalidateProp is called like so,

invalidateProp(0, name = "Whirl");

the only things that will happen are

1) $$.ctx[0] will equal "Whirl"
2) "Whirl" will be returned.
3) Nothing anywhere will consume that return value in the current app.

Hypothesis #3: `$$.bound` and `ready` Will Come Into Play If There's Some Interactive UI

Furthermore, I'm thinking $$invalidate's return value might actually get used somewhere. There's plenty more to explore ahead of this test, but let's forge ahead and see what magic the compiler has for us.

Testing Hypothesis #3

Let's make a button to increment number:

<!-- src/App.svelte -->
<script>
    export let name;
    export let number;
</script>

<main>
    <h1>Hello {name}!</h1>
    <p>Your lucky number is {number}.</p>
    <p>Visit the <a href="https://svelte.dev/tutorial">Svelte tutorial</a> to learn how to build Svelte apps.</p>
    <!-- 👇👇👇 new -->
    <button on:click={() => number += 1}>
        Make Number More Bigger
    </button>
</main>

Isn't Svelte so beautifully straightforward and succinct? 😍😍😍

Then, npm run build. To view the app, run python3 -m http.server from the app folder, then navigate to http://localhost:8000. Click that beautiful button and watch your lucky number mutate!

Before hypothesis validating, bundle.js has some new toys for us! Here are the new lines in the resulting bundle.js:

function listen(node, event, handler, options) {
    node.addEventListener(event, handler, options);
    return () => node.removeEventListener(event, handler, options);

c() {
    ...
    t11 = space();
    button = element("button");
    button.textContent = "Make Number More Bigger";
    ...
    dispose = listen(button, "click", /*click_handler*/ ctx[2]);
    // and m() mounts 👆👆👆 them
}

d(detaching) {
    if (detaching) detach(main);
    dispose();
}

function instance($$self, $$props, $$invalidate) {
    ...
    const click_handler = () => $$invalidate(0, number += 1);
    ...
    return [number, name, click_handler];
}

Running through the changes, there's nothing shocking. The button is added, its click handler is added to the DOM, and the click handler is set up to be removed if App is detached. click_handler is added to $$.ctx.

I don't see any of my hypotheses proving out here! $$.bound remains an empty object, $$invalidate's return value remains un-consumed, and there's still only one spot where ready is set to true, and that's after instance is called.

Sidenote: There was one other change, and it's really weird:

class App extends SvelteComponent {
    constructor(options) {
        super();
        // 👇👇👇 before 
        // init(this, options, instance, create_fragment, safe_not_equal, { name: 0, number: 1 });
        // 👇👇👇 after 
        init(this, options, instance, create_fragment, safe_not_equal, { name: 1, number: 0 });
    }
}

The props have inexplicably swapped their indices. What madness is this?! Restraining myself from tweeting at Svelte about this one... for now. Maybe it will explain itself as we go.

The Most Basic Knowledge We're Missing

I still don't know what actually triggers the function update to run!

As a reminder, we studied update in part 1 because it was the only place where $$.fragment—the object containing the methods for creating, destroying, and updating the rendered elements—was updated. As a further reminder, the key line in update calls $$.fragment.p($$.ctx, $$.dirty), fragment.p being short for fragment.update: It checks the value of dirty and potentially updates the node identified by dirty, if the provided new value differs from the node's current one.

...
function update($$) {
    if ($$.fragment !== null) {
        $$.update();
        run_all($$.before_update);
        $$.fragment && $$.fragment.p($$.ctx, $$.dirty);
        $$.dirty = [-1];
        $$.after_update.forEach(add_render_callback);
    }
}
...
function create_fragment(ctx) {
    ...
    // === $$.fragment.p
    p(ctx, [dirty]) {
        if (dirty & /*name*/ 2) set_data(t1, /*name*/ ctx[1]);
        if (dirty & /*number*/ 1) set_data(t5, /*number*/ ctx[0]);
    },
...
}

Conveniently, the function update (distinct from the method $$.fragment.p) is called in only one place:

function flush() {
    const seen_callbacks = new Set();
    do {
        // first, call beforeUpdate functions
        // and update components
        while (dirty_components.length) {
            const component = dirty_components.shift();
            set_current_component(component);
            update(component.$$);
        }
        while (binding_callbacks.length)
            binding_callbacks.pop()();
        // then, once components are updated, call
        // afterUpdate functions. This may cause
        // subsequent updates...
        for (let i = 0; i < render_callbacks.length; i += 1) {
            const callback = render_callbacks[i];
            if (!seen_callbacks.has(callback)) {
                callback();
                // ...so guard against infinite loops
                seen_callbacks.add(callback);
            }
        }
        render_callbacks.length = 0;
    } while (dirty_components.length);
    while (flush_callbacks.length) {
        flush_callbacks.pop()();
    }
    update_scheduled = false;
}

There's a lot here, but as with much of the code generated by Svelte with a basic app, we don't have to concern ourselves with any of the callback-related parts. This is the relevant section:

...
while (dirty_components.length) {
    const component = dirty_components.shift();
    set_current_component(component);
    update(component.$$);
...

This is the first time we're dealing with dirty_components: Where is this array modified? Why, in the aptly named make_dirty function.

function make_dirty(component, i) {
    if (component.$$.dirty[0] === -1) {
        dirty_components.push(component);
        schedule_update();
        component.$$.dirty.fill(0);
    }
    component.$$.dirty[(i / 31) | 0] |= (1 << (i % 31));
}

And where is make_dirty called?

function init(component, options, instance, create_fragment, not_equal, props, dirty = [-1]) {
    ...
    let ready = false;
    $$.ctx = instance
        ? instance(component, prop_values, (i, ret, value = ret) => {
            if ($$.ctx && not_equal($$.ctx[i], $$.ctx[i] = value)) {
                if ($$.bound[i])
                    $$.bound[i](value);
                if (ready)
                    make_dirty(component, i);
            }
            return ret;
        })
        : [];
    $$.update();
    ready = true;
...
}

This is a bit confusing, though, because make_dirty is only called if ready is true, and ready appears to be false when the $$invalidate callback is defined.

𝒵𝑒𝓋 𝒜𝓋𝑒𝓇𝒷𝒶𝒸𝒽

@zevav

I have a little JS puzzle for you, Twitter:

08:30 AM - 09 Dec 2019

~~For now we're going to skate right past the scope monster and accept the mystery of ready's truthiness.~~

Thank you to my fellow Recurse Center alum Thomas Ballinger for demystifying this:

Thomas Ballinger

@ballingt

@zevav @sveltejs apologies if I'm misunderstanding, are you on board with

let a = 0;
function foo() {
return a;
}
a = 1;
foo()
// evaluates to 1

14:05 PM - 09 Dec 2019

Yup, it was a scope monster, or to put it another way, "scope". To spell it out, that 👆👆👆 ready = true takes effect for the ready inside the anonymous function's body, well before it's called as $$invalidate.

Here's more clarification from Svelte themselves:

Svelte

@sveltejs

@zevav ready will be false if an invalidation happens during initialisation

23:38 PM - 09 Dec 2019

🤔 Methinks this will make more sense when I'm older. I mean, in part X of our little adventure...

Okay then, ready is true for our purposes, which means when invalidateProp is called, make_dirty gets called with the App instance for component and the index of whatever prop has been modified. So, something like make_dirty(App, 1).

The function make_dirty

1) adds the component (App instance here) to an array called dirty_components
2) does something confusing in schedule_update, but, importantly, calls flush
3) updates component.$$.dirty

function make_dirty(component, i) {
    ...
    dirty_components.push(component);
    schedule_update();
    component.$$.dirty.fill(0);
    ...
}
...
function schedule_update() {
    ...
    resolved_promise.then(flush)
}

So we've found where dirty_components is mutated, and as a bonus, where flush gets called (schedule_update).

We'll study the Array.fill later, but returning to flush's definition, it appears that dirty_components is being used as a queue:

const component = dirty_components.shift();
set_current_component(component);
update(component.$$);

The meaning of current_component doesn't seem important in this early moment when there's only a single component in our app. This makes the only interesting code here the call to update, which as we already know triggers set_data on any relevant elements.

The Last Piece

We nearly have the full picture of how prop changes make their way through bundle.js. There is a curveball with respect to our current understanding, though:

What is `$set` For???

app.$set is never called in our app: It appears to be a debugging utility, or maybe a hook for external tools. So this function isn't part of the pipeline of functions handling prop changes. Sorry about that; one day the knowledge we built about $set will have its moment, I assure you!

So Where? Sneaky Succinctness

It took me a bit of console.log-ing to realize where the pipeline begins:

    $$.ctx = instance
        ? instance(component, prop_values, (i, ret, value = ret) => {
            //                             right here 👇👇👇
            if ($$.ctx && not_equal($$.ctx[i], $$.ctx[i] = value)) {
                if ($$.bound[i])
                    $$.bound[i](value);
                if (ready)
                    make_dirty(component, i);
            }
            return ret;
        })
        : [];

It isn't clear to me at all why this anonymous function runs more than once (on App instantiation), but it can be demonstrated that it does with

    ...
    if ($$.ctx && not_equal($$.ctx[i], $$.ctx[i] = "default")) {
    ...

Which functions like so:

This is a bit sneaky, stuffing a variable reassignment inside a function call! Nevertheless, we now have the pipeline mapped:

Props Pipeline

1) init: if(not_equal($$.ctx[i], $$.ctx[i] = value)) (the assignment there)
2) init: make_dirty(component, i);
3) make_dirty: dirty_components.push(component);
4) make_dirty: schedule_update();
5) make_dirty: component.$$.dirty[(i / 31) | 0] |= (1 << (i % 31));
5) schedule_update: resolved_promise.then(flush); (equivalent to flush())
6) flush: while (dirty_components.length) update(component.$$);
7) update: $$.fragment.p($$.ctx, $$.dirty);
8) $$.fragment.p: if (dirty & /*number*/ 1) set_data(t5, /*number*/ ctx[0]);
9) set_data: if (text.data !== data) text.data = data;

A note about resolved_promised.then: It isn't exactly the same as calling flush, as even though schedule_update is called before component.$$.dirty is mutated, that mutation has taken place by the time flush executes here. The mechanism for this isn't clear to me yet.

There's actually a variation on the start of this pipeline, as $$invalidate is actually called in the click handler:

function instance($$self, $$props, $$invalidate) {
    ...
    const click_handler = () => $$invalidate(0, number += 1);
    ...
}

function create_fragment(ctx) {
    ...
    c() {
        dispose = listen(button, "click", /*click_handler*/ ctx[2]);
        ...
    }
...
}

1) click
2) instance: const click_handler = () => $$invalidate(0, number += 1);
3) 1-9 above

Next Time

Join us next time for the further adventures of Svelte Spelunker! Discover the secrets of the runtime, such as

1) How does bundle.js change with increasing app complexity? Multiple components, for example, or dynamically re-rendering props/state.
2) Can dirty ever contain anything besides a single integer? In other words, are elements updated one after the next ("ATOMIC"), or can update sometimes operate on more than one element at a run?
3) When are components and elements destroyed? Are Svelte and Rollup as efficient about unnecessary re-renders as billed?
4) Is it a security problem that the user can directly manipulate the state from a browser's JavaScript console?

New Questions

1) What's up with step 1 of the props pipeline? How and why does it run when a prop changes?
2) how do $$.bound and ready figure into init?
3) This has been there all along, but why the space()s in block?
4) What is outroing? An inside joke? Obscure British slang? Both? 🎩
5) Why does $set even exist? Hint:

Svelte

@sveltejs

@zevav No, $set is non-dev API. It will differ as soon as a component has private state

23:34 PM - 09 Dec 2019

5) Why does Rollup/Svelte compiler include "name" in the $set binding (in instance) when it currently will never be mutated?
6) Why did number and name swap position in the call to init when number got attached to an event listener?
7) What's this business doing exactly? component.$$.dirty[(i / 31) | 0] |= (1 << (i % 31));

How Does Svelte Actually Work? part 1

Zev Averbach — Thu, 05 Dec 2019 16:36:05 +0000

Here's part 2:

How Does Svelte Actually Work? part 2

Zev Averbach ・ Dec 9 '19

#svelte #javascript

A friend put Svelte on the map for me this summer. Rather than tout its performance relative to the frameworks of the day, he touted the bite-sizedness and readability of the JavaScript it generates when compiled.

I'm writing a course that uses Svelte (and FastAPI and some other snazzy things) and am realizing that I could use some deeper knowledge of how Svelte operates: Specifically, how the code works that Svelte compiles to.

I'll post my insights as they come about, so this is part 1 of x.

First Steps

I used the template provided by the Svelte project by doing
npx degit sveltejs/template my-svelte-project; cd $_; npm install.

Then I ran npm run dev to compile the included component and start the development server.

This produced build/bundle.js, the beast we'll be dissecting.

Start at the Bottom

// build/bundle.js (all code blocks are from this file unless otherwise specified)
...
const app = new App({
    target: document.body,
    props: {
        name: 'world'
    }
});

return app;

}());
//# sourceMappingURL=bundle.js.map

I didn't know what a source map is, but having Googled it and inspected bundle.js.map a little, I've decided not to try to decipher it just yet!

Those parens at the end tell me that the app var on line 3 of bundle.js

...
var app = (function () {
...

stores the result of return app, as everything on the right-hand side of that 👆👆 = is an anonymous function which immediately calls itself.

Then, the above block, starting with const app, is identical to the logic in main.js.

// src/main.js

import App from './App.svelte';

const app = new App({
    target: document.body,
    props: {
    name: 'world',
    }
});

export default app;

Searching for main.js in the Rollup config file that came with this sample app, I see

// rollup.config.js
...
    input: 'src/main.js',
...

Okay, I'm reminded that this is where the Svelte app is defined, as configured in rollup.config.js.

The App: First Hypothesis

It looks like the App class has get and set methods on it, each called name.

...
class App extends SvelteComponentDev {
    constructor(options) {
        super(options);
        init(this, options, instance, create_fragment, safe_not_equal, { name: 0 });

        dispatch_dev("SvelteRegisterComponent", {
            component: this,
            tagName: "App",
            options,
            id: create_fragment.name
        });

        const { ctx } = this.$$;
        const props = options.props || ({});

        if (/*name*/ ctx[0] === undefined && !("name" in props)) {
            console.warn("<App> was created without expected prop 'name'");
        }
    }

    get name() {
        throw new Error("<App>: Props cannot be read directly from the component instance unless compiling with 'accessors: true' or '<svelte:options accessors/>'");
    }

    set name(value) {
        throw new Error("<App>: Props cannot be set directly on the component instance unless compiling with 'accessors: true' or '<svelte:options accessors/>'");
    }
}

...

I hypothesize that if I give App another prop, there will be a pair of get and set for that as well.

Testing Hypothesis #1

<!-- src/App.svelte -->

<script>
    export let name; 
    export let number; // new
</script>

Sure enough, these methods have appeared:

...
get name() {
    throw new Error("<App>: Props cannot be read directly from the component instance unless compiling with 'accessors: true' or '<svelte:options accessors/>'");
}

set name(value) {
    throw new Error("<App>: Props cannot be set directly on the component instance unless compiling with 'accessors: true' or '<svelte:options accessors/>'");
}

get number() {
    throw new Error("<App>: Props cannot be read directly from the component instance unless compiling with 'accessors: true' or '<svelte:options accessors/>'");
}

set number(value) {
    throw new Error("<App>: Props cannot be set directly on the component instance unless compiling with 'accessors: true' or '<svelte:options accessors/>'");
}
...

So that's how that works. I don't know much about how getters/setters work in JS classes, but I'm guessing it's like in Python: They trigger when you try to get or set an instance attribute.

Then there's this in the constructor of App:

if (/*name*/ ctx[0] === undefined && !("name" in props)) {
    console.warn("<App> was created without expected prop 'name'");
}

if (/*number*/ ctx[1] === undefined && !("number" in props)) {
    console.warn("<App> was created without expected prop 'number'");
}

This ctx thing is mysterious, and it's popped off of the even more mysterious this.$$.

class App extends SvelteComponentDev {
    constructor(options) {
        ...
        const { ctx } = this.$$;
...

We'll come back to these.

Before continuing, let's update main.js to provide a value for the number prop.

// src/main.js
...
const app = new App({
    target: document.body,
    props: {
        name: 'world',
        number: 42
    }
});

Everything Starts in `create_fragment`

function create_fragment(ctx) {
    let main;
    let h1;
    let t0;
    let t1;
    let t2;
    let t3;
    let p;
    let t4;
    let a;
    let t6;

    const block = {
        c: function create() {
            main = element("main");
            h1 = element("h1");
            t0 = text("Hello ");
            t1 = text(/*name*/ ctx[0]);
            t2 = text("!");
            t3 = space();
            p = element("p");
            t4 = text("Visit the ");
            a = element("a");
            a.textContent = "Svelte tutorial";
            t6 = text(" to learn how to build Svelte apps.");
            attr_dev(h1, "class", "svelte-1tky8bj");
            add_location(h1, file, 5, 1, 46);
            attr_dev(a, "href", "https://svelte.dev/tutorial");
            add_location(a, file, 6, 14, 83);
            add_location(p, file, 6, 1, 70);
            attr_dev(main, "class", "svelte-1tky8bj");
            add_location(main, file, 4, 0, 38);
        },
        l: function claim(nodes) {
            throw new Error("options.hydrate only works if the component was compiled with the `hydratable: true` option");
        },
        m: function mount(target, anchor) {
            insert_dev(target, main, anchor);
            append_dev(main, h1);
            append_dev(h1, t0);
            append_dev(h1, t1);
            append_dev(h1, t2);
            append_dev(main, t3);
            append_dev(main, p);
            append_dev(p, t4);
            append_dev(p, a);
            append_dev(p, t6);
        },
        p: function update(ctx, [dirty]) {
            if (dirty & /*name*/ 1) set_data_dev(t1, /*name*/ ctx[0]);
        },
        i: noop,
        o: noop,
        d: function destroy(detaching) {
            if (detaching) detach_dev(main);
        }
    };

    dispatch_dev("SvelteRegisterBlock", {
        block,
        id: create_fragment.name,
        type: "component",
        source: "",
        ctx
    });

    return block;
}

create_fragment is a function that takes a single argument ctx, and its job is primarily to create and render DOM elements; it returns block.

`block`

block is an object whose most important attributes are c (create), m (mount), p (update), d (destroy).

`c` (create)

block.c's value is a factory function called create, which

    c: function create() {
        main = element("main");
        h1 = element("h1");
        t0 = text("Hello ");
        t1 = text(/*name*/ ctx[0]);
        t2 = text("!");
        t3 = space();
        p = element("p");
        t4 = text("Visit the ");
        a = element("a");
        a.textContent = "Svelte tutorial";
        t6 = text(" to learn how to build Svelte apps.")
        ...

1) creates a bunch of DOM elements and text nodes
2) assigns them each to a variable declared at the start of create_fragment

Then it

    ...
    attr_dev(h1, "class", "svelte-1tky8bj");
    add_location(h1, file, 5, 1, 46);
    attr_dev(a, "href", "https://svelte.dev/tutorial");
    add_location(a, file, 6, 14, 83);
    add_location(p, file, 6, 1, 70);
    attr_dev(main, "class", "svelte-1tky8bj");
    add_location(main, file, 4, 0, 38);
}

3) sets attributes (like 'class' and 'href') on the elements
4) dispatches an event for each attribute-setting (more on that later: we can safely ignore these events forever).
5) adds metadata to each element (__svelte_meta) detailing exactly where it's defined in the src modules.

`m` (mount)

block.m's value is a factory function called mount, which, y'know, adds each element and text node to the DOM in the appropriate place.

    m: function mount(target, anchor) {
        insert_dev(target, main, anchor);
        append_dev(main, h1);
        append_dev(h1, t0);
        append_dev(h1, t1);
        append_dev(h1, t2);
        append_dev(main, t3);
        append_dev(main, p);
        append_dev(p, t4);
        append_dev(p, a);
        append_dev(p, t6);
    },

`p` (update)

block.p's value is not a factory function, but a plain old function which seems to

    p: function update(ctx, [dirty]) {
        if (dirty & /*name*/ 1) set_data_dev(t1, /*name*/ ctx[0]);
    },

1) do something with bits that I don't understand, but probably just checks whether there's anything to update (dirty)
2) if the new value (ctx[0]) differs from t1's value (undefined by default),
3) update t1's value -- it's a text node, as a reminder

Hypothesis #2

I notice here that the prop we added in the first hypothesis, number, doesn't appear in the update function. I'm thinking this is because it's not used anywhere in the component: It's an unused prop.

Testing Hypothesis #2

<!-- src/App.svelte -->
...
<main>
    <h1>Hello {name}!</h1>
    <p>Your lucky number is {number}.</p> <!-- 👈👈👈 new -->
    <p>Visit the <a href="https://svelte.dev/tutorial">Svelte tutorial</a> to learn how to build Svelte apps.</p>
</main>
...

// build/bundle.js
...
    p: function update(ctx, [dirty]) {
        if (dirty & /*name*/ 1) set_data_dev(t1, /*name*/ ctx[0]);
        if (dirty & /*number*/ 2) set_data_dev(t5, /*number*/ ctx[1]);
    },
...

Ding ding ding! I'm still not sure about this if (dirty & 2) business; we'll kick that can for now.

`d` (destroy)

block.d's value is a function which -- shock and awe -- removes an element from the DOM.

    d: function destroy(detaching) {
        if (detaching) detach_dev(main);

Where is `block` consumed?

create_fragment is only called once in bundle.js, which makes sleuthing pretty easy:

    ...
    $$.fragment = create_fragment ? create_fragment($$.ctx) : false;
    ...

This is inside of the monster init function, which is itself called only in the constructor of the class App definition. What is this create_fragment ? ... ternary about? It seems like create_fragment will always be truthy, given that it... exists? The more fruitful question is probably where and how is $$.fragment used? Where? In three places, it turns out. How?

`init`

...
   function init(component, options, instance, create_fragment, not_equal, props, dirty = [-1]) {
    const parent_component = current_component;
    set_current_component(component);
    const prop_values = options.props || {};
    const $$ = component.$$ = {
        fragment: null,
        ctx: null,
        // state
        props,
        update: noop,
        not_equal,
        bound: blank_object(),
        // lifecycle
        on_mount: [],
        on_destroy: [],
        before_update: [],
        after_update: [],
        context: new Map(parent_component ? parent_component.$$.context : []),
        // everything else
        callbacks: blank_object(),
        dirty
    };
    let ready = false;
    $$.ctx = instance
        ? instance(component, prop_values, (i, ret, value = ret) => {
            if ($$.ctx && not_equal($$.ctx[i], $$.ctx[i] = value)) {
                if ($$.bound[i])
                    $$.bound[i](value);
                if (ready)
                    make_dirty(component, i);
            }
            return ret;
        })
        : [];
    $$.update();
    ready = true;
    run_all($$.before_update);
    // `false` as a special case of no DOM component
    $$.fragment = create_fragment ? create_fragment($$.ctx) : false;
    if (options.target) {
        if (options.hydrate) {
            // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
            $$.fragment && $$.fragment.l(children(options.target));
        }
        else {
            // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
            $$.fragment && $$.fragment.c();
        }
        if (options.intro)
            transition_in(component.$$.fragment);
        mount_component(component, options.target, options.anchor);
        flush();
    }
    set_current_component(parent_component);
}

...

$$.fragment is referred to three times directly after its creation in init. Since only target is in the options of the sample app, we'll ignore all but the second, $$.fragment && $$.fragment.c();. Similar to the previous step, I don't understand the boolean check here of $$.fragment && ..., but what's notable is that fragment's c method is called, which will create—but not mount—all the elements and text nodes, giving the elements metadata about their pre-compiled location in App.svelte.

Since init is called inside the constructor of App, we know the above will be executed at runtime.

Backtracking: What About `$$`?

Real quick: $$ is defined early in init.

...
        const $$ = component.$$ = {
            fragment: null,
            ctx: null,
            // state
            props,
            update: noop,
            not_equal,
            bound: blank_object(),
            // lifecycle
            on_mount: [],
            on_destroy: [],
            before_update: [],
            after_update: [],
            context: new Map(parent_component ? parent_component.$$.context : []),
            // everything else
            callbacks: blank_object(),
            dirty
        };
...

Mystery solved!

`update`

    function update($$) {
        if ($$.fragment !== null) {
            $$.update();
            run_all($$.before_update);
            $$.fragment && $$.fragment.p($$.ctx, $$.dirty);
            $$.dirty = [-1];
            $$.after_update.forEach(add_render_callback);
        }
    }

We can ignore almost all of this. $$.update is assigned to noop which does nothing at all. We'll also assume $$.fragment isn't null (how could it be??). Then, $$.before_update is currently an empty array, so we'll wait for more app complexity before studying run_all($$.before_update). Similarly, $$.after_update.forEach(add_render_callback) we can ignore because $$.after_update is also an empty array.

That leaves only

    $$.fragment && $$.fragment.p($$.ctx, $$.dirty);
    $$.dirty = [-1];

Looking around bundle.js I'm pretty confident that $$.dirty = [-1] means there are no pending changes to the app's state. This means that after updating the DOM in the line above it, $$.fragment.p($$.ctx, $$.dirty), we're indicating that all necessary changes have been made.

That makes the only action-packed line $$.fragment.p($$.ctx, $$.dirty), to update the DOM with any changes to
$$.ctx.

`$$.ctx`

$$.ctx seems to be where the app's state lives. Its calculation is a little complex:

    $$.ctx = instance
        ? instance(component, prop_values, (i, ret, value = ret) => {
            if ($$.ctx && not_equal($$.ctx[i], $$.ctx[i] = value)) {
                if ($$.bound[i])
                    $$.bound[i](value);
                if (ready)
                    make_dirty(component, i);
            }
            return ret;
        })

The instance function is what generates it:

    function instance($$self, $$props, $$invalidate) {
        let { name } = $$props;
        let { number } = $$props;
        const writable_props = ["name", "number"];

        Object.keys($$props).forEach(key => {
            if (!~writable_props.indexOf(key) && key.slice(0, 2) !== "$$") console.warn(`<App> was created with unknown prop '${key}'`);
        });

        $$self.$set = $$props => {
            if ("name" in $$props) $$invalidate(0, name = $$props.name);
            if ("number" in $$props) $$invalidate(1, number = $$props.number);
        };

        $$self.$capture_state = () => {
            return { name, number };
        };

        $$self.$inject_state = $$props => {
            if ("name" in $$props) $$invalidate(0, name = $$props.name);
            if ("number" in $$props) $$invalidate(1, number = $$props.number);
        };

        return [name, number];
    }

instance destructures our props, name and number, and passes them right through, unchanged, to $$.ctx.

Therefore, $$.ctx is equal to ["world", 42]: Not as complex as I expected; we'll come back to all these side effects happening here between the seeming pass-through of props.

As seen earlier, $$.fragment.p($$.ctx, $$.dirty) is calling this function:

    function update(ctx, [dirty]) {
      if (dirty & /*name*/ 1) set_data_dev(t1, /*name*/ ctx[0]);
      if (dirty & /*number*/ 2) set_data_dev(t5, /*number*/ ctx[1]);
    }

Okay, time to figure out what this dirty & x business is about. It seems like dirty contains indices of what elements need updating, but why not find out the specifics?:

    p: function update(ctx, [dirty]) {
        if (dirty & /*name*/ 1) {
            console.log(`dirty 1 was dirty: ${dirty}`)
            set_data_dev(t1, /*name*/ ctx[0]);
        } else {
            console.log(`dirty 1 wasn't dirty: ${dirty}`)
        }
        if (dirty & /*name*/ 2) {
            console.log(`dirty 2 was dirty: ${dirty}`)
            set_data_dev(t5, /*name*/ ctx[0]);
        } else {
            console.log(`dirty 2 wasn't dirty: ${dirty}`)
        }
        console.log(typeof dirty)
    },

In order to trigger update without building some UI, to trigger these informative console.logs, we need to manipulate the app's state manually:

`app` in Action

Circling back to the instance function, the more meaningful work it performs (the "side effects") is in binding three methods—$set, $capture_state, and $inject_state—to $$self, which is App.

Did I mention we can inspect our App instance, app, in the console? It's another lovely feature of Svelte: Since it compiles down to vanilla Javascript, app is in the global scope of a browser rendering it, without any special plugins or other somersaults! Armed with that knowledge, let's play with these new methods in the Javascript console:

>> app.$capture_state()
   ► Object { name: "world", number: 42 }
>> app.$set({name: "Whirl"})
   undefined
   dirty 1 was dirty: 1
   dirty 2 wasn't dirty: 1
   number
>> app.$capture_state()
   ► Object { name: "Whirl", number: 42 }
>> app.$inject_state({number: 24})
   undefined
   undefined
   dirty 1 wasn't dirty: 2 
   dirty 2 was dirty: 2
   number
>> app.$capture_state()
   ► Object { name: "Whirl", number: 24 }

The page looks like this now:

Several discoveries here:

1) $capture_state gives the current state of the app as an object.
2) $set and $inject_state seem to both update the app's state via an object.
3) dirty, when it's not equal to [-1], is a positive integer seemingly referring to the props by a 1-based index.
4) These props are updated in the rendered page.

One more mystery to unravel:

>> app.name
   Error: <App>: Props cannot be read directly from the component instance unless compiling with 'accessors: true' or
   '<svelte:options accessors/>'
>> app.name = 'hi'
   Error: <App>: Props cannot be set directly on the component instance unless compiling with 'accessors: true' or
   '<  svelte:options accessors/>'

That's the purpose of the set and get methods from earlier: Enforce that the compiled code doesn't set and get props directly on the App instance, but that it uses... the included machinery?

Next Time

Join us next time to unwrap the mysteries of

1) What is the difference between app.$set and app.$inject_state, if any?
2) How does bundle.js change with increasing app complexity? Multiple components, for example, or dynamically re-rendering props/state.
3) What is __svelte_meta for?
4) Where and when does mount actually get called?
5) Can dirty ever contain anything besides a single integer? In other words, are elements updated one after the next, or can update sometimes operate on more than one element at a run?
6) When are components and elements destroyed? Are Svelte and Rollup as efficient about unnecessary re-renders as billed?
7) How does all this fit together? Asked another way, is it possible to have a basic understanding of how a web framework we use actually works?

Random Notes

According to Svelte's tweet response to me, the events emitted at various points in bundle.js are strictly for dev tooling. This is why we can ignore them.

How I Got "Go To Definition" Working in Vim in 2019

Zev Averbach — Tue, 30 Jul 2019 15:07:33 +0000

👇👇👇tl;dr 
# .vimrc
set tags=tags
autocmd BufWritePost *.py silent! !ctags -R --python-kinds=-i --languages=python 2&gt; /dev/null &amp;

$ brew install ctags

In the last two years I've spent a significant amount of time in PyCharm and a somewhat shorter time with Visual Studio Code, both with vim keyboard bindings, writing mostly Python and JS. Ultimately, I came back to some flavor of terminal-based vim: First Vim 8, now Neovim. There were too many missing features in "vim mode" in both of the IDEs, and the speed difference—brain-to-screen—was noticeable compared to vim.

What I Miss From IDEs

Because I'm not yet a sed/grep expert in the context of search and replace, I still miss the refactoring tools in PyCharm for renaming and reorganizing code. Automatic imports is also something I haven't replicated in vim.

However, what I was missing most was "go to definition", which I had mapped in VSC and PyCharm to ctrl-] as it is by default in vim.

Getting It To Work + Timesucks to Avoid

As with most of my efforts to make vim more ergonomic/IDE-like, getting "go to definition" working took longer than I had hoped.

For the uninitiated, while vim comes with ctrl-] out of the box, it doesn't actually know where something is defined unless there's at least one tags file and you've told vim where to find it/them. You can run ctags manually, but this can get tiresome if you want vim to always have an updated idea of where all the definitions are in your project: functions, modules, constants, classes, types, etc.

The first option I found for automatically updating the tags file didn't work as advertised: I tried to set up git hooks like Mr. Pope suggested, but for whatever reason the tags file never refreshed on commit. Avoid this rabbit hole! And anyway, don't you want "go to definition" to work between commits too?

What ended up working to refresh the tags file *on every save* was a modified version of something I found in the comments of a StackOverflow answer. However, without python-kinds=-i, "go to definition" didn't work as expected on MacOS (it was fine on an Ubuntu droplet). Inspecting my tags file, it was including imports, which caused my ctrl-] invocations to only jump to the top of the current module, where the import was, not to the definition of the entity.

Final Product + Developer Experience

I can now very quickly navigate to the definition of whatever's under my cursor with a single keystroke. What's nice about this usage of ctags is 1) it runs in the background and never interrupts you, 2) it's fast, and 3) it runs every time you save. I haven't tried this at all with JS yet (will be trying this), but it is lightning fast and accurate for jumping around in large Python-based projects.

Bonus: `gf`

gf bridges a significant gap that ctags don't cover: It stands for "go to file". Type gf in normal mode when your cursor's over a filename, and it opens it!

Next Thing to Try: Tagging dependencies and the Python standard lib

As alluded to above, you can tell vim about multiple tags files in multiple paths. It doesn't come up as often that I want to "go to definition" of library code, but I can see how it might come in handy.

Forem: Zev Averbach

How To Get Black Working in PyCharm in 2021

Immediate Performance Issues

Plugin?

A Little Bit of Configuration

How Do PostCSS + Tailwind Work? Part 1: Shrinking the Build

Foundations

Experiment #1: postcss-cli

Experiment #2: postcss.config.js

Shrink the Build

Experiment #3: How Small Will It Get?

Experiment #4: Adding One Line To The Build

Next Time

Patch Your Idols: Making OSS Work the Way Your Team Does

Case Study: Apache Airflow

Challenges + Patches

Challenge: Where Is My Running DAG?

Solution: Sort the DAGs by "Running"

Challenge: When Did My DAG Actually Last Run??

Challenge: Do You Speak CRON?

Conclusion: Optimize the Widely Used Tools!

How Does Svelte Actually Work? part 2

How Does Svelte Actually Work? part 1

Zev Averbach ・ Dec 5 '19 ・ 11 min read

zevaverbach / how-svelte-works

Contains the markdown and code for the blog series.

Episode Recap

What We've Learned So Far

Outstanding Questions

A Skinnier Bundle

Q: Where are the elements created in create_fragment mounted?

Disappearing Questions

What is __svelte_meta for?

What is the difference between app.$set and app.$inject_state, if any?

The $$invalidate Callback

Hypothesis #3: $$.bound and ready Will Come Into Play If There's Some Interactive UI

Testing Hypothesis #3

The Most Basic Knowledge We're Missing

The Last Piece

What is $set For???

So Where? Sneaky Succinctness

Props Pipeline

Next Time

New Questions

How Does Svelte Actually Work? part 1

How Does Svelte Actually Work? part 2

Zev Averbach ・ Dec 9 '19

First Steps

Start at the Bottom

The App: First Hypothesis

Testing Hypothesis #1

Everything Starts in create_fragment

block

c (create)

m (mount)

p (update)

Hypothesis #2

Testing Hypothesis #2

d (destroy)

Where is block consumed?

init

Backtracking: What About $$?

update

$$.ctx

app in Action

Next Time

Random Notes

How I Got "Go To Definition" Working in Vim in 2019

What I Miss From IDEs

Getting It To Work + Timesucks to Avoid

Final Product + Developer Experience

Bonus: gf

Next Thing to Try: Tagging dependencies and the Python standard lib

Experiment #1: `postcss-cli`

Experiment #2: `postcss.config.js`

Q: Where are the elements created in `create_fragment` mounted?

What is `__svelte_meta` for?

What is the difference between `app.$set` and `app.$inject_state`, if any?

The `$$invalidate` Callback

Hypothesis #3: `$$.bound` and `ready` Will Come Into Play If There's Some Interactive UI

What is `$set` For???

Everything Starts in `create_fragment`

`block`

`c` (create)

`m` (mount)

`p` (update)

`d` (destroy)

Where is `block` consumed?

`init`

Backtracking: What About `$$`?

`update`

`$$.ctx`

`app` in Action

Bonus: `gf`