Forem: Brady Holt

Database deadlocks

Brady Holt — Mon, 11 May 2020 00:00:00 +0000

In a database system, a deadlock happens when two or more transactions wait for one another to release a lock. When this happens, the only course of action for the database engine is to kill one of the transaction to unblock the others.

Here is a simple visualization of a deadlock:

Mitigations

Here are some mitigation steps to avoid deadlocks:

Take locks in a consistent order. Deadlocks happen when competing transactions take locks in opposite order. In the PostgreSQL Explicit Locking documentation, the following deadlock example is given:

Session 1:
```
  UPDATE accounts SET balance = balance + 100.00 WHERE acctnum = 11111;
```
Session 2:
```
  UPDATE accounts SET balance = balance + 100.00 WHERE acctnum = 22222;
  UPDATE accounts SET balance = balance - 100.00 WHERE acctnum = 11111;
```
Session 1:
```
  -- DEADLOCK WILL HAPPEN!
  UPDATE accounts SET balance = balance - 100.00 WHERE acctnum = 22222;
  -- DEADLOCK HAPPENED!
```
If Session 2 had updated the accounts record WHERE account = 11111 first and then WHERE account = 22222, which is the same order as Session 1, a deadlock would be avoided.
Find long running transactions and make them faster by optimizing or breaking them up into smaller transactions. The longer transactions run, the greater the chance of a deadlock.
Move statements that obtain a lock as far down in a transation as possible, so that the locks are obtained for a shorter duration of time.
Use explicit locking to take less aggressive locks where possible. For example, in SQL Server, evaluate using READ UNCOMMITTED isolation level where possible.
Depending upon the database system, adding an index may help prevent deadlocks. For example, in SQL Server, a clustered index scan could be blocked by a row lock from another transaction. Adding another index could prevent this.

As a last resort, you could add retry logic to the application. The application could catch a deadlock exception and then let the application gracefully retry until it succeeds. This option should be a last resort since it is working around the root cause of the problem.

Postgres composite types for tables

Brady Holt — Fri, 08 May 2020 00:00:00 +0000

A nice feature in PostgreSQL is that when you create a table, “a composite type is also automatically created, with the same name as the table, to represent the table’s row type” (quoted from the documentation). A composite type represents the structure of a row or record. This means you can work with a single record of a table much like an object in an OOP language.

So, if you create an example table:

CREATE TABLE example (
  id int,
  name text
);

You also get a composite type (also known as a “user defined type” in other database systems) named example.Then, in PL/pgSQL, you can then do things like:

Create a function that returns an example record:

CREATE OR REPLACE FUNCTION construct_example (p_name text)
RETURNS example
AS $$
DECLARE
  v_example example;
BEGIN
  v_example.id = 1;
  v_example.name = p_name;
  RETURN v_example;
END;
$$
LANGUAGE plpgsql;

Use an example record as a single record source for an INSERT INTO statement:

INSERT INTO example VALUES ((SELECT construct_example('ABC')).*);

Use ROW to construct a record on the fly:

SELECT ROW(1::int, 'ABC'::text)::example;

Use JSON to hydrate a record:

v_json_object = '{"id": 1, "name": "ABC"}'::json;
v_example = json_populate_record(NULL::example, v_json_object);

It’s actually quite handy to be able to work with a single record that is of the same type as a table.

IS NULL and IS NOT NULL

One curious thing about composite types is how the IS NULL and IS NOT NULL constructs work on them.

IS NULL is TRUE if a variable of the composite type is NULL or if all the fields of the record are NULL.

So, both of these evaluate to TRUE:

SELECT ROW(NULL::int, NULL::text)::example IS NULL;
SELECT NULL::example IS NULL;

IS NOT NULL, on the other hand, is only TRUE if all the fields in the record are not NULL.

SELECT ROW(1::int, NULL::text)::example IS NOT NULL;
-- false!
SELECT ROW(1::int, 'ABC'::text)::example IS NOT NULL;
-- true

Because of this, a composite type variable can have both IS NOT NULL and IS NULL constructs equal to FALSE at the same time!

Teaching Kids to Code

Brady Holt — Tue, 28 Apr 2020 00:00:00 +0000

A few months ago, I decided to build an app with my 9 year old daughter and her friend Noah. They seemed interested and I was excited at the prospect of inspiring them to learn more about programming.

We setup a time and about an hour before I figured I better come up with a plan. When I starting thinking about it, I had no idea where to start!

Questions on my mind:

What language should I use?
What type of app should be build? Game? Terminal app? Website?
How do I teach them? Do I start with basics and then build up? Or, should we just build something and fill in the gaps later?
No, really, do I teach these 9 year olds what an array is or just start typing and tell them what I’m doing?
What if I go too fast? Will I know? Will they tell me? Or, will they just get lost and feel like they aren’t understanding and give up.

I was surprised how hard it seemed to simply teach something to them. It was particularly challenging because they were starting from scratch. When I’ve given talks at meetups and conferences, my audience has been professional developers. Bridging the gap between their experience and what I’m teaching them is mucheasier than bridging the gap between being a professional developer and a 9 year old that knows nothing about computer programming.

I still don’t know the answer to most of those questions, admittedly. But, I do have a newfound respect for teachers. Especially teachers teaching elementary topics.

I decided that I wanted to do something to keep them engaged. I knew I would lose them if I started talking about variables and data structures right away. So,I just asked them what they wanted to build. “A game”. Of course, right?

So, we talked about the details of the game. What would be the objective? How will you win? How does the game play work and what are the rules? Their wheels were spinning and I had to rein them in a bit because this little game app was getting complicated fast.

We eventually came up with a simple plan: The name of the game is Tresure Castle. You are asked a series of questions and are given keys as you answer questions correctly. Once you get enough keys, you can unlock the castle and you win the game.

I opened up VSCode, created an index.js file and start coding in JavaScript. I told them what I was doing along the way but didn’t get too detailed. I figured they would ask if they really wanted to know. I stopped along the way and ask some questions.

Noah - see this variable that tells me how many keys I have and this other variable that defines how many keys you need to win? How do you think I can find out how many more keys they need. [Subtract them?] Yes! Now, we can subtract them and store the result in another variable. Then, we can use that.

Kayla - See how we’re looping through the questions we ask the user? What do you think we should do once they get enough keys? Should we keep looping? [No?] Right, we need to stop and tell them they’ve won! Here’s how you do that…

Honestly, I felt like I didn’t know what I was doing but it worked out fine. We just tried to have fun and I tried to adjust as we went. When they got lost, I slowed down. When I felt things were clicking I sped up a bit. I tried to make it fun and engaging to keep their interest. All along the way, I kept focusing on my main goal: to enspire them. I really hoped they would clearly make the connection that we are telling the computer what to do and it does it. And, we could tell it to do anything! That’s amazing to me and I hoped they would see that too.

Time will tell if they do :)

Here’s the code repository for our humble little game: https://github.com/bradymholt/treasure-castle.

How we use OpenAPI / Swagger for the YNAB API

Brady Holt — Mon, 16 Mar 2020 21:27:46 +0000

OpenAPI Specification	➡	✅ Tests ✅ Documentation ✅ Client Libraries

YNAB combines software with 4 simple rules to help our users gain control their money. Back in 2018 we released an API to help our community build things to connect their budget to other apps and services.

We built the YNAB API using the OpenAPI Specification and Swagger tooling. In retrospect we think it was a good decision and has provided many benefits. This post gives a landscape of our setup and how the tooling works.

The specification file

Everything starts with our OpenAPI Specification file: https://api.youneedabudget.com/papi/spec-v1-swagger.json. This is a fairly simple JSON file that we edit when anything changes on the API.
This file defines the endpoints and the shape of request and response data.

With this spec file in place, we get access to some great tooling.

Tests

We use a Ruby gem called Apivore to test our actual API implementation against our OpenAPI spec. When our tests run, they exercise all the spec defined endpoints and ensure they are present and work as expected. With Apivore, you have a validate_all_paths method, which does a simple validation to ensure endpoints are available and return the expected statuses. But, you can also exercise the API by passing certain data and Apivore will ensure the shape of the data responses match your spec. So, if your spec says an endpoint returns an array of sharks: [{"id": 1, "name": "megamouth"},{"id": 2, "name": "hammerhead"}] but a particular request returns a single shark: {"id": 1, "name": "megamouth"} a test will fail. Another example: If you were to change the name of a field in the JSON response, a test would fail.

Examples

Here, one of our tests ensures that requesting a non-existent budget at /budgets/:id returns a 404.

it 'response with 404 for a non-existent budget' do
  expect(subject).to validate(
    :get, '/budgets/{budget_id}', 404, params.merge({
      'budget_id' => SecureRandom.uuid
    })
  )
end

Here, we ensure the /budgets/:id/payees/:id endpoint returns a payee, when requested, and also that the shape of that payee conforms to our spec:

it 'returns a single payee' do
  expect(subject).to validate(
    :get, '/budgets/{budget_id}/payees/{payee_id}', 200, params
  )
end

The ability to use a tool like Apivore against our spec is a huge win because we get automatic testing. Having this alone would be a case for using an OpenAPI spec.

Documentation

Using Swagger UI we are able to automatically generate our documentation page just by pointing to our spec. We did make some customizations to suit our preferences but any changes to our spec file are automatically reflected on that page.

For example, the documentation for GET /budgets/:id shows the query parameters that can be passed, expected response status code, and the shape of the response data.

Swagger UI also has the ability to actually use the API on the page itself which is a great help for developers wanting to test something or quickly see the API responses.

For example, this is what the page looks like when requesting a specific payee:

Client Libraries

Arguably, Swagger tooling is most known for Swagger Codegen, which generates client libraries for interfacing with an API.

We use Codegen to generate our JavaScript client and our Ruby client. And others in our community use it to build clients for other languages.

One of the nice things about Codegen is the ability to customize the generated client with the use of templates. For example, we specify a number of templates to override the defaults on our JavaScript client. This allows us to customize things to our suit our preferences.

TypeScript Definitions

Our JavaScript client uses the typescript-fetch generator which, along with generating a JavaScript client usable from both Node.js and the browser, generates TypeScript definition files. This is really useful because developers who are using TypeScript tooling (like VS Code) can get develop-time support.

Examples

IntelliSense support so a developer can clearly see the available fields:

Develop-time errors (a.k.a. red squiggles) so a developer can see when they have accessed a field that does not exist:

Enums selection so a developer can easily select from a list of supported values:

All of this support is coming directly from the original OpenAPI specification file!

Overall Experience

We've been pleased with our usage of an OpenAPI specification and Swagger tooling to build out our API and the ecosystem around it. Of course, there have been a few bumps along the way and we still would like to tweak some things but the amount of benefit this tooling brings is significant and allows us to ship things more rapidly.

Playing with MediaStream API

Brady Holt — Wed, 26 Feb 2020 00:00:00 +0000

I recently purchased a new headset for use on video conference calls at work. I wanted to test out the microphone to see how it sounded compared to my internal MacBook microphone. I started fiddling around with using macOS Voice Memos and it worked but it wasn’t long until I was looking on the web for “test microphone” site. Sadly, the sites I found were full of ads and bloated with things I didn’t want.

Since I’m a developer and can’t help myself, I set out to build something to my own liking. It was time to play with the MediaStream API, something I really haven’t developed for before.

The way I like to learn something new is to get the big picture first. Then, dive into the details. Here in this case, this meant getting a super simple script working that would start recording when the page is opened, stop after 5 seconds, and automatically play it back. It ended up like this:

// This script starts recording when the page is loaded, stops after 5 seconds
// and then automatically plays back what was recorded.
navigator.mediaDevices.getUserMedia({ audio: true }).then(function(stream) {
  let chunks = [];
  const mediaRecorder = new MediaRecorder(stream);

  mediaRecorder.ondataavailable = function(e) {
    // Store data stream chunks for future playback
    chunks.push(e.data);
  };

  mediaRecorder.onstop = function(e) {
    // Playback recording
    const blob = new Blob(chunks, { type: "audio/ogg; codecs=opus" });
    const audio = document.createElement("audio");
    audio.src = window.URL.createObjectURL(blob);
    document.body.appendChild(audio);
    audio.play();

    // Clear recording
    chunks = [];
  };

  // Start recording!
  mediaRecorder.start();

  // Record for 5 seconds then stop and playback
  setTimeout(()=>{
    mediaRecorder.stop();
  }, 5000);
});

If you drop the above code in a <script> tag you have a working recorder / playback. Easy and simple!

For the record, this MDN article and corresponding demo were really helpful in understanding how the MediaStream API works.

Once I had that working, I iterated on the details and eventually landed with something I like and find useful: an app where you click ‘Record’, make some sound, click ‘Stop’ and then it automaitcally plays it back. Simple, right? Also, it shows a nice little realtime graph of the audio input.

It looks like this:

You can see the working app here: https://bradymholt.github.io/test-microphone/ and view the source here: https://github.com/bradymholt/test-microphone.

YNAB at KotlinConf 2019

Brady Holt — Mon, 06 Jan 2020 14:38:07 +0000

A few weeks ago some of the YNAB team attended KotlinConf 2019.

We have been using Kotlin in our Android app for some time and are particularly interested in its multiplatform capability as a way to share code across Android, iOS and the Web.

After attending quite a few multiplatform focused sessions, chatting with folks at the conference, and discussing things as a team, we're planning to start experimenting with multiplatform usage across Android and iOS and then eventually on the Web. There is significant potential here that we want to explore!

Left to Right: Kevin, Graham, Taylor, Jeff, Brady

Updating My ASP.NET Core Template to .NET Core 3.1

Brady Holt — Mon, 16 Dec 2019 00:00:00 +0000

Back in 2016, I decided to build an ASP.NET Core template app so I could play with the framework and get a feel for working with .NET Core. It turned out to be a fun process but took a long time! Admittedly, I took it further than I initially planned to, by adding email functionality, scaffolding tests, building in database migrations, provisioning / deployment with Ansible, and much more. I learned a lot.

It become helpful to some others, to my joy. I got some contributions, questions, and some link backs to my repository as a reference. That type of interest and involvement in the open-source community is really what makes it fun and gives me encouragement to keep contributing.

Back in September, Victor Cardins asked if I planned to upgrade the template to .NET 3.0. It had been awhile since I worked with this project and the idea seemed to be too much to tackle with all the other things I was juggling at the time. But I had some spare time last week, and a burst of interest, so I decided to take the plunge. It took longer than I thought it would (of course) but I finally finished it. This upgrade was certainly rougher than than the upgrade from 2.0. to 2.1 as quite a few foundational APIs have changed. By far the biggest change was the deprecation of Microsoft.AspNetCore.SpaServices because I had to restructure how Webpack was integrated for on-demand bundling and HMR.

Repository Link

https://github.com/bradymholt/aspnet-core-react-template

KotlinConf 2019

Brady Holt — Sat, 07 Dec 2019 00:00:00 +0000

This week I had the opportunity to attend KotlinConf 2019 in Copenhagen, Denmark with 4 other fellow colleagues at YNAB. It was a great time of learning new things, meeting new people, seeing the city of Copenhagen, and spending time with my fellow coworkers and friends.

Kotlin is not a language I use (yet) but at YNAB we are evaluating using its multiplatform capability to enable writing code in Kotlin that can be compiled down to JVM (for Andriod), Native (for iOS), and JavaScript (for Web). The prospect of us being able to share code in this way is compelling so we want to learn more.

I walked away admiring the Kotlin language, impressed with its community, and hopeful for the future in regards to multiplatform support and features.

Sessions I Attended

Day 1

Go multiplatform with Kotlin Workshop - Salomon BRYS

Day 2

Building Progressive Web Apps in Kotlin - Erik Hellman
MPP in 1.3.X and beyond - Dmitry Savvinov & Liliia Abdulina
What the F(p) is Kotlin? Shelby Cohen & Katie Levy
Bridge the physical world: Kotlin/Native on Raspberry Pi - Qian Jin
Your Multiplatform Kaptain has Arrived - Ahmed El-Helw
Shipping a Mobile Multiplatform Project on iOS & Android - Ben Asher & Alec Strong
Kotlin Native Concurrency Explained - Kevin Galligan

Day 3

Asynchronous Data Streams with Kotlin Flow - Roman Elizarov
Kotlin Uncovered - Chet Haase & Romain Guy
What’s new in Java 19: The end of Kotlin? - Jake Wharton
Kotlin in Space - Maxim Mazin
Do it in code (not YAML)! Unlock power of Kotlin DSL for Kubernetes - Fedor Korotkov
Kotlin puzzlers, vol 3 - Anton Keks

Communicating Visually While Working Remotely

Brady Holt — Mon, 25 Nov 2019 00:00:00 +0000

One thing I have learned in since I started working remotely about five years ago is that communicating clearly is very important. Remote working has many benefits but inherently better communication is not one of them.

Context can be easily lost. The other person may be in a different timezone. Cultural jargon you understand can be lost with other cultures. There is a time delay when using asynchronous means of communicating (chat, comments).

There are things I’ve learned to help overcome these challenges. Communicating clearly, with concrete examples is one of them. Making sure to “spell things out” and not assume my reader will know insider jargon or acronyms. But, one of my favorite things to do is communicate with screenshots, annotations, animated GIFs and videos often.

I’ve spent quite awhile finding tools I like and setting things up to make me productive. I’d like to share my setup in hopes it spurs some ideas in others.

Tools

macOS screenshots - The built in tool to “Copy picture of selected area to the clipboard” (mapped to Command + Control + Shift + 4 by default) is easy, fast, and a good tool for quickly selecting an area of my screen. I use it when I don’t need to annotate and I can copy/paste from clipboard to my target app.
Monosnap - When adding some text, drawing a box, or pointing with an arrow will help communicate something more clearly, I use Monosnap. Also, I like using Monosnap for making videos since it supports easy sharing.
Gifox - This is a simple tool for record animated GIFs and has drag/drop support which makes it suitable for quick sharing on GitHub and Slack.

Keyboard Shortcuts

I have keyboard shortcuts setup for these tools, in a progressive order. This makes using this quick and easy to remember.

Screenshot: Command + Shift + 1 (macOS Copy picture of selected area to the clipboard)
Annotated Screenshot: Command + Shift + 2 (Monosnap)
Animated GIF: Command + Shift + 3 (Gifox)
Video: Command + Shift + 4 (Monosnap)

Sharing

If the app I am sharing to supports copy/paste or drag/drop I will use that. For example, I do this when sharing screenshots and animated GIFs to Slack or GitHub. This allows my audience to see the visual quickly without having to click a link.

For all other cases (including email), I will upload the media to Google Drive and share the link. Monosnap and Gifox both support sharing to Google Drive.

GitHub Actions Version and Release to npm

Brady Holt — Mon, 11 Nov 2019 00:00:00 +0000

I have been playing with GitHub Actions quite a bit recently at YNAB and for some personal projects and it has been fun. Most recently I got xertz, my static site generator project, running on GitHub Actions for build/test and releases to the npm registry.

I wanted to walk through the workflow I wrote to release a new version to the npm registry. With this workflow, when I merge a PR, a new release will be tagged and pushed up to npm. This is one less thing I have to do manually when I want to get a change out to a package I manage. Automated scripts for deploying to npm are certainly not a new things but getting this working on GitHub Actions was a new thing for me.

The full workflow script is in the xertz repository so if you just want to take a quick look, please do.

Below, I will work through the script.

Walkthrough

The first bit is the trigger clause:

name: Publish to npm Registry

on:
  pull_request:
    types: closed

When should GitHub Actions run this workflow? Any time a Pull Request is closed. But, I actually only want it to run when a PR is merged and I want it to run on the master branch. Those additional filters will come later.

Now, the next bit simply names the first (and only) job in the workflow “publish” and tells GitHub Actions to run it on the latest supported version of Ubuntu. This is pretty standard stuff:

jobs:
  publish:
    runs-on: ubuntu-latest

Now we get to the main part of the workflow. The steps. These are the steps the “publish” job will run, in sequential order. Above, I told this workflow to trigger on pull requests being closed. Here, I will tell it to only trigger if a pull request is closed because it was merged (if: github.event.pull_request.merged). Also, I’ll specify that I want the workflow to checkout and use the master branch (refs/heads/master).

steps:
  - uses: actions/checkout@v1
    if: github.event.pull_request.merged
    with:
      ref: refs/heads/master

So far, my workflow will run using the latest commit on the master branch, after a Pull Request is merged. Next, I’ll specify I want it to use Node.js version 8 and to run npm install to make sure all the dependent packages are installed:

- name: Install Node.js
  uses: actions/setup-node@v1
  with:
    node-version: 8
- name: npm install
  run: npm install

Next, I’m going to setup some git config (user.name and user.email) in preparation of pushing new git tags up to GitHub. $GITHUB_ACTOR is an automatically available environment variable and is the “the name of the person or app that initiated the workflow”, which in this case will be my down GitHub username (bradymholt) since I will be the one merging Pull Requests which triggers the workflow.

- name: version and publish
  run: |
    git config user.name $GITHUB_ACTOR
    git config user.email gh-actions-${GITHUB_ACTOR}@github.com

Next, I’ll login to npm through the CLI. $NPM_API_TOKEN is a token I generated on my npm account and stored as a GitHub Secret on the repository, and then exposed as an environment variable (see below).

npm config set //registry.npmjs.org/:_authToken=$NPM_API_TOKEN

Ok, now I’ll do some standard npm commands to version and publish my package:

npm version minor --force -m "Version %s"
npm publish

By default, when you use npm version (as I did above), it will create a git version commit and tag. I want to push both of these back up to GitHub to keep track of them. So, here, I will add the GitHub remote and push to it. $GITHUB_TOKEN is a built-in secret, exposed as an environment variable (see below), with access to the repository. $GITHUB_REPOSITORY is also a built-in environment variable referencing the, well, repository name which in this case is “bradymholt/xertz”.

git remote add gh-origin https://${GITHUB_ACTOR}:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git
git push gh-origin HEAD:master --tags

Finally, the env: section specifies environment variables to be made available for the duration of the workflow. Above, I referenced $NPM_API_TOKEN (manually added secret) and $GITHUB_TOKEN (automatically added secret). You have to explicitly make secrets available as environment variables by using the following syntax:

env:
  NPM_API_TOKEN: ${{ secrets.NPM_API_TOKEN }}
  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Only then can you use them in the steps. Do note that other automatically added environment variables that are not considered secrets, such as $GITHUB_REPOSITORY, do not require this explicit syntax.

That’s it. I hope this proves useful to others.

Progressive Conversion of TypeScript Namespaces to Modules

Brady Holt — Mon, 16 Sep 2019 00:00:00 +0000

We love TypeScript at YNAB. One of our main modules is something we call the “Shared Library” and it is a quite large TypeScript project. Actually, it’s comprised of 3 library projects and 3 test projects. It’s big. And, it was initially written using TypeScript namespaces, before TypeScript had support for ES modules.

We wanted to start converting this library over to using ES modules for the various benefits that gives including the ability to tree-shake and better development tooling. But, it became obvious we needed to find a progressive way to do this because a few attempts at an all-or-nothing approach proved daunting. Thousands of errors and no simple or obvious way to automate the conversion.

Guidance for progressively converting a project from namespaces to modules is slim. There is this GitHub issue where some discuss approaches but there are still some gaps the approaches.

To setup progressive migration, we ended up doing the following which has been working well for us.

Add "exclude": ["**/*.m.ts"] to the tsconfig.json file in the original global / namespace project. This allows you to create modules with .m.ts extention and keep the outFile config.
Create a tsconfig.module.json file adjacent to the main tsconfig.json file.
Use a project reference in the tsconfig.module.json file pointing to the namespace project: "references": [{ "path": "./tsconfig.json" }]. This makes the global / namespace types available from within the modules project. It doesn’t emit the code for this project but it tells TypeScript to assume these types will be available at runtime.

It looks like this:

tsconfig.json

{
  "compilerOptions": {
    "outFile": "../dist/package.js",
    "composite": true
  },
  "include": ["**/*.ts"],
  "exclude": ["**/*.m.ts"]
}

tsconfig.module.json

{
  "compilerOptions": {
    "module": "esnext"
  }
  "files": ["./MyClass.m.ts"],
  "references": [{ "path": "./tsconfig.json" }]
}

With the above setup, you can create module files with the extension .m.ts adjacent to the existing namespace files like this:

- MyClass.ts
- MyClass.m.ts

Then, as @danielrosenwasser pointed out, you can use existing code from the namespaced code and wrap it in the module file. You can also just export it as is:

export import MyClass = my_namespace.MyClass;

Since this setup is using ES modules, you can now use a bundler like webpack to prepare it for web consumption. In the following example, webpackwill be used with ts-loader. Notice ts-loader is configured to use the tsconfig.module.json config file.

webpack.config.js

module.exports = {
  entry: './src/MyClass.m.ts',
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: [
          {
            loader: 'ts-loader',
            options: {
              configFile: "./tsconfig.module.json" // Use the module project config!
            }
          }
        ]
      }
    ]
  },
  output: {
    filename: 'bundle.js',
    path: path.resolve(__dirname, '../dist')
  }
};

Finally, in your entry webpage, you can include a reference to the original namespace outFile and also the webpack bundle.

index.html

<head>
  <script src="dist/package.js"></script> <!-- outFile output from tsc -->
  <script src="dist/bundle.js"></script> <!-- webpack bundle -->
</head>

Once you’ve converted over all the namespaced source files to modules, you can remove the original namespace project and stop loading it.

A more exhaustive example can be found in this repository: https://github.com/bradymholt/ts-progressive-convert-namespace-modules.

I found a use for a zero-width space

Brady Holt — Fri, 30 Aug 2019 03:29:07 +0000

You know about zero-width spaces right? It's an invisible character that seems not so useful at first glace.
Most of my experience with them has been negative because they usually show up in a random file I'm parsing and it is the cause of a bizarre bug. Or, it's been the cause of a copy/paste search that yields no results even though there are matches sitting in front of my eyes.

Well, despite my apathy for them, I actually found a good use for them.

I've recently been building out xertz, a static site generator written in TypeScript. It's being used to build this here site. Something that had been bugging me was the indentation of the rendered HTML. I use Handlebars.js templates and include raw HTML which has been converted from Markdown. My templates looks something like the following, where {{ content_html }} is the raw HTML bit:

<body>
  {{> header }}
  {{> sidebar }}
  <article class="post">
    {{{ content_html }}}
  </article>
  {{> footer }}
</body>

The problem is, newlines in {{ content_html }} do not get indented so the actual output looks something like this:

<body>
  <header>My Site</header>
  <aside>Sidebar here</aside>
  <article class="post">
    <p>Lorem ipsum dolor sit amet, usu an justo deterruisset. Est ad discere nominati,
erroribus dissentias mei ne, appetere qualisque eloquentiam sea et.</p><img alt="An image" src="my-image.jpg"/><p>Lorem ipsum
dolor sit amet, usu an justo deterruisset. Est ad discere nominati</p>
  </article>
  <footer>My footer</footer>
</body>

That's a simple example. In practice it's much worse. Yes, this has no effect on the actual layout and rendering of the webpage
but I'm a developer and care about the source and what it looks like.

I tried to create a Handlebars helper called "indent" so I could indent each newline. I called it like this:

...
<article class="post">
  {{{ indent content_html 2 }}}
</article>
...

The helper is pretty simple. It just replaces newlines (\n) with a newline followed by a number of spaces:

function indent(input: string, width: number) {
  const intendation = input.replace(/\n/g, "\n" + new Array(width).join(" "));
  return input.replace(/\n/g, match => match.replace(/\n/, `\n${intendation}`)
}

That worked pretty well until my <pre> code blocks started looking like this:

const my_var;
    const another_var;

The HTML looked like this:

<body>
  <header>My Site</header>
  <aside>Sidebar here</aside>
  <article class="post">
    <p>Below is a code block</p><pre>const my_var;
    const another_var;</pre><p>Lorem ipsum dolor sit amet, usu an justo deterruisset. Est ad discere nominati,
    dissentias mei ne</p>
  </article>
  <footer>My footer</footer>
</body>

Uh oh, I'm getting indentation in my code blocks. Yes, since my code blocks are using <pre> any spaces inside of them will be rendered as is. <pre> means "preformatted" after all.

So then I thought I needed hint to signal to the indent helper to skip indentation in these <pre> tags.

Adding hints in the <pre> tags seemed feasible because I use Prism with Marked to convert Markdown code blocks like:

```javascript
const my_var = "Hello";
```

into <pre> blocks. It's quite easy to modify the output of these tags because you provide a function that returns something like this:

return `<pre class="${className}"><code class="${className}">${code}</code></pre>`;

Easy to modify, yes. I thought, "can I add some character(s) to end of <pre> tags lines that my indent helper could skip?" But since I'm using Regex to add the indentation in my indent helper, I can only use a single character to be able to include a negated character (e.g. [^!]) in my RegEx without having to do a negative look-behind (Javascript doesn't support these anyway).

Ok, so, I just need Prism to add a single character that will not be visible to the end of lines that are inside of <pre> blocks. Then my indent helper can ignore these. How do I do this?

Zero-width spaces, of course!

Now, my code formatting function preceeds newlines in my code blocks with a zero-width space. It looks like this:

const codeWithNewlineHints = code.replace(
  /\n/g,
  // Prepend each newline with a zero-width space character so we can signal to any upstream formatting to leave the formatted code alone.
  "\u200b\n"
);

return `<pre class="${className}"><code class="${className}">${codeWithNewlineHints}</code></pre>`;

In my indent helper, I simply ignore lines containing these characters preceding a newline.

const intendation = input.replace(/\n/g, "\n" + new Array(width).join(" "));
return input.replace(/\n/g, match => match.replace(/[^\u200b]\n/, `\n${intendation}`)

See that RegEx there? /[^\u200b]\n/ means only match newlines if they are not preceded by a zero-width character (\u200b). So, with this, indentation will only be added to lines not preceeded by these characters.

I've gained a newfound respect zero-width spaces.