Forem: Jonathan Creamer

The JavaScript Developer's Guide to Ubuntu on Windows 11 with WSL

Jonathan Creamer — Fri, 05 Jan 2024 16:34:09 +0000

This is a bit of a re-hash of an earlier post on setting up WSL in Windows. It is just so easy now, that I wanted to make a whole new post.

It's seriously as easy as wsl --install now!

Seriously just check this out... 👇👇

Learning how to Learn

Jonathan Creamer — Fri, 19 May 2023 14:43:54 +0000

As silly as it sounds, the ability to learn is not necessarily easy.

We all as engineers put ourselves in a state where we put guilt on ourselves when we stop what we're doing, and learn something because it feels like that's not "work".

💡

"Be passionate and bold. Always keep learning. You stop doing useful things if you don’t learn" Satya Nadella Microsoft CEO

The reality is, learning is everything, we should be perpetual learners in every aspect of work, and life. I'd even go so far as to say we're doing our company's a disservice by NOT taking time to learn!

The big question then is, "how do I learn how to learn?!"

Create systems not goals

I heard Cory House reference a book a few years ago in his keynote at Connect.tech called "How to Fail at Everything and Still Win Big" and one idea in that book says that instead of trying to create new goals such as "read x number of books" or "exercise more", we should create systems that enable us to make those goals a reality.

We should approach learning that way! Rather than saying things like, "I want to learn a new language like Rust", or "I want to read 3 new books this year", we should create a system for ourselves that allows us to complete those goals.

For example, use an app such as To Do, Todoist, Apple Reminders, and create yourself a morning checklist that covers a daily few things you want to do.

If you want to learn a new technology, block off time on your calendar daily or weekly in order to spend a bit of time reading a book about it. Take an hour every Friday to stop what you're doing, read a book, watch some content, etc.

Learning Resources

There are so many resources out there for developers to learn from, but here are a few of my favorite

Start a blog

Wait wait? A blog? YES! One of the absolute BEST ways to learn is simply to write down things down. I love the quote, "Writing is thinking". It's so true, often times taking the time to write down something that you've been learning and thinking about will offer clarity and understanding that you didn't expect. It doesn't have to be a complicated blog setup, it's 2023, use Wordpress, Github Pages, Azure Static Webapps, Notepad, just something to get your mind working.

Social Media

This one sometimes makes people giggle, but the reality of it is, Twitter is still a massive source of great free content. Social media, in spite of its obvious pitfalls, is still a great place to go to consume new ideas, and occasionally when the mood strikes, put your own new learnings and ideas out there.

Use Tweetdeck to create a List of top news sites that cover your interests, and add a column for it.

You can even follow certain hashtags if you want, and join them with an OR.

Online courses

Occasionally a company you work for will offer a free education site like Pluralsight, Frontend Masters, Egghead.io, Udemy, LinkedIn Learning and more, but even if they don't offer it to you as a free resource, consider joining one of them anyways! Most of them only cost roughly $30/month, and are full of great content. Here is where you could leverage that hour block of time you schedule on Friday's to go watch some courses on that new language, or learn more about advanced React Hooks usage. Whatever that thing is, take the time to go watch some content on it.

News Sites and Newsletters

Dev.to is one of the best ways to get a stream of content, and also turn around, and write about your own thoughts and ideas.

There are lots of great online news sites like Reddit's r/javascript, or r/rust.

If that's all too overwhelming, just go subscribe to one of the dozens of great newsletters out there that aggregate all that data into a weekly digest.

JavaScript Weekly: The JavaScript Email Newsletter

React Status

Node Weekly

ES.next News: the latest in JavaScript and cross-platform tools

Go forth and learn

Learning is key to growth. Make sure to create the systems you need to foster learning as a part of your day to day life. You will better yourself, better your company, and better those around you.

Tip for writing nodejs scripts using only built-in modules and async

Jonathan Creamer — Tue, 03 Jan 2023 21:42:35 +0000

For the last few years, I've spent a huge majority of my time writing scripts in nodejs as my default scripting language. I've dabbled with bash a bit too, but it's still so weird, and hard to test, and I'm such a Typescript fan that I generally prefer to go the typescript route as often as possible.

Below is a list of tips to make writing nodejs scripts fun, fast, and easy.

Use ES Modules

Yes! You can do it now! However, thar be dragons.

In general tho, if you want to use ES modules in node, you can...

Name your file with .mjs on the end of it
Use a .js file in a place where the closest package.json has "type": "module"
Use node --eval and pass --input-type=module

Similar rules apply to keeping a file use commonjs

Name your file with .cjs
Use a .js file in a place where the closest package.json has "type": "commonjs"
Use node --eval and pass --input-type=commonjs

There are a couple of provisos , a, a couple of quid pro quos

When you switch to using ES Modules in node, you have to:

Always include the extension in your imports...

import { foo } from "./bar.js";

You have to use import() instead of require
This doesn't work... __dirname

But, this does...

import path from 'node:path';
import { fileURLToPath } from 'node:url';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

You might also run into issues with some of the node_modules that you import, so, just be careful. But, if you're generally sticking to native node modules and utilities anyways, then you should be pretty good to go.

Use top level await

This became available in 14.8. For the longest time, I'd always write a wrapper every time I got ready to write a node.js script like...

const main = async () => {
  // await some stuff in here
};

main().catch((err) => {
  console.error(err);
  process.exit(1);
});

I did that so often that I wrote VSCode snippets for it. Heh.

But, cool thing is, you can just await anywhere in a JS file now.

NOTE: you can only do this in a "module" context, i.e. see "Use ES Modules" above

import fs from 'node:fs/promises';

const pkg = await fs.readFile('./package.json');

console.log(JSON.parse(pkg).name);

Use the new-ish promise based APIs along with the node prefix

As of node v14.18 and v16, you can now require native node modules, i.e. those things like path, fs, etc which are built into node, by using the node: protocol in the import or require statement. This will prevent script writers from accidentally importing 3rd party versions of utilities.

import fs from "node:fs/promises";
import stream from "node:stream/promises";
import dns from "node:dns/promises";
import timers from "node:timers/promises";

Also, for everything else that takes a standard node (err, results) => {} callback, you can always fall back on the promisify method.

import fs from 'node:fs/promises';
import { exec as _exec } from 'node:child_process';
import { promisify } from 'node:util';

const exec = promisify(_exec);

const out = await exec('git status');

console.log(out.stdout);

Use child_process effectively

The child_process module is one of the most useful modules in a script writer's backpack. You can run any arbitrary thing on the machine using its spawn or exec methods. However, knowing when, and how to use either of them, is essential.

The most fundamental difference between spawn and exec is...

spawn creates and returns a child process that you can interact with easily via streams

exec creates a runs a subprocess returning the output and error streams with a max of 200kb

Therefore, most of the time exec will do what you need, but if you need some sort of interactivity or have a bunch of output, then you can use spawn.

One of the previous examples above used exec to call git status. Another example could be an abstraction over some sort of unix method like find.

import fs from 'node:fs/promises';
import { exec as _exec } from 'node:child_process';
import { promisify } from 'node:util';

const exec = promisify(_exec);
const [,, filename, directory] = process.argv;

const out = await exec(`find ${directory} -type f -name ${filename} `);

console.log(out.stdout);

// run with...
// node ./script.mjs filename ./directory

Here's an example of using spawn to do something.

In this case, we're going to spawn an npm install in a different package in our repo.

import { spawn } from 'node:child_process';

const install = () => {
  return new Promise((resolve, reject) => {
    const proc = spawn('npm', ['install'], {
      stdio: ['inherit', 'pipe', 'pipe'],
      env: {
        ...process.env,
      }
    });

    proc.stdout.on('data', (data) => {
      if (data.includes('npm WARN')) {
        // Do something
      }
    });

    proc.stderr.on('data', (data) => {
      console.error(`stderr: ${data}`);
    });

    proc.on('close', (code) => {
      if (code !== 0) {
        return reject(code);
      }

      resolve();
    });
  });
};

try {
  await install();
} catch(e) {
  console.error("Something went wrong with the install.");
  process.exit(1);
}

First thing to note here is that we're going create a function and use the Promise constructor in the return. That way we can await it at the top level.

Let's take a look at the options we're passing to spawn. (You can see the entire API of spawn here) The first is the top level command, in this case npm. The second option is an array of the arguments we want to pass to npm in this case just install. But, if we wanted pass multiple things to npm, we'd need to add them each as elements to that array such as ['install', '--save-dev', '--legacy-peer-deps'].

The third argument is the options object. By default, when you spawn a process, the stdio option is set to pipe.

The pipe option means that the proc which is returned from spawn will have proc.stdout, proc.stdin, and proc.stderr on it. These are streams so that you can use methods like on and one to respond to events such as data.

The stdin argument can either take pipe, inherit, or an array corresponding to which parts of the process you want to control...

If you don't need to use stdin or stderr for example, you can switch from pipe to inherit.

stdin: [
  'inherit' /* stdin */,
  'pipe' /* stdout */,
  'inherit' /* stderr */ 
],

Passing stdin like that 👆 would provide proc.stdout but not proc.stdin or proc.stderr.

Lastly, you can watch for the process you're spawning to close by using proc.on('close', () => {});.

One more nice option is that you can control which working directory the process is spawned in via the cwd option. This allows you to use either process.cwd(), or set some other path where you actually want to spawn the process.

Use Typescript for an even cooler script writing experience

Generally I prefer to use Typescript when writing just about anything these days, including scripts. There's a couple of different ways when writing scripts like this which you can use to leverage Typescript for the extra security blanket of checks in your code.

First of all, if you're going to be using Typescript and Node, make sure that you have the @types/node package installed as a "devDependency". That will ensure that all the native modules have correct type information.

Intellisense for parameters

Intellisense for options

That in and of itself is super useful.

What's really worth noting though is that, you don't HAVE to have your file end in a .ts extension. You can add a couple of options to a tsconfig.json. They are "allowJs" and "checkJs". Set them both to true, and you're good to go.

Typescript as of the 4.7 release now supports ES Modules too by reading the "type": "module" that can be added to package.json files. And you can also manually tell Typescript to use modules by setting the "module" option to node16 or nodenext. All that is documented here.

If you want to use .ts instead of using allowJs and checkJs, you'll probably also want to install something like @babel/register, or ts-node, maybe even esbuild-runner. I still personaly prefer the Babel route.

Conclusion

In the world of Frontend and DevOps aka DivOps, writing scripts is inevitable. Writing scripts in Javascript / Typescript allows you to use the JS skills you have as a Frontend dev to interact with the machine to do all kinds of automations, pipelines, etc. And bonus points because you can also write unit tests too. Go forth and write all your scripts in JavaScript!

Calling an API with authorization using native nodejs with zero external dependencies

Jonathan Creamer — Fri, 19 Aug 2022 19:17:48 +0000

Fetching from an API is one of the most repeated tasks for anyone writing JavaScript backends with node.js. There are dozens of libraries for doing this including node-fetch, isomorphic-fetch, axios, etc etc, however, there's actually a couple of native ways that are fairly simple to use and require no external dependencies at all.

Let's talk about a few of them!

The 'https' way

In a world prior to node 18, many people may or may not realize that it's actually not that hard to simply use the built in node.js https module to do a simple API request with ZERO external dependencies.

const https = require("node:https");

const request = (options) => {
  return new Promise((resolve, reject) => {
    const req = https.request(options, (res) => {
      let data = "";

      res.on("data", (d) => {
        data += d;
      });

      res.on("end", () => {
        data = JSON.parse(data.toString().trim());

        if (res.statusCode === 200) {
          resolve(data);
        } else {
          reject(new Error(`Error code: ${res.statusCode}.`));
        }
      });

      res.on("error", (error) => {
        reject(new Error(error));
      });
    });

    req.end();
  });
};

const auth = `Basic ${Buffer.from(`${user}:${password}`).toString("base64")}`;

request({
 hostname: "foobar.com",
  port: 443,
  path: "/api/users/1",
  method: "GET",
  headers: {
    Accept: "application/json",
    Authorization: auth,
  },
})
.then((data) => {
  console.log(data);
})
.catch((error) => {
  // Do something with the error
});

Here you can see we used the native via require("node:http").

You can optionally start by creating your own function that returns a promise, here above there's a request function of our own.

The https.request function returns an instance of the ClientRequest class. And takes options, and a callback function. The callback function contains a parameter for an instance the IncomingMessage class, or basically, the response.

You can use the res.on("data", callback) like you would when you're reading a stream. There's a few event handers you can listen to, build the full response data by just building up a big response string.

Then in res.on("end", callback) you can check the status code, and send back data as JSON or however you want to send back data to the calling function by calling resolve and resolving the promise. If anything goes wrong such as the status not being 200, you can reject with an error, or a string, just something to let the caller know what's going on.

There's also an res.on("error", callback) handler in case something else goes wrong in the response.

Finally, you just call req.end().

Now you have your own custom request function to make API requests with!

The last piece is just building up the correct Authorization headers to send along with the headers option. That's often just a base64 encoded string in the format of username:password along with Basic ${auth}.

As seen above...

const auth = Basic ${Buffer.from(${user}:${password}).toString("base64")};

The 'fetch' way

For a while now, the fetch API has been available in most browsers. However, it wasn't until recently that you have access to the fetch API in node. As of 18.0.0, you are now able to simply use fetch in basically the same way you would in the browser. In 17.5.0 you could use the --experimental-fetch flag, but that will no longer be needed in 18+.

fetch("https://foobar.com/api/users/1", {
  headers: new Headers({
     'Authorization': `Basic ${Buffer.from(`${user}:${password}`).toString("base64")}`, 
     'Content-Type': 'application/json'
   }), 
})
.then((resp) => {
  if (response.status >= 200 && response.status <= 299) {
      return response.json();
    } else {
      throw Error(response.statusText);
    }
})
.catch(() => {
  return resp.text();
})

Well, that was a lot simpler. That's thanks to the work of the folks who put together the new undici library that powers fetch in node!

Conclusion

tldr; if you're in node 18, use fetch, otherwise, use https.

The key takeaway here is that you probably don't need to add some big fetching library just to make a simple HTTP request in node. It's simple enough to make your own thing.

Running Typescript scripts in node using esbuild

Jonathan Creamer — Mon, 11 Jul 2022 18:22:00 +0000

Typescript is hands down my favorite way to write Javascript these days, but sometimes when I just want to run a quick JS script to do something, I'll still reach for node ./foo/script.js. I'll sometimes reach for babel-node, or@babel/register`, but that can be pretty difficult to configure sometimes, and has some perf issues.

It does work though, and I have a post on it...

[

Running Typescript programs with babel-node aka how does babel register work

Running Typescript programs on demand with babel-node basically uses babel register, but how does babel register work? This post will detail it.

Jonathan CreamerJonathan Creamer

](https://www.jonathancreamer.com/running-programs-with-babel-node/)

However, I recently discovered that ESBuild has a similar mechanism to babel-node called esbuild-runner.

[

esbuild-runner

Super-fast on-the-fly transpilation of modern JS, TypeScript and JSX using esbuild. Latest version: 2.2.1, last published: 10 months ago. Start using esbuild-runner in your project by running npm i esbuild-runner. There are 13 other projects in the npm registry using esbuild-runner.

npm

](https://www.npmjs.com/package/esbuild-runner)

It's SUPER easy to use as well.

npm i esbuild esbuild-runner
`

Then, anytime you run a .ts file, you do...

node -r esbuild-runner/register ./path/to/script.ts

And it's stupid fast...

I hope to employ esbuild-runner in some more local developer tooling so developers don't have to pre-compile or build their tools.

Inside the pain of monorepos and hoisting

Jonathan Creamer — Fri, 10 Dec 2021 23:01:51 +0000

Working in a monorepo comes with a long list of pros, and a few cons. One of the most painful cons, when it comes to working in a specifically JavaScript based monorepo, is the pain that comes from hoisting.

What's this hoisting thing?

As we all know, node_modules is a deep dark place with lots and lots of stuff. That problem is even more massive in a large monorepo.

Let's take a step back for a sec and take a look at what happens when you require something.

node module resolution

If you take a read through the docs, you'll find this...

require(X) from module at path Y
1. If X is a core module,
   a. return the core module
   b. STOP
2. If X begins with '/'
   a. set Y to be the filesystem root
3. If X begins with './' or '/' or '../'
   a. LOAD_AS_FILE(Y + X)
   b. LOAD_AS_DIRECTORY(Y + X)
   c. THROW "not found"
4. If X begins with '#'
   a. LOAD_PACKAGE_IMPORTS(X, dirname(Y))
5. LOAD_PACKAGE_SELF(X, dirname(Y))
6. LOAD_NODE_MODULES(X, dirname(Y))
7. THROW "not found"

What this is basically saying is...

If you require X, see if it exists in node, things like fs, child_process, etc.

If you start require('/X'); you're at the file system root.

If you require('./X'); then see if .X is file, then a directory.

...and this is the interesting one...

If you do require('X'); and it's not a node thing, then keep traversing the file system looking in every node_modules along the way.

Hoisting

Package managers such as yarn and npm implemented a hoisting algorithm as a part of their different workspaces implementations.

What this hoisting does is, it scans your package.json files across your workspaces and figures out what the most common versions of dependencies are.

If you have 50 packages, and 47 of them are using react 16.9.0, but 3 are using 16.13.0, it'll "hoist" the common version of react, 16.19.0 to the top level ./node_modules directory. That way you don't have to have 50 different versions of react in your project, the import React or require('react') calls will simply pull from the root node_modules/react one in the case of 16.9.0, or ./packages/*/node_modules/react for the 3 cases of 16.13.0.

Voila, space saved.

However, the plot thickens...

Monorepo structure

So, most monorepo structures have a root package.json, and a packages folder.

./packages/foo/
./packages/foo/node_modules
./packages/bar/
./packages/bar/node_modules
./package.json
./node_modules

Let's say we're working with the ./packages/foo/ and it does an import React from 'react';.

Following the node module resolution from above, it'll eventually look in the ./packages/foo/node_modules directory. If it's not there, it'll look at ../packages/node_modules/, which definitely should not be there, and then it'll look at the ./node_modules directory and see if it's there.

Ok, so that seems fine, where can we go wrong here?

Well, a couple of different bad things can happen here.

Let's say the foo/package.json has "react": "^16.9.0".

Then, let's say over in bar, they forgot to add react in the package.json, but in bar/src/index.ts someone does an import React..., and it also happens that bar used a feature that only exists in the 16.13.0 version of React.

What's gonna happen? Welp, because of the fact that bar doesn't have a react in its package.json, the node module resolution algorithm will kick in and look up to the ../../node_modules/ directory and grab react from there.

But OH NOES, that code won't work at runtime because the code written in bar needs that fancy new feature in React 16.13! This is a problem that is also nicely summed up in the Rush docs as a "Phantom Dependency" and a "Doppleganger".

Real world ouchies

Here's a real world example of how this came into play in a recent release pipeline failure.

In the monorepo supported by my team, the most common version of jest-environment-jsdom that was hoisted to the root node_modules folder in the repo was 24.9.0.

A pull request when in which added jest-environment-jsdom 26.10.0 to one of the packages in the ./packages folder. Well, what happened was, there were a couple of other places using that same version across the repo so, yarn in its attempt to de-duplicate by hoisting decided to switch the hoisted version to 26.10.0!

Here's where things got bad.

Let's say the package which added jest-environment-jsdom was called, cool-button. The pull request for adding the new dependency will get pushed to CI, and the CI server does a check for all the places where cool-button is used, and start running tests, builds, etc on the dependencies in the repo to make sure that the changes cool-button doesn't break any of it's downstream dependencies.

Ok, PR is green, all looks great!

Well, let's say there's another package called cool-color-picker. And cool-color-picker had some tests which was more or less like...

it('should pull from memory storage when localStorage isnt there', () => {
  expect(storageUtilThing.get('item')).toBeTruthy();
})

Well, in this crazy insane case... cool-color-picker was relying on the hoisted jest-environment-jsdom version, 24.9.0. Well, that particular version used jsdom 11.11.0. In that particular version of jsdom there was NO local storage support in the jsdom environment, so the test would pass.

However! In the new version of 26.10.0 of jest-environment-jsdom, the version of jsdom gets bumped to 16.x.x and it just so happens every so ironically that version 11.12.0 of jsdom implemented localStorage. See Changelog.

All the sudden, the test is now failing because the test had previously assumed that there was no localStorage support in jsdom without some kind of 3rd party mocking like jest-localstorage-mock.

What's worse here is, because cool-color-picker isn't in the dependency graph of cool-button, it never got tested as a part of the pull request.

Therefore, the change landed and attempted to go through the release pipeline.

THANKFULLY, although also painfully, the release pipeline currently does a "build the world" strategy regardless of what changed.

It's here where the release pipeline which always builds all the packages that the failure happened and broke releases for a few hours until we figured out what happened which involved LOTS of code spelunking.

So what next?

Well, in an ideal world we'll soon be switching to a "strict" package manager like the one we're currently building called midgard-yarn-strict, which will eventually migrate much of it's feature set into the implementation of Isolated Mode in NPM. Yarn v2 or later can also solve this problem with plug and play, or pnpm as well, but we're currently focusing on the former of the solutions with NPM and midgard yarn strict.

A strict package manager such as these options will do a couple of things.

First of all, the packages are forced to correctly declare their dependencies in the package.json so as to eliminate the phantom dependency / doppleganger problem.

Secondly, the root node_modules folder can be moved to a different location on disk, and pointed to with symlinks.

./packages/foo/
./packages/foo/node_modules/react -> node_modules/.store/react-16.9.0
./packages/bar/
./packages/bar/node_modules/react -> node_modules/.store/react-16.13.0
./package.json

# This breaks the node module resoution
./node_modules/.store
./node_modules/.store/react-16.13.0
./node_modules/.store/react-16.9.0

By moving the node_modules folder somewhere else, and simply symlinking all of the package's dependencies to the new location, the hoisting algorithm breaks.

So, in the previous case where the bar package had forgotten to declare react in its package, bar can no longer rely on hoisting!

Other things you can do to help are implementing a couple of solutions like DepCheck to make based on scanning your code that your dependencies are all correctly declared. There's also a tool called TypeSync which does something similar for the @types packages to make sure they're present. The hoisting problem with@types packages presents a whole different set of complexities because their usage isn't in code necessarily, but only in the TypeScript setup.

Conclusion

Hoisting helped a lot in solving some problems for many repos out there. However, the moment your monorepo starts to scale a bit, you'll inevitably run into these problems.

Setting up WSL2 and the VSCode Remote Containers WSL Plugin

Jonathan Creamer — Mon, 13 Sep 2021 17:11:52 +0000

For years, macOS has been the gold standard for tooling when it comes to Web Development. The unix basis of macOS, iTerm, bash, VIM, etc are all things that folks building for the web platform rely on day to day.

That all changes with WSL2.

WSL2 allows web dev folks working in Windows to get the same goodness of macOS's unix based tooling running "more natively" on Windows.

Steps to install WSL2

Step 1

Turn on WSL in a command prompt or Powershell.

dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart

Step 2

Enable the Virtual Machine Platform

dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

Probably should restart after this.

Step 3

Install this X64 Patch.

Step 4

Make sure WSL2.0 is default.

wsl --set-default-version 2

Step 4

Install Linux!

Go to the Windows Store, and find your favorite Linux.

e.g. Ubuntu 20, Alpine Linux

Optional other good stuff

That's it for WSL2 install, but there's more goodness you can get with...

Install Windows Terminal

Windows Terminal is a brilliant app that manages all kinds of command prompts, terminals, etc. Read more about how to make Powershell Pretty.

By default it will work with Command Prompt and Powershell, but once you have WSL, and Linux installed, it'll also allow you to open Terminals straight into your Linux!

Install Docker Desktop

Installing Docker Desktop allows you to install a new WSL2 based backend that speeds up working with Docker on Windows, adds a nice GUI on top of Docker that allows you to see your running containers visually, installs all the command line tools for docker like docker-compose, and allows you to install a small kubernetes cluster as well.

VSCode Remote Containers

Step 1

Install the Remote Containers - WSL plugin.

Step 2

Clone a Repo, or something in your Linux Distro.

Step 3

Use the Command Palette (Control + Shift + P) and search for Remote-WSL to Open a Folder.

Choose the folder you want, and VSCode will now open the project connected directly to your Linux Distro.

Alternatively you can navigate to the repo in Terminal, and use code . which will also open VSCode connected directly to the Linux Distro.

Conclusion

This post tried to be as clear and concise along the way to get up and running with WSL2 and VSCode as quick as possible. It's a huge productivity win for Web Dev folks working on Windows to be able to develop against Linux, so hopefully this will help accelerate those gains.

BONUS PROTIP

You can also install the Remote - Containersplugin and test out different programming languages with the power of Docker.

Making powershell pretty

Jonathan Creamer — Wed, 21 Jul 2021 19:17:34 +0000

For the first time in a long time, Windows has re-entered my life now that I work at Microsoft.

In macOS land, I use iTerm and oh-my-zsh to make a pretty shell environment with all the colors and such, so when I got to Windows again and lanched into the blue Powershell Window, I figured there had to be something better...

This is what Powershell looks like now!

Windows Terminal

The first thing to make sure you have installed is the Windows Terminal.

That is a fantastic product, and even has built in support for WSL, CMD, Azure, and launching into Ubuntu or whatever your Linux Environment is...

Another #protip is you can hold Ctrl + Shift when launching Windows Terminal and it'll open as an Administrator.

Windows Terminal is super easy to configure as well, and there are themes for it as well at >_TerminalSplash - Windows Terminal Themes or Windows Terminal Themes. You simply copy the JSON from those sites, and open the Windows Terminal Settings and Click "Open JSON File"...

Oh My Posh

The tool that adds the bar at the bottom of the Powershell window is called Oh My Posh of course inspired by oh-my-zsh for mac / linux.

To install, simply run...

Install-Module oh-my-posh -Scope CurrentUser

Then you can list all the themes with...

Get-PoshThemes

Once you find one you like, you can edit your $PROFILE in your editor of choice, e.g. notepad $PROFILE, code -a $PROFILE.

And you can set your theme up with...

Set-PoshPrompt -Theme jandedobbeleer

The last thing you'll more than likely also have to do is download the right fonts that supports "Powerline" fonts, the Oh My Posh docstalk about that. More than likely, you'll choose a font from Nerd Fonts, and update your Windows Terminal settings to point to the new font.

"profiles": 
    {
        "defaults": 
        {
            "fontFace": "MesloLGL Nerd Font"
        }
     }

It'll even show you what version of node you're in as well as many other what they call "Segments"...

There ya go! You'll be the envy of all your Windows code demos in meetings going forward.

Getting started with Frontend Architecture and DivOps

Jonathan Creamer — Wed, 31 Mar 2021 17:24:59 +0000

Trying to understand the world of Frontend Software Development is like trying to solve a Rubix cube that automatically scrambles itself when you get close to solving it. You'll never fully understand it, and that's ok! What you can do though is maybe stop worrying about the entire cube, and focus on one side or even one row, and just be happy you solved that part of it.

My friend Cory House tweeted this a while ago, and I heard him give a talk on it as well...

Why specialize? Because the industry is changing faster than ever.

Today, the half-life of an engineering degree is between 2.5 and 5 years. You must study 10 - 20 hours a week just to keep up with the weekly changes.

Critical read for every dev: https://t.co/lDaDAYA1RH pic.twitter.com/869LlGzlJd

— Cory House (@housecor) March 19, 2018

Specializing in something can be just as if not more rewarding for your long term career growth. Sometimes it's ok to stop learning a little about a lot and learn a lot about a little.

The important point to make here is, you don't have to know everything there is to know about Frontend Development to be successful. In fact, you might be more successful if you simply pick the area of frontend you want to specialize in, and stick with that! Inside of the frontend world, as Brad Frost put it so well recently, there's the Front of the frontend, and the Back of the frontend. If you're reading this and you find yourself leaning more and more into the "Back of the frontend", aka things like bundlers, testing, CI, Docker, deployments, APIs, etc, then maybe you should consider specializing in Frontend Architecture aka DivOps.

A lot of folks who start to specialize in DivOps at one point were on the Front side of frontend, using jQuery, then into Backbone, require.js, maybe Angular, then React or Vue, etc etc. Along that journey a few folks might have decided that actually enjoyed spending time configuring all those different tools to work. As time has passed, it's become increasingly more and more difficult and tedious to manage all that configuration and infrastructure.

If the journey you're on feels like you've started venturing into the Divops world, then here are some great resources to get you started!

Weekly newsletters such as node, esnext, and JS Weekly are expertly curated with swaths of knowledge delivered to your inbox weekly
Ya know that node_modules folder? Actually try looking into it some times. You can spend a few minutes reading over some babel plugins and learn a ton about important topics like ASTs
Speaking of which, learn some ASTs! Start with the Babel Handbook, ASTs are how most modern tools work under the hood and can be extremely powerful in manipulating thousands of lines of code in seconds through things like babel plugins and "codemods" with jscodeshift.
While yes, there are a TON of tools out there, you don't have to learn ALL of them at once, pick a few at a time, find out which one you like, try to learn the ins and outs of webpack vs rollup vs snowpack. In the end of the day, there are reasons why you might choose one over the other, but there's no technically wrong choice. Pick what works for your problem at that moment.
Learn about monorepo tools like yarn workspaces, lerna, bolt, rush, etc. Monorepos can be extremely useful in a large scale frontend architecture
Read code, like lots of it, particularly in code bases like Babel, Typescript ESLint, or just go through trending Github languages like... http://github.com/trending/javascript, http://github.com/trending/javascript.
Learn Docker. Docker is huge in the DivOps, and DevOps worlds alike.
Learn a CI tool, or a few... CircleCI, Travis CI, Github Actions, and Azure Pipelines are all good ones to get started with and basically all do the same thing, just slightly differently
Typescript is probably a good idea now. :)
Talk to people, engage, join a community, join Twitter to meet folks doing this same type of stuff, go to conferences, even virtual ones can offer you a chance to meet others
Write. Things. Down. This cannot be stressed this enough. Journal, blog, create a Notion blog, Evernote, etc. Write down everything, ideas, day to day work logs, tips and tricks, proposals that you can share with other engineers or leadership folks, etc. Our brains can hold a finite amount of information in them at once.

Stay connected, specialize, and have fun!

DivOps is a great path to tread, and is an ever evolving discipline.

Running Typescript programs with babel-node aka how does babel register work

Jonathan Creamer — Wed, 09 Dec 2020 18:32:31 +0000

As a member of a frontend infrastructure team, (aka DivOps), I write a lot of Javascript tooling. I also really really like Typescript. In fact most of the tooling I write these days is written in Typescript.

The problem is, when the tools we work on are handed off to our end users, they have to either be compiled with tsc, or babel first. Generally we typecheck with tsc and transpile with the @babel/preset-typescript preset.

Sometimes though, that compilation step causes a delay, and we'd rather just run the code as is, Typescript, ES modules and all. That's where tools like @babel/node come in.

@babel/node allows you to "transpile" code from Typescript, or "ES Next" JavaScript with modules, or whatever to Javascript that can run in nodejs, and it does this at runtime as opposed to pre-transpiling.

Aka you can do stuff like this, and run Typescript files without "pre" compiling them.

babel-node -x .ts -- ./path/to/script.ts

So, how does @babel/node actually work?

npm i -D @babel/node

When you install this pacakage, it will add a babel-node command to your ./node_modules/.bin folder. This .bin folder is used for all CLI tools you install, things like webpack, babel-node, etc. The .bin then just symlinks the babel-node command to ./node_modules/@babel/node/bin/babel-node. This works because of the "bin" being set to "babel-node": "./bin/babel-node.js" right here.

Then, babel-node is simply a program with a shebang that tells the system to what interpreter to use for the program, in this case the /usr/bin/env command is what actually runs first, and makes sure that it can find node.

#!/usr/bin/env node

The babel-node program itself then just imports require("../lib/babel-node");

This is where we start getting into the actual runtime of @babel/node.

The lib/babel-node.js script is responsible for reading in any runtime arguments for v8 runtime stuff like --harmon or node env variables like --require.

Ultimately the lib/babel-node.js uses a program called kexec, a C++ programwhich swaps out a process at runtime, to actually run ./lib/_babel-node.js

./lib/_babel-node.js is where things really start to happen as this is really the meat of what @babel/node is actually doing. It uses a CLI helper utility called commander and sets it up with the following possible args.

You'll notice a few of those are similar to ones you'd use when working directly with node itself. Things like -e, -p, and -r are all flags that node itself also accepts, other flags like --inspect fall through to the node process that will ultimately get spun up.

Eventually after some args parsing, an internal Module.runMain function) happens and runs your script by overwriting the arguments passed with process.argv.

process.argv = [
 '/path/to/node/v12.17.0/bin/node',
 '/path/to/node_modules/@babel/node/lib/_babel-node',
 ...args,
];

Then what happens in this line is, node will now run /path/to/_babel-node, and send it your actual runtime arguments.

So, that's how _babel-node is actually invoked, but how does it transpile Typescript to Javascript on the fly?

The answer there is @babel/register.

@babel/register

@babel/register is the special sauce of how @babel/node is able to actually read in Typescript, or ES Next Javascript, transpile it to commonjs, and run it on the fly.

Using @babel/register is one way the docs say to setup your code.

Basically instead of babel-node -- path/to/your/script.ts you can manually call the @babel/register module itself by importing it in your code and call it with node /path/to/your/script.ts, in the case of Typescript though

require("@babel/register")({
  extension: ['.ts', '.tsx', '.js', '.jsx']
});
const { main } = require("./src/cli");

main().catch((e) => console.error(e));

Keep in mind with register, you have to separate your entry point from your code because register has to run and add the require hooks to the NEXT files you'll load, aka you can't have import and require("@babel/register"); in the same file. Note to use Typescript and Typescript with React, you'll have to add the "extensions". The other arguments you can pass to register are the same as the regular babel options.

In terms of @babel/node, the register function gets called here and is passed the command line args, and some additional configuration for babel.

@babel/register uses a library called pirates to add a compile hook using some internal Module.__extensions magic to change the underlying Javascript loader in node. It does that here.

So basically every time a file gets require'd going forward in your program, it'll first run through the compile function, get compiled, and finally executed.

Running and debugging babel-node

There's a couple of different ways to run a script with babel-node, just like with node itself.

Passing it the name of a script will invoke the script just like it would with node.

babel-node -x .ts -- ./path/to/script.ts

👆note the -x .ts or --extensions .ts there to make it recognize the Typescript .ts extension as well as .js

You can also add the --inspect flag and in VS Code or using chrome://inspect, you can debug your command line script. In VS Code make sure you have on Auto Inspect, and you'll then be able to throw breakpoints into your code.

If you want to use the Chrome developer tools to debug your script, make sure you throw a debugger; statement somewhere in your code, then you can run...

babel-node -x .ts --inspect-brk -- ./path/to/script.ts

The --inspect-brk will force the process to pause at the very first line of execution, which actually in this case will pause in the _babel-node.js script since that's technically the first line of code that gets executed. Then you can visit chrome://inspect and find your node script in the list, and click inspect.

You'll then get dropped into a Chrome Debugger.

You can run babel-node with -e and execute code on the fly.

babel-node -e "console.log('oh hai');"

You can run a script in combo with -p and print the results...

babel-node -e "new Date().getTime()" -p
// 1607531838009

Lastly you can run babel-node by itself, to spin up a repl for playing around in node with an environment that will transpile code on the fly for you.

Gotchas

One thing in particular that has bit me many times, and partly is the reason why I am writing this post to begin with is,

When using babel register, call the register function before your code

As mentioned previously, make sure when using @babel/register, call the require("@babel/register") first, then require your actual code. Otherwise it won't work at all.

babel-node's command line ignore overrides the ignores in your babel.config.js

This one seems like a bug, but basically, if your babel.config.js has any kind of ignore in it...

module.exports = {
  /* ... blah blah babel stuff */
  ignore: [function(filepath) {
    return filepath.includes('some-string-you-wanna-ignore');
  }]
}

You have to pass an empty ignore in your babel-node --ignore ' ' path/to/script.ts. It probably has to do with this line. Register is expecting ignore not to be undefined, and it's not checking that against the ignore from the babel.config.js.

babel-node will add your process.cwd() location to the --only param by default.

This might be fine in your codebase, but if you're expecting babel-node to be able to transpile code from outside of your cwd, it simply won't work because @babel/register . This goes back to the same line above. If no --only flag is passed at run time, babel will by default ignore anything outside your current working directory AND anything in $(cwd)/node_modules.

So, say you have a project structure like this.

packages/tools/my-awesome-tool/
packages/utils/
package.json

If you have a script in my-awesome-tool/scripts/cool-script.ts which imports from packages/utils and you invoke it FROM the directory, i.e. if you did cd packages/tools/my-awesome-tool, the scripts inpackages/utils will NOT get transpiled by default.

There are 2 ways around this. First, just run all your scripts from the root of your project. Second, you can pass --only /path/to/project. That way it won't ignore stuff.

Make sure to add the `--` thing when running a script

If you forget to add -- in between your options i.e. babel-node -x .ts path/to/scripts.ts instead of babel-node -x .ts -- path/to/script.ts, then babel-node is going to try to parse through the options you passed to your OWN script. This is especially important when the script your calling takes it args.

For example...

babel-node -x .ts -- ./node_modules/.bin/build-storybook \
  -c packages/tools/storybook/src/config \
  -o packages/tools/storybook/dist/storybook

In this case, we're calling the build-storybook command with babel-node, and the -o flag is setting the output dir for storybook. If you forget the --, the -o param will actually be parsed as the --only flag for babel!

Conclusion

@babel/node is a nice convenient way to run scripts on the fly without pre-compiling them. Just be aware of some its nuances and you'll be able to build some amazing tools.

This post was written primarily because many of the things mentioned are things I've dealt with in real life, and I kept running into them over and over again, and finally decided it was time to write them down for myself! Hopefully it can help you too.

Dealing with async jest tests

Jonathan Creamer — Fri, 25 Sep 2020 18:11:28 +0000

In jest, there are a couple of different ways to handle tests which deal with async code.

Let's say way have a test that calls fetchSomeAsyncThing().

it('should call my api', () => {
  const result = fetchSomeAsyncThing();

  expect(result.foo).toBe('hello world')
});

This will fail.

Why?

Because of the way JavaScript handles async code.

When you call a function known to be async, whether it's something that returns a Promise, calls a setTimeout, etc, that function get pushed onto a call stack, and called wayyy after your tests have actually ran.

A stack is a data structure kinda like an array which basically means, the last thing that got put into it, is the first thing to come out. Read more about stacks and async Javacript.

Use the done callback

If your async function returns a callback, several of the jest functions provide a done callback which you can use like this...

it('should call my api', (done) => {
  const result = fetchSomeAsyncThing((result) => {
    expect(result.foo).toBe('hello world');
      done();
  });
});

As you can see, the callback to the it method above returns a done function.

This done function will tell jest that your async method is done and it will wait for that all to finish before wrapping up that test.

Use promises

If your async function returns promises you can do one of 2 things.

it('should call my api', (done) => {
  const result = fetchSomeAsyncThing().then((result) => {
    expect(result.foo).toBe('hello world');
      done();
  });
});

Similar to above, use the done function inside the .then.

Or, you can simply return the entire promise.

it('should call my api', () => {
  return fetchSomeAsyncThing((result) => {
    expect(result.foo).toBe('hello world');
  });
});

In this case, jest will realize that the return value of the test was itself a promise, and will therefore wait until that promise fully resolves before wrapping up the test.

Use async / await

Once again, if you know that your async function returns a promise, you can use the async and await features of modern Javascript.

it('should call my api', async () => {
  const result = await fetchSomeAsyncThing();
  expect(result.foo).toBe('hello world');
});

The async / await method is a nice, clean way to read the code.

For of loops in Javascript one loop to rule them all

Jonathan Creamer — Tue, 17 Mar 2020 19:04:37 +0000

For the longest time, for in and for were the main loops you could use to iterate over collections of things in JavaScript. Then we got some fancy array methods like forEach, map, filter, etc. It starts to get a bit confusing of when to use each loop type. For example, you can't use for in on array's, only on objects. Then, how do I loop over an object? Well, you can use for in, but only if you check hasOwnProperty or whatever, or use...

Object.keys(obj).map((key) => {
  const value = map[key];
});

Which is all weird because you have to get the keys, then grab the value, etc.

Now, we have a new thing as of ES6 called for of. It's becoming more and more used as knowledge of how to use it has grown, but there's still occasional confusion around how/when to use it. Below is a quick cheatseat of some usages of for of, one loop to rule them all.

Arrays

const arrayOfStuff = ['thing one', 'thing two', 'thing three'];

for (const thing of arrayOfStuff) {
  console.log(thing);
}

For arrays, it's pretty simple. It looks like a for in, but you can't for in an array. The point here is, the thing becomes each item in the array.

Arrays of Objects

const arrayOfObjectsOfStuff = [{ name: 'thing one' }, {name: 'thing two' }, { name: 'thing three' }];

for (const { name } of arrayOfObjectsOfStuff) {
  console.log(name);
}

Here you'll notice when iterating an array of objects, you can utilize destructuring to pull the value of the key name off of every item in the array. Note, the descructuring here uses {}'s because we're descructuring an object, vs [] in the case of desctructuring an array.

Objects

const userMap = {
  '123': 'user 1',
  '456': 'user 2',
  '789': 'user 3',
};

for (const [id, name] of Object.entries(userMap)) {
  console.log(id, name);
}

Things get even cooler here now, thanks to the magic of Object.entries. Object.entries returns an array of key value pairs, so in this case basically...

[
  [123, 'user 1'],
  [456, 'user 2'],
  [789, 'user 3'],
]

So, you're in one line converting the object to an array of key value arrays, and then using destructuring to get the id, and name values!

Maps

const actualMapOfUsers = new Map();

actualMapOfUsers.set('123', 'user 1');
actualMapOfUsers.set('456', 'user 2');
actualMapOfUsers.set('7899', 'user 3');

for (const [id, name] of Array.from(actualMapOfUsers)) {
  console.log(id, name);
}

With ES6 Map objects, you can just use the Array.from method to convert the Map into, you guessed it, an array of key values pairs again.

Promises

const getUser = async (name) => {
  const response = await fetch(`https://api.github.com/users/${name}`);
  const json = await response.json();
  return json;
};

const arrayOfPromises = [];

const usernames = ['jcreamer898', 'kwelch', 'AlexSwensen'];
for (const user of usernames) {
  arrayOfPromises.push(getUser(user));
}

Promise.all(arrayOfPromises).then((users) => {
  for (const user of users) {
    console.log(user.name);
  }
});

The final crazy cool thing you can do is handle promises or async await inside of for of loops. In the above example, we're actually creating an array of promises which we then resolve with Promise.all, so this will add a bunch of stuff into the event loop and then once they're all resolved, call the .then on the Promise.all.

Note in this case, there's no use of async / await, so the code will transpile to much less than that of the code that would require babel polyfill, etc from using async await. That said, you probably already have a polyfill like babel installed, so alternatively you can still async/await the Promise.all with...

const main = async () => {
    const users = await Promise.all(arrayOfPromises);
};

The other option is to use await in an async function and actually await each response.

const getUser = async (name) => {
  const response = await fetch(`https://api.github.com/users/${name}`);
  const json = await response.json();
  return json;
};

const getUsers = async () => {
    const users = [];
    const usernames = ['jcreamer898', 'kwelch', 'AlexSwensen'];

    for (const name of usernames) {
      const user = await getUser(name);
      users.push(user);
    }

    return users;
};

const main = async () => {
  await getUsers();
};

In this case the code will pause and wait for each getUser response to come back before moving on to the next one.

Here is a code sandbox where you can see all of this running!

Hopefully this article helps clear up any confusion in for of loops going forward.

Forem: Jonathan Creamer

The JavaScript Developer's Guide to Ubuntu on Windows 11 with WSL

Learning how to Learn

Create systems not goals

Learning Resources

Start a blog

Social Media

Online courses

News Sites and Newsletters

Go forth and learn

Tip for writing nodejs scripts using only built-in modules and async

Use ES Modules

Use top level await

Use the new-ish promise based APIs along with the node prefix

Use child_process effectively

Use Typescript for an even cooler script writing experience

Conclusion

Calling an API with authorization using native nodejs with zero external dependencies

The 'https' way

The 'fetch' way

Conclusion

Running Typescript scripts in node using esbuild

Inside the pain of monorepos and hoisting

What's this hoisting thing?

node module resolution

Hoisting

Monorepo structure

Real world ouchies

So what next?

Conclusion

Setting up WSL2 and the VSCode Remote Containers WSL Plugin

Steps to install WSL2

Step 1

Step 2

Step 3

Step 4

Step 4

Optional other good stuff

Install Windows Terminal

Install Docker Desktop

VSCode Remote Containers

Step 1

Step 2

Step 3

Conclusion

BONUS PROTIP

Making powershell pretty

Windows Terminal

Oh My Posh

Getting started with Frontend Architecture and DivOps

Running Typescript programs with babel-node aka how does babel register work

npm i -D @babel/node

@babel/register

Running and debugging babel-node

Gotchas

When using babel register, call the register function before your code

babel-node's command line ignore overrides the ignores in your babel.config.js

babel-node will add your process.cwd() location to the --only param by default.

Make sure to add the -- thing when running a script

Conclusion

Dealing with async jest tests

Use the done callback

Use promises

Use async / await

For of loops in Javascript one loop to rule them all

Arrays

Arrays of Objects

Objects

Maps

Promises

Make sure to add the `--` thing when running a script