Forem: Jordan Nielson

Why I started a newsletter

Jordan Nielson — Thu, 23 Apr 2020 06:20:24 +0000

Originally published on jnielson.com

Hi there! I started a newsletter! You can sign up by heading over to my newsletter page!

Why a newsletter?

First things first, I like newsletters. I prefer to get an email about important things over any other communication method, in most cases. If someone that I think does great work has a newsletter, I sign up for it. I have over 20 people I've signed up to receive newsletters in my personal inbox from! If you have a newsletter, add it to this twitter thread!

Secondly, I think I have something useful that you might want to read about. That's why I started blogging in the first place! Blogging is great, but sometimes it's hard for people to find what I write. I have an RSS feed for my blog, but not many people I know still use RSS readers (even though I do). I generally post about what I write on my twitter, but twitter is chaotic and hard to find things on. From what I've seen in the newsletters I'm signed up for, newsletters are a great way to share the content you produce to a group that actually wants it enough to give you their email address.

Dealing with inbox clutter?

A common thing I think of when I see a newsletter signup form is "Do I want another piece of email coming my way?"... usually yes! I chatted with Ryan Warner and a few other people in the party corgi discord (which you should join!) about adding newsletters to personal websites, and learned that not everyone has the same approach I do to their email inboxes. For over two years now, I've subscribed to the "inbox zero" lifestyle. I think there hasn't been an entire week pass where I've left something in my inbox that couldn't be read and taken care of, or snoozed for later when it was actionable. For the most part, newsletters fall into the category of "at least skim, but regardless I'll archive it when I get to the bottom", which has helped me stay up with inbox zero while signing up for newsletters wherever I can find them. One of the nicest things about newsletters is their capability to link out to other things, which if I have time and interest I can click on... but there's nothing that says I have to click!

Why not post more on your blog?

I'm working on it! Taking a page out of Chris Biscardi's book, I'm going to use my blog as a place to post all sorts of content. If I have a time-bound newsletter post, it'll probably also end up on my blog. A shadow newsletter... likely wouldn't end up on my blog...? I dunno yet. I personally like newsletters that have a link out to a personal site to read them on to break up looking at my email reader, so maybe it'll end up as a section of my site. 🤷‍♂️

Either way, the blog and newsletter are both priorities for me. The reason for that, is I enjoy writing! I find writing to be an excellent way to marshal the thoughts that I have and get them out into the world in a communicable way. It helps me to get out of my box of like 30 people that I work with regularly, which I think is important enough to put time into.

I'm going to be a bit surprised if anyone reads this post all the way through, so if you do... tweet me about it! I'm excited to see where this newsletter journey goes from here. 🥳

Why I cross-post

Jordan Nielson — Fri, 10 Apr 2020 18:49:21 +0000

originally posted on jnielson.com

Some history

Since writing my first blog post over a year ago on my new gatsy-powered site, I've gone back and forth with how I want to share the content that I create. The main channel I've used so far is twitter, since that's pretty much the only platform that I'm active on social media wise. So, my first couple posts here were only shared on there.

At some point last year I heard about dev.to, and was intrigued by the idea of posting on there but didn't want to let my recently created site languish with no content or changes! So, I made the decision to post to both places at the same time, but only really share the link on dev.to. An example of that is this twitter post sharing my Sharpen your ax article, there was no twitter post sharing the link to the post on my site... which at this point I think was a mistake.

Throughout the rest of 2019, posts were available on both platforms at roughly the same time. At the start of 2020, I started to reconsider this pattern as I was trying to evaluate my goals with cross-posting and the first article I wrote this year, didn't ever get posted to dev.to (that was my intro to react hooks article). I rather enjoyed writing that article! Not posting it on dev.to was probably a mistake though, since at least the tweet analytics make it seem like there wasn't much traction to it (though I'm probably just missing a lot of best practices for tweets...).

So, with that singular try at not posting on dev.to I decided to re-evaluate my relationship with dev.to. This evaluation helped me realize that while I do follow a ton of blogs using RSS feeds through feedly (and my blog supports one! RSS Feed) I also do a good amount of my reading on dev.to, especially for when I'm looking to read about a topic in general. So, in order to tap into that while still giving some benefit to subscribing to my blog I decided to go back to cross-posting, but with a delay. The delay has been totally random so far and not automated at all... which has come with a couple benefits (though I'll probably automate it at some point...). The biggest benefit so far has been the extra, organic, sharing opportunity since I'll do a tweet when I post it on my website and then throw out another one from dev when it gets posted on there!

Why not just post on dev.to?

Some might ask why I want to post on my own site at all... and the biggest reason is that I like having a place to tinker, but without a reason to come back to it regularly (like writing a new post) it languishes without any care or change.

Another reason to keep it on my site is that I recently started streaming on twitch at my channel there, and I'd eventually like to find a way to integrate that into my site. That might end up looking like posting recent streams, since I've been throwing them on my youtube channel to avoid the twitch auto-delete time of their video-on-demand. It might also end up looking like using react-livestream from Ryan Harris... who knows!

Also, I just launched a newsletter that people can join from my site! I'll probably start including that link when I cross-post to dev.to... but each post on my site has the signup form embedded at the end of the post too.

Why not just post on my own site?

There's a ton of reasons to cross-post articles, so I'm just gonna list a few off the top of my head:

Network effects for discovery (aka the dev.to feeds)
People can customize the reading experience on the platform (at least on dev.to)
On platform following (I have 180 dev.to followers apparently)
Comments... I don't want them on my site since I don't like them, but some folks do and dev.to has them

Current strategy summarized

After doing the delayed posting thing for a couple months now, I can say that I like it a lot. It's nice to have a chance for people to see the post at least twice, and there's a lot of benefits for readers by posting on dev.to. There also isn't much friction there, since I generally write my posts in markdown and dev.to accepts markdown.

What's next?

I think I'm going to go back to posting on both platforms at the same time. Why? Because what I learned from reflecting on what I've tried so far (while writing this post out to see what I wanted to do) is that the bigger issue is that I don't really share the posts in a consistent way, not so much the platform that I post on. So, what I really need to adjust is my marketing strategy, not the posting strategy. I don't really see a reason to have it "exclusively" on my website, but what I should do is formulate a marketing strategy that takes advantage of both platforms.

So, what you'll probably see is something like this:

Write the post on my site and publish it to both platforms. There's a slight delay since I link to the published version on my site in the dev.to post.

Linking to my site:

Put out a quick tweet about it right when I publish to write up the sum'd up version
Schedule a newsletter post about it to go out the next morning
Schedule a tweet for the second day it's up, similar to the first one but without the "just published" sentiment
Retweet ^ at the 1 week or so mark
Schedule a tweet for 3 weeks out to re-share
Schedule a tweet for 3 months out to share it again

Linking to dev.to version:

Put out a quick tweet from dev.to right when I publish there
Schedule a tweet for the fourth day it's up, similar to the first one but without the "just published" sentiment
Retweet ^ at the 2 week or so mark
Schedule a tweet for 2 months out to share it again

I'm not set on a tool yet to help manage these scheduled things, so I'd love to hear your thoughts on twitter! Reply to this thread on it! If you have other thoughts on how to market blog posts, especially posts you've written, I'd love to read them!

Banner image courtesy of undraw.co

Git Rebase... with Merges?

Jordan Nielson — Wed, 08 Apr 2020 16:49:59 +0000

Originally published on jnielson.com.

Did you know that git rebase -i will drop merge commits by default? I certainly didn't realize that was happening until a time at work when I had to keep the merge commits in as I rebased one branch to not have a feature anymore, which had been merged in as a series of small pull requests... much like what Sarah Drasner talked about in a sweet post on scoping down PRs.

So, how do you keep merge commits when you need to rebase a branch and pull some commits out? To learn more about why I wanted to keep the merge commits, I'll push that to the end and get right into the main point:

Note: Needs git --version to be greater than v2.22.0 in order to have this feature

Generally to rebase a branch, I'll do something like:

git rebase -i [some commit]

Which gives some pretty neat output (generated during a livestream where I worked up an example for this post, on Twitch or archived on youtube):

pick c349469 Add some proper punctuation
pick 282af3c Add some descriptive text to the readme
pick 1daf229 Remove sample text and clarify that this is in progress.

# Rebase d5941f3..2587717 onto d5941f3 (3 commands)
#
# I removed the instructions that usually go here in both examples

There's a lot of options of things you can do in a rebase, that git helpfully outputs when you're working on it. But, the main thing to notice here is that it's a list of normal commits, with no indication if there was a merge commit in there.

So, if you need to keep merge commits around, here's your new friend:

git rebase -i --rebase-merges [some commit]

When you add the --rebase-merges option to git rebase it knows that you actually care about those merge commits and doesn't throw them away. In doing so, git rebase basically re-creates the entire branching history starting from that commit. It calls the starting point onto (as you'll notice in the example below), and adds some cool new commands to the interactive rebase options. These are covered pretty well in the git rebase documentation. Here's the gist, ideally in plainer english than the docs:

The label command creates a label, basically a pointer just like a branch, that is deleted when the rebase is done.

The reset command resets the HEAD (pointer), index and worktree to the specified revision (usually a label you made). Works like git reset --hard <label> but without as much... force.

The merge command merges... Usually you want the -C option, which uses the original message (and basically the original merge commit). Make it lower-case (-c) if you want to adjust the message. If you leave off the -C option, it thinks you're making a new merge commit.

So, here's an example of what that list might look like, again from the stream (on Twitch or archived on youtube):

label onto

# Branch jnielson94-jn-quick-fix
reset onto
pick c349469 Add some proper punctuation
label jnielson94-jn-quick-fix

# Branch jnielson94-jn-add-more-text
reset onto
pick 282af3c Add some descriptive text to the readme
label jnielson94-jn-add-more-text

# Branch jnielson94-jn-clarify-text
reset onto
merge -C f041aa8 jnielson94-jn-quick-fix # Merge pull request #1 from jnielson94/jn-quick-fix
merge -C eb0f351 jnielson94-jn-add-more-text # Merge pull request #2 from jnielson94/jn-add-more-text
label branch-point
pick 1daf229 Remove sample text and clarify that this is in progress.
label jnielson94-jn-clarify-text

reset branch-point # Merge pull request #2 from jnielson94/jn-add-more-text
merge -C 2587717 jnielson94-jn-clarify-text # Merge pull request #3 from jnielson94/jn-clarify-text

Once you're in this view, you can interactively adjust whatever you needed to! On the stream we switched the order the pull requests were merged in, by swapping those two lines in the list.

But why?

Alright, after all that you're still wondering why I wanted to do this. Here's the story:

We've been working on a major new feature utilizing a feature branch, feature/xyz for example. When working on things related to that, we branch off feature/xyz (usually using a branch name like jn-small-task) and then merge it into feature/xyz using merge commits in case we need to backport anything over to production (master branch). We've avoided rebasing on master and feature/xyz to be able to keep the commit history of the two interleaved (though I rebase heavily on my little branches and occasionally for things that don't need to be on master). So, in order to keep the ability to backport we've kept merge commits on feature/xyz as much as possible. So, I rebased feature/xyz to remove a feature (which had been merged in as a bunch of little jn-small-thing branches), but I wanted to keep jn-medium-feature which had all of them, but was rebased onto the new end of feature/xyz. Since I kept the merge commits in both instances, git rebase did a great job handling pulling them all to the end of the jn-medium-feature branch 🎉

And that is the story of why I needed this obscure/advanced git feature 😎

Cover image courtesy of undraw.co

Getting my feet wet with TypeScript

Jordan Nielson — Tue, 03 Mar 2020 17:50:05 +0000

Originally posted on jnielson.com. Cover image courtesy of the awesome undraw.co.

Hi there! I'm writing this post with the goal of helping people who are in a similar spot to where I am, at the time of writing. As mentioned in my changing jobs post, my new job mostly uses TypeScript. Prior to this job I'd written a total of zero characters of production TypeScript! Starting at this position and wanting to jump right in, I've written some lines of production TypeScript now! In this post I hope to share a few things that tripped me up as I've been slowly working in TypeScript. Before I get into it though, I want to share a couple thoughts on learning...

Thoughts on learning

Awhile back I wrote a post about sharpening your axe where I shared a few of my thoughts on learning. To reference that a bit, with TypeScript I have a general idea of how it works, and I know where to find and have access to the documentation and some sample (existing production) code in a few projects. Since I have these resources, I was able to jump straight to, for the most part, knowing what I don't know. There's a huge number of features available in TypeScript that I haven't needed to dig into yet... but at least I know where I can find out about them. For instance, I have had no need, so far, of Generics, but I know exactly where to look if I do need them: The TypeScript docs. Also, since:

TypeScript is a typed superset of JavaScript that compiles to plain JavaScript

TypeScriptlang homepage (emphasis added)

I'm able to utilize the vast JavaScript knowledge that I do have, and if I do something in my work that TypeScript doesn't like the compiler will tell me. Since there's this existing bed of knowledge, I went ahead and read through the TypeScript in 5 minutes document that they have available to have some general knowledge in my head... but outside of that it's been a "learn as it comes up" approach, especially since I'm mainly working on existing projects.

With that background, let's dive into some of the things I've learned while "getting my feet wet with TypeScript"!

Getting my feet wet with TypeScript through Create-React-App and Next.js

One of the first things that I had heard about TypeScript in the past was how painful it was to get integrated into the tools you were already using if you started a project without it. Well, prior to my arrival the projects I've been working on were changed to use TypeScript. From what I understand, that was made way easier by the built-in support of TypeScript in the tools we mainly use - Create React App and Next.js. So far as I can tell, we're pretty much using the built-in support with both of these tools... though there might also have been some customization since I haven't actually dug into it. I haven't needed to... we'll see if that changes? Either way, the experience has been seamless so far! I haven't noticed any drastic difference in hot reloading times when working on things, and I've appreciated the error messages so far (make sure to read those)!

Using examples of things already working in the project

One point that I mentioned earlier in the section about learning, is how useful it's been to have existing production code to lean on. There's something to be said for caution there, even if you trust the developer(s) who wrote the code... but I'm not going to dig into that now.

One example that I did want to share of this was when I was trying to use a ref for the first time. As I usually do, I utilized the useRef hook with an initial value of null... but TypeScript didn't like that one! My first thought for solving it was to do a search... and the first StackOverflow answer made the compilier happy! In my younger years I probably would have called that good... but I had this thought that I didn't love the look of that answer. So, I decided to search the project to see if there was an existing case of using a ref with an initial value of null, and there was! It was way simpler than the complex typing that I had found on StackOverflow... though I later pivoted and didn't end up needing a ref after all that.

Trying something new

Another example of something that I reached for without knowing was an enum. I wanted a string enum to define the available sizes for a new component I was writing, and while I wanted to use my experience I just talked about and find something in the project to model after... I had no idea what to search for. So, I knew that an enum pretty closely matched the concept of what I was looking for... but I didn't love using it! The enum definition looked something like:

export enum Size {
  small = "small",
  medium = "medium",
  large = "large"
}
interface SomeComponentProps {
  size: Size;
}

To satisfy the typechecker when using an enum, you had to import the enum that had been defined in the type... at least that's what the docs made it seem like. It looked something like this:

import { SomeComponent, Size } from "../some-component";

const component = () => <SomeComponent size={Size.small} />;

As I mentioned in my tips and tricks post, I get to do regular pair programming sessions with my lead dev. In one of these sessions, I brought up the enum deal and they had never seen an enum before (in TypeScript)! He suggested the way they usually type something like that, using a string union like so:

interface SomeComponentProps {
  size: "small" | "medium" | "large";
}

There's probably some case where an enum would make more sense, but so far as my team is concerned a union of strings makes more sense in this case. It still gives autocompletion, which is probably the biggest benefit I've seen so far in TypeScript!

Pros and Cons?

This post is definitely not a pros and cons type post, but I do want to point out that there are definitely pros and cons to the approaches you can take to learning something. There's a wide spectrum of strategies, depth, planning, and winging it that come together and end up with you being at the place where you can actually work on a project that ships to customers. For me in the case of TypeScript, I leaned pretty heavily towards the "wing it and only go as deep as needed" side, but there are tradeoffs there. I probably won't be writing too many posts focused on TypeScript with this approach, since I'm not currently planning to dive deep into the more advanced features... but I guess we'll see.

Thanks for reading! Hope to see you around here again soon 😊

Tips and Tricks - Starting a new [Thing]

Jordan Nielson — Fri, 28 Feb 2020 19:08:41 +0000

Originally posted on my website, jnielson.com

Hi there! As I mentioned in my last post that I was changing jobs, I've recently had the pleasure of starting in on some new projects at a new job. It's been a rollercoaster trying to absorb as much information as possible while also trying to contribute to the forward progress of the company and the projects that I get to work on. It's really easy when starting on a new [Thing], be that a project, employment opportunity, or something entirely different, to be discouraged and overwhelmed at the sheer amount of information you need to gather. In this post I'll outline some of the thoughts I've had as I have dove head first into my new job where I've worked on a major project and at least 3 smaller projects in my first two weeks.

The high-level thoughts:

find someone you can ask all your questions to

Don't be afraid to clarify processes

Put up half-finished work to get feedback

See if you can setup some regular pair programming

Find something you can do during work to connect with people

Let's dive in!

The first thing I've appreciated with this new employment opportunity has been the focus on helping me become familiar with the projects and productive as quickly as possible. It's not fun to sit for a couple hours not knowing how to move forward on a large project, since even the smallest task can be daunting in a large codebase. The major project I'm working on has had a few devs working on it for a while now, which means that it's super quick for them to do the small task I was assigned... but if they do it I can't learn nearly as well! So, I'm grateful that my company has a structure where there's a lead developer (lead dev) who has it as a part of their job description to directly support other developers, especially the new ones. Sometimes this means that the lead dev spends way more time supporting that new person on a task then it would take for them to do it. A quick example...

This screenshot is actually from a w3schools example since I was working on an internal project

Tale of the dropdown search field

One of the first small tasks I worked on was updating a dropdown search field to automatically focus the search box when it is opened. While my lead dev could have finished this task in probably 30 seconds, he assigned it to me knowing it would take me way longer than that to finish. As is pretty typical when working on a project for the first while, it can take a good amount of extra time to track down where things are happening and how to adjust existing code (that was written by someone else) to accomplish the task at hand. So, in this example I poked around on the running site (locally) to figure out where the code for this dropdown search field lived... and eventually found it. While I could have asked my lead what file to make the change in, I've found it super helpful since then to have spent some time digging around in the project, as it built up some context on where the various pieces are coming from and how they fit together.

Things I've done

Now let's focus on some things that I've tried to do as "the new person" to have more fun and be more useful and productive! The first, and in my opinion, most important is I've tried to find a fun way to be involved and connect with the people at work. Connecting with people has a number of benefits, like making it more enjoyable to be at work even if work things are hard. The way that I've found most fun to connect with the people in my office has been foosball! It's fantastic, especially since there was already a decently active group before I started making it easier to jump in and meet people. I've also tried to be involved in the smaller lunch group that hangs around the office and chills. Today a part of that group played Super Smash Bros, which I joined and had a blast! Basically, find things to do with people, in my case one on one or in small groups to get to know them.

Looping back to the list

Of the high-level thoughts I listed above I've really only talked about the first and last ones. The middle ones are a bit harder at least for me since they involve leaving my comfort zone and admitting that I don't know everything.

For example, on my first day I was told what the basic outline of the dev process was and that it applied to most projects. A few days later, I was working on a project doing what I thought was the process and trying to follow my advice of opening a PR early with half-finished work, and then noticed... the PR from another person that was up was to an entirely different branch! So, I quickly reached out to my lead dev sharing the process that I thought I was following and asked if it applied to this project. He responded promptly (thankfully) and filled in the gaps, since this particular project has a stricter process than most of the other ones at the company due to a large upcoming release that I was helping with. This experience helped me realize how important it is to clarify things, especially when jumping between different projects.

One of my favorite things about this position has been the scheduled, Monday through Thursday, hour long pair programming block with my lead dev. It's super useful! It's a time that I can ask pretty much any question (related to work) that I've run into since the last one, if I didn't get to catch him more quickly. It's also a time where I know I have their full attention, since they don't even bring their laptop in. This time has also been critical in orienting myself with the various tasks I've worked on in these first two weeks, helping me to make progress faster by leaning on his experience with the codebase.

What do you do?

These are some of the things that I've noticed have helped a ton during these first two weeks at my new job. I'd love to hear about what you've done! Feel free to reach out on twitter or use the comments below.

Evaluating JavaScript Open Source Packages

Jordan Nielson — Tue, 17 Sep 2019 02:20:11 +0000

Originally published on jnielson.com

In almost every project there will come a point where you reach for an existing package to augment your source code to make it simpler, more robust, or to reduce the amount of maintenance. When you reach for a libary it is important to evaluate the different options available, on a number of different fronts. If you simply evaluate a package based on if it will accomplish the task you have for it, you might miss out on some important considerations that can come back and hurt you later. For instance, if you pick a package that only has 1 line of actual source code and no tests does that package really give you any benefits? I would say it doesn't, but each project is different and it might be helpful to your project to be able to import that 1 line from a package instead of a local file.

So, what do you consider when evaluating a package? In my experience, it's helpful to have a list ready to go through for each package that you think might work for the need you have. My list usually looks something like the following:

License permissiveness
Stability
Maintenance Level
Popularity
Documentation Quality
Amount of code in the library
Amount of code it will require/save in my project
Open and Closed Pull Request trends
Contributing Documentation

In using this list, I have a number of questions I ask myself to evaluate each category, which I'll include below. You should generally answer as many questions as you need to ensure a package isn't a definite "no", and write down your answers! By writing down the answers that you come up with to each question you pose for the various packages you're providing notes and documentation for future you in order to answer the question 6 months from now, "Why did you choose [this package] over [that package]?" (which comes up more often than not).

I want to point out that this list is definitely personal opinions that I have found useful in my projects. If you have other suggestions, feel free to share them with me! I'd love to know what other people do when evaluating packages, since there might be critical steps I'm missing.

License permissiveness

The first point that I usually check when evaluating a potential solution to my problem is the license that it uses, and how permissive that license is. When evaluating packages for use in a work project it's super important to follow the guidelines of your organization, as it can cause some major issues if you use a license incorrectly. So, the questions that I ask myself regarding the license:

Is the license on the short-list of pre-approved ones for the organization?
Is the license something I'm comfortable with using in a personal project?
Does the license require any sort of attribution or having my project adopt that license?
Is using this package worth the license requirements?

Stability

In order to properly evaluate a package for use, I feel like looking at how stable the package seems is critical to future efforts to stay up to date. I recently wrote about upgrading your project dependencies and the work that it takes to do that regularly is significantly less if you use stable packages. How do you determine a package's stability? I usually use questions like the following:

Are there frequent releases? (This could be good or bad depending on the package)
Do they utilize alpha/beta/canary releases to test out changes? (and do people use those?)
Does it seem like people usually stay on the latest version? (This is a hard question to find data on quickly, but super worthwhile)
Is there a cycle for making breaking changes (major versions when using semver)?
Is there a large number of open issues? (Raw issue counts don't always share useful information, but it can be helpful to check)

There's all sorts of different metrics and things you can check on to try and determine how stable a package is, but I find it important to consider because some of the other aspects become significantly less important if a package is in a stable state and doesn't require any major changes or bugfixes.

Maintenance Level

If you have a project that isn't particularly stable, it becomes vital to determine what the current maintainer's level of involvement is. If there's just a single person working on the package, is that their full-time job? Or do they have other higher-priority things? If there's multiple maintainers you might consider if it seems like there's a "lead maintainer" who if they dropped off the package would stop moving forward. Particularly with less stable packages, having changes and improvements stop is not what you want to have happen since that could lead to critical bugs not being able to be fixed without forking the package. In determining the maintenance level I tend to ask myself questions like:

How many contributors are there to the package source code?
Does it look like there's a single person with permission to release updates, or can multiple people?
Are there any people working on it full-time? (Not every package needs full-time people working on it, but core ones I like to have it)
How quickly are issues and pull requests addressed? Does it seem like the maintainers work on it in spurts?
Is there a way to contribute funding/support to the maintainers? Can you do that?

Popularity

An important consideration in tandem with the Maintenance level is the popularity of the package. If a package is popular enough, it becomes more important to have a higher maintenance level through full-time maintainers or other means. But, important packages are useful because the maintainers tend to develop better habits relating to releases and documentation as the package increases in popularity. Picking a package purely because it is popular isn't always the best choice for a project, but it can make a difference in the resources that are available to help you figure out how to use the package. It can also impact how likely it is that the package maintenance will be picked up should something happen to prevent the current maintainers from keeping up with it. An important consideration with popularity is usually the package download count, but I don't find that metric to be particularly useful since it doesn't give any insight into how many projects are actually utilizing the package. I like the new Github "Used By" badge a little more but it still has some serious limitations. For now, I will just point out the things that I use to try and approximate how popular a package is:

What do the comparative download counts look like? (It's important to use relative amounts, since raw download counts are meaningless)
What does the github "Used By" badge say? How does that compare with other options?
How does the project compare in google result counts?
Are the questions on stackoverflow answered and accepted? Does it look like it is a community effort to answer questions?

For comparative download counts I usually use npmtrends, which doesn't provide a complete picture but can give you a decent idea of what the package looks like from that perspective. StackOverflow allows you to view tags, so you could check the styled-components tag to get a feel for if those questions are being answered and who is answering them.

Documentation Quality

An important and often overlooked aspect of an open source package is how good the documentation is. For simple packages the documentation might be sufficient as a simple readme that details the exports and how to use them. For more complex packages you'll often find they have an entire site dedicated to documentation. In either case, it is important to consider the documentation quality relative to the code you're pulling in. I generally use questions like the following to assess the quality of the docs:

When was the README last updated?
Does the README include all the exports from the package or are there undocumented features?
Is there a changelog or place for release notes?
Are the changelog/release notes more clear than the list of commits?

If they utilize a documentation website I'll check that out (ideally the source code for it in addition to the live site), using questions like the following (in addition to looking at the in-code documentation):

Does the website get updated when things change?
Is the website organized to make it easy to find what I need?
Does the website have missing information?
Are there terms on the website that are unclear and not explained?

Evaluating documentation quality is very personal, since what might make perfect sense to me has the possibility of completely confusing the next developer to come along. This is a step that I think it's super useful to get more than one person's opinion on, since usually people will generate an opinion on documentation as soon as they look at it the first time. Doing a review of the available documentation is pretty critical in making sure that the library actually solves the problem you're trying to solve, but it also helps to evaluate how easy it will be to figure out how to use the library and handle the weird things that will inevitably come up.

Amount of code in the library

Sometimes pulling in a package is helpful, but other times if you're just going to use it in a couple places you might be better off with writing more specific code for your project. In a lot of cases packages have extra abstractions that make them more reusable but add more code than you might need in your use. It might also be the case that there's a built-in language feature that can do most, and sometimes all, of what the package does. For example, it used to be common to use the left-pad package (and it still has 4 million weekly downloads), but JavaScript has a built-in padStart() method which does the same thing and doesn't require you to pull in a package. So, in some cases you want to make sure to ask yourself questions like:

Can this be done using a built-in language feature?
Does the package provide extra features I don't need?
Would there be a significant maintenance increase in my project to write the functionality myself? (Usually this is a yes, but some packages are trivial)
Does the package have tests and other code quality tools used that I don't want to replicate?

Amount of code it will require/save in my project

Something that isn't talked about often enough when using packages is the idea of how much code is required to consume the package. In some cases, it might be worthwhile to build a project-specific version of a package simply to reduce the amount of code needed to use the package. Dan Abramov wrote a post titled "Optimized for Change" which mentions this idea of "second order" API design - how does code using this API will look and evolve. In order to evaluate this idea for packages I look at, I tend to address questions like the following:

How long are the examples?
How much difference is there between the examples? Is there a lot of boilerplate?
Does the amount of code the package saves outweighs the amount of code it requires me to write?

Open and Closed Pull Request trends

Another thing to glance at related to Maintenance level is the open and closed pull requests for the package. If a package has a bunch of open pull requests that sit there for a really long time with no movement then there's likely an underlying issue with the maintenance level that I wouldn't want to commit to using the package. If there's no open pull requests that's usually a good sign, but if there haven't been any pull requests in the recent time span I'd want to make sure I had evaluated how stable the project is earlier on. If there's a lot of open pull requests but also lots of recently closed pull requests it could simply be an incredibly active package, but in most cases I've come across the issue is usually related to a lack of maintainers. So, I ask questions like:

How long have open pull requests been open?
Are there any recently closed pull requests?
Does it look like pull requests are closed in batches? How frequently does that happen?
Are pull requests usually closed (declined) or merged?
Is there a lot of chatter on pull requests before they're approved or declined?

Contributing Documentation

One of the last things that I check is if the project has contributing processes documented. Usually this isn't too big a deal since most people will document them as a favor to future them, but sometimes a project only has a single maintainer who hasn't cared to document the process used to make changes to the package in a reliable way. The questions here are usually:

Does contributing documentation exist?
Is there a pull request or issue template?

Conclusion

Once you put a package "through the ringer" and evaluate it, then what? How do you determine what package to use? Usually it comes down to a personal/project preference of what package makes the most sense. In almost all cases it will be a trade-off as one package might be smaller but less robust or another might be more feature-complete but add some extra bundle size that you'd prefer not to add. In any case, what packages you ultimately choose should be a decision that you are able to back up, so ideally you write down the answers to the questions you posed to each package as you do it in order to have some documentation for future you! I find that there's great value in being able to answer the question, "Why did you choose this package over the other option?" 6 months down the road by having notes, particularly since if things have changed in those 6 months you have the notes from last time and you can re-evaluate with ease!

Outline / Prep

What matters with a JavaScript Open Source Package? (Not ordered)

TL;DR Points:

Check the stability and popularity - are there frequent releases? Do people use it?
What's the Maintenance level? Are there people working on it full-time?
Code/Documentation/Contributing quality - are things documented and working?
License check is vital! Make sure it is a license you can use
Consider writing it yourself if it's insignificant?
Stability (How often do they release? Do they do alpha/beta releases? Do people use those?)
Maintenance level (are people working on it full-time? Does it have multiple contributors? etc)
Popularity (How many other projects depend on it? What do the download counts look like?)
Code Quality (Do they have tests? CI builds?)
Documentation quality (Is it up to date? Do they have previous versions available? Is it a separate site or just a readme?)
Contributing Documentation (Does it exist? Do they have Issue/PR Templates in github?)
Open and Closed Pull Requests (Did they sit for a long time? Was there back-and-forth conversation about them?)
What license(s) does it have involved in it? (Is it clear why it has those? Does your organization allow it?)
How much code is actually there? (bundlephobia & looking at the source - it might be better to inline)

Banner image courtesy of undraw.co

Updating Your JavaScript Project Dependencies - A Why/How Approach

Jordan Nielson — Thu, 12 Sep 2019 00:00:00 +0000

Originally published on jnielson.com.

Why Should we Update?
How often should we update?
What is the best way to update?
When should we not update?
Why is updating hard?
What makes it easier?

Some time ago, I set off to answer the above questions. In my position at work we interact with a lot of developers on a number of different projects. Some projects are consistently up to date, while others are always at least 6 months behind. From what we've seen, the difference between these two groups of projects is usually not in how they view updates (generally people want to update), but in how much care they give to maintaining their project. The projects that are up to date, are also usually the ones where the developers are encouraged to spend time refactoring code when they're working in a section of the application, or the ones with the best documentation. On the other hand, the projects that are consistently out of date are the ones where developers are encouraged to go as fast as possible in adding new features, and rarely go back to refactor or document.

A Lawn Analogy?

This is a similar idea to taking care of your lawn, since if you only work in your lawn in the place where you're planting a new flower, tree, or bush, the other areas just grow and grow without ever being pruned or cared for.

Eventually, you might get a notice from someone that you have a tree about to hit a power line and you have to have it trimmed or they'll cut it down at the end of the month. At this point, the work to trim the tree is significantly greater than if you would have pruned the tree a few months ago and spent some time keeping the tree well maintained. Some people when faced with this situation will let the company come cut it down and then pay the charges that come from it, while others decide it is worth the effort to trim the tree back themselves, and still others hire someone else to do it for them in an effort to save the tree.

How does this relate to updating your project dependencies? Well sometimes you have a project where you do a great job of regularly updating your dependencies and it takes a little effort regularly but more often than not it's straightforward. Other times you have a project where you only work on it occasionally and so it never receives any updates and when you do decide to update it takes forever and a ton of work (like this blog, which hasn't been updated in a while and has some 44/55 dependencies out of date). Since I've come clean and pointed out that my blog site hasn't been updated in a while, I'm going to take this chance to outline my thoughts on answers to the above questions (essentially who, what, when, why, and how - the basic questions of life they taught me in school) and hopefully that motivates me to update my own blog.

Why update?

So, why should we update? From what I've seen the typical answers for why update include things like the following:

Bug fixes
Security patches
Cool new features
Easier to submit bug reports
Keeping up with major changes to make future updates easier

In my experience, the number one reason to update is to receive bug fixes. In pretty much every release of the libraries I follow there are at least some sort of bug being fixed, even if it doesn't impact me right then. Related to bug fixes are the important security patches, which are occasionally back-ported but more often you just need to be on the latest version in order to get them. Sometimes they are so vital that it's worth being on the "bleeding edge" or the latest release of a library to get an update that fixes some bug or security issue that has actually impacted you, and I know that libraries appreciate those who are willing to risk being on the "bleeding edge" and trying their alpha or canary builds in order to get a good test of it before promoting it.

Another reason to update is to make future updating easier. If you take the time to make it over that major update spot you'll be able to more easily update in the future since usually libraries and tools release changes in an incremental fashion that gives you an easy set of steps to follow if you're on the latest version. For instance, React recently released a version that deprecates some component lifecycle method versions, requiring you to rename your uses of them to an UNSAFE_ variant. But, they've also provided a codemod that you can run which will do it for you automagically! In cases like this, the codemod would theoretically work on any version of the library that has support for the feature. In the case of storybook.js, in version 5.2 (which is in a release candidate version right now) they introduced a new way to write your stories called CSF. They also provided a codemod that allows you to automagically update most of your stories, but there are cases that it doesn't work for. In cases like that, being on the latest version of the library ensures that you can receive the best support since that's the targeted upgrade path they've been working on.

From the above cases, you can see that there are some huge upsides to using popular and stable libraries that try to minimize breaking changes. They're motivated to keep people on the latest version, and so they provide tools to make upgrading easier or change things less drastically since drastic changes make for harder upgrades!

How often?

Now that you've been convinced that you should update (and if you're not convinced, reach out to me with suggestions on how I could improve that section), how often do you update things? Well, it really depends on your project! From what I've seen, there are about 4 categories you could split into:

Active development applications
Maintenance ("every so often") applications
Libraries
Rarely touched and even more rarely organized projects

In my experience with applications you can get into a pretty good rhythm based on how much you work on the project. From what I've seen, projects that are actively worked on (daily in most cases) should be trying to update their dependencies every "sprint" near the start to give time to naturally test the updates instead of needing to run through all the test cases for your application right before releasing things. In other words, if you update early in a cycle you can spend some time to fix things that crop up instead of trying to do it late in a cycle and having to rollback if you run into any issues. Some people might find updating every sprint to be overkill, but I'd suggest that it's better to have a small update reguarly instead of waiting until there are a large amount of updates.

In a different case you have the applications that are worked on "every so often" when there's a major bug or some feature that needs to be added/adjusted. In this case, I find it best to check for updates but not actually do them unless they're minor and easy to verify. In the case of projects like this I tend to let updates build up a little bit, so that once you've got enough updates that look useful/major/helpful you can add a task to do the updates with a full verification that things are still working after. Ideally you'd have written automated tests that can be used to speed up that verification, but I haven't seen many projects that actually do that effectively.

For libraries I've found that as often as there are updates to dependencies it's worth at least looking into updating to use it. In most cases there is benefit to updating and releasing a version since those updates will include bug fixes that you might not have run into but your consumers might.

The last category of suggestion that I've found is inactive side projects, or things that are worked on so in-frequently that there isn't even a system in place to track what needs to be done. Since I've been blogging more regularly lately this project doesn't fall into that category anymore, but for a while I didn't even have a list of things that I wanted to blog about or do to my blog since I hadn't touched it in months. In the case of projects that haven't been touched in months usually I've found it more useful to do what needs to be done and get out, or get back into a regular cadence of working on it before trying to update. In the case of my blog site, I've been enjoying the current authoring experience enough that I've been doing it in a more regular manner which makes it easier to justify updating things.

How to update?

Once you've determined how often to update, how do you actually do it? How do you make it happen on that schedule that you just determined? In my opinion, using one of the tools that will automatically check your package.json for updates is the best way to go if your source is open and available. I haven't investigated them too much, but I've seen projects use tools like greenkeeper or renovate with what seems to be great success. In most cases, these tools can submit a pull request to your project whenever a new version is released and if you've got it setup correctly that can trigger a run of your test suite and a test deployment of the application. In cases that you have closed source or want to do it more manually, there are great tools like depcheck or npm-check (which is best run as npx npm-check -u in the project folder, in my opinion) which can programmatically read your manifest file and check for the latest versions. Either of these approaches are preferred in my mind over manually check with npm show [package] or similar going through your dependencies one by one to check for a newer version and update to it. In my opinion, this is a case where automation can really shine though, so embrace it!

For the npm-check tool (the one I use most frequently), if you use it with npx npm-check -u in your project, the output would look something like this if you have no updates:

If you have a few updates it would look something like this:

While if you have a lot of updates it provides some nice pagination for you:

In the case of the minor and patch updates that it splits out for you, those are usually pretty safe if the libraries you use follow semver. If they don't that makes things more difficult, but most JavaScript packages (the topic for this post) do use semver and try to stick to it. If you update a patch/minor update and it causes issues, most libraries will work with you to fix them and release a new minor/patch update.

In the cases of more major updates, libraries or tools will frequently include a codemod, or a script that can be run that allows for automatic updating to a new syntax, name, etc. These codemods are super handy, and I'd highly recommend using them! If you have a case where one isn't provided, write your own! Most codemods I've seen use jscodeshift under the hood, and I would think most library maintainers would appreciate some help getting people to update (I know I do!).

Outside of automated tooling, you'll want to make sure that you read through any provided documentation like changelogs or release notes that can give you insight into things you might need to update. For example, if you're running into issues installing a library like gulp-sass or node-sass the first place you'd want to go is to check their release notes to make sure that the version of node you're on is compatible with the version of the library you're trying to install. In this specific case, the library depends on some native hooks in node, and so if it isn't compatible with your node version it won't work at all. So, you'd go to their releases page, like the one for gulp-sass which usually looks to just say it updated to X version of node-sass, which has a releases page as well, which details what versions of node are supported.

Are there times to not update?

Sometimes, especially when projects use the automated tools mentioned above I've seen some of the update pull requests sitting for a long while. I've occasionally wondered why that is, since these projects have a great test suite that ensure things are still working on the new version of the library, but for whatever reason the maintainers have chosen not to merge the pull request. In my thinking, there are a few reasons for this:

Some feature was removed without a replacement (usually results in tests failing though)
Want to allow major versions time to "bake" (get bugs out and let others find regressions and issues)
Want to hold off on dependency updates and batch them
Higher priority projects or tasks don't leave time for lower priority projects or updates

If you know of other reasons that people choose not to update, let me know and I'll add to this list! I'd love to understand the why behind them, especially in cases where the tests (automated and manual) still pass. From what I've seen most of the time it's to allow "baking" of the update, and the pull request gets merged after a couple weeks (and run with the latest project code). This time allows others in the community to find issues for you, and ideally gives you a better experience in updating. Some people have enough risk tolerance to update immediately though, and those are the people that I appreciate for their efforts in stabilizing the dependencies.

What makes updating hard?

Outside of the times that you choose not to update for whatever reason (if it isn't captured I'd appreciate you sharing it with me!), there are times where updating is just plain hard. That's okay. Sometimes the project is so large and complex, or there are too many things that need updates, or you simply don't have time to test all the things in your application that could potentially be impacted by an update. In all of these cases, there are valid reasons to not update or to put it off, but from what I've seen people usually regret those decisions-especially in projects that are in active development since they come due later. So, some things that I've found that make updating hard or more difficult:

Requires ensuring nothing in your code has broken
Incomplete documentation on what changed
Updates are super frequent (some libraries release new versions almost daily)
Requires checking for the updates (manually or automatically)
Some libraries don't follow a versioning scheme (semver)

As I said earlier, each of these is a totally fair reason to not want to update. In most of these cases there are things you can do to make it easier, but sometimes it is just plain hard. The hardest I've had to deal with is libraries that don't follow any sort of versioning scheme, so you never really know what you're going to get when you update. Most projects I work with follow semver, which provides a nice framework to make your update decision within. But, the ones that don't (or say they do but are still on 0.x.x versions) are incredibly difficult to keep track of.

What makes it easier to update?

Outside of the truly hard cases, and in spite of them, there are some things that you can do in your application to make it easier to update. For example:

Automated tests that give you confidence
Automated tools that make it less manual to check for updates
Reading documentation and changelogs (and submitting pull requests that update/create them if they don't exist!)
Making it a regular task is easier than doing a huge update every couple months

One of the best things that you can do to make updating easier is to create a suite of tests that can be automated and run whenever you update. When you've got tests that give you confidence it makes it way easier to catch when an update breaks things (expectedly or unexpectedly) and also speeds up the update process. To some extent, automated tests are a requirement for regular updating since without them you'll likely fall into the trap of feeling like you need to manually test your entire application after any updates, regardless of their size. Ideally your tests are written in such a way that if a dependency changes it doesn't fail unless your application legitimately breaks, but that's a different issue.

In addition to tests, using the automated tools I mentioned above make it way easier to manage a large number of dependencies. For instance, npm-check provides links to the readme/changelog/repository if it can find it, which is super useful to go check and see what changed.

To reiterate, having a regular task of updating makes a huge difference in how much effort it takes! It is way easier to justify and make time to read 3 or 4 changelogs for libraries then to go through 40+ and do a major update on most of them. In any case, you're way more likely to quickly diagnose issues when updating 2 or 3 dependencies at a time, even if you eventually have to update 44. There are times where you have dependencies that need to be updated at the same time in order to ensure you don't run into larger issues. For example, storybook and next both use babel under the hood. A while back they were updating to babel 7, but they did it at different times and so it was significantly easier to wait until they were both on babel 7 and update our projects after that. The alternative would have been to update one and adjust configs and things to isolate them to support babel 7 in one case and babel 6 in another, which is doable... but work that would then need to be un-done once both were on babel 7. So again, there are times where it makes sense to put off updating or just not do it.

Another point to making updating easier is to have less dependencies. It might seem obvious, but updating is less painful if there is less of it to do! One great way to accomplish this is using tools like gatsby or next which provide a lot of the tooling that you need to get up and running for you, and hide a huge number of dependencies (and update them for you!) to make it easier to work on your actual project. One feature of gatsby that I'm loving is themes, since they can abstract even more away for you. Since I originally wrote this post I spent about a day updating to use the @eggheadio/gatsby-theme-egghead-blog and got to rip out most of the things I had in my blog codebase from the @eggheadio/gatsby-starter-egghead-blog, since they now exist in the theme! It was actually a pretty fun experience updating from the starter to the theme, especially since I hadn't done all that much to customize it and could just rip most of it out.

That's all that I've got for now on updating dependencies, feel free to reach out if you have other thoughts on the matter as I'd love to hear them! Thanks for reading! For those who might appreciate seeing a bit of the process, I left the outline for this post below (it also provides a nice summary for those who might have skimmed). If you like that, let me know and I can do it more often!

Outline (for reference if someone was curious or wanted an overview)

Why Should we Update?
- Bug fixes from dependencies
- Security patches from platforms/libraries (ex. node)
- Cool new features (ex. react hooks)
- Easier to submit bug reports (most libraries require you to be on the latest version to ensure the bug still exists)
How often should we update?
- It depends!
- Active applications should at least check for updates each "sprint" (every couple weeks) to avoid build-up, and should do updates at least monthly
- Maintenance applications should do minor updates regularly-every 2 months or so (#semver), with tickets/notes to complete major updates with more testing
- Libraries should update as often as needed (usually they have less dependencies), ensuring to communicate semver changes as needed
- Side projects (like my blog) get an update when there's time - making progress is more important than updating unless you've hit a bug or need a feature
What is the best way to update?
- Use automated tools or scripts like greenkeeper, renovate, depcheck or npm-check
- Other options include manually installing the latest version of packages
- Automation is great at this though
When should we not update?
- Feature removed that you need without a replacement
- Major version changes sometimes need some "baking" time to find and fix major bugs or regressions (ex next 9.0.1 fixed some dynamic routes issues)
Why is updating hard?
- Requires ensuring nothing in your code has broken
- Incomplete documentation on what changed
- Updates are super frequent (some libraries release new versions almost daily)
- Requires checking for the updates (manually or automatically)
- Some libraries don't follow a versioning scheme (semver)
What makes it easier?
- Automated tests that give you confidence
- Automated tools that make it less manual to check for updates
- Reading documentation and changelogs (and submitting pull requests that update/create them if they don't exist!)
- Making it a regular task is easier than doing a huge update every couple months

Banner image courtesy of undraw.co, lawn image from Wikimedia commons

Demystifying webpack - What's a Bundler doing?

Jordan Nielson — Thu, 29 Aug 2019 23:46:11 +0000

Originally publised on jnielson.com

In my introduction to this series on Demystifying Build Tools, I introduced the core concepts of webpack and babel. I've created a couple other posts on various facets of babel, like @babel/preset-env and babel plugins more generally. If you haven't read those, I'd highly recommend them (obviously!). In this post I'll shift and cover a little more about webpack. In the talk I'm prepping for, I'm intending to spend more time on babel and less time on webpack, which you might have guessed from the blog coverage difference.

Why less on webpack?

I haven't had nearly as much in our projects to manage with webpack since we're using the defaults provided by next.js (thanks Next team!). But, the things that I have found valuable to be aware of include a knowledge of what webpack is at a little more depth than the concepts docs referenced in the introduction post and also how to use and read the webpack-bundle-analyzer plugin. In my opinion, having a knowledge of webpack makes it simpler to work with as the core concepts build together masterfully, and then the bundle-analyzer plugin is super useful to examine what webpack is outputting that I can't imagine doing a project where I don't use it at least once to sanity check that nothing I don't expect is included in the bundle.

So, to learn more about webpack where do you start? First, I'd start with breaking down the description they use for webpack in their docs:

"At its core, webpack is a static module bundler for modern JavaScript applications."

webpack docs

That statement is relatively simple, but can be broken down to emphasize the key features and goals of webpack. I'll talk more to each of the following ideas:

Bundler
Module
Static
Modern JavaScript
Applications (including libraries)

Bundler

At its core, webpack is a bundler. Not a task runner or a compiler, a bundler. What is a bundler? In the context of webpack, it takes all files referenced from the entry point(s) and spits out at least 1 file called "the bundle". The goal of the bundle is to package code in a way that makes sense for the target environment, in most cases that's the browser. With HTTP 1.1, it tends to be best to serve as much of the application in a single file, to reduce the number of round-trips needed to get the code for the browser to execute. But, with HTTP 2 as well as in environments where you want heavier caching it makes sense to split your "bundle" into multiple files that can be cached and served independently and in parallel.

How does webpack's role as a bundler impact you? Well, for the most part it doesn't. Since it's a bundler it usually does its thing just fine, and once setup in an application it doesn't take much maintenance unless you add a new file type or want to process something differently. More on that later though!

Module

In stating its place as a bundler, the webpack docs clarify that it is a module bundler. In that aspect, it treats everything as a module: JavaScript Code, Images, Raw files, you name it and it is a module in webpack. Modules are loaded into webpack through a variety of loaders, which you can read more about on the loaders concepts page. Essentially in order to support a large variety of file types you'll need to add loaders for them so that webpack can understand them. Out of the box it supports JavaScript and JSON "modules", much like Node itself. In webpack 4 at least, the module type you use greatly impacts the extra features webpack is able to enable, such as Tree Shaking. Modules are key in webpack, since that is how it determines what code to include in the bundle that it creates. It starts from your "entry point" (which is a module) and pulls in everything referenced by that module. In order to pull it in, it needs to be a module! So, anything that you import in that entry module will end up in your bundle that is created. Without module boundaries, webpack wouldn't be able to determine code that can be left out, and we'd be back to including entire directories in what we serve to the browser.

Static

One of the best features of webpack, in my opinion, is the static analysis capabilities that are unlocked by it being a static (in other words, build time) module bundler. A runtime bundler could probably work, but it wouldn't be able to do Tree Shaking or Dead Code Elimination. This would be a pretty large drawback for me, since it is pretty common in my projects to only use part of the aspects that a library or component exposes. In my opinion, the word static in this context also implies that the build output won't change unless the build input does (assuming you have things configured correctly), which gives me some confidence in being able to run builds as many times as needed. Related to that, another benefit of static in this context is that it allows the build process to support plugins that act on those static assets to transform, adjust, or otherwise do something to the code.

There are some downsides to it being a static module bundler. One of the largest I've run into is the inability to dynamically use require.context in storybook to get just the stories that I want with some sort of option string. This led to us re-writing our storybook config file whenever we want a different set of components to work on, which thankfully was relatively easy to implement.

Modern JavaScript

Since the docs statement says "modern JavaScript applications", I decided that there should be a comma in there and broke it down even further. Modern can be made to indicate that it is something up to date, but I think when you combine it with JavaScript you usually get the idea of ESNext or ES2015. In the case of new language features, that job is actually handled by babel, which webpack can run on your code as it bundles it. This interplay is something that I wanted to highlight since it illustrates the capability of the module bundler to take in anything that you can tell it how to handle. Since it runs in node, webpack can be default handle whatever syntax your version of node can. Since you can run it with babel, webpack can optionally handle whatever syntax you throw at it (within the limits of babel of course). These two libraries work together to output your code in a manner that's suitable for browser consumption. In the simplest configuration, babel will take your files and output them, one for one or all to one, transformed according to the plugins you use. Using webpack, it can be a little smarter than that and only run babel on the files that it is bundling, allowing you to have other files in your src directory (or however you organize yourself) that don't need to be processed by babel.

Splitting this up further, Modern is also a good descriptor of webpack itself. The team there does a great job adding new features/plugins, fixing things, and overall keeping the tool modern in the sense of up to date and useful! JavaScript by itself doesn't mean all that much though, it does indicate that webpack is focused on that language (though if I understand correctly it supports web assembly to some extent).

Applications (including libraries)

The core use case for webpack is definitely applications that are served to the browser, but it can also be used for libraries if they have a desire to do so. There is support for libraries in a similar way to applications, and they have an awesome guide on their docs site about how to use webpack to bundle your library code. Since webpack focuses on the application level, there are tons of plugins that support that use providing things like aliasing, loading all the file types you use, and others.

The Bundle Analyzer

After you've got webpack setup and outputting some wonderful files to serve to the browser, you might run into a case where you're curious what is in there. In most cases, your bundle will be minified and uglified so it won't be much good to try and read what's there, though there are some things that don't uglify very well that you can use if you're trying to check to see if something is there quickly. But, outside of that the webpack-bundle-analyzer is a fantastic tool. For use in next.js, it's as simple as installing the Next.js plugin and following the instructions in the readme to add it to your project. Since Next produces two bundles, one for the server and another for the client, it can be pretty intimidating to set up any webpack things from scratch. So, I'm super grateful for the team that added this plugin since it's already setup to create a bundle analyzer for both bundles. Most of the time I just use the client bundle, but the server bundle is also quite helpful. The bundle analyzer looks pretty overwhelming when you first look at it, since it shows in some manner every file that is included in the bundle. There's a number of things to look at when using the bundle analyzer, but there are a few that I want to call out:

Different Size Settings
Hiding chunks
Outputting a JSON file (not currently supported by the next-bundle-analyzer plugin)

Different Size Settings

One of the first things you might wonder is "where does this size information come from?", since in most cases you won't be seeing what your file explorer told you the size was. In the sidebar menu when analyzing your bundle, you can select between stat, parsed, and gzip. These are described in detail on the documentation page linked above, but I think it's useful to point out that stat should be close to your file system output, parsed should be the post-webpack size (minified/uglified) and then gzip is the compressed size of the post-webpack file. By default the parsed size is pulled up, which is why I pointed out that they might look different than you might expect. In most cases I've seen, parsed is the most useful number, since stat doesn't help much as it's pre-webpack and gzip is useful... but I don't want to spend my time optimizing my code for gzip compression since the time the browser spends parsing it is usually longer than the network time a few more bytes off would save. There's more information on this in the documentation.

Hiding Chunks

In most cases, the output from the bundle analyzer will be entirely too much to handle as most projects that care to analyze their bundle will have hundreds of modules. If you haven't used it before, clicking on a module/section will zoom in on it, but that doesn't actually hide the ones that now can't be seen. To do that, you can uncheck them in the sidebar menu, which will actually re-draw the entire page in most cases. There are a number of things that you might want to hide, like a node_module that you're stuck with and can't reduce the size of or a section of your application that you're not working on right now and is distracting from the actual part you are inspecting. There's more information on this in the documentation.

Outputting a JSON file

In a lot of cases, webpack has way more information available then even the bundle analyzer shows, and in that case I find the bundle analyzer's capability to output the stats.json file from webpack for you to be wonderful. Since the bundle analyzer already uses a lot of the stats options (and webpack does slow down a bit when you use a bunch of stats options), it's helpful to be able to re-use those and output them to a file. Sadly the next-bundle-analyzer plugin doesn't currently support passing any options to the bundle analyzer (they'd probably add it, but I haven't cared enough yet since it isn't terribly hard to use for a one-off case). So, if you want to do this in a next context you'd need to manually adjust your next.config.js to use the bundle analyzer (in a similar way to what the plugin does ideally) to pass the generateStatsFile: true option to the bundle analyzer, with the statsFilename changed based off which build is running. The stats file is a bit of a beast to handle, so we're not going to talk about it much here, but it is super useful if you think webpack is doing something weird!

Thanks for reading! Ideally this helps you understand a little bit more about webpack, in combination with going through their core concepts docs. I'd highly recommend spending some time on doing so, since even if you're using an awesome tool like next.js there's still benefits that come from understanding what is happening to bundle your code.

Cover image courtesy of undraw.co

Demystifying Babel Plugins: A Debugging Story

Jordan Nielson — Tue, 27 Aug 2019 22:38:53 +0000

This post was previously published on jnielson.com. Thanks for reading!

In previous posts in this series, Build Tools Demystified and Demystifying @babel/preset-env I introduced the ideas I wanted to cover and then dove into @babel/preset-env to see what we could learn. In this post, I'm going to do a little more general treatment of babel plugins, though the examples I use will largely be babel-plugin-styled-components and @babel/preset-env since those are the ones that I most commonly utilize in my projects.

As part of understanding more about babel plugins, it's important to realize that the amount of things you need to know to work with plugins is significantly lower than what you need to write plugins. For writing plugins, definitely check out the babel handbook on plugins but there isn't a similar handbook of instructions that you can follow to work with or understand an existing plugin. If it was written following the handbook then you should be at a pretty solid place, but I'm going to assume most people haven't read through that awesome (and long) handbook and roll with that.

Major Concepts

There's 2 major concepts involved in babel plugins: Abstract Syntax Trees (ASTs) and the Visitor Pattern. The basic idea of ASTs it to take the code that you write and parse it to just the important language constructs and generate a tree structure of "nodes" that allows you to traverse the "code" to manipulate it. In order to manipulate it, babel utilizes the Visitor pattern which basically allows plugins to be aware of all the node types, select ones they care about and do their thing to them, before passing it back to babel. The combination of these two ideas is incredibly powerful and unlocks the wide range of plugins that exist in the babel ecosystem. The two plugins I'm most familiar with are @babel/preset-env and babel-plugin-styled-components, as mentioned earlier. In @babel/preset-env there's a wide array of plugins that are enabled based off the targets that are passed to it, more on @babel/preset-env here. Since preset-env is a preset and not a plugin, strictly speaking, the rest of this post will focus on babel-plugin-styled-components and the experiences I've had with it.

Using a Plugin

In my experience there's usually 3 ways that a plugin is used:

Part of a preset
Directly to enable a language feature/syntax
Directly to support a library/code style

In each of these cases your approach with the plugin usually varies. In the first case, you often don't know the plugin is even being used. Unless you dig into the preset source or dependencies you often have no idea what plugins are included in it, and what impact they're having on your output. In the second, you include a plugin because you want to enable some new feature or syntax to use in your codebase. Usually teams will evaluate these additions pretty strictly, because the more plugins you have the longer your build process will take (by nature of each plugin running on the entire tree, even if it only actually does something on specific nodes). Then with the third type you have plugins like babel-plugin-styled-components that enable cool features of the library like SSR. The reason to call out these different uses of plugins is it greatly impacts how you approach the plugin if you run into issues with it.

Issues when trying to use a Plugin

In my experience there's usually 3 kinds of issues you run into:

Plugin not doing something you want
Plugin doing something you didn't expect or don't want
Plugin running at all

In these three cases the way you approach debugging is a little different. In each case, it's super useful to have a minimal reproduction you're working with. Minimal reproductions make problems easier to notice and trace, as well as can be submitted to projects that accept bug reports if you happen to determine it's actually an issue with the project. There are of course issues that don't occur in a smaller reproduction, but in the majority of cases I've seen there's benefits to trying to boil it down to the simplest case possible. Once you have a simple case, there's a few things you can check for to figure out what might be going on. They depend on what's happening of course, but let's run through some possibilities:

Plugin isn't doing something you want it to

The first, and usually easiest to notice case is where you a plugin that isn't doing something you'd expect. An example of this that I ran into was with babel-plugin-styled-components with the ssr option flipped to true still producing a className mismatch warning. This warning indicates that something isn't quite right in the build process, since the babel-plugin is designed to generate consistent classNames for the server and client renders of a component. First, I had to rule out that it wasn't an issue of the plugin not running at all, and that was pretty easy to do by digging into node_modules and adding a simple console.log statement. Then, after spending some time trying to get a minimal reproduction, the smallest I could get still involved my code, a shared component library, babel-plugin-styled-components and @babel/preset-env - not a very small reproduction if you ask me! But, as I turned things off and back on checking for the issue, I realized that my code wasn't actually a part of the issue since it was getting transformed correctly and not producing the warning if the shared library code wasn't present. At this point I knew that it wasn't an issue with the actual application, so I was able to strip away pretty much all the application and boil it down to the simplest Next.js SSR setup I could get with the simplest library component that the issue would happen to. After playing with the options of various plugins for a while, I realized that babel-plugin-styled-components wasn't actually visiting any of the nodes that are output from the shared library component. Since we pre-build the shared library components, it turns out that we weren't using babel-plugin-styled-components but we were using @babel/preset-env to build them. The targets (something I talked about at length in my post on preset-env) were setup to have @babel/preset-env transform the template literals that babel-plugin-styled-components looks for and made it so that babel-plugin-styled-components didn't recognize the styled-components out of the shared library. This meant that it wasn't doing what I wanted it to, transform the code to be SSR compatible, because another plugin had already transformed the code previously. In this case, I learned that the order matters for babel plugins, since it's possible for one plugin to transform a node type into a different one and cause other plugins expecting the original type to not work.

Plugin is doing something that you don't want

Related to the previously discussed issue of a plugin not doing what you want, but significantly harder to notice in most cases, are plugins doing something that you don't want. I was recently involved in debugging an issue on the storybookjs repo where someone was running into an _esmodule property being added to their exported object by @babel/preset-env, something they weren't aware was happening until it started showing up in their storybook. In this case, they had forgotten to switch and tell @babel/preset-env to avoid transforming modules, and so it was transforming them. In most cases that will work just fine, but it does cause issues with other tools like webpack (for application code) being unable to use features like tree-shaking. As such, they had spent a couple hours trying to debug why this unexpected thing was happening and it boiled down to a plugin option. In this example, the plugin was doing something they didn't want anymore, but it took some prior knowledge on my part to know that _esmodule is a property used by @babel/preset-env to indicate that it is a transformed esModule file. In the cases of plugins doing unexpected things that you don't know about, the best way to recognize it that I've found is to occasionally glance at your output from the babel process (before minification/uglification ideally) to see if there's anything in there that you wouldn't expect given the options and targets that you've used. I find that checking on that a couple times a year is sufficient, since in most cases your targets and options don't need to change, but if you never check it then there's a pretty good chance you have a plugin doing something that you don't know about. On the bright side, issues of this type indicate that the plugin is actually running (although it might not be configured properly) so you can rule out the third type of issue!

Plugin is not running at all

Sometimes it's hard to tell if you have a plugin that isn't running or a plugin that isn't doing something you want, but in most cases it's usually pretty easy to debug and see if a plugin is being run. Since this is JavaScript and we use npm or yarn for our packages, you can usually go into the node_modules folder and edit the built library/plugin code to check and see if it's running - especially useful if it doesn't provide a debug option (which @babel/preset-env provides). In the cases of plugins not running, usually it's a config issue or picking the wrong plugin. I talked briefly in the overview post about the difference between transform and syntax plugins since one will actually adjust the output code while the other enables the parser to recognize the input. I also noted that transform plugins are supposed to enable the required syntax plugins for you. So, in most cases that I've seen issues of this type the config is the place to look and make sure there aren't plugins running before that are causing the plugin you want to not run, or to check and make sure that the options you pass the plugin are valid and correct.

Conclusion

Overall, babel plugins from a users perspective are quite simple. They're designed to take code and transform it into other code based on it's defined use case, options, and the targets of the build. These simple plugins have been built up into presets that enable you to turn on whole lists of plugins at once, and it's entirely possible (and relatively simple) to build your own preset! It could be a "smart" preset like @babel/preset-env or it could be a simple preset that enables the various plugins you need in your project.

Thanks for reading! I hope you've learned something! If you have, or if you have any comments or suggestions feel free to reach out and let me know. Thanks!

Banner photo courtesy of undraw.co

Five Thoughts From Getting Into Open Source Maintenance

Jordan Nielson — Sun, 25 Aug 2019 08:49:16 +0000

Originally published on jnielson.com

Normally I wouldn't be writing, let alone publishing, a blog post on a Sunday since I typically spend time with Family or at Church. Today's a bit unique since I got stranded in Dallas due to a weather delay causing me to miss the connecting flight home last night. So, I'm a bit tired at this point as you can see:

I noticed while I was here that it's August 25th, a date that in most cases isn't particularly notable. There is one point of note for me though - it was 6 months ago today that I opened my first Pull Request to the node-notifier project on github! Through the process of getting that pull request merged, and later reverted and pushed to a v6 branch that we're working on now something pushed Mikael (the project owner) to offer to have me help out as a contributor, something that I'd never considered doing before. As this is my first foray into real open source I wanted to record some thoughts that are hopefully helpful to others. I say real open source to mean a project that is actually consumed outside of itself (according to the github numbers there are 1.5 million dependents on node-notifier), instead of something like my personal website which is open source on github but doesn't impact anyone but me.

There's 5 main thoughts that I've experienced from getting into Open Source Maintenance:

Issues happen, and can't always be dealt with instantly
Taking time to communicate clearly can make a huge difference
Prioritization - of the project and my time
Jumping in to help
Empathy is key

Github Issues

The first thing that I tried to tackle when Mikael mentioned that he'd like some help was the issue queue of open things that people had filed on github. As you can see on the repository there are lots of outstanding issues. I usually strive to reach inbox zero with my emails, and since I've been doing that for a some time now it's relatively easy to keep up with each day. But, one of the first things I had to accept when getting into open source is the infinite nature of it. In most cases, there will be an endless supply of issues that consumers could file, especially in projects used in a wide variety of circumstances. If you happen to have one of the 52 issues open at the time of writing and it is still impacting you, please feel free to reach out to me on twitter and I'll take a look at it when I have time. Once I realized that the issues would likely never be at "inbox zero" without some serious dedicated time and lots of communication, I determined to do a good job responding quickly to new (or updated) issues since those people are more likely to respond back and have the relevant context still available.

Communication Clarity

The next thing I realized quite quickly, as I was triaging the issues that were open (until I decided to take the approach I decided above), was how important taking some extra time to re-read the words I had written and ensure they communicated what I wanted (something that I don't usually do for blog posts/tweets/emails/etc). Because I've found so much benefit from coming across old issues in other projects that I use, I wanted the responses I write to be useful to people who might stumble upon the issue down the road. But, more importantly, lots of the people interacted with on Github don't speak English as their first language and do the best they can. Sometimes there are misunderstandings, but trying to use clear and concise language will help someone down the road, and while they might not thank you for it, knowing you put in the little bit of effort to make your response more clear will leave you (hopefully, since it does me) with a nice feeling. Being clearer in my communications is something that I'll need to continue working on, but I feel like I've definitely made some progress.

Prioritization of Effort

Another thing that's been on my mind since being invited to collaborate on this project is the idea of Prioritization. How does this fit into family/work/blogging/etc? How do the issues raised compare to those things? How do the issues rank compared to each other? How does an open source project decide what to accept or work on? I don't have the answers to these questions, but as I've been trying to wayfind in this new area of life I have found that I don't necessarily need definitive answers to them. Prioritization is a shifting sea of options and trade-offs, with something that might be the most important thing today becoming significantly less important tomorrow. As such, I've found that throwing an hour or two each week towards this helps me to keep a pulse on it without feeling overwhelmed. Another perk of my current job is that I get to work on open source that is used at our company, and since we use Jest throughout our JavaScript projects, and Jest uses node-notifier, I can easily justify taking the occasional break from other work tasks to tackle a few things for this project. So far as comparing the issues amongst the other issues to determine what should actually be worked on, I have tended towards prioritizing the most recent issues since those are the most likely ones to still be issues. There will continue to be improvements made in this area, but I think one that could give some serious return is to clarify for node-notifier what platforms/node versions we want to support and ensure that we hit those the best that we can with a unified and consistent API where possible.

Jump Right In

Related to Prioritization is the idea of Jumping In. Sometimes, it's useful to just try and do the best you can. In this case, I tried to jump right in and help with the open issues to get a feel for what could be improved in the project since outside of the thing that I was working on it seemed to be in good shape from when I was using it. But, I decided that the most useful thing I could do was to get digging into the issues. I usually tried to jump in and see if I could reproduce the issue on one of my machines, which isn't super sustainable, but it worked for some initial testing. One of the hardest things was making sure that my communication (as noted above) clearly indicated that I was not an expert so it was just a suggestion. While I do have some solid knowledge of the project at this point, I don't know the history or the context of the choices that were made in the past. So, I try my best to provide some sort of direction for the issue opener to try and leave questions for Mikael whenever I come across them.

Empathy is Key

The last thought I had while reflecting on this increased involvement in open source was how impressive it is that there are people who maintain tens or even hundreds and possibly thousands of open source projects. I personally don't aspire to do much more than a couple at any time, particularly now that I'm more familiar with how much work actively maintaining a project can be. While this isn't a huge project, it is still a project that I've found useful in my programming and that I want to make better for the other users of it. In this effort I've gained so much more empathy for other maintainers of open source projects and how much incredible effort they've put in to get the community to the point it is with some incredible resources created by fantastic people.

As I continue to work in the open source community I have a number of goals and things to continue to work on, but that will probably become a later post :) Thanks for reading! Let me know if you have any feedback, and I'd love to hear about your (hopefully positive) experiences getting into Open Source!

Banner image courtesy of undraw.co

P.S. I'm hopeful that tomorrow (Monday) I'll get back to work and have time to write another post in the series I'm working on demystifying babel and webpack - Check out the first post introducing the series if you haven't yet.

Demystifying @babel/preset-env

Jordan Nielson — Fri, 16 Aug 2019 19:38:02 +0000

Originally published on jnielson.com.

Ideally my post Build Tools Demystified helped to clarify some of the basic ideas in play when using babel and webpack. If not, please let me know things that can be clarified or added further! I'd love to have this series by helpful to someone outside of the talk I give based on it. I know for me reading is significantly more digestable than listening to talks so I hope that this format helps someone.

One of the topics that I find really interesting when talking to people about Babel is the preset-env that they provide. Since Babel is an open source project, you can find the code for @babel/preset-env here, but in this post we'll dig into a little bit why it is such a popular preset for those using Babel. For reference, there are docs specifically for this preset which detail the options that it supports. Generally your usage of preset-env (hereafter I'll just call it "the preset"), could be as simple as having a babel config file that contains:

{
  "presets": ["@babel/preset-env"]
}

While this simple usage works, their docs state:

"We don't recommend using preset-env this way because it doesn't take advantage of its ability to target specific browsers."

I would add to that, since using it this way will result in transforming all your code to ES5 which in a lot of cases you don't need-unless you get to support really old environments. In most cases, at least where I've worked, we generally only supported relatively recent browsers, and one of the coolest features of the preset is that it integrates with Browserslist to allow you to use the same targets that other tools utilizes. For instance, you can use a .browserslistrc file to specify your targets with something like:

last 1 version
> 1%
maintained node versions
not dead

One of the benefits of using an approach like this is that you don't have to manually maintain what versions of the browsers and such that you are targeting, letting you instead focus on the code that you write.

Using the Debug option

In addition to the fabulous Browserslist integration, there's a number of other options that you can pass to the preset in order to customize or even debug it. For instance if you use it like:

{
  "presets": [
    [
      "@babel/preset-env",
      {
        "debug": true
      }
    ]
  ]
}

You'll get a bunch of useful debug information to help you know what plugins are being applied (and what browsers you're targeting that require them), something like the following:

@babel/preset-env: `DEBUG` option
Using targets:
{
  "android": "67",
  "chrome": "74",
  "edge": "17",
  "firefox": "66",
  "ie": "10",
  "ios": "12",
  "node": "10.16",
  "opera": "12.1",
  "safari": "12",
  "samsung": "8.2"
}
Using modules transform: false
Using plugins:
  transform-template-literals { "android":"67", "ie":"10", "ios":"12", "opera":"12.1", "safari":"12" }
  transform-literals { "android":"67", "ie":"10", "opera":"12.1" }
  transform-function-name { "android":"67", "edge":"17", "ie":"10", "opera":"12.1" }
  transform-arrow-functions { "android":"67", "ie":"10", "opera":"12.1" }
  transform-block-scoped-functions { "android":"67", "ie":"10", "opera":"12.1" }
  transform-classes { "android":"67", "ie":"10", "opera":"12.1" }
  transform-object-super { "android":"67", "ie":"10", "opera":"12.1" }
  transform-shorthand-properties { "android":"67", "ie":"10", "opera":"12.1" }
  transform-duplicate-keys { "android":"67", "ie":"10", "opera":"12.1" }
  transform-computed-properties { "android":"67", "ie":"10", "opera":"12.1" }
  transform-for-of { "android":"67", "ie":"10", "opera":"12.1" }
  transform-sticky-regex { "android":"67", "ie":"10", "opera":"12.1" }
  transform-dotall-regex { "android":"67", "edge":"17", "firefox":"66", "ie":"10", "opera":"12.1" }
  transform-unicode-regex { "android":"67", "ie":"10", "opera":"12.1" }
  transform-spread { "android":"67", "ie":"10", "opera":"12.1" }
  transform-parameters { "android":"67", "edge":"17", "ie":"10", "opera":"12.1" }
  transform-destructuring { "android":"67", "ie":"10", "opera":"12.1" }
  transform-block-scoping { "android":"67", "ie":"10", "opera":"12.1" }
  transform-typeof-symbol { "android":"67", "ie":"10", "opera":"12.1" }
  transform-new-target { "android":"67", "ie":"10", "opera":"12.1" }
  transform-regenerator { "android":"67", "ie":"10", "opera":"12.1" }
  transform-exponentiation-operator { "android":"67", "ie":"10", "opera":"12.1" }
  transform-async-to-generator { "android":"67", "ie":"10", "opera":"12.1" }
  proposal-async-generator-functions { "android":"67", "edge":"17", "ie":"10", "opera":"12.1" }
  proposal-object-rest-spread { "android":"67", "edge":"17", "ie":"10", "opera":"12.1" }
  proposal-unicode-property-regex { "android":"67", "edge":"17", "firefox":"66", "ie":"10", "opera":"12.1", "samsung":"8.2" }
  proposal-json-strings { "android":"67", "edge":"17", "ie":"10", "opera":"12.1", "samsung":"8.2" }
  proposal-optional-catch-binding { "android":"67", "edge":"17", "ie":"10", "opera":"12.1", "samsung":"8.2" }
  transform-named-capturing-groups-regex { "android":"67", "edge":"17", "firefox":"66", "ie":"10", "opera":"12.1", "samsung":"8.2" }
Using polyfills: No polyfills were added, since the `useBuiltIns` option was not set.

The above was generated using the following Browserslist query: last 2 versions, current node. As you can probably guess, in most applications this includes way more than you actually need to support, in fact using npx browserslist 'last 2 versions, current node' prints out the following list right now:

and_chr 75
and_ff 67
and_qq 1.2
and_uc 11.8
android 67
baidu 7.12
bb 10
bb 7
chrome 75
chrome 74
edge 18
edge 17
firefox 67
firefox 66
ie 11
ie 10
ie_mob 11
ie_mob 10
ios_saf 12.2
ios_saf 12.0-12.1
kaios 2.5
node 10.16.0
op_mini all
op_mob 46
op_mob 12.1
opera 58
opera 57
safari 12.1
safari 12
samsung 9.2
samsung 8.2

Why are targets so useful and important?

If you don't need to support things like ie10, you should probably adjust your query to be something like the example used in the .browserslistrc file above. Running that query, npx browserslist 'last 1 version, > 1%, maintained node versions, not dead' gives the following output:

and_chr 75
and_ff 67
and_qq 1.2
and_uc 11.8
android 67
baidu 7.12
chrome 75
chrome 74
chrome 73
edge 18
edge 17
firefox 67
firefox 66
ie 11
ie_mob 11
ios_saf 12.2
ios_saf 12.0-12.1
kaios 2.5
node 8.16.0
node 12.5.0
node 10.16.0
op_mini all
op_mob 46
opera 58
safari 12.1
samsung 9.2

Doing this change we dropped support for some old and dead things, like bb 10 and bb 7 (the blackberry browser), and added support for more node versions (8 and 12). We also grabbed an extra chrome version, probably due to its current usage amount.

The preset's debug output for this list looks like this right now:

@babel/preset-env: `DEBUG` option
Using targets:
{
  "android": "67",
  "chrome": "73",
  "edge": "17",
  "firefox": "66",
  "ie": "11",
  "ios": "12",
  "node": "8.16",
  "opera": "46",
  "safari": "12.1",
  "samsung": "9.2"
}
Using modules transform: false
Using plugins:
  transform-template-literals { "android":"67", "ie":"11", "ios":"12", "safari":"12.1" }
  transform-literals { "android":"67", "ie":"11" }
  transform-function-name { "android":"67", "edge":"17", "ie":"11" }
  transform-arrow-functions { "android":"67", "ie":"11" }
  transform-block-scoped-functions { "android":"67" }
  transform-classes { "android":"67", "ie":"11" }
  transform-object-super { "android":"67", "ie":"11" }
  transform-shorthand-properties { "android":"67", "ie":"11" }
  transform-duplicate-keys { "android":"67", "ie":"11" }
  transform-computed-properties { "android":"67", "ie":"11" }
  transform-for-of { "android":"67", "ie":"11" }
  transform-sticky-regex { "android":"67", "ie":"11" }
  transform-dotall-regex { "android":"67", "edge":"17", "firefox":"66", "ie":"11", "opera":"46" }
  transform-unicode-regex { "android":"67", "ie":"11" }
  transform-spread { "android":"67", "ie":"11" }
  transform-parameters { "android":"67", "edge":"17", "ie":"11" }
  transform-destructuring { "android":"67", "ie":"11" }
  transform-block-scoping { "android":"67", "ie":"11" }
  transform-typeof-symbol { "android":"67", "ie":"11" }
  transform-new-target { "android":"67", "ie":"11" }
  transform-regenerator { "android":"67", "ie":"11" }
  transform-exponentiation-operator { "android":"67", "ie":"11" }
  transform-async-to-generator { "android":"67", "ie":"11" }
  proposal-async-generator-functions { "android":"67", "edge":"17", "ie":"11", "node":"8.16", "opera":"46" }
  proposal-object-rest-spread { "android":"67", "edge":"17", "ie":"11", "opera":"46" }
  proposal-unicode-property-regex { "android":"67", "edge":"17", "firefox":"66", "ie":"11", "node":"8.16", "opera":"46", "samsung":"9.2" }
  proposal-json-strings { "android":"67", "edge":"17", "ie":"11", "node":"8.16", "opera":"46", "samsung":"9.2" }
  proposal-optional-catch-binding { "android":"67", "edge":"17", "ie":"11", "node":"8.16", "opera":"46", "samsung":"9.2" }
  transform-named-capturing-groups-regex { "android":"67", "edge":"17", "firefox":"66", "ie":"11", "node":"8.16", "opera":"46", "samsung":"9.2" }
Using polyfills: No polyfills were added, since the `useBuiltIns` option was not set.

If your organization has decided to drop support for Internet Explorer entirely, you could append an exclusion to your query not ie 11, not ie_mob 11 and take those off the list as well. If you're going to do that, you might even be able to convince your organization to drop what is called "Android Browser" in caniuse ("android 67" above) since it has 0% usage, to reduce the amount of transforms you apply even more. So, add not android 67 to your query. Once you've done that, the preset debug output looks more like this:

@babel/preset-env: `DEBUG` option
Using targets:
{
  "chrome": "73",
  "edge": "17",
  "firefox": "66",
  "ios": "12",
  "node": "8.16",
  "opera": "46",
  "safari": "12.1",
  "samsung": "9.2"
}
Using modules transform: false
Using plugins:
  transform-template-literals { "ios":"12", "safari":"12.1" }
  transform-function-name { "edge":"17" }
  transform-dotall-regex { "edge":"17", "firefox":"66", "opera":"46" }
  transform-parameters { "edge":"17" }
  proposal-async-generator-functions { "edge":"17", "node":"8.16", "opera":"46" }
  proposal-object-rest-spread { "edge":"17", "opera":"46" }
  proposal-unicode-property-regex { "edge":"17", "firefox":"66", "node":"8.16", "opera":"46", "samsung":"9.2" }
  proposal-json-strings { "edge":"17", "node":"8.16", "opera":"46", "samsung":"9.2" }
  proposal-optional-catch-binding { "edge":"17", "node":"8.16", "opera":"46", "samsung":"9.2" }
  transform-named-capturing-groups-regex { "edge":"17", "firefox":"66", "node":"8.16", "opera":"46", "samsung":"9.2" }
Using polyfills: No polyfills were added, since the `useBuiltIns` option was not set.

These adjustments are important for a lot of reasons, but the biggest one is that each plugin that you use in Babel contributes to how long the process takes. While that might not seem like a huge deal in your application, it can add up. For those who didn't feel like counting, adjusting our targets reduced our list of transforms that we use from 28 to 9. While this does exclude some possible users, you'll want to work with your analytics to determine if that actually matters. If it does, you might look into the module/nomodule split to produce two different bundles, something that Jake Archibald has an excellent post on.

Another feature that the preset supports is the modules transform (you might have noticed that mentioned in the debug logs above). There's a number of modes for this transform, with the default being "auto" (usually ends up as commonjs). For those that use webpack to bundle your code, you'll want to set modules: false in order to allow webpack's cool features like tree shaking to work. If you don't set modules: false, babel will transform the import/export statements into require/module.exports statements (aka not ES6 modules), which webpack can't statically analyze. In cases for library code, you'll probably want to produce an ES6 modules build, and a commonjs build, but maybe not.

To sum up, @babel/preset-env is a smart preset - a collection of plugins that are enabled or disabled based on the targets you give it to transform your code into something compatible with your targets. Hopefully you learned something from this, I certainly did while writing it!

Cover Image courtesy of undraw.co

JavaScript Build Tools Demystified - Overall Thoughts

Jordan Nielson — Fri, 16 Aug 2019 00:30:08 +0000

Originally published on jnielson.com

Hi again! Another prep post for a work talk I'm giving. The talk isn't until the start of October, so if you're reading this post after that point I hope it helps you. This one will probably be followed up by another one that takes a more "talk in a post" approach, whereas this one will be "thoughts regarding the topic that are unorganized". It's probably pretty rough, but I figured I would put it out there in case there's interest. If that's your jam, feel free to read on and I'd appreciate any feedback that you have!

The topic that I selected to share at this internal work conference is focused on demystifying the build tools that are used in the JavaScript stack at work. We utilize Next.js, a fantastic library that makes it so that we rarely have to interact with our build tools since they do a fantastic job of updating them in a way that doesn't usually break our applications. Sometimes though we have projects that aren't using Next.js (either they're using an old version of the stack before Next.js was adopted or they're not using the stack at all) or they have run into something that they need to adjust to fit their specific project and they come to us asking various questions about build tools because they're having issues. In those cases, it's helpful to have some background knowledge about what the tools are doing and why they are there.

When using Next.js it includes configuration and support for Babel and webpack. Their out-of-the-box configurations are pretty solid, and in most cases easily extendable. For instance, we've included a babel configuration file in our stack that sets up the styled-components babel plugin, and that has been relatively painless to have an keep up to date. Other build tools that we've helped projects work with include Gulp and Grunt, though we don't have many projects with changing Gulp/Grunt setups at this point. Most of those are the legacy AngularJS projects and since they don't change all that often we're not going to talk about them in this post. As such, I'll be focusing on Babel and webpack.

For those who aren't familiar with the typical web development stack, we utilize Babel and webpack to take the code that we write and change it (compile/transpile/whatever else you want to call it) to the ideal format for the intended consumption method. Generally JavaScript code is consumed in the browser, with different browsers supporting different features.

Babel is a JavaScript Compiler that lets you write code using the latest language features and will transform that according to the configuration you give it. By default, it doesn't actually do any transforming-it only transforms using the plugins that you supply it.

webpack is a Bundler that will take the files you give it (usually called "entries") and gather all the assets/code that are referenced into a set of files that are ready to serve to the browser.

One of the greatest features of webpack and babel is extensibility through a vast plugin ecosystem. In a lot of cases, people will utilize Babel within webpack through something called babel-loader. Loaders are used by webpack to customize how file types are bundled in the pipeline, and so babel-loader is used to run files through Babel before being bundled by webpack.

In either case (using Babel directly or within webpack), the configuration files are where a lot of the "magic" happens that I'll be de-mystifying through this talk I'm going to give. The goal of the talk is for people to have a working vocabulary they can use when googling the different things they might need to while working with these tools as well as provide them with some strategies for working with them. One of my biggest goals is to have people be less scared of tinkering with the tools, since more often than not you just need to give it a try and see if it works.

Both of these fantastic tools have thorough documentation, and one of my favorite sections of the webpack docs is called "Concepts" since it gives a great high-level overview of the main things going on with webpack. Those main concepts include:

Their descriptions of them are spot on, but to tie them together I'll add my own. Essentially webpack takes an entry point (or points), usually a JavaScript file but doesn't have to be, and builds out a map of the dependencies that it has, utilizing the Loaders based on the file paths, running it all through the plugins, and spits out a set of output files with the mode allowing you to utilize some preset things.

While Babel doesn't have a "Concepts" page, there are still some major concepts that are important to understand in order to work with Babel from a consumer standpoint. In my opinion that includes ideas and things like the following. Note that I wouldn't recommend sitting and reading through any of these links since they're pretty dense.

So, while Babel might not have a dedicated docs page to describing their major Concepts, these are the things I find people would get the most benefit from understanding. Since the linked to docs pages are a bit dense, here are some working definitions/views that I have for these topics. Plugins are the things that actually give Babel behavior and capability. If you don't use any plugins, Babel will just give you the code you gave it. There's a couple types of plugins-Transform and Syntax. Syntax plugins are the more straightforward ones to understand since they enable Babel to even parse the feature. While they're simpler to understand, they're rarely used directly since merely parsing a language feature doesn't do much good for the consumer of that code. Therefore Transform plugins will load the Syntax plugins they need and transform the relevant code to the desired output. Because it can be tedious to configure and load all the plugins you might want in each project, Babel allows for Presets, which are groups of plugins that are pre-configured. In both Presets and Plugins, along with lots of other things about Babel the idea of passing in options is super useful. Options allow for plugins and presets to adapt based on the target environments that you have, the "mode" that you're in (similar to webpack), or any other thing that they've been setup to handle. With babel, there's a multitude of tools that utilize it and passing presets and plugins through the command line can get pretty tedious, so there are a number of config file types available to you, with two major kinds - Project Wide and File Relative. The one that you'll generally want to use in application-type projects or monorepos as of this writing is the Project Wide babel.config.js file. Libraries will generally use a File Relative config, like a .babelrc file (or something in their package.json file depending on how complex their configurations are). If you do ever decide to write anything that other people will use with Babel, the handbook linked above is an excellent resource. The plugin handbook is super helpful if you're trying to debug someone else's plugin, since they likely used that while writing it. In most projects that I've seen, they utilize the preset-env provided by the babel team. The docs on that are excellent, so I'd recommend checking those out if you're going to be using it directly. Thankfully, Next.js handle a lot of the configuration for us and even provides a preset we can use to extend their configuration (which we did to add styled-components support).

I think that's all I'm going to get to so far as my thoughts on the overall topic today. I might throw out some other, more detailed and specific posts as I continue to refine what I'll be talking about. Thanks for reading!

Cover image courtesy of undraw.com