Forem: Travis Fischer

The Absolute Best JS Dev Tools in 2022

Travis Fischer — Mon, 11 Apr 2022 19:09:14 +0000

A breakdown of the most important JS dev tools in 2022, including their most relevant tradeoffs, and some opinionated advice sprinkled on top.

In the world of software engineering, it’s important to have a firm understanding of the tools at your disposal.

But the landscape of JS tooling is always changing at such a rapid pace.

And 2022 is no different.

So I decided to break down the most important dev tools that you should be aware of in 2022.

We’ll start with the lowest-level tools and work our way up to higher-level tools from there. Let’s get started 💪

Compilers

Compilers are responsible for transforming your input code into some target output format. For our purposes, we’re focusing on compilers which support transpiling modern JavaScript and TypeScript into specific versions of ECMAscript that are compatible with browsers and recent versions of Node.js.

The most important thing to understand about this space is that it’s in the middle of undergoing a massive shift from compilers like tsc and babel, which are written in higher-level interpreted languages, to compilers like swc and esbuild, which are written in much faster, compiled languages.

This shift results in 10-100x faster compilation times as shown in this esbuild demo.

If you’re updating your devtools stack or starting a new project in 2022, then you’ll want to consider using one of these next-gen compilers under the hood. They may not be as mature as the official typescript compiler or babel, but the benefits of having 100x faster builds cannot be understated.

Note that neither swc nor esbuild perform type checking. They simply transpile code into the desired output format as quickly and efficiently as possible. For the time being, if you’re working with TypeScript, you’ll almost always need to have the official TypeScript compiler as part of your toolchain to guarantee that you’re getting the most out of TypeScript’s static type checking. It’s also worth mentioning that the author of swc, kdy1dev is working on porting tsc to Go in order to remove the need for tsc in many cases, as it tends to be the bottleneck in most toolchains.

SWC vs esbuild

swc and esbuild are both excellent, blazing fast, open source JS / TS compilers. They have comparable performance and are both used regularly in production by some of the world’s largest companies.

It’s likely that your choice between the two will be dictated more by the higher-level tools built on top of these compilers as opposed to choosing between them directly.

Notable projects using swc:

Notable projects using esbuild:

💡 In software engineering, convenient statements such as “technology A is better than technology B” rarely hold much value. Rather, you should try to always keep context in mind. In this case, there are many scenarios where you’d be better off using tsc or babel.

Becoming a better software engineer often boils down to thoroughly understanding the tradeoffs involved with these types of decisions, and balancing those tradeoffs against the particular constraints of your project, team, and business’ needs.

Bundlers

Bundlers are responsible for taking all of your input source files and bundling them together into an easy-to-consume output format. The two most common use cases for bundlers are bundling libraries and bundling resources for web applications.

Bundlers like webpack and rollup are the swiss-army knives of modern JS toolchains. They are both extremely extensible, with well-maintained plugins covering major use cases. It’s relatively straightforward, for example, to use any of the popular compilers listed above to transpile TS code with either webpack or rollup.

Parcel, on the other hand, provides a mostly zero-config approach to bundling. It focuses on simplicity as opposed to extensibility and uses swc as a compiler under the hood.

Note that swc and esbuild both provide basic bundling capabilities as well, though compared with these alternatives, they’re not full-featured enough to be included on this list.

For a much more detailed comparison of these bundlers, check out tooling.report.

Package Managers

Package managers are responsible for managing dependencies in the form of NPM packages.

There’s a lot of history here, but the TL;DR is:

All of these package managers have similar features and perf nowadays, so don’t be too worried about which one you’re using.
pnpm seems to be gaining a lot of traction very quickly. 💪
The way yarn berry was rolled out and the subsequent deprecation of yarn v1 turned a lot of people off from using yarn, though yarn berry has come a long way in the past few years.
yarn plug’n’play is an interesting approach, but in practice, it has only seen adoption in cases with very large monorepos.
- I can’t tell you the number of times I’ve wanted to inspect or add console.log statements to my node_modules, and not being able to do so is a real disadvantage.

Adoption by popular projects

^ A breakdown of the package managers chosen by popular projects. This image is from Sebastian Weber’s excellent package manager deep dive. Note that at the time of this writing, none of these projects are using Yarn PnP.

Library Development

These tools are meant to help library authors bundle and publish modern NPM packages.

If you’re developing a new library in 2022, you’ll likely want to use one of these higher-level tools to simplify your workflow.

If you have a TS package and want to take advantage of extremely fast build times courtesy of esbuild, then tsup is a great option.
If you have a TS package and need some additional features, then tsdx is a great option.
If you have a TS or JS package, then microbundle is also a great option.
Vite is mainly a tool for building frontend web apps, but it also includes support for outputting libraries and is a very solid all-around option.
For monorepos, nx looks really promising.

My personal preference will be to use tsup for TS packages, mainly because once you’ve experienced 100x faster builds, it’s really difficult to consider switching back to anything else.

More Info

Most of these tools don’t currently provide great support for TS monorepos which take advantage of composite project references. For the time being, my recommendation for this case is to use tsc for type checking and generating .d.ts typings (with emitDeclarationOnly: true) and tsup for compiling code in each of the sub-packages. For an example of this approach, check out the react-notion-x monorepo (one of my OSS projects).

Publishing modern NPM packages is a nuanced topic that goes well beyond the scope of this article. For more info on ESM, commonjs, dual-package publishing, exports, and more see:

What does it take to support Node.js ESM?
Sindresorhus’ notes on publishing and consuming ESM packages

Web App Development

These higher-level tools and frameworks are intended to help developers build modern web applications without worrying about all the details.

If you’re developing a new web app in 2022 using React, then I would highly recommend using Next.js. It has the best support, the most active community, and close integration with Vercel, the world’s leading deployment platform for modern web apps.

Remix provides a really compelling alternative to Next.js. It’s from the makers of react-router, and though it’s relatively new, they’re definitely a framework to keep an eye on.

If you’re developing a new web app using Vue, then Nuxt.js and Vite are both great options.

And last, but certainly not least, if you want something more light-weight, then give Parcel a try. 🤗

💡 There seems to be a roughly equal number of projects building on top of swc and esbuild. The same observation goes for webpack and rollup.

Conclusion

Modern web development has evolved significantly over the past 10 years. Developers today are lucky to have such a wide range of amazing, well-maintained tools to choose from.

This is, however, by no means a comprehensive list of dev tools. If there’s something missing that you’d like to see added, let me know on twitter.

Hopefully this breakdown has helped you parse the most important aspects of the current JS / TS dev tools landscape, and hopefully it’ll help you to make more informed decisions going forwards.

If you found this article helpful, then consider following me on twitter for more @transitive_bs

Saasify's Approach to OSS

Travis Fischer — Thu, 26 Mar 2020 22:43:04 +0000

I'm very proud to announce that Saasify's open source GitHub repo has just hit 600 stars. ⭐

This is a small but important milestone for us that has taken over a year of consistent, daily work including contributions across over a dozen related projects in the more general JS and TS communities.

I'm a big believer in the power of open source and can say from experience that a lot of people underestimate the second & third-order effects of building a strong open source community.

With all this in mind, I thought this would be the perfect time to share some general thoughts on our approach to open source.

Saasify's Open Core

Saasify is what's known as an open core company. This just means that some of our code is open source, and some of it is closed source.

Our platform's codebase is split into two monorepos, one for our publicly-facing, MIT-licensed open source work, and one that's private, for proprietary closed source work.

Saasify Open Source

User-facing parts of the platform (e.g., our Node.js CLI, React component library, marketing website / admin webapp, JS client API, etc)
All platform documentation (source and generated docs)
Lots of generally useful functionality that other open source projects and companies may benefit from (OpenAPI utilities, FaaS utilities, badge generator powered by headless Chrome, etc)
20 or so example projects showcasing different use cases for Saasify
A standalone project for Functional TypeScript which transforms normal TS functions (and jsdoc-annotated JS functions) into JSON Schemas and OpenAPI specs

Saasify Closed Source

Our platform's main backend API
Our API gateway for proxying FaaS requests
Various serverless functions whose business logic is built on top of OSS packages from Saasify's public repo
The template-based SaaS web client (most of the React components that comprise this webapp are from our open source react-saasify package)

Choosing what to open source versus what to keep proprietary with the ultimate goal of building a sustainable business is a difficult balancing act, and there's no one right way to go about it. We're sharing Saasify's approach so far in the spirit of transparency and in the hope that it will help other founders make similar decisions in the future.

The Open Source Advantage

Working out in the open has some important, fundamental advantages over more traditional proprietary approaches.

For starters, it's one thing to claim that your company's transparent, has a solid platform that customers can rely on, and that customers can trust you, but it's really a different thing entirely to show all of these things by putting your code where your mouth is and open sourcing much of your core platform. Having a reputable presence in the open source community is increasingly becoming a major advantage for aspiring indie hackers.

Another advantage of investing time in open source is that it will eventually attract other talented developers who want to have an understanding of what things look like under the hood before considering whether or not they'd want to work with you. If you can show publicly that your platform is reasonably clean and follows modern best practices, you'll be able to have conversations with many top developers who would otherwise not give your company a second glance.

Aside from the very rare occasional viral HN post or influencer plug, successful open source projects tend to follow a very similar lifecycle as bootstrapped businesses.

They both generally take a slow, methodical approach to gaining traction and eventual success or failure can't be pinpointed to any one particular feature, strategy, or decision. It's a combination of consistent grinding, iterating based on user feedback, and ultimately persistence that separates both successful open source projects and indie products from unsuccessful ones.

Onwards & Upwards

If there's one thing that I'd love for other startup founders and indie hackers to take from our experience so far, it's an appreciation for how powerful open source can be as a natural content marketing and business development strategy especially for early adopters who tend to be more technically savvy.

As we continue our journey towards becoming the Shopify for SaaS, we're actively looking for other developers, designers, marketers, and like-minded business people to help us reach our goals. If this type of thing gets you excited, please reach out.

On a related note, COSS.media is one of my favorite resources discussing the tradeoffs and advantages of mixing open source with commercial business. If this stuff interests you, definitely check out their blog.

And finally, check out our open source repo for inspiration and let us know what you think. 🙏

Making Your Code Beautiful

Travis Fischer — Sun, 16 Feb 2020 08:44:00 +0000

The best techniques for sharing code snippets and screencasts that will help propel your open source projects to success.

Creating your own open source projects can be extremely rewarding, but it can be hard to break through the noise and get other developers to trust and use your software. You can gain a lot of ground by following common best practices like including solid documentation, adding unit tests, integrating with a CI/CD oriented towards open-source projects (like travis-ci or circle-ci), and enforcing consistent style conventions.

One of the most effective and easiest ways I’ve found to make open source projects really stand out from the crowd is adding quality screenshots or animated demos. Whenever I see this attention to detail, not only does it prove to me that the author cares about the project, but it’s the absolute fastest way to convey what the project actually does.

Animation Credit: CSS-only coding animation by Chris Dermody

Including quality screenshots and demos is becoming an increasingly important part of what I’d call Developer UX , that is the flow a prospective developer takes from considering adding your project as a dependency all the way through successful integration and future maintenance.

Towards that end, we’ll be looking at three common use cases for improving the developer UX of your open source projects with media:

Static code snippets (images)
Animated code demos (GIFs or animated SVGs)
Project screencasts (videos)

Static Code Snippets

Sharing small bits of static code is definitely the most common and important use case on this list. Every open source project’s readme should include some easily parseable example usage snippet, so let’s start there.

GitHub-Flavored Markdown Snippets

At the easiest end of the spectrum, GitHub allows syntax highlighting in markdown code snippets. Hopefully, this style of embedding is familiar to you, and if not, I would definitely recommend starting here.

Javascript snippet embedded in markdown from one of my favorite npm modules, p-map by sindresorhus.

GitHub Gists

The code snippet above also provides an example of an extremely popular way of sharing static code snippets via GitHub Gists, which have the following advantages:

Linkable
Support versioning
Support discussion via comments
Syntax highlighting

Carbon

Markdown snippets and GitHub gists are both useful, but if you really want to make your code pop, then look no further than Carbon.

Image Credit: Carbon

Carbon is a very popular open source project that allows you to easily create aesthetically pleasing code screenshots, along with a plethora of customization options and community plugins. It’s a great choice for making a hero image stand out in your readme, increasing engagement on social media, or for writing engineering-related blog posts like this one 😛.

Animated Code Demos

Including a high quality, inline demo that quickly demonstrates your project’s core use case is the single most important suggestion I have to give.

There are a ton of different ways to go about creating these types of demos, however, so I’d like to discuss what I’ve found to be the best approach here.

Asciinema is a free tool that lets you record and share your terminal sessions, the right way.

Asciinema provides a lightweight, purely text-based approach to terminal recording, which allows you to make lossless recordings that can then be shared directly or rendered to animated SVG, animated GIF, or video. It surprised me just how many popular open source projects on GitHub make use of Asciinema — I would highly recommend checking it out.

Example Asciinema screencast converted to a GIF (credit: create-react-library) Note that the quality of this embedded GIF is much lower than the animated SVG in the linked readme as discussed below.

Animated SVG or GIFs?

We all know GIFs are a horribly inefficient, lossy format, but let’s dig a little deeper into this particular use case.

Compare the above embedded screencast gif to the animated SVG of the same screencast from the readme. It’s difficult to embed an inline comparison side-by-side, but the animated SVG is significantly sharper and smaller , coming in at 73kb vs 4.4MB for the lower quality GIF.

Why is this even a discussion then? Well, you can’t exactly include custom HTML everywhere on the web, now can you? Unfortunately for the foreseeable future, GIFs will live on as a necessary evil for these use cases. But open source authors, please consider using animated SVG versus GIFs for your GitHub projects!

There are some very popular open source projects on GitHub that have started using more efficient animated SVGs for their demos, such as create-react-app, but in general, you’ll find GIFs to be much more common.

Examples of using the excellent svg-term-cli to generate our lossless animated SVG.

It’s important to note that when discussing animated SVGs, we’re really talking about embedding an HTML snippet into GitHub-flavored markdown that links to an SVG file encoded with each frame as an SVG group and the animation defined via CSS keyframes (example SVG source).

Insert this HTML snippet into any GitHub-flavored markdown file to embed the linked animated SVG with optimal sharpness and low size overhead compared with a comparable GIF.

For reference, here is the screencast from create-react-library we’ve been using as an example in several different formats:

Original asciicast
High quality animated SVG created with svg-term-cli
Low quality GIF created with asciicast2gif

Capturing and Optimizing GIFs

Asciinema is great for terminal-based recording, but what if you want to record a UI component or website? Well, my first and foremost answer here is to always include a usable demo if possible alongside your project, especially if it’s a frontend web project. It’s really easy to get started with GitHub Page’s free hosting!

If you do want to include a GIF, I’d recommend using either GIPHY Capture or Kap to record your screen and output a GIF. Alternatively, if you have a video recorded from another source, I’d recommend using Gifski to convert the video to an as-optimized-as-possible GIF for ease of embedding.

Example of embedding a GIF into GitHub-flavored markdown files. (see the resulting gif below)

Quality demo GIF embedded in readme using the snippet above. (image credit: prompts by terkelg)

Project Screencasts

If your project is becoming more involved or you’re launching your project to a wider audience, including walkthrough video(s) can really help with user onboarding and support.

Screenflow

My go-to screen recording software is ScreenFlow, which is not cheap at $129, but gives you a lot of powerful, quality tools for the price, including precise rectangular screen recording, video and audio track mixing, audio voiceovers, transition effects, and more. This type of screencast is quite a bit more complicated than the screenshots and terminal session recordings we were looking at earlier.

Conclusion

Developer UX is important for promoting and marketing your work, which can in turn lead to very real consequences, as getting noticed for your open source contributions is definitely one of the best ways to gain notoriety and land big job opportunities as a software engineer.

I hope some of the techniques I’ve covered help you to promote your open source projects. If you’ve found this article useful and end up creating a snazzy screenshot or animated demo, add a comment linking to your project and let me know!

And as always, don’t forget to spread the ❤️… in the form of beautiful coding demos, of course!

Gaming CS Interviews

Travis Fischer — Tue, 04 Feb 2020 01:18:00 +0000

Tips for Kicking Ass at Whiteboard Interviews for Non-CS Peeps.

Let me start by saying that many companies in the tech industry have started moving away from traditional, technical whiteboard interviews, myself included, because they tend to bare little relevance to an employee’s day-to-day development work. Most companies are better off focusing on testing practical skills and the ability to deliver as opposed to algorithmic, computer science questions, and that’s coming from someone who genuinely loves those types of questions. There are exceptions to this, of course, but I believe most engineering jobs today fall into this category.

With that being said, the largest and most esteemed tech companies such as Google, Facebook, Amazon, Microsoft , etc. all still employ very similar technical interview loops that tend to greatly favor candidates with standard computer science backgrounds over candidates who are either self-taught or who prefer to focus on software engineering over the “science” aspect of computer science.

Google: 90% of our engineers use the software you wrote (Homebrew), but you can’t invert a binary tree on a whiteboard so fuck off.

— Max Howell (@mxcl) June 10, 2015

Regardless of your views on whether or not this process is fair or optimal, I have a lot of friends who fall into this latter category of being self-taught or software-engineering heavy and scoff at the thought of interviewing with one of these bigger players, even though I know from experience that they would fit in just fine once they made it past the interviews. Given that these are also some of the better, more passionate devs I’ve had the pleasure of working with, I wanted to share some no-bullshit advice I’ve accumulated over the years in the hopes of encouraging other engineers out there to consider advancing their careers by spending time at one or more of the bigger tech companies.

I sincerely believe that most developers who are proficient at developing code in their language of choice are capable of passing a Google-style interview loop by adopting the right mindset and studying a few key topics and question archetypes ahead of time.

So, with that goal in mind, let’s dive into that whiteboard…

Here’s a picture of a generic whiteboard… because it’s on topic and not at all redundant…

General Tips

When given a programming problem, never start coding right away. Always talk through the problem, by first verifying that your assumptions and thought processes are on the right track.

I highly recommend trying to get comfortable verbalizing your thought process at all times but especially when you’re not sure how to proceed. Oftentimes, the interviewer cares more about your thought process than the solution and/or will give you guidance according to your thoughts. Guidance is expected; a great interview should be more of a conversation than a one-sided question and one-sided answer.

Generally start out with the most naive, straight-forward approach to a problem you can think of, even if you think it’s really inefficient. Verbalize your thought process in doing so, and either the interviewer will say that's great and you can start coding, or you’ll get confirmation that they want to dig deeper into a more optimal solution which generally leads to a conversation about where the most inefficient part of the algorithm is (like the innermost loop) and how you could potentially mitigate its runtime.

Always use the programming language you’re most comfortable with; never use a “harder” language because you think it will make you look more legit.

At the end of the interview, your assessment will be highly subjective, so keep that in mind and try to have some fun and cold read the interviewer to play off of his or her interests. Almost always asking them early on about what they do at company X will help you understand the type of person they are and also helps to put them in a good mood because people love to talk about themselves. For instance, I recently interviewed with a developer who works on a compiler team at company X which adjusted the way I approached certain parts of the conversation to be more low-level and joke at one point about something all compiler peeps can relate to. If they like you as a person, they’ll be more lenient in their assessment whether they’re aware of it or not; that’s just human nature.

Credit: xkcd

Interview Topics

There are some very common archetypes in algorithmic interviews that tend to account for the vast majority of questions you’ll encounter.

If you understand these core question types and can solve some example problems from each of them, you’ll have a much better eye for solving similar problems during a real interview and subsequently solving real problems on the job.

Algorithmic Complexity

This topic boils down to understanding big-O notation . Even though there are other, more rare measures of complexity (like little-o, theta…) and topics like NP completeness, I would recommend skimming them, as they’re unlikely to appear in a typical technical interview.

For almost every problem you’re asked to solve in an interview, you’ll either be asked explicitly about the big-O runtime of a proposed solution or be implicitly expected to bring it up during your discussion.

This part can definitely be gamed somewhat by just practicing a bit on a representative set of problems ahead of time. You’ll both get the hang of it and also generally be able to fairly easily say that problem X looks like problem Y so they’re likely to have similar runtimes.

I love how they portray algorithms in movies, don’t you? (Credit: The Winter Soldier, 2014)

Note that with big-O complexity, it’s most common to think about the problem in terms of runtime, but it can also come into play in terms of space storage. For example, a sorting algorithm may take O(n log(n)) runtime which is pretty common but be able to operate on an array in-place, only requiring O(n) storage. Sometimes this can be an important factor when considering between alternative approaches or an interviewer will add that you are memory-bound or something.

I recommend reviewing and understanding the big-O runtime of the most common data structure operations, such as:

add / remove / get / find from an array
add / remove / find from a linked list
add / remove / peek from a stack
add / remove / peek from a queue
add / remove / get from a hashmap
add / remove / get from a balanced binary tree
add / remove / get from a heap (though heaps are less common…)

You should be intimately familiar with the runtime of each of these operations, since many algorithms will use these as building blocks. It is extremely worth it to not only memorize these runtimes, but to have a solid understanding of how they are derived.

This topic can be difficult to grasp under different circumstances for even the most qualified candidates, so don’t worry if you’re able to come up with a solution but have trouble fleshing out its runtime. Also note that this is one of the easiest topics to “game” by practicing on examples ahead of time.

Understanding Big-O complexity will affect your ability to answer interview questions on all of the following topics, which is why it is the single most important base topic to focus on before proceeding.

One common subtopic I would recommend having a basic familiarity with is amortized big-O, aka expected big-O, whereby you use some neat probability theory to say that the expected value of an operation is, for instance, O(1) even though sometimes it may be O(n) for individual calls. The most common examples of amortized / expected big-O in practice are hashmap lookups being amortized O(1) and quicksort being amortized O(n log(n)). In Javascript, for instance, all object lookups such as myObject.foo or window.document are amortized O(1) hashmap lookups (aside from special cases where the compiler is able to optimize these operations under the hood).

Graphs and Trees

Example graph composed of nodes and edges.

Graphs are one area where there is a lot of potential complexity and bullshit to wade through, but at the end of the day, almost all graph-related interview questions are really pretty simple once you understand the basics. It can just be overwhelming sometimes when you’re not sure what “the basics” are and you’re trying to understand something like Dijkstra's algorithm which is definitely beyond the scope of what most interviews will delve into.

Terminology

A graph is a set of nodes and edges between some of those nodes. Nodes and edges often have payloads like a label or weight associated with them.
The most common graph distinction is between undirected vs directed graphs. E.g., when you have an edge between two nodes, is it a directed, one-way street, or is an undirected, two-way street where you can go in both directions when going from node to node.
A tree is a very common type of graph with some interesting constraints, so anything you learn about graphs in general also applies to trees like binary search trees and the DOM.
Traversing a graph is the process of visiting nodes in a graph, usually starting from a root node and expanding out from there recursively based on each node’s neighbors.
The two main algorithms to understand w.r.t. graphs that 95% of graph questions boil down to, are breadth-first-search (BFS), and depth-first-search (DFS), visualized briefly below.

BFS vs DFS visualization (credit: https://github.com/kdn251/interviews).

Advice

When working with graphs, it can be especially useful to visualize them by drawing examples on a whiteboard, which is one of the only good uses I can think of for a whiteboard during a generic technical interview...

There are lots of different types of graphs and specializations that you may come across during studying, but their distinctions are rarely important for interviews.

The Document Object Model (DOM) is an important example of a common graph used by many frontend developers.

You should be very comfortable coding BFS and DFS from scratch. Even if the question isn’t directly “code BFS”, lots of questions will indirectly involve you traversing a graph starting from a given node of interest and making sure you’re not visiting nodes multiple times which is exactly what BFS/DFS excel at.

Notice how interchangeably I use BFS/DFS; they are very slight variations on each other and most of the time it doesn’t matter if you use BFS or DFS, but you should still understand the difference between the two and be able to draw example traversals on a whiteboard.

BFS and DFS can both be implemented iteratively or recursively (any so-called “tail-recursive” function can be rewritten iteratively). The recursive mindset is much more powerful so I’d focus your efforts there first.

Most of the time, it’s totally up to you in terms of how you define the graph you’ll be working with. For example, here is a very succinct way of representing a graph by defining a single Node:

Node-centric example graph representation.

A common distinction with graphs is whether the data structure you use is “node-centric” or “graph-centric”. The previous Node definition is node-centric because each node is smart and encapsulates information about its adjacent edges. Here’s an alternative graph-centric example, where we also use integers to represent nodes:

Graph-centric example graph representation.

Example question:

Given a graph where nodes represent major US cities and edges represent flights between those cities, write a function that takes in two cities and returns a valid flight path between those two cities or null if no such flight path exists.

The most direct solution to this problem uses DFS.
A harder variant of this type of question would be to find the shortest path if each edge (flight) had a number associated with it that represented distance, which is where Djikstra’s algorithm would come into play.

Graph visualization of flights between major US cities (credit: databricks).

Sorting

Sorting numbers, strings, etc. is a very common sub-problem in solving many interview questions. It won’t be common for an interviewer to ask you to write mergesort or quicksort or any other type of sort, but it will be quite common to either have to sort some part of your input as a piece of the puzzle or have the solution very closely resemble a widely-known sorting algorithm. For this reason, it’s useful to review and be able to code the most common ones.

Note: please don’t take this algorithm seriously. (Credit: Reddit)

Common Sorting Algorithms

Mergesort ; in particular, its recursive “divide & conquer” approach comes up often. O(n log(n))
Quicksort ; generally considered the most robust, general-purpose sorting algorithm. generally amortized O(n log(n))
Radixsort ; only works on numbers using bit hacks but is significantly more efficient. O(n)

Radix sort is too advanced to implement in any interview that’s not from Hell, so don’t worry about its internals, but it can come in handy knowing that it exists and being able to make use of it.

Example question:

> Given an array of integers, write a function that will remove all duplicates. Be sure to add the obligatory followup, what is its runtime?

The “aha” moment here comes if you realize that by sorting the input, you can just walk along the array with all duplicates being next to each other, resulting in an efficient solution.

Strings

Review string primitive operations in your preferred language. Eg., for javascript, slice, substr, substring, toLowerCase, toUpperCase, charAt, and very basic regex stuff using match.

Luckily, most interview questions focus on ascii.

Notes

Strings are just arrays of characters, so any algorithms you learn for arrays also applies to strings.
A very common type of string problem involves finding all possible substrings of a given input string.

Example question:

Given a password string of length n as input and a mapping from all 26 lowercase characters to possible alternates (like “e” maps to [“e”, “E”, “3”]), implement a function that returns all possible password combinations you could generate of length n.

For example, “haxor” could be “Haxor”, “hax0r”, “HAX0r”, etc.

Recursion

Writing recursive functions should flow like bread & butter and has a lot of overlap with all the other topics listed here.

Credit: The Internets

Example question:

Implement a function that will return the nth number in the fibonacci sequence (1, 1, 2, 3, 5, 8, 13, …).

A common follow-up is that the straight-forward solution is typically pretty inefficient, so how could you optimize the recursion?

Example question:

Given a binary tree, implement a “visitor” pattern function that takes in a node and visits all children. Implement prefix, infix, and postfix traversals.

The difference in the traversal order is just moving the order you visit the “current” node around, either before the children, after the left child, or after the right child.

Example question:

Given a simplified DOM node defined as: class Node { string className; Array<Node> children; }, write a function that takes in a Node along with a target css class and returns a list of all subnodes which match that CSS class.

Aside from the traversal, which you’ll likely do recursively, the logic to visit each node needs to take into account the fact that DOM nodes can have multiple classnames, so it’s not enough just to do a direct comparison between the target CSS class and a Node’s className.
This is exactly what the built-in function getElementsByClassName does.

Brainteasers (Abstract Shit)

Brainteasers aren’t as common as they used to be, and these types of questions are more common for PMs (project / program managers), but they still come up occasionally in developer interviews as well.

They typically involve asking you to solve some impossible or outlandishly difficult problem, epitomizing the mantra that your thought process is more important than the solution you come up with.

One of the most famous examples comes from Google back in the day asking candidates “How would you move Mt. Fuji?”

Advice

Realize that the goal isn’t to come up with the best possible solution, but rather a reasonable, viable solution that is backed by reasoning.
Ask clarifying questions; “Where are we moving Mt. Fuji to?”, “What resources do we have to accomplish the task?”, etc.
One common subset of brainteasers is to ask “How many X exist?” such as “How many gas stations are there in the US?”
The goal here is to be able to guesstimate some numbers that give an idea of the order of magnitude of the response, so if we estimate that there are 10 gas stations per town and 2000 towns per state and 50 states, … which should be more than enough to get the ball rolling.

Less Common Topics

These topics aren’t as common as the core algorithm and data structure topics above, but depending on the position you’re applying for, it’s still a good idea to understand the high-level categories and be able to recognize a certain type of question when you encounter it.

Concurrency
Databases
More generic data structures
Dynamic programming
Architecture
And sooooo many more…

Image Credit: Coding Interview University and Silicon Valley

Where To Go From Here?

The purpose of this post is to serve as a jumping off point to focus your interview prep on a few, core topics. Once you’re ready to dive into more detail, here are some great resources that’ll help you flesh out a better understanding of these core concepts with a focus on practical interview training.

Coding Interview University is one of the most starred repos on Github and for good reason. It aggregates articles, classes, videos, and other learning resources across a large number of topics relevant to CS interviews. My only word of warning is that it is pretty overwhelming and covers a lot more areas than are really necessary for standard technical interviews. Nonetheless, this is the first place I would recommend going to learn or review any of the topics I’ve outlined in this post.

Hired in Tech is an amazing, well-organized resource which covers a lot of useful high-level techniques as well as specific examples. I would thoroughly recommend checking it out.

The Tech Interview Handbook is a great resource that, in addition to covering a lot of CS material itself, also gives more practical tips for what to expect and how to approach technical interview loops.

Once you’re comfortable with the core CS concepts I’ve outlined here, I’d recommend spending most of your prep time practicing online coding problems. Just remember while practicing to consider how you’d verbalize your thought process in a real interview setting and remember to consider things like big-O in addition to solving the problems themselves. Here are some of my favorite resources for finding quality practice interview questions:

Interactive Coding Challenges — Lists a large number of interactive practice questions, many of which come with solutions and explanations.
Google Interview Questions — Great list of interview questions previously used by Google provided by Interview Cake.
Coding Interview University — Their section on coding exercises / challenges is a great meta-list of additional resources to find practice questions.

Finally, the best way to get more comfortable with interviewing is by actually interviewing. I know this sounds obvious, but one concrete piece of advice I can give is to apply anywhere and everywhere, even to companies you wouldn’t necessarily consider working for, with the tacit goal of gaining valuable experience in real-world interviews and the added benefit of possibly finding opportunities you didn’t know existed beforehand.

For example, if you’re interested in working for Google / Facebook / Twitter / etc, but you wouldn’t be too keen on working for Oracle & IBM (strictly for example purposes…), I’d encourage you to still apply to them in order to gain practical experience and get more comfortable with interviewing. This is the absolute best way I know of to hone your skills in real-world settings that are going to be fairly comparable to interview loops at the more prestigious tech companies.

❤️ Travis

Before you go, if you found this article useful, I'd love it if you checked out my latest project on GitHub. Thanks!

JavaScript Modules Worth Using 🔥

Travis Fischer — Sun, 02 Feb 2020 18:25:00 +0000

A quick breakdown of the most useful JavaScript modules that I find myself using over and over again.

This is an opinionated article that focuses on general-purpose modules and utilities that I’ve found invaluable to Node.js and frontend JavaScript development. It won’t be exhaustive or include any special-purpose modules, as those types of awesome lists are indeed awesome but tend to be a bit overwhelming.

Yeahhh we all love npmmmmm huzzahhhhhh!!! (Image Credit: npm)

Command Line Tools

Let’s get started with some extremely useful command line tools.

np - A better npm publish.

If you’re an npm author, I’d highly recommend checking out np, as it makes the process of bumping versions, adding git release tags, and publishing to npm a breeze, especially once you start having more than a couple of modules to maintain. It’s also worth noting release by Zeit as a solid alternative.

Image Credit: np by Sindre Sorhus

yarn - An alternative package manager to npm.

There’s no clear winner between npm and yarn. The recent drama over yarn v2 is probably enough to push many yarn devs back to using npm.

As a JS developer in 2020 just make sure that you’re at least familiar with both npm and yarn and be comfortable switching off between them.

prettier - An opinionated code formatter.

Prettier enforces a consistent style by parsing your code and re-printing it with its own rules that take the maximum line length into account, wrapping code when necessary.

I love eslint and have been a long-time user of JavaScript Standard Style in particular, but the idea behind automatic code formatters like prettier and gofmt is undeniably attractive.

As developers, we spend way too much time and mental energy worrying about code presentation and styling, whereas prettier alleviates the need for those thought processes and allows you to focus on what you’re writing instead of how you’re writing it.

Image Credit: prettier

now - Extremely simple deployments.

Now is absolutely the best free deployment system that exists today in terms of simplicity, reliability, and feature set. It‘s great for testing static and dynamic deployments and scales up nicely if and when you require more servers. Aaaaaaaaaand did I mention that it’s free until you want to scale up?!

It plays extremely well with Node.js and JS-powered webapps. I’d also highly recommend checking out the rest of Zeit’s offerings as well, as their team is comprised by some of the best JS devs the community has to offer.

Image Credit: Zeit

asciinema - Free tool for recording high quality terminal sessions.

See my previous blog post “Making your Code Beautiful” for a breakdown of how you can take advantage of asciinema to produce quality code demos & screencasts like the pros.

Promises

This section really deserves an entire article unto itself, especially now that async & await have started becoming the de facto standard paradigm for concurrent programming in JavaScript. With that being said, I’d highly recommend checking out Sindre Sorhus’ excellent promise-fun module collection if you haven’t already. My only gripe with these modules is that they likely won’t work out-of-the-box with most frontend toolchains like create-react-app or rollup.

Here are a few of the most useful gems which stick out for working with promises and async-style code in Node:

pify - Promisify a callback-style function.

There are a lot of ways to convert functions from old-school callback-style to Promise-style, but I’ve found pify to be the best. It’s tiny and has some niceties like automatic method binding that the built-in util.promisify is lacking.

p-map - Map over promises concurrently.

Concurrency is great, but most of the time you want to set a practical limit on parallelism whether it be throttling network bandwidth or compute resources. This is where p-map really shines. I use it 99% of the time as a drop-in replacement for Promise.all(…), which doesn’t support limiting parallelism.

Before I was aware of p-map, I created my own version async-await-parallel, but you should use p-map as it’s better. 😛

p-retry - Retry a promise-returning or async function.

I generally wrap any HTTP requests and external service calls with p-retry to add a basic level of robustness to them. Combined with p-map, you can process large batches of external requests with controlled parallelism without having to worry too much about an occasional transport error, socket hang-up, or server timeout.

p-timeout - Timeout a promise after a specified amount of time.

Alongside p-retry, p-timeout is a must for robustly working with third-party APIs and services. You can also specify an optional fallback, as oftentimes returning something is better than hanging indefinitely or returning after an inordinate amount of time.

p-cache or p-memoize - Memoize the results of an async function via an LRU cache.

The aim of many of these Promise utilities reminds me a lot of architecting robust microservices, where every external dependency can be treated with a common interface supporting retrys, timeouts, caching, circuit breakers, fallbacks, etc.

Graceful degradation of functionality is generally preferable to overwhelming the system or not responding at all, so if you’re not too familiar with microservices, check them out and see if their design decisions can help improve your Promise handling abilities as well.

Scraping

There are a lot of great scraping utilities out there, some of which operate on raw HTML like cheerio, and some of which simulate a full browser environment like puppeteer. What you decide to use really depends on your use case, as working with raw HTML is a lot quicker and more lightweight, whereas automating a headless browser is more robust at the cost of being heavier to get started.

cheerio - Fast, flexible, and lean implementation of core jQuery designed specifically for the server.

Cheerio is really great for quick & dirty web scraping where you just want to operate against raw HTML. It provides a robust jQuery-like syntax for traversing & manipulating HTML documents. Cheerio pairs particularly well with request-promise-native below for fetching remote HTML documents.

puppeteer - Headless Chrome Node API

Unlike cheerio, puppeteer is a wrapper for automating headless chrome instances, which is really useful for working with modern JS-powered SPAs. Since you’re working with Chrome itself, it also has best-in-class support for parsing / rendering / scripting conformance. Headless Chrome is still relatively new, but it will likely phase out older approaches such as PhantomJS in the years to come.

If you need to faithfully scrape websites, automate web-based workflows, or capture screenshots, puppeteer is a clear winner that will only become more popular with time.

Node.js

dotenv-safe - Load environment variables from .env and ensure they’re all present.

This module extends the hugely popular dotenv module to enforce the existence of expected environment variables via a .env.example file. Like the original, it provides fast, secure, and robust environment variable support for Node.

It also plays well with Zeit’s now.sh deployments with the ”dotenv”: true option set in now.json.

request and request-promise-native - Simplified HTTP request client.

Making HTTP requests is an extremely common operation, and my goto module here is request-promise-native which wraps the original request module with native ES6 promise support. 95% of the time I want to await the result of a promisified HTTP request. The other 5% of the time I want to work with the response stream directly, in which case I use the underlying request module, foregoing Promise support.

For robustness, I will oftentimes wrap request-promise-native calls in some combination of p-retry, p-timeout, and p-cache.

It’s also worth mentioning got as a newer alternative to request with promise support baked in, although I haven’t used it much personally.

Example of downloading an HTML document with request-promise-native.

consolidate - Template engine consolidation library for Node.

Consolidate is great for handling any type of backend templating (emails, tweets, raw html, etc.). I generally use handlebars as my templating engine of choice, but no matter what, I always wrap my template usage in consolidate, because it provides a simple & consistent interface to templating regardless of the template engine you decide to use under the hood.

For example, I used consolidate in create-react-library to render the boilerplate’s templates with library-specific variables.

execa - A better child_process.

Extremely useful if you need to run a shell command or spawn a child process in general.

fs-extra - A better fs with additional methods and Promise support.

It’s rare that I find myself using fs directly anymore. Try fs-extra and you won’t look back.

Math

D3 (Data-Driven Documents) is a popular frontend library for data visualization and animation. It also contains some of the best standalone packages for common math operations that I find myself consistently choosing over alternative modules.

d3-random — Generate random numbers from various distributions.

When Math.random doesn’t cut it, give d3-random a try. It supports sampling from different common distributions, including uniform, normal, and exponential.

d3-ease - Easing functions for smooth animations.

Image Credit: d3-ease

d3-interpolate - Interpolate numbers, colors, strings, arrays, objects, whatever!

This module provides a variety of interpolation methods for blending between two arbitrary values. Values may be numbers, colors, strings, arrays, or even deeply-nested objects.

Testing

ava - Awesome JS test runner.

It’s not surprising that my go-to for unit test runner for Node.js is yet another tool created by Sindre Sorhus. Ava is a newer unit test runner that takes a lot of what was good about mocha, tape, chai and other JS test runners, and bundles it all together into a quality project with sensible defaults that “just works”.

It’s worth noting that Ava’s tests are run in parallel by default, which you can disable at the file-level for use cases like database testing where the order your unit tests run may be important.

Image Credit: ava

nock - HTTP mocking and expectations library.

Nock is great for testing modules that perform HTTP requests in isolation. If your Node module makes HTTP requests and you want to provide proper unit testing, then nock is the way to go.

sinon - Function spies, stubs and mocks for JS testing.

Sinon is a very useful utility library for writing isolated tests by taking advantage of dependency injection. It should be a part of every Node developer’s tool belt.

Wrapping Up

I hope you’ve found this breakdown helpful, even if it’s just learning about one quality module that you weren’t aware of before. I know a lot of aspiring and seasoned developers alike who end up rolling their own solutions to common problems, which can be a useful practice in & of itself, but it’s also good to know when there are quality, existing solutions you should be using instead of constantly reinventing the wheel.

The size & scope of NPM’s module library is unprecedented, and imho it’s one of JavaScript’s biggest advantages compared with other programming languages. The better you become at taking advantage of npm modules, the faster and more productive you’ll be as a developer. Higher-order “soft” skills like this are one of the hallmarks of becoming a mythical 10x programmer.

Have any favorite npm modules I left out? Let me know by sharing your favorite modules in the comments! ❤️

Before you go…

If you liked this article, please leave leave a ❤

Scraping the web with Node.js

Travis Fischer — Wed, 29 Jan 2020 09:49:00 +0000

An in-depth guide to building a minimal, robust web scraper for extracting structured data on the internets.

Node.js provides a perfect, dynamic environment to quickly experiment and work with data from the web.

While there are more and more visual scraping products these days (import.io, Spider, Scrapinghub, Apify, Crawly, ……), there will always be a need for the simplicity and flexibility of writing one-off scrapers manually.

This post is intended as a tutorial for writing these types of data extraction scripts in Node.js, including some subtle best practices that I’ve learned from writing dozens of these types of crawlers over the years.

In particular, we’ll be walking through how to create a scraper for GitHub’s list of trending repositories. If you want to follow along with the code, check out the repo scrape-github-trending.

Building Blocks

One of the best features of Node.js is the extremely comprehensive community of open source modules it has to offer. For this type of task, we’ll be leaning heavily on two modules, got to robustly download raw HTML, and cheerio which provides a jQuery-inspired API for parsing and traversing those pages.

Cheerio is really great for quick & dirty web scraping where you just want to operate against raw HTML. If you’re dealing with more advanced scenarios where you want your crawler to mimic a real user as close as possible or navigate client-side scripting, you’ll likely want to use Puppeteer.

As far as got goes, there are dozens of HTTP fetching libraries available on NPM, with some of the more popular alternatives being superagent, axios, unfetch (isomorphic === usable from Node.js or browser), and finally request / request-promise-native (most popular library by far though the maintainers have officially deprecated any future development).

Getting Started

Alright, for this tutorial we’ll be writing a scraper for GitHub’s list of trending repositories.

The first thing I do when writing a scraper is to open the target page in Chrome and take a look at the how the desired data is structured in dev tools.

Chrome Devtools is very handy for scraping 💯Also, I know I know I need to charge my laptop…. 😂

Switching back and forth between the Console and Elements tabs, you can use the $$(‘.repo-list li’) selector in the console to select all of the trending repos.

What you’re looking for in creating these CSS selectors is to keep them as simple as possible while also making them as focused as possible. By looking through the Elements tab and selecting the elements you’re interested in, you’ll usually come up with some potential selectors that may work. The next step is to try them out in the Console tab using the $$() syntax to make sure you’re only selecting the elements you intended to select. One rule of thumb here is to try and avoid using aspects of the HTML’s structure or classes that may change more often in refactors or code rewrites.

Let’s write a scraper!

Now that we have a good idea of some CSS selectors that will target our desired data, let’s convert them to a Node.js script:

Note that we’re using async / await syntax here to handle downloading the external web page asynchronously in a way that looks synchronous.

Line 12: we download the remote page and extract it’s text body (HTML).
Line 14: we load that HTML into cheerio so that it’s easy to traverse and manipulate.
Line 15: we select all the repository li elements using our previous CSS selector and map over them.
Lines 16–32: we extract the relevant portions of each trending repo into a plain JSON object.
Line 33: here we’re filtering out any repos that failed to parse correctly or threw an error. These will be undefined in the array and [].filter(Boolean) is a shorthand syntax for filtering any non-truthy values.

At this point, we’ve succeeded in scraping a single web page and extracting some relevant data. Here’s some example JSON output at this point:

Crawling Deeper

Now that we’ve explored how to scrape a single page, the next logical step is to branch out and crawl multiple pages. You could even get fancy and crawl links recursively from this point on, but for now we’ll just focus on crawling one level down in this data, that is the repository URLs themselves.

We’ll follow a very similar approach to how we scraped the original trending list. First, load up an example GitHub repository in Chrome and look through some of the most useful metadata that GitHub exposes and how you might target those elements via CSS selectors.

Once you have a good handle on what data you want to extract and have some working selectors in the Console, it’s time to write a Node.js function to download and parse a single GitHub repository.

The only real difference here from our first scraping example is that we’re using some different cheerio utility methods like $.find() and also doing some additional string parsing to coerce the data to our needs.

At this point, we’re able to extract a lot of the most useful metadata about each repo individually, but we need a way of robustly mapping over all the repos we want to process. For this, we’re going to use the excellent p-map module. Most of the time you want to set a practical limit on parallelism whether it be throttling network bandwidth or compute resources. This is where p-map really shines. I use it 99% of the time as a drop-in replacement for Promise.all(…), which doesn’t support limiting parallelism.

Here, we’re mapping over each repository with a maximum concurrency of 3 requests at a time. This helps significantly in making your crawler more robust against random network and server issues.

If you want to add one more level of robustness here, I would recommend wrapping your sub-scraping async functions in p-retry and p-timeout. This is what got is actually doing under the hood to ensure more robust HTTP requests.

All together now

Here is the full executable Node.js code. You can also find the full reproducible project at scrape-github-trending.

And an example of the corresponding JSON output:

Conclusion

I have used this exact pattern dozens of times for one-off scraping tasks in Node.js. It’s simple, robust, and really easy to customize to practically any targeted crawling / scraping scenarios.

It’s worth mentioning that scrape-it also looks like a very well engineered library that is essentially doing everything under the hood in this article.

If your crawling use case requires a more distributed workflow or more complicated client-side parsing, I would highly recommend checking out Puppeteer, which is a game changing library from Google for automating headless Chrome. You may also want to check out the related crawling resources listed in awesome-puppeteer such as headless-chrome-crawler which provides a distributed crawling solution built on top of Puppeteer.

In my experience, however, 95% of the time a simple one-file script like the one in this article tends to do the job just fine. And imho, KISS is the single most important rule in software engineering.

Thanks for your time && I wish you luck on your future scraping adventures!

Data Fingerprinting in JavaScript

Travis Fischer — Sun, 26 Jan 2020 23:35:20 +0000

I want to talk a little about how you can use content-based addressing (aka data fingerprinting) as a general approach to make your applications faster and more secure with some practical JavaScript examples.

First off, I find the concept of content-based addressing dope af. 👀

It's an extremely powerful tool for building services that are fundamentally more performant, scalable and secure. 💪

It's related to immutability, decentralization, data integrity, and more buzzwords...

But it's also so useful and under-appreciated in general that I wanted to write a practical intro to show how it works alongside some real-world JavaScript.

What the hell are you talking about?

You can think of content-based addressing as fingerprinting for data.

Just like how fingerprints allow you to:

Identify a person based on their fingerprint
Refer to a fingerprint as a unique ID for the person
Tell if two people are the same person based on their fingerprints
Quickly test to see if a person is in a database using just their fingerprint

Just replace "person" with "data" in the above descriptions and you have a rough overview of what content-based addressing enables.

Do you know where your data's been? 👀

Put another way, content-based addressing allows you to uniquely and efficiently reference data based on it's actual content as opposed to something external like an ID or a URL.

Database-generated IDs, random GUIDs, and URLs are all useful in their own right, but they're not quite as powerful as data fingerprinting.

Shut up and show me some code

Let's see how this looks with some real-world code that I've used for reals:

const pick = require('lodash.pick')
const stableStringify = require('fast-json-stable-stringify')

const data = pick(myData, ['keyFoo', 'keyBar'])
const fingerprint = hash(stableStringify(data))

This snippet leaves out the hash function (more on that below), but it does represent the core algorithm pretty clearly.

It creates a content-based hash fingerprint of any JavaScript object myData that is a unique representation of that object based on the keys we care about [ 'keyFoo', 'keyBar' ].

In short, this fingerprint offers you a very efficient way of telling when two JavaScript objects are the same.

If two content-based IDs are the same, the data in those objects is the same.

No need for a deep comparison. No need for Redux. Just pure immutable goodness.

So how does this actually work?

We'll break this process into three distinct steps: 1) Input data 2) data cleaning 3) simplification.

Let's take another look at our JavaScript code:

const pick = require('lodash.pick')
const stableStringify = require('fast-json-stable-stringify')

const data = pick(myData, ['keyFoo', 'keyBar'])
const fingerprint = hash(stableStringify(data))

First, we take as input any JavaScript object myData. This could be a model from your database or some object containing Redux-like app state, for instance.

Second, we clean our data to ensure that we're only considering parts of the data we actually care about via lodash.pick. This step is optional but usually you'll want to clean your data like this before proceeding. I've found in practice that most of the time there will be parts of your data that aren't actually representative of the uniqueness of your model (we'll refer to this extra stuff as metadata 😉).

As an example, let's say I want to create unique IDs for all of the rows in a SQL table. Most SQL implementations will add metadata to your table like the date an entry was created or modified, and it's unlikely we'd want this metadata to affect our notion of uniqueness. In other words, if two rows were inserted into the table at different times but have the exact same values according to our application's business logic, then we want to treat them as having the same fingerprint so we filter out this extra metadata.

Third, we simplify our cleaned data into a stable, efficient representation that we can store and use for quick comparisons. Most of the time this step involves some sort of cryptographic hash to normalize the way we refer to our content in a unique, concise manner.

In the code above, we want to make sure that our hashing is stable, which is made easy for us by the fast-json-stable-stringify package.

This awesome package recursively makes sure that no matter how our JavaScript object was constructed or what order its keys may be in, it will always output the same string representation for any two objects that have deep equality.

There are some details this explanation is glossing over, but that's the beauty of the NPM ecosystem – we don't have to understand all the bits & pieces to take advantage of their abstractions.

Let's hash this thing out

Whatsha up to?

Up until now, we've glossed over the hashing aspect of things, so let's see what this looks like in code:

const hasha = require('hasha')

const hash = (input) => hasha(input, { algorithm: 'sha256' })

Note that there are lots of different ways you could define your hash function. This example uses a very common SHA256 hash function and outputs a 64-character hex encoding of the results.

Here is an example output fingerprint: 2d3ea73f0faacebbb4a437ff758c84c8ef7fd6cce45c07bee1ff59deae3f67f5

Here is an alternative hash implementation that uses the Node.js crypto package directly:

const crypto = require('crypto')

const hash = (d) => {
  const buffer = Buffer.isBuffer(d) ? d : Buffer.from(d.toString())
  return crypto.createHash('sha256').update(buffer).digest('hex')
}

Both of these hash implementations are equivalent for our purposes.

The most important thing to keep in mind here is that we want to use a cryptographic hash function to output a compact, unique fingerprint that changes if our input data changes and remains the same if our input data remains the same.

So where should I go from here?

Once you start thinking about how data can be uniquely defined by its content, the applications are really endless.

Here are a few use cases where I've personally found this approach useful:

Generating unique identifiers for immutable deployments of serverless functions at Saasify. I know ZEIT uses a very similar approach to optimize their lambda deployments and package dependencies.
Generating unique identifiers for videos based on the database schema we used to generate them at Automagical. If two videos have the same fingerprint, they should have the same content. One note here is that it's often useful to add a version number to your data before hashing since changes in our video renderer resulted in changes to the output videos.
Caching Stripe plans and coupons that have the same parameters across different projects and accounts.
Caching client-side models and HTTP metadata in a React webapp.

We've really only started to scratch the surface of what you can do with content-based addressing. Hopefully, I've shown how simple this mindset shift can be done in JavaScript and touched on a bit on the benefits this approach brings to the table.

If you enjoy this stuff, I would recommend checking out:

The power of content-based addressing - An awesome intro to the topic with a focus on content identifiers (CIDs) as they're used in IPFS.
Multihashes - Self-describing hashes. 💪
Merkle trees - A recursive data structure built on top of content-based hashes.
Rabin fingerprinting - An efficient string searching algorithm that uses content-based hashing.
IPFS - InterPlanetary File System.
libp2p - Modular building blocks for decentralized applications.
Saasify - An easier way for devs to earn passive income... Oh wait, that's my company and it's not really related to content-based addressing but cut me some slack haha 😂

Thanks! 🙏

How interested are you in working remotely?

Travis Fischer — Sun, 26 Jan 2020 04:44:07 +0000

Lately, I've been having some fun pushing the limits of Typeform's free tier with the goal of doing some market research for Saasify.

Since we're really all about helping developers launch their own remote passive income streams, I'd love it if some of you fine folks took a few minutes to help me with some market research. (10 questions, ~2 minutes)

And because your time is valuable, if you fill out the survey and respond to this thread pointing me to a project of yours, I will personally review your project / website / landing page / idea and give you the best feedback I can offer. Maybe it'll be worthwhile, maybe it won't but I promise that I'll try to help you out with targeted feedback.

Also, if any hackers out there have had success with their own forms of market research, I'd love to hear what's worked for you.

Thanks! ❤

Modern web accessibility 💪

Travis Fischer — Wed, 15 Jan 2020 00:16:51 +0000

We're excited to help enable a more accessible & inclusive web via Ta11y, an open source suite of modern web accessibility testing tools.

Intro

We've tried to design Ta11y to be as simple to use as possible.

The easiest way to get started is via the CLI, but you can also use it programmatically from Node.js via @ta11y/core.

Ta11y uses Puppeteer to crawl and extract content from websites, so you can be sure you're testing any dynamic, JavaScript-powered content that your users will see.

Ta11y supports a large number of output formats such as JSON, Excel, CSV, and HTML.

Features

Accessibility as a service
- Audit your websites with a range of test suites including WCAG 2.0/2.1 A, AA, AAA, Section 508, HTML validation, as well as our own best practices.
Flexible and automated
- Run manual tests during development and then integrate into any CI pipeline. Supports generating reports in XLS, XLSX, CSV, JSON, HTML, and more.
Runs in any environment
- Easy integration that supports localhost, firewalls, custom auth, as well as any public production environment.
Modern dynamic websites
- Ta11y treats all websites as dynamic with full JavaScript support, so you'll test pages as your users actually experience them.
Free to try
- Simple to get started for free, then sign up once you're ready to remove rate limits. Have a non-profit use case? Get in touch.
Private & secure
- Ta11y is built using serverless functions and never stores any of your data or audit results.

Examples

This example runs the wcag2a and wcag2aa audit test suites on the given URL and outputs the results to an Excel spreadsheet (supports any xls, xlsx, or csv file).

ta11y https://example.com -o audit.xls

This example runs the wcag2a and wcag2aa audit test suites on the given URL and outputs the results to a comma-separated-value file (csv).

ta11y https://example.com --suites wcag2a,wcag2aa -o audit.csv

This example will crawl all pages of a local site and then perform an audit of the results.

Note that the local site does not have to be publicly accessible as content extraction happens locally.

ta11y http://localhost:3000 --crawl -o audit.json

Feedback

Ta11y is a new open source project written in JavaScript and built on the shoulders of other OSS giants.

We're looking for feedback and contributors, so please try it out and let us know if you have any thoughts. Thanks! 🙏

A better way to fund OSS

Travis Fischer — Mon, 02 Dec 2019 20:00:42 +0000

Hi everyone! 😄

For the past year, I've been trying to make it easier for OSS devs to earn sustainable funding.

I'm very proud to be working on Saasify, a platform which enables devs to monetize open source projects via serverless APIs. 😍

Our mission at Saasify is to provide sustainable funding for open source.

If you've ever struggled to make a living working on open source, I hear you.

OSS can be extremely powerful and rewarding, but there is a huge problem that exists in the market between how valuable open source is to every business in existence versus how difficult it is for OSS developers to capture any of that value.

Donation platforms like Patreon and GitHub Sponsors help with a very small percentage of this problem because the business model of voluntarily giving back to OSS is inherently weak.

Contrast this with the standard SaaS API business model of paying based on usage that aligns much more naturally with how businesses actually pay for software critical to their core business needs.

Saasify is aiming to bridge the gap between this disparity by reducing the barrier to entry for developers who are interested in earning passive income via SaaS.

On a personal level, I care deeply about the problem of OSS sustainability and am looking to connect with other devs who feel similarly. And while Saasify won't work for every project, it's my sincere hope that it will help other independent OSS developers like myself earn passive income.

Looking forward to hearing your thoughts on Saasify && thanks for reading!

WordCloud

Travis Fischer — Tue, 19 Nov 2019 09:02:34 +0000

Very excited to be launching WordCloud today on PH!

It's a simple API to create attractive, unique preview images from any URL.

This SaaS API wraps the open source stylecloud and wordcloud packages, with 80% of any revenue from the API being set aside for these OSS authors.

Our core mission at Saasify is to provide sustainable funding for open source developers, and today marks a big step in that direction.

I'm particularly excited as this is our first launch for an API written in Python, where we're relying heavily on FastAPI to generate the OpenAPI schema that powers the API.

It's also open source -- check out the relatively simple code we used to generate this webapp.

If you have any questions or comments, please let me know -- thanks!

One SaaS PH Launch a Week...

Travis Fischer — Fri, 08 Nov 2019 01:14:55 +0000

Our team at Saasify is busy preparing for our YC interview in a few weeks.

In order to validate our product, our main objective right now is to launch one fully polished FaaS API every week for the next two months!

We recently had our first two PH launches for products built on top of Saasify:

Puppet Master - Headless chrome as a service.
Synopsis - Summarize any website or article automagically.

These should give you a good idea for the types of SaaS API products we're targeting.

One of the things that I'm most excited about is how Saasify enables us to launch many product experiments very quickly. Too many founders put all their eggs into one basket and inevitably fail to find product / market fit, and with Saasify, we're hoping that each of these product launches will teach us about more about our market and what to focus on.

Our thesis is that many of these individual FaaS products will fail to gain traction, but if each one has a certain probability of finding product / market fit, then N independent product launches improves our chances linearly by N, which imho is extremely powerful in the aggregate.

Eventually, these products will all blend into a cohesive marketplace of standardized FaaS, but for now we're focusing on marketing them as standalone SaaS products to optimize messaging and inbound channels.

We have a strong pipeline of projects and OSS authors we're working with to meet our aggressive launch schedule, but we need YOUR help!

If you have a project you'd like to saasify and start monetizing, we'd LOVE to hear from you!

Feel free to DM me on our open slack && thanks for reading!