Forem: Fred K. Schott

6 More Things I Learned Building Snowpack to 20,000 Stars (Part 2)

Fred K. Schott — Wed, 15 Sep 2021 22:58:01 +0000

This article is the second entry in a two-part series. In the first post, I went over the early history of Snowpack and how we grew an open source project to find our first set of users. In this post, I want to focus on what happened next: how do you maintain and continue to grow a large project at this scale?

This will be an entertaining read for anyone interested in open source software. The highlighted lessons are for current (or aspiring!) open source maintainers of large and/or growing open source projects.

Background

If the first post in this series was about everything that I did right with Snowpack, then this post is about everything that went wrong.

We came into the year with big expectations: Voted "Productivity Booster of the Year" in the OS Awards. Voted #1 & #2 placement in the "2020 State of JavaScript" survey on build tools. Exploding from 200,000 downloads in 2020 to 1.3M in 2021.

When you do something like this for the first time, you're never going to get it 100% right. This was my first experience maintaining an open source project of this scale. I had started plenty of new repos in the past, and some of them were even well liked, but none had ever grown this big. We didn't have a roadmap for this transition, and I made plenty of mistakes that I now see in hindsight.

I want to make it clear that I'm incredibly proud of this project and the people who have contributed to it. Snowpack pushed the entire web development industry forward, and that's pretty cool. Even if you never use Snowpack directly, the work that we pioneered -- specifically around npm package handling for ESM and unbundled development -- is being built on and improved on across the entire web tooling landscape in projects like Vite, Skypack, JSPM CDN and others.

This post is my attempt to create a guide for anyone who finds themselves in a similar position one day.

Lesson 1: Dogfood large, real-world projects

Real-world testing is super important. I'm sure that sounds cliche, but its true. We had a few starter projects that we could test Snowpack against, but they were all small and simple. This created a huge experience gap between our internal projects and our actual users.

People tend to think of "dogfooding" as a way to prevent bugs, but I've found it to be most useful as a way to align with your users. It's impossible to make good decisions about something that you don't know well. Without some kind of real-world dogfooding, you often end up prioritizing the wrong features and fixes.

This is unfortunately one thing that large-scale corporate open source does well. Facebook is able to test a new React feature or bugfix across a codebase of 30,000+ components. They can try things out internally, at scale, before sharing publicly.

If your project isn't owned by a tech giant, you still have options. If you work somewhere full-time, try to dogfood things within your company. Rich Harris often talks about how using Svelte at The New York Times benefits the framework. Your company could be a real-world playground for new features, API changes, and even entire pre-release projects.

Snowpack never had a company playground. Yet, we still could have been better about talking to our users and getting feedback before working on features. In retrospect I would have sought out invites to real-world codebases in exchange for some testing and support.

Lesson: Dogfood large projects to prevent bugs and useless feature work.

Lesson 2: Painless developer experience is essential

In the early days of a project, you'll be forgiven for a few bugs and odd behaviors. As your project matures, this patience tends to run out. The real problem doesn't have to be a single big bug, but the sum of multiple "poor" user experiences.

For example, you should always have a clear error message when something breaks. Yes, even if you think that it was the user's fault:

As our audience transitioned from early-adopters to a larger "mainstream" audience, users became less likely to track down odd errors (undefined is not a function 😱). Instead, they would abandon the project for more familiar/stable alternatives.

This is also relevant to how you choose new features. "Bundling should be optional" was a core idea baked into Snowpack from the start. If you remember back to the first post in this series, that was the idea that our first users fell in love with. As we grew, mainstream users didn't love it so much. They were mostly confused why they had to implement such a simple feature themselves.

Lesson: As your audience grows, understand how your users change. Invest in testing, clear error messages, and overall stability. Make sure that the default user experience is good before investing in advanced features.

Lesson 3: Your users won't tell you everything

Snowpack almost powered SvelteKit.

Rich Harris announced it at the Svelte Summit conference, and published a blog about how excited he was about our project. We were ecstatic. But right before SvelteKit's public release, they switched out Snowpack for an alternative tool called Vite. We found out tool late. The decision had already been made. Their team was unhappy with Snowpack, and we hadn't even noticed!

You tend to have a strong connection to your users on smaller projects. But as the audience grows, you lose touch a bit. I had gotten so used to this feedback cycle that I hadn't even thought to check in. I had missed the rough edges that the Svelte team was encountering every day, and only got their feedback after it was too late to change any minds.

It's important for open source leaders to invest in feedback channels. We learned this too late.

Lesson: Don't wait for your users to tell you what's wrong. Be proactive about gathering feedback and fixing issues.

Lesson 4: Be consistent

The best part of open source development is the community. As your project grows, you'll see more people stopping by to chat, comment on issues, and maybe even contribute some code. Repeat contributors can become life-long friends.

Consistency is the best way to build trust in your community. Bursts of productivity are fine for personal projects, but the long stretches of quiet that usually follow are poison to a growing community. This might be the most common mistake that I see large open source projects make. When you step away from your project, contributors and potential future contributors notice. There's nothing worse than putting time into a PR and then having it sit around, uncommented and unnoticed for weeks or months.

I want to stress that the solution here is not "just spend more time." That's a guaranteed path to burnout. Instead, spend your time better. It's better to spend an hour or two every week than it is to spend a full day, once a month.

For what it's worth, this is something that I'm still working on myself.

Lesson: Be consistent. Don't leave your contributors hanging on code reviews and pull requests.

Lesson 5: Be present and use Discord (seriously.)

I mentioned it before but it's important enough to say again: Use Discord. Create a community server as soon as your get your first users. If you already have a Slack community, start thinking about moving. Seriously, it's that much better.

A new Discord server will only ever be as active as you are. If you never visit it, don't expect much to happen. If people never get a response, don't expect them to stay for long. Calling back to the previous two sections: Consistent presence is the best way to build a community and get valuable feedback from your users.

Discord also is great at encouraging experimentation. Is someone recommending a great bot (aka integration) for your server? Try it out! Ask them to help integrate, customize, or even teach you how Discord works. If your codebase is daunting, Discord can be a great middle-ground where you can collaborate with (and even learn from) your community.

Lesson: Use Discord. Be present + consistent. Embrace the fun side of the platform (emotes/emojis, bots, stickers, etc).

Lesson 6: You can't do it all on your own

It's important to realize when your project has grown beyond your ability to maintain it alone. At that point you'll have a decision: bring on more people, or burn out.

"It will just be faster if I do it myself" might be true short-term thinking, but its dangerous over the long-term.

Despite accepting plenty of contributions over the years, I still fell into this trap with Snowpack. A part of me wanted to run the project all by myself, and refused to encouraged larger contributions. I shipped some great stuff during that period, but I also rushed my work. Code quality suffered. I skipped code reviews because I felt I didn't have time for it. And then when I did step away to recover, I would stay away for longer periods and the project would go quiet.

Ever been so burnt out that you didn't have the energy to realize it? Yeah. It's tough.

Lesson: You can't do it all yourself. Building a community can be the most fun part of open source, if you invest in it. Read up on good open source governance to learn how others do it.

Epilogue: What's Next for Snowpack?

If you're currently an open source maintainer or contributor, I hope that you have found this honest guide useful! This last year has been a wild journey, but I wouldn't trade a moment of it.

Painful mistakes tend to stick. I've already started to apply these lessons to our newest project, Astro. We've already invested in an active Discord, a healthy governance model, a solid test suite, a focus on stability, and a community of amazing maintainers.

It's a great feeling to step away and know that your project is in good hands.

To be honest, I'm not sure where Snowpack goes from here. I burnt out on it at the end of last year, and haven't found the energy to return. Usage and downloads began to trend down and the community has gotten quieter.

At the same time, Vite (that Snowpack alternative that now powers SvelteKit) is taking off. To their credit, they do a lot of things really well. The good news is that two tools are very similar and easy to switch out. Even Astro is experimenting with moving from Snowpack to Vite in a future release.

So maybe it makes sense to wind things down. We asked our community if anyone wanted to get involved in long-term maintenance. But new contributor onboarding takes time that we just can't seem to find any on our end. It's a bit of a Catch-22.

One other idea would be going back to basics, and bring this story full-circle. The ESM package installer that our first users fell in love with still exists as its own package. The audience for a utility like that would be smaller. It might even be fun!

Whatever happens, I know that we'll keep learning and keep improving.

Thanks for reading! Follow me on Twitter for more updates. If you missed it, check out the first post in the series.

5 Things I Learned Building Snowpack to 20,000 Stars

Fred K. Schott — Mon, 13 Sep 2021 17:27:18 +0000

Update: Part two has been posted!

My name is Fred, and I created Snowpack. If you're not familiar, Snowpack is a web build tool that fundamentally unlocked the "unbundled web development" movement that Snowpack, Vite, SvelteKit and other modern dev tools leverage today.

In this post, I want to share 5 things that I learned growing Snowpack from the initial commit to almost 20,000 GitHub stars and over 1,000,000+ downloads.

This post is meant for anyone interested in Open Source software. The highlighted lessons are directed at anyone who is interested in starting their own open source project or contributing to an existing project.

This will be a 2-part series: In this first post I focus on lessons learned creating Snowpack from scratch and finding our first set of users. In part two, I will focus what it's like to maintain a popular open source project, at scale.

Background

A couple of years ago, I started an experimental JavaScript project. Codename: Pika. It had a cute, blue mouse mascot and a fun vibe that a bunch of smaller experimental projects could live under. Its unifying mission could be best summarized as, "ESM is this cool new technology, lets do more stuff with it."

That first year of Pika may have been the most productive year of my life. I created @pika/pack (a publishing tool for npm package authors), Pika CI (a Github action that let you npm install or even import() any GitHub PR), a failed in-browser code editor, and a next-gen JavaScript CDN that went on become Skypack.

The biggest standout of the bunch was @pika/web, which let you install any npm package to run directly in the browser without a bundler (ex: react -> /react.js). You probably know this project better under its newer name: Snowpack.

Below are five lessons that I learned while growing Snowpack from its first commit to the official v1.0 release, and how we found our first set of users.

Lesson 1: Start with a personal frustration

Snowpack began as a tool to convert any npm package to a single JavaScript file that you could run in the browser. Sounds boring, right? Wrong!

This small, straightforward tool would unlock an entirely new mode of web development that is now referred to as "Unbundled Web Development". Unbundled development introduced features like instant reloads and near-instant startup time during development, using a process that wouldn't slow down as your project grows to 1,000 or even 10,000+ files. Compare this to more traditional bundled dev environments, where multi-second startup and reload times are still the norm today.

The original idea for Snowpack came out of a simple, personal frustration that I had been having at work. I was working on the Polymer team at Google, where I had helped create some alterative build tools for the (now dead) HTML Imports spec. The tool itself was lovely, but it didn't work well with npm and very few people ever used it.

I eventually left the Polymer team, but that problem still stuck in my head: Why had bundlers like webpack become the only way to use npm packages in the browser? Something has to solve the problem of getting npm packages to run in the browser, but did it have to involve bundling your entire website? Snowpack was my attempt to find out whether another path was possible.

Lesson for open source maintainers: Build for yourself, first. If you're frustrated by something, chances are other developers are too. Question everything.

Lesson 2: move fast, stay small

When you're working on a new project, you rarely know what code will be important long-term and what code is about to be deleted. I've thrown away enough code in my career to have learned that there's sometimes value in fast, messy coding. When you're starting a new project, it's okay to be a bit messy.

Internally, almost all of Snowpack's complexity was handled by Rollup. Snowpack was really just a wrapper around Rollup that would bundle only your npm packages instead of your entire website. Realizing that Snowpack could leverage Rollup internally saved me weeks (or maybe even months) of development time.

To be honest, Snowpack was just a single index.js file for the majority of its life.

To keep the project on track, I focused on a single feature: "give me a package name, and I'll convert it to ESM." This would be too low-level for most web developers. In fairness, Snowpack really didn't take off with a large audience until we added a built-in dev server and build command in v2.0. But even without the dev server, Snowpack's small v1.0 focus was enough for a certain kind of low-tooling/no-tooling web developer.

My overeall recommendation is to test your idea and get together a working demo as quickly as possible. In practice, this means four things:

Use existing tools! Fork a similar open source project or use an existing tool internally if it can save you time.
Write messy code! At the earliest stage, you probably don't know exactly know what you're building. Premature refactoring can sometimes be worse than never refactoring at all, so keep your code flexible for as long as possible.
Keep scope small! Don't build half-baked, half-working features. Instead, focus on doing one thing very well.
Skip tests! Confirm that you're building something useful before spending your free time writing tests for it. Nothing is worse than writing tests for something that you end up never using.

I know that some of this could be considered a hot/controversial take. "No testing??? Blasphemy!" All I can say is that this strategy has worked well for me over several popular projects and countless unpopular projects that went nowhere.

The obvious downside to this "move fast" advise is that it is not sustainable long-term. Moving fast means taking on tech debt, which you will absolutely need to repay at some point if your project becomes successful. As soon as you have some users who like what you're doing, you should begin to re-prioritize stability, refactoring and testing. More on this in the next post.

Paying down tech debt can be a slog. But, silver lining: If your project never takes off, then congratulations! You didn't waste any time testing something that no one wanted :)

Lesson for open source maintainers: Write messy code, keep scope small, and skip any unnecessary work until you know that you're building something useful.

Lesson 3: Fix fast

You just received your first bug report. Oh no, someone tried your project out and it broke! But what matters is that they cared enough to tell you about it.

One of the best things that you can do in a new open source project is fix bug reports right as they come in. Prioritizing fixes over everything else becomes much harder as a project grows, so enjoy the freedom to move quickly while you still can.

Sebastian McKenzie (creator of Babel, Yarn, and now Rome) summarizes this theory well:

One of the reasons Babel was successful is how quickly I was able to quickly fix bugs and release new versions. I would regularly have releases out within minutes of a bug report. This was critical during the early days when adoption was low. Being able to unblock users quickly would often make them more excited to use Babel even though they ran into a bug. -- Sebastian McKenzie, Rome

Lesson for open source maintainers: Your first users are essential. Prioritize fixing their issues over everything else.

Lesson 4: Practice good storytelling

Something about marketing always seems to make developers squeamish. Unfortunately, if you want people to use your project you eventually need to tell them about it. Even organic, viral word-of-mouth sensations tend to have a cheerleader acting behind-the-scenes.

To start, just share your project with a friend or colleague and ask them for their thoughts. It's okay if you don't hit the front page of Hacker News on day one, all you're looking for is early feedback and maybe your first users.

When you're ready to share your project with a larger audience, you have a few options for how to do it. One popular choice is to tell your story in small, visual pieces over time. This is how Sebastian built excitement for Rome for almost 2 years before its launch. Mark Dalgleish has also done a great job of promoting vanilla-extract on Twitter this way.

You can also get creative, and do something unique. Ben Holmes has been having a ton of fun recording announcement videos in front of a whiteboard for his new project, Slinkity.

With Snowpack, I decided to tell our story in one big blog post. This took some serious time to write, but gave us the space to really explain the "why" of the project. I figured that we had to make an emotion connection to our bigger goal to change the web. Even though this was just a single post, it made a big splash when it was released and then got re-shared and referenced over the next year.

Lesson for open source maintainers: Promote your work. Find a style of storytelling that fits you and your project.

Lesson 5: Ignore your haters, listen to your users

If your work ever gets posted to Hacker News, expect some haters.

If you're lucky, you can tell the difference between ignorant criticism and constructive criticism. Ignore the ignorant stuff (aka haters) but listen to the constructive feedback if you can. If someone shows that they took the time to at least attempt to understand your project in good faith, their feedback will usually have some value if you can spot.

A common constructive criticism is when someone clearly tried understand your work, but still misunderstood some key part of it. It's easy to call that person dumb and move on, but remember that clear communication is your responsibility. When someone misunderstands your work, it usually means that you could improve your README or documentation or general storytelling in some way.

Lesson for open source maintainers: Haters gonna hate, ignore them. Learn how to spot the good-faith, constructive criticism.

Main Takeaway: Support your early users

If you're starting a new open source project, the best thing that you can do is make sure that your first 10 users are happy. All of the lessons above are really just about finding and supporting these first users in some way. If you do right by them, then you've already built something meaningful.

And if this all sounds like too much work, remember that open source software has no rules. It's supposed to be fun. Building for yourself or a smaller audience is totally fine, in which case you can go ahead and ignore most of this advice.

Introducing: Pika CDN + Deno

Fred K. Schott — Wed, 05 Feb 2020 23:37:00 +0000

Deno isn't just for TypeScript, anymore! Did you know that you can also use Deno to import and run JavaScript packages from npm?

As long as your package is written as ESM (JavaScript's native module syntax) and doesn't rely on any Node-specific behavior/imports, Deno can load and run JavaScript from any URL without breaking a sweat. Pika already hosts a search catalog of 80,000+ ESM packages on npm with more being added every day.

All of this makes the Pika CDN -- our modern CDN for the npm ecosystem -- a great fit for Deno. With the Pika CDN, you can import any npm package from a single, hosted URL. This has always been a good fit for websites, but now it is an even better fit for Deno. That's because Deno has no package manager, so importing packages from URLs is already built in to the Deno workflow.

Introducing "X-TypeScript-Types"

But what about type declarations? Deno is a TypeScript runtime, after all. It would be a shame if you couldn't also fetch the type declarations that exist inside of each npm package (or via the community-generated @types/ project by DefinitelyTyped) whenever you import a JS npm package.

Well, now you can!

In collaboration with the Deno team for their recent v0.32.0 release, Pika CDN now hosts type declarations for every package that provides them! Import an npm package from our CDN and Deno will automatically fetch its type declarations along with it.

With this new feature, your programs get an extra level of stability with automatic warnings from Deno when a package is used incorrectly.

This is all thanks to Deno's new "X-TypeScript-Types" response header support. Here's how it works:

Hosting the type declarations: For Deno to read package type declarations, they have to exist somewhere on our CDN. We do the work at build-time to find the type files for your package and then host them at the /types/* package URL subpath.
Telling Deno where to find them: Each package on the CDN that has valid types responds with a X-TypeScript-Types header. When Deno sees this header from an import, it automatically fetches the types and connects them to the import.

If no type declarations exist on npm, we don't send the header. In that case, Deno will read the JS files and make its best guess at what its interface looks like. We could try to do the same on the CDN, but for now we feel pretty confident that Deno is better at this than we are :)

Exciting Times!

None of this would have been possible without the efforts of @kitsonk and the entire team of Deno contributors.

With these changes, Pika CDN is now THE BEST PLACE to get npm packages that run on Deno. This is something that we're extremely proud of, and a goal that we'll keep improving on as Deno works towards their v1.0 release. Exciting times, indeed!

We can't wait to see what you build with Deno and our CDN. Reach out to us on Twitter with any questions, comments, and links to what you've built.

✌️
- @fks

Tree-Shaking Without a Bundler? Here's How Snowpack Does It!

Fred K. Schott — Thu, 16 Jan 2020 00:10:20 +0000

Until Snowpack came on the scene, application bundling was the only way to do tree-shaking (dead-code elimination) on the web.

But Snowpack is the unbundler. It runs only on your dependencies without ever touching your application code. So how could it tree-shake?

Here's a quick post about how we made this production optimization happen for faster production apps, thanks to some essential help from the RollupJS team 🙌

The Setup

Bundlers "tree-shake" your web application by finding and removing unused code, shaving off valuable bytes when you deploy to production.

Since Snowpack never touches your application code, how could we do the same? We had to get creative.

Snowpack v1.0 added an initial "scan" phase behind the --optimize flag. It runs a quick scan of your app to learn how you use each dependency.

... and that's it! We pass that info to our internal installer (powered by Rollup) and eliminate any dead code at install time.

The Challenge

"Passing that info to our Rollup-powered installer" was an interesting challenge: How do you tell Rollup that code in your top-level entrypoints is unused?

This was an unusual request, and something the Rollup team (rightfully!) didn't want to build into the bundler itself.

We created an issue on the Rollup repo to outline the problem, and with @lukastaegert's help came up with a really clever solution via a new plugin...

Here's how we solved it: We create an in-memory "envelope" wrapper for Rollup of every dep & fill it with the exact imports used by your app.

Rollup sees the envelope, reads the scanned imports to the real dep, & then automatically tree-shakes out everything not found in the ✉️
You can see exactly how it works here, coming in at only ~50 LOC.

This all runs behind the production-only --optimize flag, so you can add & remove imports as you develop your app without worrying about tree-shaking or needing to re-install as your code changes.

But even when optimizing for production, just reading & scanning your application code should be a hell of a lot faster than reading, scanning AND BUNDLING your app code with a traditional bundler (so you can still expect fast production deployments).

What's Next?

We're experimenting with replacing our current import scanner with the even faster es-module-lexer by @guybedford, a WASM-powered scanner written in C.

guybedford / es-module-lexer

Low-overhead lexer dedicated to ES module parsing for fast analysis

ES Module Lexer

A JS module syntax lexer used in es-module-shims.

Outputs the list of exports and locations of import specifiers, including dynamic import and import meta handling.

Supports new syntax features including import attributes and source phase imports.

A very small single JS file (4KiB gzipped) that includes inlined Web Assembly for very fast source analysis of ECMAScript module syntax only.

For an example of the performance, Angular 1 (720KiB) is fully parsed in 5ms, in comparison to the fastest JS parser, Acorn which takes over 100ms.

Comprehensively handles the JS language grammar while remaining small and fast. - ~10ms per MB of JS cold and ~5ms per MB of JS warm, see benchmarks for more info.

Built with Chomp

Usage

npm install es-module-lexer

See types/lexer.d.ts for the type definitions.

For use in CommonJS:

const { init, parse } = require('es-module-lexer');
(async

…

View on GitHub

Thanks for Reading!

Enjoyed this post? Throw Snowpack a 🌟on GH to help spread the word:

FredKSchott / snowpack

ESM-powered frontend build tool. Instant, lightweight, unbundled development. ✌️

Update (April 20, 2022): Snowpack is no longer actively maintained and is not recommended for new projects.

Check out Vite for a well-maintained Snowpack alternative.
See also: esbuild, parcel

Snowpack

Snowpack is a lightning-fast frontend build tool, designed to leverage JavaScript's native module system (known as ESM). It is an alternative to heavier, more complex bundlers like webpack or Parcel in your development workflow.

Key Features

Develop faster, with a dev server that starts up in 50ms or less.
See changes reflected instantly in the browser.
Integrate your favorite bundler for a production-optimized build.
Enjoy out-of-the-box support for TypeScript, JSX, CSS Modules and more.
Connect your favorite tools with third-party plugins.

💁 More info at the official Snowpack website ➞

Contributor Guidelines: CONTRIBUTING.md
License: MIT

View on GitHub

Learn more about Snowpack: https://www.snowpack.dev/
Follow us on Twitter for updates: https://twitter.com/pikapkg
Read & share the original Twitter thread that this was based on: https://twitter.com/pikapkg/status/1217509391668133888

Unrolled: Introducing the Pika REPL

Fred K. Schott — Tue, 06 Aug 2019 16:24:51 +0000

Today we launched the new rewrite of pika.dev, which included a new Package REPL for the package view page. This was just announced on Twitter, but I figured I'd unroll and repost the entire thread here quickly for anyone interested.

You can read our entire release thread including all of todays releases here: https://twitter.com/pikapkg/status/1158765492191911936

What's a Package REPL? Keep reading to find out...

Pika 📦

@pikapkg

✨ INTRODUCING: The Pika Package REPL

Test packages directly in the browser, without installing anything locally. Perfect for quickly evaluating packages before bringing them into your important projects.

Try it today: pika.dev/packages/path-…

16:05 PM - 06 Aug 2019

3 7

Pika 📦

@pikapkg

True story: The second we had this feature working, we ended up needing it to quickly test our own site's path parsing logic.

Instead of debugging the entire app, we could quickly iterate on just the code snippet... RIGHT IN THE BROWSER🤘

16:05 PM - 06 Aug 2019

0 0

Pika 📦

@pikapkg

This is powered internally by ristretto: a really neat, modular testing library from the Polymer team.

github.com/PolymerLabs/ri…

16:05 PM - 06 Aug 2019

0 1

PolymerLabs / ristretto

An extensible test runner ☕️

🚨 PROJECT STATUS: EXPERIMENTAL 🚨This product is in the Experimentation phase. Someone on the team thinks it’s an idea worth exploring, but it may not go any further than this. Use at your own risk.

Ristretto

Ristretto is an extensible test runner.

There are several established test runners in the JavaScript ecosystem (Mocha, Jasmine and Jest to name a few of the more prominent ones). Ristretto was created to address a feature sweet spot left unaddressed by incumbent projects.

Ristretto has the following qualities:

Simple, concise, class-based factorization
Consumable as modules with browser-compatible path specifiers OOB
Batteries included for the most popular testing syntaxes (BDD/TDD)
Designed for extensibility (mixins, specialized reporters and a spec-as-data-structure philosophy)
Ships with mixins that enhance specs with powerful features (e.g., fixtures and test isolation)
Authored in TypeScript
No build tooling required to use
"JavaScript is the native language, The Web is the…

View on GitHub

Pika 📦

@pikapkg

Your code snippet is run as a fake ristretto "test", in a sandboxed iframe, directly in the browser. Open up dev tools and you'll see any console output!

16:05 PM - 06 Aug 2019

0 1

Pika 📦

@pikapkg

This is just the start, and we'll keep adding to this over the next few months! Two additions I can't wait for:
1⃣ Package type definitions & type checking
2⃣ Logging output/errors to the page if you're on mobile

16:15 PM - 06 Aug 2019

0 0

BONUS: Curious about how a package works after using it in the REPL?

Pika 📦

@pikapkg

The SOURCE tab is another incredibly useful, but subtle addition: An @unpkg browser built right in to the page.

View any package source without leaving Pika, or click the link at the top to go directly to unpkg.com.

16:36 PM - 06 Aug 2019

0 0

A Future Without Webpack

Fred K. Schott — Fri, 02 Aug 2019 16:04:37 +0000

Note: This piece was originally published to pika.dev.

The year is 1941. Your name is Richard Hubbell. You work at an experimental New York television studio owned by CBS. You are about to give one of the world's first major TV news broadcasts, with 15 minutes of airtime to fill. What do you do?

In a world that has only known radio, you stick to what you know. That is, you read the news. "Most of the [televised] newscasts featured Hubbell reading a script with only occasional cutaways to a map or still photograph."^[1] It would be a while before anyone would show actual video clips on the TV news.

As a JavaScript developer in 2019, I can relate. We have this new JavaScript module system (ESM) that runs natively on the web. Yet we continue to use bundlers for every single thing that we build. Why?

Over the last several years, JavaScript bundling has morphed from a production-only optimization into a required build step for most web applications. Whether you love this or hate it, it's hard to deny that bundlers have added a ton of new complexity to web development -- a field of development that has always taken pride in its view-source, easy-to-get-started ethos.

@pika/web is an attempt to free web development from the bundler requirement. In 2019, you should use a bundler because you want to, not because you need to.

Credit: @stylishandy

Why We Bundle

JavaScript bundling is a modern take on an old concept. Back in the day (lol ~6 years ago) it was common to minify and concatenate JavaScript files together in production. This would speed up your site and get around HTTP/1.1's 2+ parallel request bottleneck.

How did this nice-to-have optimization become an absolute dev requirement? Well, that's the craziest part: Most web developers never specifically asked for bundling. Instead, we got bundling as a side-effect of something else, something that we wanted realllllllly badly: npm.

npm -- which at the time stood for "Node.js Package Manager" -- was on its way to becoming the largest code registry ever created. Frontend developers wanted in on the action. The only problem was that its Node.js-flavored module system (Common.js or CJS) wouldn't run on the web without bundling. So Browserify, Webpack, and the modern web bundler were all born.

Create React App visualized: 1,300 dependencies to run "Hello World"

Complexity Stockholm Syndrome

Today, it's nearly impossible to build for the web without using a bundler like Webpack. Hopefully, you use something like Create React App (CRA) to get started quickly, but even this will install a complex, 200.9MB node_modules/ directory of 1,300+ different dependencies just to run "Hello World!"

Like Richard Hubbell, we are all so steeped in this world of bundlers that it's easy to miss how things could be different. We have these great, modern ESM dependencies now (almost 50,000 on npm!). What's stopping us from running them directly on the web?

Well, a few things. 😕 It's easy enough to write web-native ESM code yourself, and it is true that some npm packages without dependencies can run directly on the web. Unfortunately, most will still fail to run. This can be due to either legacy dependencies of the package itself or the special way in which npm packages import dependencies by name.

This is why @pika/web was created.

@pika/web: Web Apps Without the Bundler

FredKSchott / snowpack

ESM-powered frontend build tool. Instant, lightweight, unbundled development. ✌️

Update (April 20, 2022): Snowpack is no longer actively maintained and is not recommended for new projects.

Check out Vite for a well-maintained Snowpack alternative.
See also: esbuild, parcel

Snowpack

Key Features

Develop faster, with a dev server that starts up in 50ms or less.
See changes reflected instantly in the browser.
Integrate your favorite bundler for a production-optimized build.
Enjoy out-of-the-box support for TypeScript, JSX, CSS Modules and more.
Connect your favorite tools with third-party plugins.

💁 More info at the official Snowpack website ➞

Contributor Guidelines: CONTRIBUTING.md
License: MIT

View on GitHub

@pika/web installs modern npm dependencies in a way that lets them run natively in the browser, even if they have dependencies themselves. That's it. It's not a build tool and it's not a bundler (in the traditional sense, anyway). @pika/web is a dependency install-time tool that lets you dramatically reduce the need for other tooling and even skip Webpack or Parcel entirely.

npm install && npx @pika/web
✔ @pika/web installed web-native dependencies. [0.41s]

@pika/web checks your package.json manifest for any "dependencies" that export a valid ESM "module" entry point, and then installs them to a local web_modules/ directory. @pika/web works on any ESM package, even ones with ESM & Common.js internal dependencies.

Installed packages run in the browser because @pika/web bundles each package into a single, web-ready ESM .js file. For example: The entire "preact" package is installed to web_modules/preact.js. This takes care of anything bad that the package may be doing internally, while preserving the original package interface.

"Ah ha!" you might say. "That just hides bundling in a different place!"

Exactly! @pika/web leverages bundling internally to output web-native npm dependencies, which was the main reason that many of us started using bundlers in the first place!

With @pika/web all the complexity of the bundler is internalized in a single install-time tool. You never need to touch another line of bundler configuration if you don't want to. But of course, you can continue to use whatever other tools you like: Beef up your dev experience (Babel, TypeScript) or optimize how you ship in production (Webpack, Rollup).

This is the entire point of @pika/web: Bundle because you want to, not because you need to.

"view-source" is back!

Performance

Installing each dependency this way (as a single JS file) gets you one big performance boost over most bundler setups: dependency caching. When you bundle all of your dependencies together into a single large vendor.js file, updating one dependency can force your users to re-download the entire bundle. Instead, with @pika/web, updating a single package won't bust the rest of the user's cache.

@pika/web saves you from this entire class of performance footguns introduced by bundlers. Duplicated code across bundles, slow first page load due to unused/unrelated code, gotchas and bugs across upgrades to Webpack's ecosystem... Entire articles and tools are devoted to solving these issues.

To be clear, leaving your application source unbundled isn't all sunshine and roses, either. Large JavaScript files do compress better over the wire than smaller, more granular files. And while multiple smaller files load just as well over HTTP/2, the browser loses time parsing before then making follow-up requests for imports.

It all comes down to a tradeoff between performance, caching efficiency, and how much complexity you feel comfortable with. And again, this is the entire point of @pika/web: Add a bundler because it makes sense to your situation, not because you have no other choice.

The Pika Web App Strategy

@pika/web has completely changed our approach to web development. Here is the process we used to build pika.dev, and how we recommend you build your next web application in 2019:

For new projects, skip the bundler. Write you application using modern ESM syntax and use @pika/web to install npm dependencies that runs natively on the web. No tooling required.
Add tooling as you go. Add TypeScript if you want a type system, add Babel if you want to use experimental JavaScript features, and add Terser if you want JS minification. After 6+ months, pika.dev is still happily at this phase.
When you feel the need & have the time, experiment by adding a simple bundler for your application source code. Performance test it. Is it faster on first page load? Second page load? If so, ship it!
Keep optimizing your bundler config as your application grows.
When you have enough money, hire a Webpack expert. Congratulations! If you have the resources to hire a Webpack expert you have officially made it.

@pika/pack - Publish great npm packages.

Fred K. Schott — Fri, 02 Aug 2019 16:03:59 +0000

Note: This piece was originally published to pika.dev.

If you've recently published a package to npm, you know how much work goes into a modern build process. Transpile JavaScript, compile TypeScript, convert ES Module syntax (ESM) to Common.js, configure your package.json manifest... and that's just the basics.

What about web browsers and bundlers? Is your package optimized for them? Can users load your package from UNPKG? Does it include type definitions for VSCode & TypeScript users? Has it been stripped of all unnecessary files to be as small as possible? Your users care about these optimizations, but they all require even more knowledge, configuration, tooling, time and effort to get right.

@pika/pack builds amazing packages without the hassle:

Simple ⚡️ Use pre-configured plugins to build your entire package.
Flexible 🏋️‍♀️ Choose plugins and optimizations that match your needs.
Holistic ⚛️ Let us handle your code, assets, and package.json config.

The Package Build Pipeline

@pika/pack connects pre-configured plugins to build and optimize your package for you. Plugins wrap already-popular tools like Babel and Rollup with options already optimized for npm. This lets @pika/pack build your package without much (if any) configuration required on your part.

@pika/pack even creates and configures a package.json manifest for your package, automatically.

This works because @pika/pack builds your entire package: code, assets, and even package.json manifest. By building the entire package, you end up with a fully-built pkg/ directory, ready to publish. Entry points like "main", "module", "umd:main", "types", "unpkg", and even advanced options like "sideEffects" and "files" are all handled by your build plugins.

Getting started is easy:

// 1. Install @pika/pack!
$ npm install -g @pika/pack
// 2. Add this to your package.json manifest:
"@pika/pack": {"pipeline": []},
// 3. Run @pika/pack!
$ pack build

😎 🆒

So now what? If you run pack build with an empty pipeline, you'll get an empty package. That's not very useful.

To get you started, here are our five favorite things to do with @pika/pack:

1. Publish Modern, ES2018 JavaScript By Default (1 Line)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-standard-pkg", {"exclude": ["__tests__/**/*"]}]
    ]
}

The first plugin that most packages will use is @pika/plugin-standard-pkg. This is our standard "source" builder that builds any JavaScript & TypeScript source code to the latest ES2018 language spec. TypeScript is supported by default, and any custom Babel plugins in your local .babelrc will be used automatically (the plugin is powered by Babel internally).

This gives the rest of our pipeline a standard ES2018 target to build from. BONUS: some of your users may support modern JavaScript, and can leverage this ES2018 distribution directly for a smaller, faster, less-bloated package.

2. Publish Node.js & Web-Optimized Builds (1 Line Each)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-standard-pkg"],
        ["@pika/plugin-build-node"],
        ["@pika/plugin-build-web"]
    ]
}

After creating your modern ES2018 build, it becomes trivial to add Node.js- and web-optimized distributions to your package. @pika/plugin-build-node will transpile a Node.js-ready distribution to run on any supported Node.js version. @pika/plugin-build-web will build a more modern, ES module (ESM) distribution: optimized for bundlers and compiled to run on all browsers that support ESM syntax natively.

Both plugins use Rollup internally, but you don't have to configure any bundler logic yourself. With just three lines of JSON you get a package that is fully optimized for the two most popular JS platforms. And because @pika/pack builds your entire package, your package.json manifest is automatically configured for you with "main", "module", and "esnext" entry points.

3. Automatically Generate TypeScript Definitions (1 Line)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-standard-pkg"],
        ["@pika/plugin-build-node"],
        ["@pika/plugin-build-web"],
        ["@pika/plugin-build-types"]
    ]
},

With @pika/pack, every package automatically gets .d.ts TypeScript definition files thanks to the @pika/plugin-build-types plugin. Even if you're writing JavaScript, this plugin will use TypeScript to automatically generate type definitions from your JavaScript & JSDoc!

4. Publish WASM! Or Reason, or Rust, or C++, or... (1-2 Lines)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-wasm-assemblyscript"],
        ["@pika/plugin-wasm-bindings"]
    ]
},

The pipeline concept is simple AND flexible. So flexible that your package source doesn't even need to be JavaScript:

@pika/plugin-source-bucklescript - Compile Reason or OCaml to ES2018 JavaScript via BuckleScript
@pika/plugin-wasm-assemblyscript - Compile TypeScript to WASM via AssemblyScript
@pika/plugin-wasm-emscripten - Compile C/C++ to WASM via Emscripten (Coming Soon!).
@pika/plugin-wasm-bindings - Add simple JavaScript bindings for any generated WASM.

5. Automatically Enhance Your Package (1 Line)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-standard-pkg"],
        ["@pika/plugin-build-node"],
        ["@pika/plugin-simple-bin", {"bin": "my-cli"}]
    ]
}

Build plugins can also enhance existing builds in some really exciting ways. Our favorite enhancement right now is @pika/plugin-simple-bin, which injects a simple CLI wrapper into any package and configures your package.json to point to it automatically.

We use @pika/pack to build @pika/pack, and we use this plugin specifically to add a command-line interface with none of the package.json configuration and setup hassle.

We can't wait to see what's possible with this wrapping concept, especially for CLIs: Automatically restart your program on failure, check for package updates after running, log usage statistics... the possibilities are endless!

Publishing Your Package

Publishing your package should be as easy as building it. So we brought our favorite parts of np (a self-described "better npm publish") into @pika/pack. With the publish command there's no need to worry about how to publish your built package. Just run pack publish in your top-level project and @pika/pack will walk you through cutting a new version and publishing your package.

It even includes a handy unpkg.com at the end so that you can immediately view your new package.

Try @pika/pack Today!

npm install -g @pika/pack

Don't wait, try @pika/pack in your next package (or any old ones that could a 2019 upgrade). And when you do, please let us know how it went!

Pika is a project to move the JavaScript ecosystem forward. Pika's mission is to make modern JavaScript more approachable by making it easier to find, publish, install, and use modern packages on npm. Hopefully @pika/pack moves us one step closer towards that goal.

Projects Already Using @pika/pack

react-async - Flexible promise-based React data loader.
hookuspocus - hooks for all the functions!
foundationjs - Full-featured NodeJS framework.
graphql-rest-proxy - Turn your REST API into GraphQL.
custom-attributes - Define custom attributes the same way you define custom elements.
@pika/pack & @pika/web - yes, we even use @pika/pack to build @pika/pack! 🤯
And hundreds more! Check out our full set of example & starter projects →
Have you used @pika/pack in a package? Let us know and we'll add your package to the list!