Forem: Havoc Pennington

If your open source dependencies are a mess, we’ve got you. Introducing catalogs.

Havoc Pennington — Thu, 09 Jul 2020 18:46:45 +0000

Today we’ve added a new feature we are calling catalogs to the Tidelift Subscription. Catalogs bring managed open source to life by providing a mechanism for customers to create and maintain an organization-wide inventory of open source package releases that just work.

Simultaneously, catalogs provide a mechanism for Tidelift, working with our network of maintainers, to pre-build a data-enriched inventory of known-good, issue free open source packages that feeds into each subscriber’s customized catalogs. Tidelift subscribers receive a feed of data and updates from Tidelift-managed catalogs, helping them keep their own catalogs high-quality and up-to-date.

So that’s the what. Now let us go back to the why.

Over the past few years, we’ve talked to hundreds of organizations about how they manage their open source dependencies. Most of them fall along a spectrum between one of these two extremes.

1. Distributed approach (aka “move fast”)

Developers in your organization bring in new open source components on their own, without many controls. After all, you don’t want to set roadblocks in the way of your developers being able to deploy as quickly as possible.

But as you multiply this by hundreds or thousands of applications, each using a large number of open source dependencies, it creates the potential for a maintenance and security nightmare. You often don’t know which dependencies are being used and how they are (or are not) being secured and maintained, and by whom.

You’ve resisted putting in place too many controls, but the risks are getting higher, and the maintenance headaches are getting worse.

2. Centralized approach (aka “stay safe”)

Your organization can’t tolerate the risk of a maintenance, security, or licensing emergency with an open source dependency. No one wants to be the next Equifax. So you’ve put strict controls in place. Scanning tools flag issues with the components you are using and block builds. Approvals for introducing new dependencies take days, weeks, or even months to weave through the bureaucracy.

The end result: Cranky developers who can’t get much done. Builds blocked at the last minute. A backlog of unresolved issues flagged by scanning tools that no one knows how to fix. Meanwhile, development slows, good developers get discouraged, and no one is happy with the status quo.

It’s become clear: scanning by itself isn’t enough...

We hear from organizations every day that while scanning tools are useful for identifying issues, identification on its own is not enough without a clear way to help resolve those issues.

Scanning tools take one problem with an open source package (say, a security vulnerability or missing license), and create an issue for every application (and every developer), using that package. The result: work proportional to M packages times N applications. Ouch. Moreover, the issues arise late in the development lifecycle.

So we asked ourselves, what might a better approach look like? How can we help organizations solve the issues that their scanners flag, while getting the benefits of a distributed approach (move fast) AND the benefits of a centralized approach (stay safe) at the same time?

Staying safe without sacrificing development speed

The biggest and most well-funded Internet giants have identified the need to move fast and stay safe, and have come up with a solution to do both at once proactively. Here is an article that describes Google’s approach, for example.

These large organizations often create a library of pre-vetted, known-good open source package releases. Developers can use these without fear of late-in-the-game deployment blockers. Vulnerabilities and license concerns can be reviewed once, centrally, and addressed for the entire organization at once.

To work in this way, organizations need to solve several problems:

A way to tackle the sheer amount of review work, especially for initial adoption when there are thousands of packages already in use;
Efficient workflow for developers and reviewers;
Accurate data to power workflow automation and policy compliance.

This approach takes a lot of time and people power—which is why only the richest technology companies have been able to afford it—until now.

Finally, good answers to basic questions about open source

Tidelift catalogs provide a way for any organization to get issue-free open source packages without the expense of vetting them wholly on its own. Instead, the Tidelift Subscription allows you to offload that responsibility to Tidelift and our network of independent maintainers—saving you time and allowing you to focus on building your apps.

With Tidelift catalogs in place, you can now definitively answer questions like these:

"Can I use this package? Just give me a clear yes or no."
"What’s the single source of truth for which packages and versions are OK?"
"Is there a repository of known-good artifacts that everyone can use?"
“Who’s on the hook for maintaining our open source components?”

With the backing of Tidelift and our network of independent open source maintainers, you will have reliable, timely, and often proactive fixes in-hand for the components you rely on.

How catalogs work

Create your first catalog. Import packages from an artifact manager (such as JFrog Artifactory) or your existing applications’ bill of materials. Subscribe your catalog to one or more Tidelift-managed catalogs, backed by our network of open source maintainers, and then add your own customizations.

Define your standards. Define the security, compliance, and legal standards you’d like your open source dependencies to meet, and then achieve those goals—building on the work Tidelift has done for you.

Tidelift keeps your catalog current. Tidelift, working together with our partnered independent open source maintainers, will continuously provide security updates and track maintenance and licensing data, along with recommended fixes we arrive at for our Tidelift-managed catalogs.

Add new packages. Your developers have a streamlined workflow to request new additions to the catalog as needed, and your security, licensing, and technology experts have a streamlined workflow to evaluate each concern only once no matter how many applications are affected.

Create more catalogs. If desired, the Tidelift service allows you to create specialized catalogs for different teams or deployment scenarios, and to share common work across all your catalogs.

Learn more

Watch a free demo
Review the documentation
Talk to one of our open source experts

Cloud providers manage your compute, storage, and network. But who manages your open source libraries? 🤔

Havoc Pennington — Thu, 09 May 2019 14:27:16 +0000

Application dependencies are code. Like all code, this code needs care and feeding.

The packages in between your application and the cloud infrastructure layer—the packages you're loading into the actual address space of your application, that go into the container image that you build—are almost all run exclusively by volunteers, usually a single person.

The dilemma teams face today:

These packages and frameworks in the middle make it much, much faster to ship working apps. Implementing HTTP from scratch isn't the sort of thing your team should be doing! Rapid application development depends on open source code re-use.
The packages aren't always in great shape. 10-20% of them aren't seeing any maintenance activity at all. Many more are under-maintained. Few have adequate security procedures, with a number of high-profile incidents as a result.
If legal asks for the licenses on all the code you're using… have fun, because there's no reliable annotation.
Meanwhile the packages that are maintained can be fast-moving, and there's a need to keep up… or you won't have bugfixes, security fixes, or new features.

When engineering teams want to manage technical, legal, and security risks in these open source libraries they rely on, they have to use homegrown or costly tooling that surfaces lists of problems...and then dedicate engineering time to tackling them. They also have to research and monitor package status (quality, roadmap, licensing, releases) themselves, package-by-package. (Many teams use thousands of packages, by the way.)

Because this is too time-consuming to be practical, in practice, teams don't do it. Inadequately maintained-and-monitored dependencies create risks, fire drills, sub-par quality, and engineering rabbit holes. Teams often come to view dependencies as a liability to be minimized, rather than taking full advantage of open source code re-use.

At many companies, there's a huge gap between stated open source policy (diligent research, monitoring, approval, and maintenance) and reality (everyone ignores the policy).

Could a managed open source approach save you money?

What if you could have a managed open source strategy for taking good care of your open source dependencies? What could that mean exactly?

The best analogy might be to go back 15 years ago or so to the world before cloud computing. In those days, if you were trying to launch a new SaaS app, you’d need to rent space from a reputable hosting facility, buy and install servers to ensure your app has appropriate backup / failover, and configure all of the software you need on those servers. Then, if something went wrong with a server, you’d have to drive or fly to the hosting facility, swap it out, install software updates, etc.

Today, you simply tap a couple of buttons or run a script, and your favorite cloud provider manages all of that for you.

But when it comes to all of the open source components your app relies on, development teams today still need to do a lot of the heavy lifting themselves.

We think a similar managed approach—managed open source—might allow you to get back the time you are currently spending wrangling open source dependencies, which frees you and your team up to focus on the truly differentiating features of your app.

We've built one approach to this at Tidelift. In brief, we keep track of the recommended versions (and recommended packages) you should be on, and give you the tools and guidance to stay on those—with the expert help of the open source maintainers who created your packages.

Without well-managed open source dependencies, what could go wrong?

As always, xkcd explains the problem well. We can't say when spottily maintained dependencies will bite you, but we know they will.

Cartoon courtesy of xkcd

The scariest risks involve security breaches and lawsuits. The most inevitable consequence of poorly maintained software, though, is wasted time. The saying goes that "open source is free if your time is worth nothing."

This is why you purchase managed cloud services for your compute, storage, and database, rather than owning and operating them yourself. Application dependencies are no different. They need care and feeding, and if you're doing it yourself, you're probably getting an inferior result at higher cost. Home brew dependency stacks are undifferentiated heavy lifting. They don't create business value.

According to our surveys, developers spend about 30% of their time on maintenance and 25% of that is open-source related. So let's say 8% of developer time is open-source related maintenance. That's a lot of time to save and you can calculate the developer salary you're spending on it using our ROI calculator.

But the real costs go even beyond developer salary; two in particular:

opportunity cost: what core business capability could your developers be working on instead, and what's the value of that?
risks: legal, technical, and security-related icebergs that arise from insufficient maintenance of your dependencies; the most common risks are technical and could cost a few months of time; bad legal and security outcomes are less common, but can be mind-meltingly expensive.

This is a significant unsolved problem for the software industry. How are you thinking about it in your organization? I'd love to hear from you in the comments.

Up to 20% of your application dependencies may be unmaintained

Havoc Pennington — Wed, 10 Apr 2019 20:39:01 +0000

We recently added a new feature Tidelift subscribers can use to discover unmaintained dependencies. Repurposing this feature to gather some general-interest numbers, it appears that about 10-20% of commonly-in-use OSS packages aren't actively maintained... meaning not even one commit in the past year, and most issues and PRs filed in the last year are still open.

This number surprised us! We expected a lot of packages to be under-maintained, but we didn't expect so many with no activity at all.

We feel this is a systemic problem. Individual maintainers are not at fault.

We also feel it's a real problem that deserves to be addressed.

Why does maintenance matter, anyway?

For this blog post, we looked first at a sample JavaScript codebase (one of our own repositories that powers Tidelift).

When we deploy an application to production, it'll have code we wrote ourselves and a bunch of code from open source dependencies. In our sample repo, we wrote about 15% of the total lines of code, and open source developers wrote about 85%. My guess is that is a fairly common ratio.

In most organizations, nobody really knows what's in the 85% (especially in the aggregate across many repos), and nobody knows whether it's maintained, or who's maintaining it, or to what standard.

Will anyone handle security vulnerabilities? Is a package at risk of handoff to a malicious actor? Does the project have sound legal practices? Will the project keep up with an evolving ecosystem, or stick you in dependency hell when it can't be upgraded along with everything else? Will the project see any useful new features or track new specifications relevant to its function?

We know it's important to maintain our 15%, but for the 85% we often don't even track what it is, let alone have a plan to take care of it. In part this is because there has been no clear answer for how to take care of it; what would that plan be?

Teams may rationalize the problem to themselves: "well, we don't really know how to keep these dependencies maintained, but everyone else is using them, so it's probably fine." This is absolutely incorrect.

Our new feature to find unmaintained dependencies provides more evidence; we ran it on the dependencies of our own sample repo, and we have a lot to improve.

How we define "unmaintained"

For a first cut, we defined unmaintained very narrowly:

The project has no commits in the last year
If there were any issues or PRs in the last year, less than half of them were closed

Using this definition, 259 of 1285 (20.15%) dependencies of our sample repository are unmaintained.

Some of these projects are explicitly annotated as unmaintained in their README, but it's more common for them to be "ghosted" (activity ceased at some point but they haven't been marked unmaintained).

Note that we're talking about widely-used packages in a pretty normal app. In fact, "Hello, World" from any of the major single-page application frameworks looks similar to our sample repo, on this metric.

If we looked at the percent of all packages (that is: including unpopular ones), it's a fair guess that the number of unmaintained would be higher than 20%. Not to be alarmist, but 20% may be a best-case scenario—it's the number if you only use currently-popular stuff. Once you start to add packages from outside the "Hello, World" stack, or once time passes and you're on an older generation of packages… it presumably gets worse.

So far, we only flag packages unmaintained when they have a known-to-us GitHub repository (because we're using GitHub's API to collect metrics). If a package isn't on GitHub or we can't determine its repository, we assume it's maintained. This could be another source of conservatism in the numbers, especially for older ecosystems like Maven.

I know what you're going to say!

Whenever I see a thread about unmaintained packages or OSS metrics on the internet, people want to argue with the details… I sympathize, but many of these arguments are rationalizations that don't hold up. Here are three common responses to hearing about unmaintained packages:

The apparently-unmaintained packages are actually "done" so don't need maintenance
This is only a problem for npm/JavaScript
The automated metrics might be missing some key detail or corner case

All three of these are truthy: there's a little bit of reality to them, that is, they move the numbers a little, but they don't change the order-of-magnitude of the situation.

These packages aren't all done

Here's the thing. There's a big difference between "feature frozen" and "unmaintained." Even if a package is feature frozen, there's baseline work to be responsive if security issues are discovered, move to new versions of the ecosystem, or at least document that the project is feature frozen!

Consider some very different scenarios:

The maintainer isn't responsive anymore—they moved on and never committed anything else, while issues are often ignored.
The maintainer documented in the README that the package is unmaintained or deprecated, and suggests switching to something else.
The maintainer documented "no new features will be accepted, but please let me know if there's a critical problem or a need for a new release and I'll take care of it."

Among the packages we're labeling unmaintained, scenario 1 is the most common by far, scenario 2 happens some, and scenario 3 is the least common—in fact it's hard to find examples. Scenario 3 is the one I'd call "done but still maintained."

Sindre Sorhus maintains over 1100 npm packages and supports 100+ of those on Tidelift; he had this to say:

Q: One comment we've heard frequently is that many JavaScript projects don't need a maintainer because they're "done." Do you have any thoughts on that philosophy?

A: That is only really true for small focused packages. Since they have a narrow scope, they can be done feature-wise. However, this argument misses all the other maintenance tasks required for a package; keeping it up to date with deprecated Node.js APIs, new better Node.js APIs, new syntax (which can improve readability and also sometimes be faster), dependency updates (especially important for reducing vulnerabilities in the dependency-tree), bug fixing, documentation improvements, issue triaging, adding TypeScript type definition, etc. In packages with a larger scope, you also need to handle feature requests and refining existing features. The work required for maintenance is often undervalued. I spend a lot of time on these tasks every day.

These packages aren't all trivial

We happen to count how many bytes of .js files each package contains—here are some numbers on that. For our sample repo, the unmaintained packages are smaller than the maintained ones, on average (11K lines of code vs. 40K). This means that though 20% of packages are unmaintained, the 20% represents less than 20% of the total code.

However, a lot of these unmaintained packages are big. I really want to link to specific projects but I decided against it because it's not the fault of these maintainers that the packages are languishing, and it seems harsh and a bit beside the point to put specific projects on the spot.

Without naming names, we are talking about parsers and other substantive libraries. Yes, there are also some short just-a-few-lines packages. That by no means changes the overall picture. To give you a sense of the spread, of 259 unmaintained packages, 91 contain under 1K in .js files, while 75 contain over 4K in .js files.

Keep in mind, for some of the risks of under-maintenance (like the event-stream-style risk or the need to keep up with the ecosystem), neither "done" nor "small size" necessarily matters. What matters is that someone's still around keeping an eye on things.

This isn't unique to the JavaScript ecosystem

Hearing these maintenance stats, some people respond with "lol JavaScript." Unless you've actually researched the dependencies you use (and recently, because maintainers ghost all the time), I'd recommend that you don't get too confident. Non-maintenance and under-maintenance are common across all of open source.

JavaScript does seem to have more fine-grained code reuse and more dispersed maintenance; that has both costs and benefits. Among other things, it makes it easier to analyze lack of maintenance (our approach can't detect unmaintained subdirectories of large projects, only unmaintained entire repositories).

Here are some of the things we've found so far:

"Hello World" for React, Angular, and Vue all have the same roughly-20%-unmaintained number we're seeing in our sample Tidelift codebase.
In a commonly-used-pypi-dependencies sample repository we created a while ago, it's 13% unmaintained packages.
a "Hello, World" Spring app seems to be about 8% unmaintained packages … but we only tag things unmaintained if we have metrics from GitHub, and some Maven packages lack metrics, so this number is low.
a "Hello, World" Rails 5.1 app has 8 of 70 unmaintained by the criteria we're using, while 5.2 has 10 of 79, so that's 11-12%.

In general, it appears we can identify more like 10% unmaintained in the non-JS ecosystems, vs. the 20% found in JS apps. A detail is that essentially all of the 10% are larger, substantive packages because these ecosystems don't have the tiny packages that are common on npm.

Also keep in mind that "Hello, World" JS apps are quite batteries-included, while some frameworks have no dependencies at all, by default. Real-world applications will pull in more dependencies (they won't stick to only the initial framework) so they may become more similar to the batteries-included JS apps.

Please don't read too much into the exact numbers above; we aren't pretending this is a scientific comparison. The point is that all ecosystems have maintainers who disappear and none of us should feel this is limited to JavaScript.

The packages are actually abandoned—it isn't a metrics artifact

We kept our definition of "unmaintained" simple and conservative, and still found 20% unmaintained in this sample JavaScript project.

We wondered about corner cases like "well, maybe the commits are on a weird branch," or "maybe none of the issues matter," so we went ahead and looked through these repositories manually. (In the Tidelift subscriber dashboard, there's a handy link to the repo for each unmaintained package we identify that makes it easier to do this.)

Short answer: the automated criteria are basically right. For the most part these repositories haven't been touched in ages. Most have been inactive for a lot longer than a year, and they tend to have numerous ignored issues.

We haven't tried to identify under-maintenance, yet, only full-on abandonment. People have proposed a lot of complex metrics for OSS projects. Since it looks like we have a lot of work cut out for us just to address completely inactive projects, there’s no need to get fancy right away.

20% is a potentially-conservative figure in two ways:

If there's even a single commit in the past year, OR most issues were closed in the past year, we mark the package maintained.
We're looking at the packages in a mainstream "Hello, World," not the entire universe of packages that exist.

For example, I have a seriously-under-maintained project out there with my own name on it. This package doesn't meet our initial criteria for unmaintained because I merge a PR occasionally, but it is 95% unmaintained—I don't have time. Issues and PRs are piling up and plenty of them deserve attention.

It's difficult to accurately flag projects like mine without pulling in more false positives or debatable judgment calls. The bad news from a state-of-open-source standpoint is that a conservative screen still flags a lot of packages and it's usually right to do so.

How can we improve this situation?

We believe the solution includes paying maintainers for value provided. Ghosted and under-maintained packages exist because people lack time and incentive to keep them maintained. It's hard to find a new maintainer for an abandoned package, and it's hard for users to port away from an abandoned package. The right solution is to enable maintainers to stick around, and give maintainers a reason to adopt orphaned packages.

Here's how we're trying to improve the open source maintenance problem at Tidelift:

We offer a monthly income to the maintainers of any package used by application development teams who purchase the Tidelift Subscription. The more Tidelift’s subscribers use a particular package, the more the maintainers of that package get paid.
Subscribers get a breakdown of packages they use which appear to be unmaintained, as well as a view into follow-on effects of under-maintenance, such as security and licensing issues.
For direct dependencies, we help subscribers find alternatives; when there's no alternative, there's also that monthly income we offer if the maintainer reappears—or if an alternative emerges.
For transitive dependencies (dependencies-of-dependencies), application development teams often can't do much themselves. So when we show subscribers security, maintenance, and licensing problems pulled in indirectly, we also show the same problems to our network of participating maintainers, and we ask those maintainers to fix problems their package pulls in. We're pushing problems down the stack to get the root cause addressed.
In addition to reporting historical problems, the fact that we're paying the maintainers of the stuff you use should cut down on the future "ghost rate" for your packages.

Every subscriber has a certain amount of "special snowflake," especially over time—quirky packages they're using, or older releases, or the like. By tracking their exact dependencies with Tidelift, each subscriber knows where they stand and that they're covered—because with Tidelift they are paying the maintainers to keep them covered.

A real solution to this problem needs to be comprehensive:

Tools so everyone can see where the problems are, and locate root causes.
Incentives so that we're paying maintainers to stick around and address problems, rather than generating a bunch of new requests for their unpaid labor.

This isn't hypothetical. We're paying maintainers today and helping subscribers improve their dependencies today.

Let us prove it

Curious which unmaintained packages you're using? Pick a sample repository (something you actually maintain and care about, ideally), send us the package manager files, and we'll email you a report with the 10 most critical unmaintained packages you're relying on.

Submit your files here.

Open source has a working-for-free problem

Havoc Pennington — Fri, 08 Mar 2019 13:48:39 +0000

It's a necessary part of open source that we do some work for free. But when it is an expectation—or at least a strong norm—to do everything for free, we have a problem.

Look around. We do have a problem, and it’s time we do something about it.

What developers can learn from designers about spec work

In the design world, there's a strong norm against spec work—see if this quote from nospec.com rings true about open source:

Why is spec work unethical?

The designers work for free and with an often falsely advertised, overinflated promise for future employment; or are given other insufficient forms of compensation. Usually these glorified prizes or “carrots” appear tantalising for designers who are just starting out, accompanied by statements such as “good for your portfolio” or “gain recognition and exposure.”

AIGA, the professional organization for design, also takes a position against spec work.

Unpaid work is one popular, sometimes-effective path to becoming a top-tier well-known developer. The lack of compensation isn't just bad for individual developers—it also creates social problems, by amplifying existing privilege. The issues with free work are well-described in this article on unpaid internships:

The broader effects of unpaid internships are (a) a tendency for employers to take advantage of young labor by offering the currency of experience in lieu of actual currency, and (b) a widening of the social inequality gap as lower-income students are implicitly barred from this so-called "educational" experience, which is their gateway to full-employment in the field of their choosing.

We can start to mitigate this impact with wonderful paid internship programs such as Outreachy, but there's a much larger structural problem, too.

Let's abandon the notion that open source is exclusively charity

In the software industry, we're normalizing spec work in a way that the design industry successfully rallied against.

The narrative around open source is that it's completely OK—even an expectation—that we're all doing this for fun and exposure; and that giant companies should get huge publicity credit for throwing peanuts-to-them donations at a small subset of open source projects.

There's nothing wrong with doing stuff for fun and exposure, or making donations, as an option. It becomes a problem when the free work is expected and the donations are seen as enough.

It actually seems to blow people's minds when open source maintainers ask for money for value provided. As Kyle Mitchell puts it, the message to open source maintainers appears to be "profit for us, sustainability for you."

A common pattern: one or a few unpaid maintainers, buried in pull requests and issues submitted by paid professionals using the software at work.

The fact is that open source isn't charity, most of the time. It's been a critical part of huge, profitable businesses for two decades. The vast majority of open source usage and work happens in a commercial context.

The charity framing truly goes off the rails when a company does actually pay for value… and nobody doing the work on the project actually gets that money, for their own personal use. Instead, the money goes into a nonprofit fund for scholarships or similar. Again, there's nothing wrong with the charity funding pool existing, but it isn't solving the same problems that actually compensating professionals for professional work can solve.

What would open source be like if we had a professional class of independent maintainers, constantly improving the code we all rely on?

Charity works best for "brand name" projects

A "Hello, World" single-page JavaScript application contains around a thousand packages… our preliminary research shows about a quarter of those have been completely ghosted for a year or more (no releases, no commits, PRs/issues ignored). On top of that quarter, many more are under-maintained, though not completely ghosted. And this isn't a JavaScript-only problem.

The well-known headline packages in this kind of app might be maintained by a big company, or have a sizable Patreon…but the other 990 packages aren't.

It is OK to confidently say no to free grunt work

In the past, exclusively pay-to-play open source projects haven't done very well; they end up forked or ignored, because requiring people to pay to participate at all makes it too hard to gain adoption and create a thriving community. There's a delicate balance to be found.

While keeping that balance in mind, there's a line where some work simply doesn't need to happen for free. For example, Mikeal Rogers suggests that we stop supporting old releases for free. Here are some other examples to consider:

Support requests. Bug reports are one thing, but there's no reason maintainers need to hand-hold anyone for free. Create a forum or use StackOverflow for people to support each other; create and advertise paid support options; if you don't want to do it for free, don't!
Extra releases. I commonly get a request like "can you make a special one-off release now on my timeline, because I want this one change;" I don't feel obligated to do that.
Extreme stability. If I'm working for fun and only thinking about community-engaged users, I'm going to break things or clean up deprecated stuff a lot more often than if I'm thinking about commercial users.
Legal checkboxes. There are license-metadata-annotation practices that are helpful for big companies trying to audit the code they use, but sort of a pain in the ass and nobody cares other than these big companies.
Extra-thorough engineering. Extremely good test coverage, security audits and hardening, whatever isn't fun for you. There's no reason to go beyond what you enjoy.
Prompt patch review and issue triage. This one is tricky, but experienced maintainers know that every pull request is a demand on their time. There's no reason you have to run a community development model open to all comers, just because that's the GitHub default; some projects are explicitly opting out of this, because the maintainer(s) want to focus on writing code themselves.
Extra mile on security. There's a pretty good learning curve on an ideal security process; we should all be using two-factor authentication everywhere; we ought to sign our packages. We may feel guilty about these things, but as a whole open source maintainers haven't reliably done them.

Every maintainer and project will want to draw this line a little differently. This is an area of active exploration and negotiation: how can communities thrive, without doing everything for free—and in particular skipping enterprise-oriented or commercial-oriented boring grunt work, when those big commercial users aren't paying for value?

Right now many users expect, and demand, that all of this will be free. As an industry, perhaps we should push back harder on that expectation. It's OK to set some boundaries.

No projects actually do all the grunt work that their users might want, by the way. But the boundaries aren't often spelled out explicitly, which leads to frustration on all sides.

One solution: charge more!

Patrick McKenzie has a catch phrase "Charge More!", which he proposes as the solution to many problems. It can help a lot with burnout.

Developers often feel guilty about asking for money, for reasons like:

they actually enjoy the work they're doing
it doesn't take that much time to do any one thing
what if someone yells at me?

Look, the businesses using your software don't consider these factors! They don't even use "cost plus" pricing—that is, they don't ask "how much does this software cost to make?" and charge that plus 10%. Instead they ask "how much is this software worth to the buyer," and they charge that. This is called value-based pricing.

Open source maintainers deserve to use value-based pricing. YOU deserve to use it. Your users are all charging their customers on value, not on cost. And many of them are also willing to pay for value, regardless of your costs.

Charging for value reduces entitlement

At Tidelift, we've had a few maintainers ask us whether companies paying for the Tidelift Subscription feel entitled to demand too much from maintainers. It's counterintuitive, but our experience is that the opposite is true; many people running a software business have commented that the less a customer pays, the more support-intensive they are. Free users are the most difficult, not the least.

For countless amusing examples of this if-I-pay-nothing-I'm-entitled-to-everything phenomenon, I recommend the ChoosingBeggars subreddit. Of course the open source maintainers among us have likely experienced some less-amusing examples.

Even if we do a lot of free work, it can be helpful to have a "list price," to set expectations. (Many maintainers could regularly and honestly say "the pull request review I just did for you would cost $5000 if I charged market rates for my time." Few of us behave as if this were true, even though it is.)

Don't forget, when you charge for value, you're giving people value

Lots of companies would love to have that extra mile of grunt work that we can't reliably get done for free. And while it's difficult to measure, imagine a couple hundred full-time, independent maintainers working on your thousand dependencies… anytime you're frustrated by missing docs, bugs, missing features, dependency hell, and the like, think about the value of paying the maintainers.

From the perspective of companies who can pay, it's often better for them if they have to pay. In general, companies are unable to make optional payments. They may also be unable to make a payment without a salesperson to walk the purchase through their own internal processes. Making it an optional “tip jar” kind of request is a disservice, because companies struggle to engage on those terms.

Of course, there are some practical barriers

If you change your project README to say "hey, I'm not doing grunt work categories A, B, and C for free," you'll probably find it's not so simple.

You'll get howls of protest from the choosing beggars of the world
Some of those protests will be from highly paid software developers, who are maybe even compensated to work on open source in some way, but who simultaneously believe that independent maintainers shouldn't be paid because open source should be charity
Your project may be too small (one of a thousand dependencies) for companies to engage with and get through their purchasing department
You'll have to set up a minimum viable business, dealing with sales, payments, accounting, and taxes

In many open source contexts, it's important to minimize friction. A developer using a project might have a PR or issue to submit, or another small request for a maintainer's time; they might be willing to pay $300 for that, in theory, but in practice there's no good way to get a one-time $300 check out of their employer. Spending money is a whole process with high overhead at most big companies. It won't make sense to go through this process over and over for every micro-interaction with an open source project.

This means that if someone files an issue or asks for something on chat, and you say "I can't do that for free," as a maintainer you've more or less told them to pound sand; there's nothing they can do. On my projects, I find this makes me reluctant to take a firm stance.

This is a worthwhile problem for the open source world to unpack

So to recap, some issues with unpaid work:

Social equity—right now, open source makes tech's diversity problem worse; we are part of the problem
Lack of maintenance, with numerous projects we all rely on ghosted and under-maintained
Entitlement: when we don't put a value on our time, people assume its value is low
Maintainers are passionate about working on the projects they know best, and just need the income to give them the time to do it!

Questions I'd love to see us all wrestle with more:

Which categories of work need to be unpaid for open source communities to thrive?
Which other categories of work need to be paid, for open source projects to have independent maintainers with reliable incomes?
How can projects make expectations clear to users and give companies a route to pay for value received?
How can the most popular, headline projects help address this for the rest of the dependency graph?

The potential benefit, restated: a more diverse, more appreciated, professional class of independent maintainers, constantly improving the code we all rely on.

Of course this relates to what we do at Tidelift—the company came out of discussions about this problem, among others. Our broad mission is to make open source work better, for everyone. In our day-to-day right now we're specifically striving to give subscribers a way to pay maintainers of their application dependencies for additional value, through the Tidelift Subscription.

But we hope to see many more efforts and discussions in this area. We believe there are unexplored avenues. Social media back-and-forth about new licenses and open source business models tends to focus on headline projects and VC-funded companies, rather than smaller independent projects and libraries. But in between a virtual tip jar and $100 million in funding, there's a vast solution space to explore. We want to be part of figuring this out, along with all of you.