Forem: Andreas Sommarström

How to use a secure private NuGet source in Visual Studio or JetBrains Rider

Andreas Sommarström — Wed, 16 Feb 2022 19:24:16 +0000

A how to guide on using a secure and private NuGet package source for your .NET dependencies in Visual Studio and JetBrains Rider.

.NET, the Microsoft supported open source framework, is celebrating 20 years! And wow, has there been a lot of changes in .NET and software development in general in those 20 years.

Where before every piece of code and functionality needed to be produced in-house, there are now millions of available packages in central repositories for users to consume — with obvious efficiency advantages.

NuGet, the package manager for .NET, allow developers to easily share and consume reusable packages dependencies for their C#, F# and Visual Basic .NET applications. With nuget.org providing easy access to over 4 millions versions, both from Microsoft and open source developers.

But using more and more open source components also puts advanced requirements on keeping control over the code used. With security attacks targeting the open source software supply chain increased by 650% in 2021 alone it is more important than ever for organizations to protect the software they build — and every developer environment, CI/CD system and server.

Need a private NuGet feed for both internal .NET packages and public dependencies? Bytesafe feeds are cloud hosted, and compatible with Visual Studio, JetBrains Rider and the NuGet CLI. You can get started instantly and build your projects securely with the tools of your choice.

Four reasons to use a private NuGet feed

A private NuGet source is a necessity for sharing internal packages and code in many organizations. But a private source also allows for control and help keep unwanted dependencies out.

Secure source for open source dependencies: Don’t allow free entry for untrusted code from public sources. Include approved dependencies, according to your rules.
Share internal packages: Authorized and personal access to your organization’s private packages.
Cache/proxy public packages: Don’t depend directly on public repositories like nuget.org. Make sure your organization’s packages are always available when you need them.
Enforce security policies: Scan for vulnerabilities and automatically block access to unwanted and untrusted dependencies.

Getting started with a private NuGet feed

Using a private NuGet feed instead of the default package source is easy. With some simple config you can have your IDE’s like Visual Studio and JetBrains Rider fetch dependencies from a private feed in place of nuget.org.

On top of IDE support the nuget package management tool has full support for private feeds, both as a target when deploying packages or as a package source for dependencies.

These steps assume users have access to a Bytesafe workspace. If not — Sign up for Bytesafe today for free.

Create a NuGet feed

To get started you need to create a NuGet registry (the Bytesafe equivalent of a feed or repository) and configure access to it in your client of choice.

After you have created your NuGet registry, you need to add a package source to your configuration.

Create an access token in Bytesafe and add it together with the registry URL to your list of approved package sources. The access token ensures only intended users have access to packages stored in Bytesafe.

Visual Studio & JetBrains Rider users can do this directly in the NuGet package tool in their IDE (see sections on IDE integrations for more details). CLI users can alternatively add the package source using nuget.



# Add the URL, username and access token (password) to your nuget sources
$ nuget sources add -Name {REGISTRY} -Source https://{WORKSPACE}.bytesafe.dev/nuget/{REGISTRY}/index.json -Username bytesafe -Password {TOKEN}

Bytesafe provides contextual and copy-paste ready instructions on how to access your private NuGet feed.

The package source information will be added to the NuGet.Config file (used by both nuget and IDE’s). For more information on sources and the NuGet config file, see NuGet in the Bytesafe documentation.

Publish a NuGet package

NuGet packages can be added to your private NuGet feed using nuget push or by uploading the package files manually.



# To publish packages using nuget set an apikey for the source
$ nuget setapikey {TOKEN} -Source {REGISTRY}
…

With your source configured you can publish packages to your private feed for other internal developers or CI/CD to access.



# Create a nuget package according to project files
$ nuget pack
…
# Publish package to registry using nuget. Replace {REGISTRY} with source name
$ nuget push {PACKAGE} -Source {REGISTRY}
…

Restoring NuGet project dependencies

With the public NuGet Gallery (nuget.org) configured as an upstream, Bytesafe will proxy public dependencies and pull any required (and allowed) version into your private NuGet feed.

To make sure security features are not bypassed it’s recommended to disable nuget.org as a package source in the NuGet.Config (the package manager fetches packages from all enabled sources).



# Disable nuget.org as a package source
$ nuget sources disable -Name nuget.org
…

There are multiple ways to specify project dependencies. I prefer package references (<PackageReference>) in the project file (.csproj).



<! — Example package reference in .csproj file -->
<ItemGroup>
   <! — … -->
   <PackageReference Include=”Newtonsoft.Json” Version=”13.0.1" />
   <! — … -->
</ItemGroup>

With project dependencies added to the project, run the nuget restore command to restore project dependencies.



# Restore package dependencies from Bytesafe
$ nuget restore -Source {REGISTRY}

Most IDE’s restore project dependencies by default on project startup or when detecting changes.

Using Visual Studio with your private NuGet feed

Visual Studio is an integral part of the .NET ecosystem and the default IDE for many .NET developers.

Private NuGet registries can easily be integrated as a package source in Visual Studio. Pre-existing NuGet.Config files will be identified by Visual Studio and used to configure package sources for NuGet Package Manager.

Adding a source manually inside Visual Studio

Add the Name, URL and credentials for the registry as the source in the Visual Studio configuration. Access Package Sources in the options (Windows: NuGet Package Manager > Package Sources / Mac: NuGet > Sources).

With the source added, packages are automatically able to be restored and updated in Visual Studio using your Bytesafe NuGet registry.

Any packages available in the private feed will also be available to browse and search in Visual Studio.

Visual Studio Code and some distribution of Visual Studio manage private sources using the nuget cli.

Using JetBrains Rider with your private NuGet feed

JetBrains Rider is the main alternative to Visual Studio for many teams. Like with Visual Studio, private NuGet feeds are easily integrated as a package source for JetBrains Rider.

It’s recommended for users to add Bytesafe as a new package source directly in JetBrains Rider to avoid conflicts. Some distributions lack support for encrypted passwords from NuGet.Config.

Adding a source manually inside JetBrains Rider

Add the Name, URL and credentials for the new feed in the NuGet Sources configuration.

Access the NuGet Tool window from the bottom toolbar or by right-clicking any project dependency and select Manage NuGet Packages.

With the new source added, packages are able to be browsed, restored and updated in Rider using your private NuGet server.

Want to know more about Bytesafe private NuGet feeds?

Visit Bytesafe for NuGet /.NET to learn more.

Want to know more about secure supply chains — read more about our firewall for dependencies.

Want to try Bytesafe? Sign up and get started today for free.

Time for secure dependencies? Private Maven repository for Java, Kotlin, Scala

Andreas Sommarström — Wed, 05 Jan 2022 17:59:12 +0000

Maven is a build tool to manage Java, Kotlin, Scala or similar JVM based languages projects - including all its dependencies, both open source and private. A great way to make sure your organization uses trusted and approved dependencies is to use a private Maven repository - and it's no more complicated than using the default repository.

The two main reasons to use private Maven repositories for your JVM packages:

Secure source for open source dependencies. With 1300+ public repositories and over 24 million Java artifacts, organizations need to control the code they are using - and not allow free entry of untrusted components.
Distribute internal components. With a wide range of applications that depend on each other, organizations require private repositories to share code between services while keeping artifacts private and secure.

Protect your software supply chain and make sure every developer and CI/CD system uses trusted dependencies from a single secure source. With hundreds of automated CI/CD pipelines and decentralized developers this is not an easy task - but definitely achievable with the right tools.

With the recent disclosures of critical vulnerabilities in the widely used open source framework log4j, this is more relevant than ever.

The exploits in log4j have once again stressed the need to keep track of open source dependencies - else risk being exploited by vulnerabilities that could give full server control.

This post describes how you easily set up a secure and private Maven repository - so you can stay in control of the dependencies you are using.

Require a private Maven repository of your own? Bytesafe repositories are compatible with both Maven and Gradle - hosted so you can get started instantly without having to think of infrastructure.

Getting started with a secure Maven repository

With minimal configuration your build clients can fetch trusted dependencies from your private repository instead of relying on (the default) Maven Central.

The popular build tools Maven & Gradle fully support private repositories, both as the target when deploying packages or as the source when fetching dependencies.

Create a Maven repository

To get started you need to create a Maven repository and configure access to it for your client of choice.

After you have created your Maven repository, you need to add an access token to your project. The access token ensures only intended users have access to packages stored in the private registry.

Create an access token and add it to your list of repositories in your configuration file for Maven (settings.xml).



<!-- Add the access token and id configuration to settings.xml -->
<servers> 
    <server>
        <id>{MAVEN-REGISTRY}</id>
        <username>{USER}</username>
        <password>{ACCESS-TOKEN}</password>
        <configuration>
            <httpConfiguration>
                <all>
                    <usePreemptive>true</usePreemptive>
                </all>
            </httpConfiguration>
        </configuration>
    </server>
</servers>

The id in settings.xml needs to match other configuration provided for uploading packages and installing dependencies.

Upload a Maven package

Maven artifacts can be added to the private repository using mvn deploy (or by uploading files manually).

To deploy using the mvn client users need to add the necessary configuration to the project pom.xml file.



<!-- Add the repository and snapshotRepository configuration to pom.xml -->
<distributionManagement>
    <repository>
        <id>{MAVEN-REGISTRY}</id>
        <url>https://{WORKSPACE}.bytesafe.dev/maven/{REGISTRY}/</url>
    </repository>
    <snapshotRepository>
        <id>{MAVEN-REGISTRY}</id
        <url>https://{WORKSPACE}.bytesafe.dev/maven/{REGISTRY}/</url>
    </snapshotRepository>
</distributionManagement>

Users can choose to deploy snapshots and release versions to either the same or different registries.

With your Maven project configured you can deploy artifacts to your private registry either manually or as part of your CI/CD chain.



# Deploy artifacts to Bytesafe using Maven
$ mvn clean deploy

As soon as the process has completed, the artifact will be available and accessible.

Resolving Maven project dependencies

A major part of building a Maven project is resolving any dependencies specified for the project. Any <dependency> specified in the pom.xml file, will be fetched when building the project.

When using private repositories from Bytesafe, public dependencies will be proxied, pulling any required (and allowed) version into your private Maven repository. Using public repositories like Maven Central as an upstream makes sure you can access your organization's required open source dependencies - while maintaining security and control.

To make sure security features are not bypassed for individual projects, it's recommended to use a mirror configuration to set the private Maven repository as the general package source.



<!-- Mirror configuration in settings.xml -->
<mirrors>
    <mirror>
        <id>{MAVEN-REGISTRY}</id>
        <mirrorOf>*</mirrorOf>
        <url>https://{WORKSPACE}.bytesafe.dev/maven/{REGISTRY}/</url>
    </mirror>
</mirrors>

With project dependencies specified in the pom.xml file, you can install dependencies using regular Maven commands like mvn install or mvn verify.



# Install Maven dependencies from Bytesafe
$ mvn clean install

Installing SNAPSHOT versions

To install snapshot versions (X.-SNAPSHOT) with Maven, users are required to explicitly specify the use of a snapshot repository. This is because there is no default snapshot repository associated with Maven.

This can be set either per project in the pom.xml file or project agnostic as part of a user's profile. See the Bytesafe documentation or the official Maven docs for more details.

Using Gradle with your Maven repository

Gradle is a popular alternative build automation tool for JVM-based projects that works with Maven compatible repositories.

To resolve project dependencies from the private Maven repository, the repository needs to be declared in the build.gradle file.




// repository configuration in build.gradle

repositories {
   maven {
      name = 'bytesafe'
      url 'https://{WORKSPACE}.bytesafe.dev/maven/{REGISTRY}/'
      credentials {
         username = 'bytesafe'
         password = '{ACCESS-TOKEN}'
      }
   }
}

// specify project dependencies

dependencies {
   implementation '{GROUP_ID}:{ARTIFACT_ID}:{VERSION}'
}

Note: Users should consider storing credentials in ~/.gradle/gradle.properties file in-place of build.gradle to avoid unintended distribution of credentials.

To publish a package using Gradle, enable the maven-publish plugin and specify the target publishing repository in the build.gradle file.



// enable the maven-publish plugin
plugins {
   id 'maven-publish'
}

publishing {
   publications {
      maven(MavenPublication) {
   }
}

repositories {
   maven {
      name = 'bytesafe'
      url = 'https://{WORKSPACE}.bytesafe.dev/maven/{REGISTRY}/'
      credentials {
        username = 'bytesafe'
        password = '{ACCESS-TOKEN}'
      }
   }
}

With these quick additions to your Gradle project, you should be able to both build and deploy your own project using a private Maven repository as well as install any required dependencies.

Want to know more about Bytesafe's Firewall for dependencies?

Secure your supply chain with private registries compatible with package managers like npm, maven and nuget. Refer to the Bytesafe documentation on package managers for more in-depth information and guides.

Enforce your security policies and automatically quarantine unwanted dependencies. Avoid compromised new versions with a set delay before newly published versions are allowed in your organization.

Want to try it for yourself? Sign up for Bytesafe and get started today - no strings attached.

Update dependencies safely - with a delay on newly published versions

Andreas Sommarström — Thu, 16 Dec 2021 15:40:36 +0000

It’s common to regularly pull the latest versions of packages from public upstreams - without review or regard for version maturity. And in most cases as a consequence of a build rather than from real intention to actually make updates to the dependency composition.

This is most apparent in ecosystems like npm, where the dependency tree with ranges of compatible versions turn each npm install into a unique fetch of what is the "latest and greatest" at that time.

But with popular packages often being targets for attacks there is every reason to be cautious. Perhaps a little friction is desirable for the sake of security?

To prevent malicious attacks like ua-parser-js, coa & rc (Edit: and intentional sabotage like colors.js/faker.js) from happening in your organization, what if newly released dependencies weren’t allowed to be used immediately? With new packages only permitted for your developers or CI/CD after a set safety period.

Good news! Easily achievable with the Dependency Firewall in Bytesafe private registries. Here's how.

Don't allow packages until a set safety delay has passed

The Delay Upstream policy allows for a custom delay before new versions are allowed in your private registries.

Until the set delay (in days) has passed, new versions are made unavailable to your organization. With other recent and allowed versions automatically selected for you - to not break your builds.

By giving millions of open source users in ecosystems like npm and maven a chance to evaluate new releases, you could prevent critical vulnerabilities and malicious packages. With the right balance you could save your organization from being compromised.

Example of how it works

A new version of a dependency, 1.3.0, is released to a public registry like npmjs or maven central. For as long as current time < publish time + safety delay the new version does not qualify and will be prevented from being used by your organization.

Actions by developers or automated systems to fetch the project's dependencies will instead receive the most recent allowed version 1.2.3 from Bytesafe.

When current time >= publish time + safety delay the new version 1.3.0 will be allowed in your organization and any subsequent fetches will receive the new version.

Customize the safety delay to match your needs

What the desired "maturity since release" is, differs between organizations. To accommodate, the delay in Bytesafe is completely customizable per registry in your workspace up to a maximum of 90 days.

Your organization can find the right balance between delay (security) and access to new functionality.
Adjust it to your needs per ecosystem and enforce a delay of 3 weeks for npm while using 2 months for maven - if it's right for you.

Patch versions intentionally

If the need arises to add a specific new security patch or functionality, consider using a separate patch registry to manually (and intentionally) add the required versions. Complete control, while keeping automated environments safe and secure!

Looking to secure your supply chain?

With the Delay Upstream policy we want to offer the option to balance flexibility with security, especially for automated environments and decentralized developer organizations.

In addition, organizations should make it a habit to review and make conscious decisions on the dependencies they are using for a secure supply chain.

Want to try delay upstreams for yourself? Sign up for Bytesafe and start for free today.

Thanks for reading!

Steering clear of the dependency trap

Andreas Sommarström — Mon, 01 Nov 2021 07:58:28 +0000

With the dust settling after the UA-parser-js, coa and rc incidents, it's the perfect opportunity to take some time and see what we can learn from it. With some small changes to how you view and work with dependencies, you can take back control - instead of letting the dependencies control and overwhelm you.

The UA-parser security issue highlights two major things for the npm ecosystem:

The dependency tree comes with security risks. Your direct dependencies might not be malicious, but your dependencies of direct dependencies may be targeted. These transitive dependencies often range in hundreds and are large weak points.
Organizations need to expand the scope of security and protect more than the CI/CD. Developer environments are often more numerous and more difficult to control, making it a more probable target to be compromised by malicious packages.

In these ransomware times, it's more important than ever to protect your whole organization - controlling what packages are allowed in your environments.

The UA-parser-js incident in short JavaScript library ua-parser-js sparked vigorous security activity, as the package was hijacked and three malicious versions were published to the public npm registry. Once again highlighting the need for more security focus in the JavaScript (and other) ecosystems.

The library, used to detect browser and user data, has close to 8M weekly downloads by developers around the world and is used as a dependency by 1200+ other packages in the public npm registry.

See the security advisory for more details.

Update: Malicious versions of packages coa and rc published on 2021-11-04. Same malware and pattern of attack (and indicating same hijacker), targeting popular support libraries. Malicious versions of both packages later unpublished by npm.

Dependency tree and levels of dependencies

Joe, the junior developer, helps his team with application testing over the course of the weekend. His environment is clean, so he diligently installs the required dependencies for the application using npm install.

The app only has a few direct dependencies, but with all the transitive dependencies he ends up adding over 700 dependencies. Unfortunately for Joe and his company, it’s already too late as he unknowingly installed a trojan into his environment…

Installing npm dependencies with package managers is easy - and we shouldn't kid ourselves and think that everyone is knowledgeable and informed on potential issues. It's easy for things to start going wrong with the amount of dependencies and with packages allowed to execute arbitrary scripts as part of the install process.

And malicious packages are a big threat for your development environment where potential user data, passwords and sensitive information is stored and therefore can be stolen by hackers.

Getting compromised without even knowing it

Let's use the ua-parser-js attack as an example. During the incident any user installing a package with a dependency resolution like ua-parser-js: ^0.7.xx would have gotten a malicious version (0.7.29). The dependency resolution tells us to fetch the latest 0.7 patch version - unless remediated by factors like locked dependency versions.

So who was affected? Projects depending directly on ua-parser-js were obviously at risk. But in the typical case the compromised dependency was added as a transitive dependency (dependency of a dependency).
Leading to users working with popular libraries and frameworks like react and angular reporting inclusion of the compromised library.

With transitive dependencies, the number of packages your projects depend on increases dramatically. To the point where it quickly becomes impossible to fully grasp what and how many dependencies your team uses.

# Example of dependency tree with ua-parser-js included as a transitive dependency. 
# 'npm ls' is used to identify the path for a specific dependency

$ npm ls ua-parser-js
    yourproject@1.0.0
      react@15.7.0
       └─┬ fbjs@0.8.18
         └── ua-parser-js@0.7.30

# Excerpt from fbjs@0.8.18 where ua-parser-js is included as a dependency
    "dependencies": {
        ...
        "ua-parser-js": "^0.7.18"
        },

# ua-parser-js: ^0.7.18 would resolve to the latest 0.7.x version. Installing a compromised version with malware during the time of the incident.

In the aftermath after ua-parser-js the biggest issue wasn't figuring out if your applications used ua-parser, instead it was trying to figure out if you were exposed - in any environment, in any way, across hundreds of developers. A task a lot of companies worked vigorously with, as they lacked proper control over packages that enter their environment.

How to avoid the trap? Control instead of being controlled

Avoiding similar issues in the future should be a priority - and any investment into proper protection would save time and money in the long run.

So, the million dollar question - How do we avoid this in the future?
We can mitigate most of the issues by inserting control over dependencies and the patch management process.

Avoid unintended dependency version changes
Use a single source of truth for dependencies

Locking dependency versions

You might be thinking, discussions about using lock-files again? Shouldn't everyone be aware of and use them by now? And I agree, everyone should use them - but they are not.

Dependency versions should be updated with intention and not as a side effect. Having consecutive npm install yield slightly different and non-deterministic results is not desired in neither CI/CD nor dev environments.

Organizations should put in place a process that updates, commits and reviews project lock-files and makes sure every subsequent installation (and user) uses the files.

Using dependency ranges, instead of pinning exact dependency versions, grants flexibility for the ecosystem - but comes with inherent security risks. Using lock-files (package-lock & yarn.lock) together with npm ci for complete and deterministic installations introduces the necessary friction that make updating dependency versions a controlled process.

Single source - The dependency firewall

Depending directly on public registries and countless GitHub repositories, instead of using a single package source, quickly makes control over the flow of dependencies an impossible task.

With multiple different sources, how are you going to make sure dependencies comply with your business policies, that they are safe and contain approved licenses?

The solution: a single hub like Bytesafe to enforce rules and monitor the flow of dependencies - for every developer, tester and build system.

To make sure everyone is using the same registry source and the intended versions, projects should include a .npmrc config file and package-lock.json or yarn.lock files that define what registry to use.

# Example .npmrc config setting the default registry to be used by npm clients
registry=https://workspace.bytesafe.dev/r/example-registry/

Keep unwanted dependencies out of your organization. Setup a firewall for your dependencies with Bytesafe!

Thanks for reading!

Open source is a valuable development resource. Give it your attention

Andreas Sommarström — Mon, 06 Sep 2021 13:34:49 +0000

Open source software (OSS) gives access to a never-ending amount of external development resources. Developing any application, it is simply more efficient to reuse building blocks from others than having to write everything yourself - a necessity for most organisations.

Businesses should see the open source they depend on as a valuable resource - that requires management.

Using OSS to build applications allows access to additional developer resources and expertise that wouldn’t otherwise be available. The downside - you have no direct influence over the persons behind the component or their actions. And there is no reason to blindly trust them.

For internal development resources the norm is to spend time and money hiring the best candidates and continuously invest in their skills and performance.

Businesses need to echo some of that effort spent finding the right in-house talent into safe use of open source - instead of taking it for granted.

Open source in short

If you’re not working with software development or open source in your day-to-day work, then perhaps the analogy in the title is not self-explanatory. Not to worry, let’s walk through it.

Today there is no need to start from scratch every time you are to deliver a feature. There are millions of already finished components ready to be used - and they are only a few letters on the keyboard away from being nested in your application and inside your environments.

What makes something open source is that the source code for all these components is freely available to everyone and anyone, to view, duplicate, work on etc. That means that as long as you adhere to licenses of components, you can use them in your applications to serve your needs.

And open source software has proven to be an explosive engine for business growth. And it’s everywhere. For ecosystems like JavaScript / npm the figures speak for themselves:

99% of projects use open source components to some extent.
An astonishing 70% of all code used to run applications are open source.

So massive upside - what’s the downside?

When your application utilize external dependencies you’ll depend on developers who you can exert no direct control over. Without control, how do you know whether the open source components in your codebase are being maintained and adhere to your security guidelines?

Core components of your business are probably relying on components that your dev team has never reviewed or seen the insides of.
Let that sink in. Eye opening, isn’t it?

And to be clear, open source is a positive thing. We wholeheartedly support it and use it every day in our own apps. But like most things in life it needs some safeguards to make sure everyone is playing by the rules.

Using a particular open source component extensively? Consider supporting it! Sponsor, buy the developers some coffee or spend some development time on improving it.

Manage the open source you use - by inserting control into your supply chain

Not everyone can be an expert - and fortunately you don’t have to be. Committing to safe use of open source can be as simple as supporting the right process and tools for your organisation. Tooling that:

Keeps track of the open source software used
Identifies for security threats - and keeps potential issues out
Highlights issues early- for easier and more cost-effective remediation

Bytesafe allows you to combine your team’s need for package management of JavaScript open source packages with security. Using Bytesafe, developers can access public open source dependencies or private proprietary components for your applications, securely. Mitigating risks for your business.

Bytesafe identifies all the components your team is using and keeps track of them for you. Your supply chain is kept secure as part of the firewall where threats are automatically quarantined. And all issues are highlighted for you - accessible to all team members.

Thanks for reading!

Automatic protection from dependency confusion

Andreas Sommarström — Wed, 19 May 2021 19:10:23 +0000

Dependency confusion occurs when a system or user is tricked into fetching a package version from a public registry, instead of the intended trusted package with the same namespace from your private registry.

Like other supply chain attacks it’s all about injecting non-intended packages into a vulnerable supply chain, where malicious code can be executed on installation. A critical security threat for your supply chain if left uncontrolled. And a problem that potentially gets even worse in automated environments like CI/CD pipelines, if left unnoticed.

My previous post on the topic was focused on how to use security focused package management to improve control of the software supply chain.

This post focuses instead on a solution that automatically protects your supply chain against dependency confusion. A scalable solution that is safe by default and prevents internal packages from being replaced by packages from external sources, without requiring complex configurations.

How to automatically protect against dependency confusion

Enforcing this protection from dependency confusion is based on using Bytesafe registries for the private and public npm packages you depend on.

Bytesafe offers fully managed and private npm registries that allow users to better control their supply chain and secure the private and public packages they depend on.

With Bytesafe any package version marked as internal, either automatically or manually, will be protected from dependency confusion.

The solution in short:

Package versions published, pushed or uploaded to an internal registry will automatically be flagged as internal
Fetching new versions of internal packages from upstream sources, will only consider upstreams with internal versions of the same package

It’s as simple as that. Package versions added directly by users are automatically marked as internal, making sure internal packages are not exposed by mistake. And the best part, all the complex logic behind resolving dependencies safely is handled by Bytesafe, instead of being pushed on to our users.

Packages without the internal flag will function as they always have, with full access to public upstreams.

Users can continue to use registries for both public and private packages, enjoying the benefits of a single source of truth for all package dependencies. While simultaneously being fully protected from dependency confusion.

Example: Publish internal packages to a private registry

Publishing internal packages to a private registry allows them to be shared and available for other team members and systems.

$ npm --registry 'https://workspace.bytesafe.dev/r/your-internal-registry/' publish

...

+ your-internal-package@4.5.6

With the target registry flagged as internal the version will automatically be flagged internal as well. Preventing it from being substituted by package versions from untrusted external sources.

Protection during dependency resolution (npm install)

The core protection of internal packages is evident when resolving dependencies from upstream. This is typically the case when using npm install to install dependencies for a project and npm attempts to fetch all available versions.

Any attempt to fetch new versions of internal packages will only use upstreams that contain other internal versions of the packages. All other sources will be ignored for the purpose of this action.

Users can securely install any internal version from their registries, without ever having to worry about the possibility of the package being replaced with a malicious package from any external registry. A protection that scales to all users and systems.

Example: Using a Bytesafe private registry together with registry.npmjs.org as an upstream for public packages

Package published to the private registry will be flagged internal
Install of internal package by other users / systems will never fetch versions from registry.npmjs.org as it does not contain internal versions
Installing any public package will work as normal, with registry.npmjs.org as a valid upstream

Getting started - How to protect your supply chain

Any new Bytesafe workspaces and registries come with protection from dependency confusion by default - No additional actions required.

Create your own Bytesafe Workspace
Publish your internal packages to the private registry. They will automatically be marked internal.
Install/add your other public dependencies. All dependencies are continuously monitored for security issues.

With both public and private dependencies now in one single source of truth, you can distribute this secure source to all team members and systems. A solution that scales across your whole organization.

Upstream issues - additional layer of defence against supply chain attacks

Bytesafe also detects upstream configuration issues that could indicate a supply chain attack.

Upstream issues are opened when a package version is found in multiple external upstreams, with non-matching contents. Issues trigger notifications, warning you of a potential dependency confusion attack.

Thanks for reading!

Different typosquatting attacks to know of - for a secure supply chain

Andreas Sommarström — Sat, 20 Mar 2021 15:08:54 +0000

Secure management of dependencies is not always a priority when compared to development speed. At the same time adding new open source software from public registries like npmjs is easier than ever.

As a consequence it's often easier for hackers to inject malicious code as part of the software supply chain instead of trying to exploit existing vulnerabilities.

This is known as a supply chain attack, an attack vector that has been further highlighted with the emergence of the dependency confusion concept
together with recent articles like typosquatting in the Go ecosystem.

The topics of typosquatting and dependency confusing are particularly interesting as they highlight two things that are central to package management.

Package names matter. Alot. All that is required to install new packages is a valid name. If you misspell or remember a name incorrectly you will worst case install another package. Possibly a malicious one.
How much trust (or risk) we put in our supply chain. We trust that packages are always available from the public registries, trust that packages are not malicious, and trust that packages has not been compromised or taken over from its original creator. Unfortunately this trust can be exploited...

To reduce some security risks, this post will look into methods that can be used to imitate legitimate package names. By being aware you stand a better chance of keeping your software supply chain secure!

What is typosquatting?

Typosquatting (in the context of package management) is the term for creating and publishing malicious packages with names imitating legitimate ones.

Through typos or mistakes, such packages are included in your supply chain. And package scripts can run during installs by default!

With package managers like npm, we can easily add new dependencies to our projects through the name alone.
Want to add that dependency you used from months ago by memory? Not really a problem, until you accidentally include a package other than you intended.

Hackers are relying on the fact that as long as their packages pass a shallow inspection the variations in package names doesn't raise any red flags.

One of the many examples include the now removed package electorn that was transposed from the legitimate package electron by switching the order of O and R.

# At a glance there's not much difference of malicious package name
$ npm install electorn
# compared to the legitimate package name
$ npm install electron

Different variations of typosquatting

What is combosquatting? Omission, repetition and transposition? Typosquatting is the collective term for imitating real package names.
But there are multiple variations on how this is achieved. Let's have a closer look, together with some real world examples from the npm ecosystem where this was used.

Combosquatting

Combosquatting consists of trying to imitate legitimate packages by appending common words, terms or letters on the original package name.

Example of combosquatting for package names:

twilio -> twilio-npm
cross-env -> cross-env.js
fabric -> node-fabric
lodash -> lodashs

Where npm, js and node are all common terms in npm and JavaScript.

Combosquatting is generally considered the most common of the permutations.

It is also common in website phishing attempts where URLs are appended with seemingly legitimate terms to fool unsuspecting users.

Omission

Omission is intentionally leaving out a character in a package name. Either a letter or some other character like a hyphen.
It is aimed at the times when we miss pressing one of the keys in a sequence, or simply to confuse developers with familiar sounding un-hyphenated names.

Example of omission for package names:

mongoose -> mongose
babel-cli -> babelcli
cross-env -> crossenv

For the npm ecosystem, the package naming rules was updated to partially cover omission in package names.
But package names are still validated on a case by case basis.

Repetition

The inverse of omission is repetition with intentionally adding multiple instances of the same characters. This plays both to our nature of pressing the same key twice by mistake but also to spelling mistakes.

Example of repetition for package names:

jquery -> jquerry

# Example of typosquatting dependency in package.json. 
# Would you notice the difference? 
...
"dependencies": {
    "jquerry": "^2.0.0",
    "react": "^17.0.1"
  }
...

Transposition

Transposition switches the position of two adjacent characters. Typos where we press keys in the wrong order, as well as common spelling mistakes are target for transposition (middle -> middel in the example below is a perfect example of this).

Example of transposition for package names:

http-proxy-middleware -> http-proxy-middelware
electron -> electorn

Control the risks by controlling your dependencies

A secure supply chain, from typosquatting or other attacks, starts with knowing what open source software you are using.

Use a service like Bytesafe to host both private and public packages. This enables a central hub with all your dependencies in one place. For both direct and transitive dependencies.

When new dependencies are added to a project, they are instantly available in Bytesafe as well. Allowing for continuous monitoring and control. Detect issues as they happen instead of waiting for point in time scans or critical issues in your build environments.

Scan dependencies for known vulnerabilities. Package typosquatting attempts are continuously being reported and flagged for vulnerabilities. Get notified or block any known vulnerabilities in your supply chain. Directly when adding them or anytime in the future.

Don't run scripts by default during install
When installing packages there are often scripts executed as part of the installation process. This is convenient and useful, but executing random scripts is also a major risk.

Make sure you know what is executed when installing packages. If you have not reviewed the scripts, you are much more likely to be safe installing with the --ignore-scripts attribute.

Want to know more about how Bytesafe can help secure your supply chain?

Visit our dedicated page on securing the software supply chain to learn more about the potential problems to look out for and how we can help.

Also, follow bytesafedev on twitter for continuous updates on everything security related. Stay safe!

Why you should host public npm packages in a private registry

Andreas Sommarström — Wed, 03 Mar 2021 14:22:27 +0000

This post talks about how using a private npm registry to proxy the public npm registry helps to identify and control the packages you use. And increase the security of your code supply chain.

Want to manage the package dependencies you use for your projects? Reduce your dependency on the public npm registry? Or perhaps enforce security policies? Then read on.

Its a good idea to know of and manage the package you use

Millions of developers use the public npm registry every day and with over 1.5 million packages it is a critical source for open source packages.

But the convenience of packages from the public registry and the development speed all those available open source packages entail, comes with downsides. It gets increasingly hard to keep track of and manage all those dependencies.

To avoid blindly trusting code from external sources, users should take steps to improve the control over dependencies and overall security when using npm. It is a necessity for any organization that wants to manage their code supply chain.

And one of the first things that is usually mentioned for improved npm security is the use of a private npm registry to host your packages.

Fully managed private npm registries with Bytesafe

Control over dependencies — How?

There are multiple ways a private registry enables better dependency management:

Central hub for all your required package versions: Private and public together, possibly from multiple upstream sources.
Identification and visualization of dependencies: With all required packages in one place it enables identification of potential issues. Additionally the proxy caches your packages, removing the worry that an essential package version will be unpublished in the future.
Single package source: With all developers using the same registry that contains the same versions, you can ensure all users build and test consistently. Removing the potential issue of unknowingly using different versions of a dependency.

Improved security — How?

Using a private registry as a proxy enables a layer of separation between your organization and the outside world.

Security scanning: continuously scan and monitor your dependencies for known vulnerabilities and security issues
Security policies: with all packages in one place you can enforce the security policies you require.
License compliance: all dependencies in one place to identify open source licenses and scan for problematic licenses

Interested in npm package security? Read my related post on avoiding dependency confusion.

Control your npm packages & avoid dependency confusion

Andreas Sommarström ・ Feb 19 '21

#npm #node #javascript #security

Setting up and using a private registry is easy

Bytesafe offers hosted private npm registries that by default can be used to proxy the public npm registry.

Allowing for access to public npm packages as well as all the additional benefits that Bytesafe offers to your workflow.

When using Bytesafe, developers configure their npm client to interact with the private registry instead of the (default) public registry. With a configured upstream, any packages not available in the private registry will be fetched from the upstream registry instead (like registry.npmjs.org in this example).

Before new packages are pulled into your registries, they are checked against any active security policies, to make sure they do not violate any rules you have configured.



# Authenticate to Bytesafe using the npm client
$ npm --registry https://example.bytesafe.dev/r/default/ login
Username: bytesafe
Password:
Email(this IS public): you@example.com
Logged in as bytesafe on https://example.bytesafe.dev/r/default/.
# Work with the regular tooling (but direct your requests to private registry)
$ npm --registry https://example.bytesafe.dev/r/default/ publish
...

With the npm client no longer directly linked to the public registry, it results in the following workflow:

Developers - publish/install package versions to/from the Bytesafe private registry — no direct interactions with the public registry. No need to change any behavior or usage patterns.
Private registry - holds all public and private packages and any required dependencies. If a package version is required that is not in the proxy registry it is pulled from upstreams.
Upstreams registries - provides package versions (when required) and is the target for push of packages from private registry. Upstreams can be either a single registry or multiple registries.

Recap

Setting up and using a private npm registry is an easy and effective way to keep your dependencies in check and improve security when using npm.
Without impacting or changing the workflow for developers.

Bytesafe offers hosted, private, reliable and private npm registries. And it's free to use for individual developers (so feel free to signup if you need a
private registry). Manage, collaborate and secure your code supply chain with Bytesafe!

Understanding open source licenses - What is Copyleft?

Andreas Sommarström — Tue, 23 Feb 2021 09:49:15 +0000

Every developer, team and organization owe it to themselves to be aware of and understand at least the basics of open source licenses.

Using open source dependencies with different types of licenses for a commercial project? Or are you looking for more information on open source licenses as part of setting a license for a package of your own?

Then you may be looking for answers to questions like:

What are open source licenses?
What are the main differences between copyleft and permissive licenses?
Why do I have to care about licenses for the packages I use?

Questions that this post aims to answer and provide some context for popular open source licenses on the way.

But wait, aren't open source packages free to use? Open source code is free and available for anyone to use — but there are limitations and restrictions.
The open source license restricts users on what they can or cannot do with a package.

And it's up to each developer or business to remain compliant.

Disclaimer: Information in this post should not be considered legal advice.

Open Source Licenses - the basics

Open Source Software Licenses allow developers to share their code and components as open source.

With the use of open source packages being the modern standard, the number of dependencies for most projects count in the hundreds.
All these open source dependencies come with license obligations.

Each license defines how other developers are allowed to use these components in their own work. Typically this can be thought of as:

Do's - things the license allows you to do
Don'ts - things you are not allowed to do
Must's - things you must do

There can be a wide variation between different licenses on what you are allowed and must do.

By using open source components you’re implicitly signing a contract that binds you (i.e. developer or company) to the do’s and don’ts of how you use that piece of code.

Standardized licenses vs non-standard

Most open source packages one encounters uses some form of standardized license.

The benefits of using standardized licenses are many: Reliable identification of licenses, recognizability, trust and more. People naturally shy away from the unknown and prefer to use open source packages with standard licenses.

There are several organizations who provide guidelines and definitions regarding open source software licenses.
The Open Source Initiative defines a list of some 80 certified open source licenses and the SPDX License List provides a list of commonly found licenses.

Non-standard licenses, on the other hand, are any license or license text that does not fit into the standardized format. It can be a completely custom license, a custom copyright provided by the author or simply some license that does not fit into the term open source.
What non-standard licenses have in common is that they usually require some kind of manual review and approval from users.

Npm package licenses

For npm packages, license(s) information has its own section in the package.json file.

But it is important to know that there are no rules forcing package maintainers to provide any license information in package.json. Licenses might just as well be found in a LICENSE.md file or as part of any other file of a project.

Multiple licenses for a single package are also something that needs to be considered. A single package can have multiple licenses that either define the obligations for a specific scenario or for an individual piece of code.

What about open source code without a license? A software component without a licence is not free to use. By default it is fully copyright protected, so developers have no legal rights to use, modify or share it.

One package. Multiple licenses
The npm package abbrev has the license ISC declared in package.json, but two different licenses are available in project files!

Want help identifying licenses in your projects? Visit Bytesafe License Compliance page!

Copyleft vs Permissive

To allow for easier categorization and understanding of the basic principles of licenses, open source licenses can generally be divided into two main categories: copyleft and permissive.

This categorization is based on the requirements and restrictions that the license places on users.

A copyleft (or strong copyleft) license allows derivative work, but requires you to release such work under a compatible copyleft license.

Copyleft licenses are linked to the principle of reciprocity where authors want to ensure that any derived work (modifications, improvements) are also made available as open source and to the benefit of the public (and not released as closed source).

A permissive (or non-copyleft) license places minimal restrictions on how others can use the open source components.

Permissive licenses allow for modifications, use in software distributed under other licenses, as well as use in non-open source (proprietary) software.
As such permissive licenses are usually the go-to type to be used by organizations with commercial intentions for their software.

Weak copyleft is a variation of the copyleft type, that is aimed at creation of software libraries. Software that uses a library with a weak copyleft license are not necessarily forced to release derived work under a compatible license (rather it depends on how it is used).

Popular Open Source Licenses

Let's look at an overview of some of the most common standardized licenses and the characteristics for them.

To read the full license text, use the links below (directing to GitHub supported choosealicense.com).

MIT & ISC - The go to permissive licenses

The MIT License (MIT) is one of the most recognizable open source licenses.
It is permissive and places minimal restrictions on what you can do and must do. The license allows distribution and use of the component for commercial use while requiring only that the original license notice and copyright is included in any copy of the software.

The ISC License (ISC) is functionally identical to the MIT License, but with some wording deemed unnecessary removed.
The ISC License is the default license used when setting up a new npm package with the npm init command.

With their permissive and simple nature, both the MIT and ISC licenses are popular for open source components. Especially if the primary intent is to be shared as dependencies for other open source projects.
As such they are among the most used licenses for packages in the npm ecosystem.

Apache-2.0 - permissive and suitable for use by larger organizations

Apache License, Version 2.0 (Apache-2.0) is another popular permissive license.

The main characteristics of Apache-2.0, compared to other permissive licenses, is that it provides users with a grant of patent rights from contributors.
This makes an open source component under this license safer to use.

The fact that Apache-2.0 addresses patent licenses, makes open source component under this license particularly appealing to any organizations that want to make sure they are not infringing patents when using a piece of software.

Gnu Public License (GPL) - copyleft at its core

The Gnu Public License is the original for the copyleft concept, where use of a GPL licensed component forces derived work to be released to the public under a compatible license as well.

Developers and organizations that promote collaboration and code sharing before anything else may prefer a license like GPL.
But for many organizations the use of open source components with a GPL License is not preferred compared to a more permissive license with fewer restrictions and limitations.

Why you need to care about licenses for the packages you use

Part of dependency management is knowing what packages your projects are using. Including the licenses attached to those dependencies.

Without this knowledge you could possibly be in breach of open source licenses agreements already.

A short-list of possible consequences for license non-compliance:

Restrictions on selling your software product
Involuntary release of your source code
Negative press coverage for non-compliance
Loss of reputation with customers or with the open source community

For a more extensive list of possible consequences, check out our Open Source License Compliance page.

Identifying non-standard licenses
The npm package jsbn has the standardized license MIT declared in package.json, but a custom license is also available in project files.

The Bytesafe License Scanner can help you identify non-standard or problematic licenses.

License compliance can be complicated

Make sure to address open source software licenses in your code supply chain. Start by:

Identify what open source licenses you are using
Identify potentially problematic unlicensed or non-standard licenses
Prune dependencies using unwanted licenses

And if you use open source components, don’t forget to support the open source communities.

Thanks for reading!

Control your npm packages & avoid dependency confusion

Andreas Sommarström — Fri, 19 Feb 2021 14:06:38 +0000

There has been a lot of great discussions on dependency management and supply chain attacks recently. Alex Birsan posted this article
on dependency confusion and npm added this post on avoiding substitution attacks (another term for the same thing).

What is dependency confusion? Term coined for the confusion facing package managers (npm, yarn, pnpm) on what package version to pull into your project, what source to use and trust, when faced with multiple options.

With Bytesafe, we have been hard at work the last year to create a service that allows users to stay in control of their package dependencies.
As such, any discussions that raises awareness on potential issues and consequences of not managing your supply chain are a top priority for us as well.

Edit: See this follow-up post with an updated, safe by default solution for dependency confusion.

How can you use Bytesafe to protect against supply chain attacks and avoid dependency confusion?

Update: Internal packages - our secure by default solution for Dependency confusion!
Update: Get notified with Issues - issues are created for package versions found in multiple external upstreams, with non-matching contents. An indicator for a possible supply chain attacks.
Use private npm registries to continuously monitor and control the dependencies you are using
Create a dependency firewall registry
Use security policies to control the flow of packages

Using a private registry for insights and control

Many teams today have limited knowledge and control over the dependencies they are using. There are many talented developers who make it their responsibility to make informed and responsible decisions for the packages used in their projects. But, the tools to get a complete top-down view are often missing.

So before you can even start to manage issues like dependency confusion, you need to address the overall dependency management:

Use private registries from Bytesafe to centralize, identify and control all the package dependencies you are using.
Continuously monitor your dependencies. Scanning them for potential security and license compliance issues.

By centralizing dependency management, users no longer have to rely on manual reviews of package.json or lock-files, or point in time scans to identify the dependencies used for particular projects.

Configure projects to use private registry be default

Your applications security is only as strong as its weakest link. So make sure private registries are used as the default registry for projects, by providing a custom project level .npmrc configuration file.

Overwriting potentially problematic individual configs when in that project space.

# Example .npmrc file where all packages are fetched from Bytesafe
registry=https://workspace.bytesafe.dev/r/example-registry/
//workspace.bytesafe.dev/r/example-registry/:_authToken=${AUTH_TOKEN}
always-auth=true

Claim organization name and use scope for internal packages

One option to close the attack vector for dependency confusion is the use of scoped packages in conjunction with blocking your organizations name in the public registry.

Users that want to deploy this tactic should claim an organization name in the public npm registry and use that scope for internal packages (@your-organization/pkg) in their private registries.

Blocking the ability for packages with the same name (inside the scope) to be pulled from the public registry by accident.

Create dependency firewall registries

The idea behind a dependency firewall is simple: create a single registry that is responsible for the link to the public npm registry and works as a single point of contact with the outside world.

All users (or other internal registries) will use this registry as a package source and npm proxy. This enables a central point where security teams can monitor and prune the registry to make sure it only includes approved packages.

If needed, the link to the public npm registry can be removed entirely, and only reinstated again when package dependencies are to be added or updated.

By using a dependency firewall in this way, new packages and versions do not automatically trickle down to all other users and applications in an organization.

Use policies to lock dependencies or block packages

Bytesafe offers a range of security policies to make sure users are in control of their dependencies.

Policies are rules that are checked and enforced before registry actions are applied. For example a policy could prevent packages with known security vulnerabilities from being pulled into a registry.

Short-list of examples:

The Freeze policy that can be used to make registries read-only (temporarily or permanently)
The Secure policy blocks packages with known security vulnerabilities
The Block policy can be used to block whole packages or ranges of versions

Going forward

Our roadmap contains many exciting security policies that we are looking to add to improve the protection against supply chain attacks.

Have any questions or suggestions on features that you would like to see? Comment below or contact us on Twitter (@bytesafedev). We appreciate all insight!

Want to improve your dependency management with Bytesafe? Sign up for free. Pull your dependencies into one of our registries and test Bytesafe yourself!

Using private npm packages with GitHub Actions

Andreas Sommarström — Mon, 15 Feb 2021 08:29:26 +0000

If you are using GitHub Actions as the CI/CD service for your build chain, you may be looking for a way to include private npm packages in your builds.

The solution: configure a custom package source and install your package dependencies from a private npm registry!

Integrating a private registry with GitHub Actions couldn't be easier.
So let's look into how we can setup GitHub Actions to install package dependencies in this way.

But first: the basics behind using a private registry

The default registry when using npm, is the public npm registry. It contains public packages that are available for any developer to pull required packages from.

But teams often require a separate and private home for their internal packages. Packages that are their own proprietary code, that is never intended to be published to any public registry.
But still need to be shared internally, and most importantly used in builds of applications.

In such cases, organizations can make use of a private registry to hold their packages. A private registry can either be something that is self-hosted or a service provided by a specialized provider.

Bytesafe offers private npm registries. Users can:

Host and cache their private and public dependencies in a central hub
Manage, overview and control their whole dependency supply chain
Scan package dependencies for security vulnerabilities, block packages that are unsecure (read our security best practices to learn more)
Identify open source licenses of packages used in their build chain and scan for unlicensed or problematic license issues

If your team requires a private registry for your CI/CD chain, signup for Bytesafe!
Our registries are hosted, secure and require minimal setup.

GitHub Actions

GitHub Actions is a workflow automation tool provided by GitHub.
With the widespread use of its repository tool, it is no surprise that the popularity of its related (and to an extent included) automation tool only continues to rise.

Users can build, test, and deploy code according to workflows they define and trigger the workflows based on certain events, like the push of code to a GitHub repo.

To get the CI/CD to install dependencies using the private registry, we need to:

Provide GitHub Actions with custom configuration for the registry to use, as well as a related access token
Define our Actions workflow for what steps are to be run as part of our build

Configuring use of private registry

Like many similar services, GitHub Actions relies on the .npmrc (npm runtime configuration) file to provide custom instructions to be used by its built-in npm cli.

If no custom configuration is provided, npm will default to fetch required package dependencies from the public registry.

But as our example uses private packages not available in any public npm registry, we will use .npmrc to set a custom parameters for:

registry (provide an URL for our private registry)
authToken (provide the authentication details required to communicate with the private registry)
always-auth=true (force authentication details to be sent alongside requests to private registry)

The authToken will be defined as an environment variable, but more on this later.

For our example we want to pull all packages from the private registry, as such we will override the default, by simply setting a new registry URL to be used for all packages.

If we want to pull only a subset of packages from a registry, we can make use of @scope and only packages matching the scope will be fetched from that registry.

Putting together the .npmrc file

First step is to provide on the registry URL that should be used. Bytesafe private registries use the following pattern:

# replace workspace and registry names to get registry URL 
registry=https://workspace.bytesafe.dev/r/example-registry/

Next we need to create an authToken to be used with requests to the private registry. Let's create a fresh token to be used solely for this CI/CD chain:

# Replace example registry URL with the url for the registry of your choice
$ npm --registry https://workspace.bytesafe.dev/r/example-registry/ token create

We add the generated token from the output to GitHub's built-in secret store (to make sure we do not leak credentials).

Create a new repository secret called NPM_TOKEN and save the created token. For more information on storing of repository secrets, refer to GitHub's official documentation on the subject.

We will configure our workflow to use the environment variable NODE_AUTH_TOKEN, which in turn links to the NPM_TOKEN secret.

At this point your .npmrc file should look something like this:

# Example .npmrc file where all packages are fetched from Bytesafe
registry=https://workspace.bytesafe.dev/r/example-registry/
//workspace.bytesafe.dev/r/example-registry/:_authToken=${NODE_AUTH_TOKEN}
always-auth=true

If you are happy with the state of your file, go ahead and push it to the related repository.

Alternative: Have GitHub Actions create the .npmrc file for you

GitHub Actions can create the .npmrc config for you, with the actions/setup-node@v1 action.

If you use this approach, enter the same registry URL that you would use in .npmrc and always-auth: true when configuring the GitHub Actions workflow yaml file.

Define the GitHub Actions workflow

Now we are ready to define the workflow and what actions should be run on our behalf.

This is done by creating a yaml file that we store in the projects .github/workflows directory. The yaml file contains the steps we want to run through.

We use it to tell GitHub Actions: what trigger to use (push), to checkout the code, what parameters to use with node and to install the dependencies as part of our example:

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: checks out code
      uses: actions/checkout@v2
    - name: Use Node.js
      uses: actions/setup-node@v1
      with:
        always-auth: true
        node-version: '14.x'
        registry-url: https://workspace.bytesafe.dev/r/example-registry/
    - name: Install dependencies using a safe clean install
      run: npm ci --ignore-scripts
      env:
        NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}

# skip 'registry-url' and 'always-auth: true' parameters if these are provided by a custom .npmrc file

Voila! With that we have now configured a GitHub Actions workflow using packages from a private registry.

With a setup like this, each time code is pushed to the registry, GitHub Actions will safely install the dependencies using npm ci with --ignore-scripts flag set.
And obviously this is only an example. Users can design the workflow to perform the actions required for their needs.

After a workflow has been triggered, users can check the status and progress of the job in the Actions tab in GitHub.

Want to integrate with other cloud platform or CI/CD services?

The basic principle of using the .npmrc file to provide custom configuration is applicable for many other similar services as well.

Users that want to integrate Bytesafe with other services can refer to our general guide on integrating services with Bytesafe on our docs site for more information.

And if you need more GitHub Actions specific details to get your build chain up and running, refer to the official GitHub Actions documentation for setting up node.js.

Thanks for reading!

P.S. Need a private registry for your packages? Start for free with Bytesafe

Installing unpublished npm modules from Git repos

Andreas Sommarström — Mon, 01 Feb 2021 20:58:06 +0000

Want to share and collaborate on internal npm modules from your Git repositories during development? Without having to:

Push changes to CI/CD systems to build and deploy packages to a registry
Store local copies of dependencies (checked out from Git) and manage symlinks for them
Manually point dependencies to a specific Git repository (and branch/commit) and maintain a development version of package.json for this

Setting up CI/CD for every feature branch just to be able to share internal packages in development is not ideal. And manual updates and coordination of versions and dependencies for each commit you want to test is not very ergonomic etiher.

Use an easier alternative, by connecting your Git repos to Bytesafe instead. Developers can install a module from Bytesafe, and the appropriate version will be packaged from the Git repository.

And by pulling both private and public npm dependencies from Bytesafe, teams can use one central source and benefit from Bytesafe plugins and policies (that can also be applied on modules sourced from Git repositories).

Git upstreams

Upstreams are linked registries that are used both as a source and destination for packages for your private registry.
When using a Git repository upstream, teams can access internal npm modules in a way that is completely transparent to developers. Without the need for developers to configure anything special, npm modules are easily accessible for your team members' project, with the tools they regularly use.

Example workflow

Using a Git repository as one of the upstreams for a registry. Changes pushed to Git are directly available to be fetched by other team members from Bytesafe(using their regular npm / yarn / pnpm tooling), together with public packages (sourced from registry.npmjs.org).

Versions, branches and tags

Branches and tags Bytesafe understands Git branches and tags, which makes it easy to install versions from feature branches or release tags
Latest The dist-tag @latest is mapped to the last commit in main (i.e "master" or "main") branch
Versions The version from package.json in the main branch can also be installed in addition to the versions that the above tags point to

# Installing a pkg from a Git upstream without specifying a version
# Installs the latest commit from the main branch
$ npm -r https://abc.bytesafe.dev/r/default/ install 'your-module'
...
# To install and test your project with a new branch of a dependency, 
# specify the branch when adding the package
$ npm -r https://abc.bytesafe.dev/r/default/ install 'your-module@feature-branch'

Multiple modules in a Git repository
Bytesafe finds all modules in a Git repository and makes them available for install (very useful for monorepos).

Configuring Git upstreams

You can add Git repositories as upstreams to a Bytesafe registry in one of two ways:

Using a URL Any repository available over HTTPS and with a URL ending with .git. For private repos you can add a username/password for authentication.
Using the Github integration you can connect one or more Github accounts to Bytesafe. Allowing you to easily add repos (both public and private) by selecting them in a drop-down list.

Private and public
Both public and private Git repositories can be added as upstreams. The Github integration makes it easy to use private repos without needing to manage tokens.

TL;DR: Treat your Git repos like any registry

Don't compromise on usability just because the source is a Git repository. Include the Git repository into your Bytesafe workflow instead!

Interested? Head over to Bytesafe to try it for yourself.