Forem: Jesse Houwing

Automatic versioning for GitHub Actions authors

Jesse Houwing — Sat, 07 Feb 2026 10:34:01 +0000

As a maintainer of 3 GitHub Actions, I've always struggled a bit with the versioning of them. GitHub's documentation provides guidance, but no easy way to verify you did everything right. Based on a quick scan of the actions on the marketplace I'm not alone, even some of GitHub's own actions have some inconsistencies when it comes to it.

This is why I cobbled together v1 of my Actions Semver Checker action a couple of years ago and it has served me well. But as I threw the action together in a short amount of time way back when, it contained a few bugs, was very hard to maintain and I always dreaded making changes to it, since I had fixed my own repos, so it was hard to test whether I broke anything or accidentally introduced unwanted side effects.

This is why I had Copilot Agent use the existing v1 implementation as a blueprint to build a new version. But with a lot of improvements. I gave Copilot the order to refactor the action with the following constraints:

Make sure the old implementation and the new one can be called side-by-side so it's easy to compare the differences.
Extract the validation rules into separate scripts so they're easier to maintain and test.
Provide manual remediation options wherever possible.
Implement an autofix functionality which tries to execute the manual remediation steps.
Add unit tests for everything.
Add integration tests for everything.
Try to remain backwards compatible with v1 as much as possible when it comes to the inputs.
I gave it links to all the relevant GitHub docs and blogs for:
- GitHub CLI release commands
- Releases REST API
- Tags REST API
- GitHub's GraphQL API
- Blogs discussing immutable releases
- GitHub Actions versioning

I used Claude Opus to generate a plan for this, since it's a pretty big effort. I went over the plan and made a few tweaks and then had the Copilot Coding Agent spend a good 45 minutes to do the work. And this resulted in my first vibe coded GitHub Action. And the result? It did mostly what it needed to do. It surfaced a couple of gaps in the initial plan and the request to keep both old and new code side-by-side caused some interesting interweaving between the 2 implementations.

But now that I had a test suite of 90+ tests, I felt much better changing the actual behavior. And after many round trips with the Copilot Coding Agent as well as a few hands-on sessions inside Visual Studio Code with Copilot, I am proud to release v2 of the GitHub Actions Semver Checker Action.

What does it do exactly?

Good question, the action is meant to be installed into the repository which hosts a GitHub action, like actions/checkout or jessehouwing/actions-semver-checker. It analyzes all branches, tags and releases and validates them based on a number of rules:

There should be a vX tag or branch (configurable) which points to the latest vX.y.z tag.
There should be a vX.Y tag or branch (configurable) which points to the latest vX.Y.z tag.
There should be a vx.y.z tag
There should be no vx.y.z branch
new - There should be a release for every vx.y.z tag
- new - That release should be immutable
- new - The release for the highest vx.y.z should be set to "latest"
new - There should be no release for vX and vX.Y tags
new - The action should be published to the marketplace
- new - The action.yml should have the correct metadata elements
- new - The latest version of the action should be published to the marketplace
new - Immutable releases should be turned on

And for almost all of these validations, I managed to build an autofix option which automatically corrects any issues found.

What's new

As you can see in the list above, v2 adds a lot of new validations and it also adds the autofix feature to automatically resolve most of the issues found. If the action is not able to fix the issue, manual remediation steps are logged to the workflows summary pages.

Autofixes available:

delete/update/create branches
delete/update/create tags
convert branch to tag
convert tag to branch
delete/update/create/publish releases
automatically set the latest version
republish releases to make them immutable

There are a few other improvements worth noting:

Fully configurable which suites of rules you want to run
You no longer need to check out the whole repo to run the action
Retry logic and handling of rate limits
Available as module on the PowerShell Gallery

There is one unfortunate change to v1 I could not work around

You should pass the ${{ secrets.GUTHUB_TOKEN }} to the token: input of the action
For some validations the token must have contents: write permissions (to read draft releases)
For autofix functionality the token must have contents: write permissions
To fix certain issues, the token needs permissions which cannot be granted to GitHub Actions, such as workflows: write. Use a GitHub App or Fine-grained Personal Access Token instead.

Recommended usage

I recommend you install the action into your action repository with its default settings, this will analyze your repo and will detect all rule violations.

Change the settings of the action to match your desired behavior:

floating-versions-use can be set to tags or branches depending on how you want to manage the vx and vx.y versions.
check-minor-version can be set to none, warning or error to validate the existence of vx.y versions.
check-releases can be set to none, warning or error to validate the existence of releases for vx.y.z tags.
check-release-immutability can be set to none, warning or error to validate whether releases are immutable.
ignore-preview-releases can be set to true or false to exclude preview releases when checking floating versions (vx an vx.y).
ignore-versions can exclude versions from validation completely. This can be used to ignore really old versions or versions that were accidentally made immutable and can no longer be fixed.

name: Check SemVer

on:
  push:
    branches: ['main']
    tags:
      - '*'
  workflow_dispatch:

permissions: {}

jobs:
  check-semver:
    permissions:
      contents: write

    concurrency:
      group: '${{ github.workflow }}'
      cancel-in-progress: true

    runs-on: ubuntu-latest

    steps:
      - uses: jessehouwing/actions-semver-checker@v2
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          check-release-immutability: none

⚠️ be extra careful with the check-release-immutability setting prior to turning on autofix. Most automatic fixes can be easily undone using the git commandline, but once a release is made immutable, there is no way back.

Once your existing tags have been cleaned up and you understand the implications, turn on release immutability checks:

    steps:
      - uses: jessehouwing/actions-semver-checker@v2
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          check-release-immutability: error

Use the action in read-only mode for a while, manually performing the suggested remediation steps.

When you're confident the action does what you want it to do, consider turning on autofix: true.

    steps:
      - uses: jessehouwing/actions-semver-checker@v2
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          autofix: true

With autofix turned on, the only thing you need to do to release a new version and update all tags and releases, is to push a new tag to to create a new release with the correct name: vx.y.z. This will trigger autofix to do the rest on your behalf.

Improved Dependency Submission for GitHub Actions

Jesse Houwing — Sat, 27 Dec 2025 18:47:09 +0000

If you've read some of my earlier blogs about GitHub Actions Security, you may have followed my advice to pin your actions and configure Dependabot. Recently someone brought to my attention that you then Security Advisories for GitHub actions do not show up in the Dependency Graph and Dependabot won't create security updates.

I decided to solve that problem.

Background

When a security vulnerability is found and a security advisory is created, it will have the action and its affected versions as metadata. Versions , not SHAs.

When Automatic Dependency Submission submits the versions of actions used by a repository it records:

@v1.2.3 as 1.2.3
@v1 as 1.*.*
8e8c483db84b4bee98b60c0593521ed34d9990e8 as 8e8c483db84b4bee98b60c0593521ed34d9990e8

The Security Advisory Database can't match 8e8c483db84b4bee98b60c0593521ed34d9990e8 against the vulnerable version range and won't show the vulnerability in the dependency graph.

Another issue exists when actions are forked to an organization (a common technique to create a private actions marketplace). The dependency is recorded using the organization name that holds the fork, not the upstream.

The Security Advisory Database can't match myorg/actions_checkout against the vulnerable action actions/checkout and won't show the vulnerably in the dependency graph.

Impacted features

Without recording the exact version of each action used, a number of features won't work as you might expect:

Dependency Graph won't show which actions you use are vulnerable.
Dependabot Security Updates won't be created for workflows using vulnerable actions.
Dependency Review Action won't block pull requests introducing vulnerable actions into your workflows.

Solution

To solve the issue I created a new manual submission action which mimics the automatic submission, but adds a few extra features:

It resolves the SHA to the highest, most specific version and records that as an additional dependency.
It resolves a wildcard version to the current specific version and records that as an additional dependency.
It can be configured to look up the upstream of a forked action and records that as an additional dependency.

With these additional dependencies recorded, the Dependency Graph now contains all the metadata to determine whether the GitHub actions you depend on are vulnerable and all features should now work as expected.

How to

Changing how Actions' dependencies are reported, you simply add my jessehouwing/actions-dependency-submission action to your workflows (and optionally add actions/dependency-review-action:

name: Submit and validate dependencies of GitHub Actions

on:
  push:
    branches:
      - main
  pull_request:
  schedule:
    - cron: '33 4 * * *'

permissions: {}

jobs:
  submit-actions-dependencies:
    runs-on: ubuntu-latest
    permissions:
      # submitting dependencies requires 'contents: write'
      contents: write

    steps:
      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
      - uses: jessehouwing/actions-dependency-submission@e848a29fd84b874cce3e45ceb00619bc72dbeca3 # 1.0.2
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
      - uses: actions/dependency-review-action@3c4e3dcb1aa7874d2c16be7d79418e9b7efd6261 # 4.8.2
        if: github.event_name == 'pull_request'
        with:
          retry-on-snapshot-warnings: true

You can test whether the action is working correctly by adding a vulnerable action to your repository:

name: Submit Dependencies
on:
  workflow_dispatch:

jobs:
  submit-dependencies:
    if: false
    runs-on: ubuntu-latest
    permissions: {}
    steps:
      - uses: actions/download-artifact@eaceaf801fd36c7dee90939fad912460b18a1ffe # v4.1.2

This should show as a high vulnerability in the Dependency graph of the repository:

Dependency Graph showing actions/download-artifact@v4.1.2 as vulnerable.

And you'll also see the original report using the SHA:

Dependency Graph showing actions/download-artifact referenced by SHA.

You can find the action and additional documentation and examples here.

GitHub Billing - Let Organization and Repository admins set the Cost Center in GitHub

Jesse Houwing — Wed, 24 Sep 2025 08:48:31 +0000

We rely on GitHub Cost Centers to split the GitHub bill across the different entities that make up our company. What resource is tied to which cost center is managed in the GitHub Enterprise Settings and for now this can only be done be a user with Enterprise Owner or Billing Administrator permissions.

I'd been searching for a way to allow Organization Owners and Repository Administrators to set the Cost Center on a repository themselves without having to involve one of these highly privileged accounts. We found a solution using GitHub Actions and Repository Custom Properties.

Since my last blog post on Cost Centers, a few small improvements were released by GitHub:

There is now an API to associate a Repository with a Cost Center.
It's now possible to associate a Repository to a different Cost Center than the Organization that owns it.

These two improvements allow us to solve this problem.

Creating the Custom Property

GitHub has a feature called Custom Properties for repositories which allows you to link arbitrary data to a repo. You can define these properties at the Enterprise, Organization and Repository level.

Since our cost centers are defined at the Enterprise level, I chose to create this custom property at the same level. To add a new custom property navigate to your enterprise, select the Policies tab and then drill down into Repository and then Custom Properties. Choose New Property to create your new property:

Navigate to the Custom Properties section and click new property

Give the property a clear Name and Description. Since our list of cost centers is clearly defined, I've chosen the Type single-select to allow our users to pick a cost center from a predefined list.

Give the property a unique name and set the type to single select

Next, add the list of cost centers you want people to be able to select from as options. I've also included an option inherit which I'll explain later:

Add the required cost-center options.

And finally make the field required and set the default value to inherit.

Set the permission to allow users with properties permission to edit this field, make it required and set the default value.

Now when you create a new repository or navigate to the settings of an existing repository, you can set the cost-center custom field:

Users are required to select a cost center when creating a new repository.

And existing repositories will now show the custom property in the repository settings:

The reason for Inherit

In the above example we added a special value inherit , when this value is set, we don't actually set any cost center on the repository and this causes all costs generated by this repository to automatically flow to the cost center associated to the Organization. If the Organization isn't associated to a cost center, the costs will flow to the Enterprise.

Making it all work using GitHub Actions

Creating the Custom Property itself doesn't do anything by itself. In order to actually associate the Repository to the Cost Center, we still need to update the configuration in the GitHub Enterprise Billing & Licensing section:

Billing Administrators can associate Organizations, Repositories and Users to Cost Centers

Without additional automation each repository needs to be manually added to a cost center by editing the Cost Center and adding the Repository. This is a tedious manual process.

Instead, to update the Cost Center configuration I've created a GitHub Actions workflow that runs once a day and compares the Cost Center configuration against the values of the Custom Properties and performs updates to the Cost Center configuration if needed.

The basic workflow is quite simple:

Query the current Cost Center configuration
Query all Organizations in our GitHub Enterprise
Query all Repositories in each GitHub Organization
Query the Custom Properties for each Repository
Update the Cost Center configuration in case it doesn't match the Custom Property

The workflow relies on the GitHub CLI t0 query the data.

$costCenters = (invoke-gh -fromJson -- api /enterprises/$enterprise/settings/billing/cost-centers).costCenters

$orgs = get-allorganizations $enterprise | % { $_.login }

update-repocostcenters

The update-repocostcenters function checks which updates are needed:

function update-repocostcenters {
    Write-Output "Updating Costcenters for repositories in $enterprise." 

    foreach ($org in $orgs) {

        Write-Output "Processing organization $org"
        $repos = invoke-gh -slurp api /orgs/$org/repos --paginate --jq '.[] | { name: .name, full_name: .full_name }' | ConvertFrom-Json

        foreach ($repo in $repos) { 

            $customProperties = invoke-gh api /repos/$($repo.full_name)/properties/values --jq '.' | ConvertFrom-Json
            $repoCostCenterProperty = $customProperties | Where-Object { $_.property_name -eq "cost-center" } | Select-Object -ExpandProperty value -ErrorAction Ignore
            $repo | Add-Member -NotePropertyName cost_center -NotePropertyValue $repoCostCenterProperty -Force

            $currentCostCenter = $costCenters | ? { $_.resources | ? { $_.type -eq "Repo" -and $_.name -eq $repo.full_name } }

            if ( $repo.cost_center -eq "inherit" ) {
                $repo.cost_center = $null
            }

            $targetCostCenter = $null

            if ($repo.cost_center) {
                $targetCostCenter = $costCenters | ? { $_.name -eq $repo.cost_center }
                if (-not $targetCostCenter) {
                    Write-Warning "Costcenter for repository $($repo.full_name) not found."
                }
            }

            if ($null -eq $currentCostCenter) {
                if ($null -ne $targetCostCenter) {
                    Write-Output "Updating costcenter for repository $($repo.full_name) from unassigned to $($targetCostCenter.name)."
                    Update-CostCenterResources -Handles $repo.full_name -ResourceType "Repo" -Action "Add" -CostCenter $targetCostCenter -Enterprise $enterprise
                }
                else {
                    Write-Verbose "Repository $($repo.full_name) does not have a cost center assigned."
                }
            }
            else {
                if ($null -eq $targetCostCenter) {
                    Write-Verbose "Repository $($repo.full_name) does not have a cost center assigned."
                    Write-Output "Updating costcenter for repository $($repo.full_name) from $($currentCostCenter.name) to unassigned."
                    Update-CostCenterResources -Handles $repo.full_name -ResourceType "Repo" -Action "Delete" -CostCenter $currentCostCenter -Enterprise $enterprise
                }
                elseif ($currentCostCenter.id -ne $targetCostCenter.id) {
                    Write-Output "Updating costcenter for repository $($repo.full_name) from $($currentCostCenter.name) to $($targetCostCenter.name)."
                    Update-CostCenterResources -Handles $repo.full_name -ResourceType "Repo" -Action "Delete" -CostCenter $currentCostCenter -Enterprise $enterprise
                    Update-CostCenterResources -Handles $repo.full_name -ResourceType "Repo" -Action "Add" -CostCenter $targetCostCenter -Enterprise $enterprise
                }
            }
        }
    }
}

Unfortunately, you can't simply update the associated Cost Center in a single call, when changing the current value, an add plus delete is required.

I shared the Update-CostCenterResources function in a previous blogpost. GitHub added the ability to set Organizations and Repositories recently, so here is the updated version:

function Update-CostCenterResources {
    param(
        [Parameter(Mandatory = $true)]
        [string[]]$Handles,

        [ValidateSet('User', 'Repo', 'Org')]
        [string]$ResourceType = "User",

        [Parameter(Mandatory = $true)]
        [ValidateSet('Add', 'Delete')]
        [string]$Action,

        [Parameter(Mandatory = $true)]
        $CostCenter,

        [Parameter(Mandatory = $true)]
        [string]$Enterprise
    )

    switch ($Action) {
        'Add' {
            $method = 'POST'
            $Handles = $Handles | Where-Object { 
                $handle = $_
                return (($costCenter.resources | ? { $_.type -eq $ResourceType } | ? { $_.name -eq $handle }).Count -eq 0)
            }
        }
        'Delete' {
            $method = 'DELETE'
            $Handles = $Handles | Where-Object { 
                $handle = $_
                return (($costCenter.resources | ? { $_.type -eq $ResourceType } | ? { $_.name -eq $handle }).Count -gt 0)
            }
        }
    }

    # Call fails when processing too many users at once. Thus batching the calls...
    $count = 0
    do {
        $batch = $Handles | Select-Object -Skip $count -First 30
        $count += $batch.Count

        if ($batch.Count -gt 0) {
            switch ($ResourceType) {
                'User' { 
                    $body = @{
                        users = [string[]]$batch
                    }
                }
                'Org' { 
                    $body = @{
                        organizations = [string[]]$batch
                    }
                }
                'Repo' { 
                    $body = @{
                        repositories = [string[]]$batch
                    }
                }
            }


            $_ = ($body | ConvertTo-Json) | gh api --method $method /enterprises/$Enterprise/settings/billing/cost-centers/$($CostCenter.id)/resource --input -
        }
    } while ($batch.Count -gt 0)
}

All of this runs from a simple GitHub workflow which has access to a Personal Access token with the required permissions to access the Cost Center information as well as sufficient permissions to query all Repositories in all Organizations in the enterprise.

name: Set Costcenters

on:
  schedule:
    - cron: "5 4 * * *"

jobs:
  update:
    permissions: 
      contents: read

    runs-on: ubuntu-latest

    steps:      
    - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
      with:
        fetch-depth: 0

    - name: Assign Costcenters
      run: |
         .\assign-costcenters.ps1
      env:
        GH_TOKEN: ${{ secrets.GH_TOKEN }}
      shell: pwsh

You can find the whole script here:

GitHub Actions learnings from the recent nx hack

Jesse Houwing — Tue, 02 Sep 2025 21:05:46 +0000

You may have seen recent reporting around the compromise of the nx project. A malicious version of their package was published to npmjs which subsequently published GitHub tokens, crypto wallets and other sensitive materials of 1000s of users.

The maintainers of the project have spelled out exactly how the project was compromised. Yet, it doesn't tell exactly what they did to prevent the same issue from happening in the future. Or how you can protect yourself from these attack vectors.

I'll walk through the attack chain and will explain what options are available to you to prevent these things from happening.

Understanding the vulnerable workflow

Let's look at the main cause of this hack. The introduction of a new GitHub Actions Workflow that had the pull_request_target trigger enabled. This trigger is dangerous, since it signals to GitHub that the workflow is safe and has access to secrets. By default only to secrets referenced in the workflow itself, but that always includes the built-in GitHub Actions token.

on:
  pull_request_target:
    types: [opened, edited, synchronize, reopened]

The second stage of the hack was possible due to script injection in that same workflow. This allowed the injection of arbitrary code in the pipeline through the title of the pull request:

      - name: Create PR message file
        run: |
          mkdir -p /tmp
          cat > /tmp/pr-message.txt << 'EOF'
          ${{ github.event.pull_request.title }}

          ${{ github.event.pull_request.body }}
          EOF

The pull request's title and body are written to the script prior to execution, so including EOF in any of them cause any subsequent text to be interpreted as arbitrary bash commands. And adding $(...) would also lead to code execution prior to writing the contents to pr-message.txt.

Since the workflow can now execute arbitrary code, and the GitHub Hosted Runner runs with sudo by default, it would also be possible to write a memory dump of the runner process to gain access to the Runner's GitHub Token.

The payload was probably hidden in the body of the pull request and could have been similar to:

# Break the multi-line content redirection by supplying the defined EOF token.
EOF
# Dump the listener process so we can get access to its secrets

sudo apt-get install -y gdb && \
sudo gcore -o k.dump "$(ps ax | grep 'Runner.Listener' | head -n 1 | awk '{ print $1 }')"

# Then, we grep the memory dump for the format of secret values—per the job data we decrypted in the network analysis:

grep -Eao '"[^"]+":\{"value":"[^"]*","issecret":true\}' k.dump*

# Optionally use CURL to send these secrets to a url you control.

This is a well-known way to exfiltrate secrets from the GitHub Hosted Runner and I speculate it was used to gain access to the GITHUB_TOKEN secret.

Since this token has write-all permissions on older repositories by default, it is possible to either write to an unprotected branch on the repo, or to run other workflows. The publish.yml workflow had the ability to be triggered manually in the master branch. This means that it can be triggered against any branch in the project repo and will use the contents of the workflow in that specific branch. Because the npm token is stored as a repository secret, it will be available to all workflows in the repository, regardless of the branch it's defined in.

In this case I could have used the GitHub Token to:

Create a new branch in the project repository
Overwrite the publish.yml in the new branch to send the npm token to a webserver I control
Trigger the publish workflow against my branch
Delete the branch
Optionally delete the workflow run

Without overwriting the publish.yml, changes in any of the build scripts executed in the publish job would have had access to the token, since it was added as a job level environment variable.

  publish:
    if: ${{ github.repository_owner == 'nrwl' }}
    env:
      GH_TOKEN: ${{ github.token }}
      NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

In the disclosure the project mentions that they have branch rulesets in place to protect their master branch. Without branch protections this would have been even simpler.

Understanding the vulnerable (human) process

This attack couldn't have happened if the workflow above wouldn't have been added, or if it had used the standard pull_request trigger instead. If you read the documentation for pull_request_target, it calls out that its dangerous.

It looks like this workflow was generated by AI:

🤖 Generated with Claude Code

And that none of the humans in the loop were aware of the dangers. Nor was there any tooling in place to scan the workflows and to alert them of the potential problems.

Once the attacker figured out the attack chain, it would have been possible to execute the attack in seconds, with only minimal observable traces. And because all actions would be performed by the github-actions[bot] user, none of the changes would stand out and any commits would even be considered verified if done correctly.

It's trivially easy to setup a repo in which to test this attack without actually having to interact with the nx repo.

Because the NPM token was stored as a repository secret, access to it would not require an explicit approval.

Not related to this attack, but worrying nonetheless. The actions referenced by the nx project are pinned to their major version tag. This is against best practices for GitHub Actions in public repos. For example:

      - uses: actions/checkout@v4

Best practices require each action to be pinned to their commit sha. In this case as far as I can tell none of the referenced actions have a known vulnerability, but this leaves the project open to attacks similar to the tj-actions/changed-files hack from a few months back.

Remediation

Let's break down the remediations that can be put in place:

Change the default permissions for GitHub Actions to read-only.
Set workflow permissions at the job level
Enable branch rulesets to require pull requests for all important branches.
Require Approval for running fork pull request workflows from contributors
Do not rely on pull_request_target.
Do not use workflow_dispatch or repository _dispatch triggers in workflows that have access to critical secrets.
Do not mix pull-request validation and continuous integration workflows
Store critical secrets in environment secrets, instead of repository secrets.
Pass secrets only to the steps that need them
Pass data to script steps using the environment
Pin all actions to their sha.
Enable GitHub Advanced Security Code Scanning for GitHub Actions and add Branch Ruleset.
Enable Dependabot for GitHub Actions to automatically update action references to the latest version.
Consider 3rd party security hardening tools like StepSecurity's Harden Runner.

Each of these remediations is discussed in more detail below.

Change the default permissions for GitHub Actions to read-only

When GitHub Actions was first released, the default permissions were set to write-all. When GitHub changed the default permission to read-all later, they did not retroactively reset the permission for existing accounts and organizations.

GitHub Enterprise Owners and GitHub Organization Owners can enforce a default permission of read-all through the Settings:

Set the default workflow permissions to: Read repository contents and package permissions.

This will break all workflows that do not explicitly declare their required permissions in case they need more than read-all.

While you're at it, you should also turn off Allow GitHub Actions to create and approve pull requests. Allowing this setting offers attackers a way to bypass branch rulesets under certain conditions.

Set workflow permissions at the job level

A workflow can declare its required permissions at the workflow or at the job level. For best security, it's recommended to deny all permissions at the workflow level and explicitly declare permissions for each job specifically:

# Opt out of any default permissions
permissions: {}

jobs:
  build:
    # explicitly request permissions
    permissions:
      contents: read

  deploy:
    permissions:
      contents: read
      packages: write

This also removes write permissions in case the default permissions are set to read and write.

Enable branch rulesets to require pull requests for all important branches.

The nx project reports that they had enabled Branch Rulesets to require a pull request for any changes to enter the master branch and other important branches in their repository.

This prevented attackers from simply pushing a new workflow directly to the main branch to exfiltrate the npm token and other secrets.

Protecting branches in your repository is a straight forward configuration change.

Require a pull request before merging

This is also a prerequisite for other protections, like requiring Code Scanning Results before merging.

There are quite a few policy settings you can enable through Branch Rulesets the following research explains why each of these settings is important and how an attacker might abuse a ruleset with a slightly less strict configuration.

Require approval for running fork pull request workflows from contributors

This is a controversial one. As it adds quite a bit of overhead to project teams receiving many 3rd party contributions. But it might have been enough for a maintainer to detect the script injection attack that was used to exfiltrate the GitHub Actions token.

GitHub Enterprise Owner, Organization Owners and Repository Admins can set a policy to require approval before running any GitHub Actions workflow.

Require approval for all external contributors

While it might feel tempting to set this policy to any of the lighter settings, there are already known cases where a person appears helpful by fixing a few typos. They'll be considered trusted after that.

Setting this policy will require a maintainer to review the Pull Request and trigger the workflows each time an external contributor pushes new commits.

Do not rely on `pull_request_target`

The pull_request_target is inherently dangerous. It opens you up to all kinds of additional attack paths and very few people are completely aware of the implications. It is very easy to make a mistake when you enable this trigger on a workflow and open your repository up for attacks.

My recommendation is to not rely on the pull_request_target trigger. For most scenarios the pull_request trigger is sufficient.

In the enterprises I control we've added a Custom Pattern for Secret Scanning to block all commits that contain pull_request_target:

Define a custom pattern to block pushes

And we enabled this pattern for Push protection:

Enable custom pattern for push protection

While there are cases where pull_request_target can be useful, it's generally not worth the security risks.

From the docs:

⚠️ Warning

For workflows that are triggered by the pull_request_target event, the GITHUB_TOKEN is granted read/write repository permission unless the permissions key is specified and the workflow can access secrets, even when it is triggered from a fork. Although the workflow runs in the context of the base of the pull request, you should make sure that you do not check out, build, or run untrusted code from the pull request with this event. Additionally, any caches share the same scope as the base branch. To help prevent cache poisoning, you should not save the cache if there is a possibility that the cache contents were altered. For more information, see Keeping your GitHub Actions and workflows secure: Preventing pwn requests on the GitHub Security Lab website.

Do not use `workflow_dispatch` or `repository_dispatch` triggers in workflows that have access to critical secrets.

Since workflow_dispatch and repository_dispatch allow an attacker to control the branch against the workflow will run, it can be used to exfiltrate repository secrets if the attacker has write access to any other branch on the repository.

The only way to protect secrets from this attack path, is to store them as an environment secret.

Do not mix pull-request validation and continuous integration workflows

Many developers will try to reduce duplication by combining multiple workflows into 1 and defining multiple triggers or conditions. These conditions then control what code is executed depending on the event that triggered the workflow.

A really common example is to have a single pull request validation workflow that also doubles as the first set of steps of the full CI build.

The publish.yml in the nx project has 3 such purposes:

Release the packages for the nx project whenever a new tag is created
A canary release based on a schedule
Creation of a test-build for pull requests

name: publish

on:
  # Automated schedule - canary releases from master
  schedule:
    - cron: "0 3 * * 2-6" # Tuesdays - Saturdays, at 3am UTC
  # Manual trigger - PR releases or dry-runs (based on workflow inputs)
  workflow_dispatch:
    inputs:
      pr:
        description: "PR Number - If set, a real release will be created for the branch associated with the given PR number. If blank, a dry-run of the currently selected branch will be performed."
        required: false
        type: number
  release:
    types: [published]

The nx project does this by restricting the build and publish jobs to only run when the triggering repository is owned by nrwl.

    build:
      if: ${{ github.repository_owner == 'nrwl' }}

    publish:
      if: ${{ github.repository_owner == 'nrwl' }}

Which would prevent the publish job from running against pull requests from a forked repository.

However, since the leaked token had write-all permission, it would have been easy to publish a new branch which would satisfy the repository_owner condition and thus trigger the publication jobs.

More restrictions could have been added, like checking against the actor to see whether it's a maintainer or preventing workflows triggered by GitHub Actions, or restricting jobs to specific branches. But when the attacker has write permissions to the repository, none of these will likely save you.

A better approach would be to split the workflow into 2 separate workflows:

publish.yaml - which publishes the production packages
pr-validation.yaml - which builds a PR validation package

These two workflows can use a 3rd callable workflow, or one or more Composite actions to reduce code duplication while also reducing chances of an attacker calling a workflow they're not meant to call.

Another advantage of callable workflows and composite actions is that they can be stored in a different repository, limiting an attacker's ability to leverage a leaked Actions Token to change the workflow code.

Store critical secrets in environment secrets, instead of repository secrets.

Secrets defined at the repository level are accessible to all workflows. There is no way to limit access to repository level secrets.

In order to protect a critical secret, such as the NPM token used to publish your package, it should be defined as an Environment Secret.

In the Environment's policy you can then set a branch policy to limit which branches have access to the secret.

Set specific branches to limit access to critical secrets.

Optionally add Required Reviewers to prevent access to the secrets without manual approval:

Add Required Reviewers to prevent access to the secrets without manual approval

Then update the workflow to reference the environment when defining the job:

jobs:
  deploy:
    environment: public
    steps:
      uses: actions/deploy@sha
      with:
        npm_token: ${{ secrets.NPM_TOKEN }}

It looks like the nx project has already implemented this in a recent change.

Pass secrets only to the steps that need them

In the nx repository the NPM_AUTH_TOKEN is defined at the job level:

  publish:
    env:
      # DO NOT USE THIS!
      GH_TOKEN: ${{ github.token }}
      NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
    steps:
      - ...

This makes the secrets available to all steps in the workflow. If the attacker can take control of any of the code executed in these steps, they would be able to exfiltrate the secret.

In this case the job also calls pnpm build:wasm which adds a number of additional places to hide exfiltration code.

It's better to explicitly pass the secrets only to the steps that need them:

  publish:
    env:
      # NO SECRETS SET HERE
    steps:
      - ...
      - ...
      - name: Publish
        env:
          VERSION: ${{ needs.resolve-required-data.outputs.version }}
          DRY_RUN: ${{ needs.resolve-required-data.outputs.dry_run_flag }}
          PUBLISH_BRANCH: ${{ needs.resolve-required-data.outputs.publish_branch }}
          NX_VERBOSE_LOGGING: true
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
        run: |
          echo ""
          # Create and check out the publish branch
          git checkout -b $PUBLISH_BRANCH
          echo ""
          echo "Version set to: $VERSION"
          echo "DRY_RUN set to: $DRY_RUN"
          echo ""
          pnpm nx-release --local=false $VERSION $DRY_RUN

Pass data to script steps using the environment

The Script Injection attack was possible because variables were in-lined in the body of a script step:

      - name: Create PR message file
        run: |
          mkdir -p /tmp
          cat > /tmp/pr-message.txt << 'EOF'
          ${{ github.event.pull_request.title }} <- unsafe variable reference

          ${{ github.event.pull_request.body }} <- unsafe variable reference
          EOF

To pass data safely to scripts, you should instead pass them through the environment:

      - name: Create PR message file
        run: |
          mkdir -p /tmp
          echo $PR_TITLE >> /tmp/pr-message.txt
          echo "" >> /tmp/pr-message.txt
          echo $PR_BODY >> /tmp/pr-message.txt
        env:
          PR_TITLE: ${{ github.event.pull_request.title }}
          PR_BODY: ${{ github.event.pull_request.body }}

Script Injection can be detected by GitHub Advanced Security Code Scanning for Actions as well as ActionLint.

Pin all actions to their sha.

The attack on nx doesn't seem to have relied on this attack vector, but the recent attack on tj-actions/changed-files has shown again that it's dangerous to rely on only the tag to reference actions of 3rd parties.

GitHub recommends that you always reference actions you do not control using their explicit full commit sha.

For example:

      - uses: actions/checkout@v4

Should have used:

    - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5

There is now a policy which requires all actions to be pinned to their commit sha:

Require actions to be pinned to a full-length commit SHA

Because this is a new policy, GitHub Administrators must explicitly opt-in to this policy.

There are several tools available to perform the pinning on your behalf:

Once, pinned, if enabled, Dependabot will automatically submit a pull request with the correct new commit sha. Unfortunately, it doesn't automatically change the tag v4 to its sha.

Enable GitHub Advanced Security Code Scanning for GitHub Actions and add Branch Ruleset.

The script injection vulnerability at the root of this attack would have been detected by GitHub Advanced Security Code Scanning for GitHub Actions.

GitHub Advanced Security is free for all public repos, so there is not really a good reason not to enable it.

Because support for GitHub Actions was added recently, GitHub Repository Owners must explicitly enable GitHub Actions in the Code Scanning configuration:

Enable GitHub Actions for CodeQL Code Scanning

Or add actions to the list of supported languages in their workflow:

jobs:
  analyze:
    name: Analyze (${{ matrix.language }})
    runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
    permissions:
      security-events: write
      packages: read
      actions: read
      contents: read

    strategy:
      fail-fast: false
      matrix:
        include:
        # Make sure Actions is added to the list of languages:
        - language: actions
          build-mode: none

This will detect a number of GitHub Actions vulnerabilities, including:

Script Injection
Unsafe checkout

In order to prevent workflows with these vulnerabilities to enter the repository, add a branch ruleset that requires the Code QL workflow to succeed:

Enable Require status checks to pass and add the CodeQL check explicitly

And require Code Scanning results:

Enable Require code scanning results and add CodeQL explicitly.

An alternative for GitHub Advanced Security Code Scanning for Actions that would also have detected the Script Injection vulnerability is ActionLint. Actionlint can also be added as a workflow.

Enable Dependabot for GitHub Actions to automatically update action references to the latest version.

To automatically keep your referenced GitHub Actions up to date, especially when you are pinning to the full commit sha, you can enable Dependabot to automatically create a pull request whenever an action is updated.

To enable Dependabot for GitHub Actions, explicitly add github-actions to your .github/dependabot.yml:

version: 2
updates:
  - package-ecosystem: "github-actions"
    directory: "/" # Location of package manifests
    schedule:
      interval: "weekly"

To reduce maintainer burden, GitHub Actions can be used to automate updates completely.

Consider 3rd party security hardening tools like StepSecurity's Harden Runner.

All of the above remediations are built-in features of the GitHub platform.

There are 3rd party solutions available to further secure GitHub Actions. Of those StepSecurity's Harden Runner is probably the best-known option. You can add harden-runner to any workflow by adding a single step to your workflow jobs:

steps:
  - name: Harden Runner
    uses: step-security/harden-runner@c6295a65d1254861815972266d5933fd6e532bdf # v2.11.1
    with:
      egress-policy: audit

The Community version of Harden runner offers the following protections to all of workflows:

CI/CD-Aware Event Correlation: Each outbound network connection, file operation, and process execution is mapped to the exact step, job, and workflow where it occurs.
Automated Baseline Creation: Harden-Runner builds a baseline for each job based on past outbound network connections.
Anomaly Detection: Once the baseline is created, any future outbound calls not in the baseline trigger a detection.
Block Network Egress Traffic with Domain Allowlist: Optionally use the automatically created baseline to control outbound network traffic by specifying allowed domains, preventing unauthorized data exfiltration.
Detect Modification of Source Code: Monitor and alert on unauthorized changes to your source code during the CI/CD pipeline.

Conclusion

GitHub offers many features to protect your project against attacks that try to abuse GitHub Actions. Unfortunately, many of these features require explicit opt-in or configuration.

This should not be an excuse to leave your open-source projects open to attack.

Dear GitHub,

We love the unlimited free environment and all the tools you provide to keep our open-source projects free and healthy. And while we love your commitment to not break our workflows, it's not helpful to make many of the protections you offer opt-in.

Please, please, please start enabling core security features automatically, at least on popular open-source projects as part of your roll-out strategy.

The new GitHub Advanced Security Code Scanning for GitHub Actions would have alerted the maintainers of the nx project to the Script Injection vulnerability. But it isn't activated by default, nor are maintainers warned they must opt-in to receive this protection.

The long ago changed default read-only permissions for GitHub Actions would have prevented the triggering of the publish workflow, but this setting is kept to the organization default, and maintainers aren't warned they should change this setting.

Requiring workflows triggered by external contributors to always be approved by a maintainer would have at least triggered a human intervention as part of the attack chain. But this new default has also not been rolled out to existing organizations, nor are maintainers warned they should really change this setting.

Pinning Actions to their full commit sha has been a recommendation for years now. Yet GitHub has not provided any tooling to make this process easier, and while the policy has finally arrived, again it has rolled out with relatively low fanfare and maintainers are not warned they should really change this setting. Nor does Dependabot or GitHub Advanced Security Code Scanning for Actions submit a pull request to pin these actions.

This hasn't been the first attack to leverage the pull_request_target trigger, I'd wish workflows with this trigger would not automatically enable and would require explicit enablement by an owner through a danger-zone action which clearly spells out the risks.

So again, please, start enabling these core security features automatically and make such roll-out part of the introduction of new features such as these.

Authenticate Connect-MgGraph using OIDC in GitHub Actions

Jesse Houwing — Tue, 10 Jun 2025 15:08:58 +0000

I'm running a number of maintenance scripts against our Azure EntraId to manage GitHub related things. Removing dormant users, asking users to setup their notification email correctly etc.

For a long time, I ran these scripts with an interactive session, before moving them over to GitHub Actions. Recently I made the move to convert my scripts to a workflow and changed the authentication mechanism to OIDC-connect, in order to remove the need to manage secrets and tokens.

The documentation is quite clear on how to set things up.

Unfortunately, while it explains how to setup an Azure PowerShell session using enable-AzPSSession: true, it doesn't explain how to authenticate using connect-mggraph.

After some experimentation (and of course some help from GitHub Copilot), I figured out I could query the access token from az, then pass that straight to connect-mggraph. So now my workflow looks like this:

- name: Azure CLI Login
  uses: azure/login@v2
    with:
      client-id: ${{ secrets.AZURE_CLIENT_ID }}
      tenant-id: ${{ secrets.AZURE_TENANT_ID }}
      allow-no-subscriptions: true

  - name: Assign Costcenters
    run: |
       $accessToken = az account get-access-token --resource https://graph.microsoft.com `
         --query accessToken --output tsv
       write-host "::add-mask::$accessToken"
       $accessToken = $accessToken | ConvertTo-SecureString -AsPlainText -Force
       Connect-MgGraph -AccessToken $accessToken -NoWelcome

       # rest of the script here ... 
    shell: pwsh

And my scripts can happily query the Microsoft Graph.

Picking the right GitHub Copilot model

Jesse Houwing — Fri, 25 Apr 2025 09:24:19 +0000

GitHub recently released documentation on all the large language models available in GitHub Copilot. The list is ever expanding, especially when you have previews turned on for your account. But it's still hard for many people to pick the right model for their task.

You can find the official guidance in the GitHub Documentation.

I wanted to make model selection even easier, so I asked GitHub Copilot to read the doc for me and suggest the right model:

#fetch https://docs.github.com/en/copilot/using-github-copilot/ai-models/choosing-the-right-ai-model-for-your-task

What model would I use to plan a large refactoring involving multiple files for which I do not yet have a working example?

Select the best 2 and explain the pros and cons between them

And it actually came back with a pretty good suggestion:

To plan a large refactoring involving multiple files without a working example, the two best models to consider are GPT-4.5 and GPT-4o. Here's a comparison of their pros and cons:

GPT-4.5

Pros:

Deep Reasoning and Contextual Understanding: GPT-4.5 excels at handling complex logic and multi-step reasoning, making it ideal for large-scale refactoring tasks.
Cross-File Context: It can retain and process context across multiple files, which is crucial for understanding dependencies and relationships in a large codebase.
Reliable and Consistent Results: Produces high-quality, structured suggestions for improving maintainability and modularity.
Nuanced Problem-Solving: Handles ambiguous or incomplete requirements effectively, which is helpful when no working example exists.

Cons:

Higher Latency: Slightly slower response times compared to smaller models like GPT-4o.
Higher Cost: More resource-intensive, which may not be ideal for cost-sensitive scenarios.

GPT-4o

Pros:

Faster Response Times: Provides quicker iterations, which is beneficial for rapid back-and-forth during refactoring.
Balanced Performance: Offers a good mix of reasoning and speed, making it suitable for lightweight tasks or smaller refactoring steps.
Cost-Effective: Less expensive than GPT-4.5, making it a better choice for budget-conscious projects.

Cons:

Limited Depth for Complex Tasks: While capable, it may struggle with deeply nuanced or highly interconnected refactoring tasks compared to GPT-4.5.
Reduced Cross-File Context: May not handle dependencies across multiple files as effectively as GPT-4.5.

Recommendation:

Use GPT-4.5 if the refactoring involves complex logic, dependencies across multiple files, or requires high-quality, structured suggestions.
Use GPT-4o if you need faster iterations, are working on smaller or less complex refactoring tasks, or are constrained by cost.

Both models are strong candidates, but the choice depends on the complexity of the refactoring and your priorities (speed vs. depth).

Say goodbye to your Personal Access Tokens

Jesse Houwing — Tue, 04 Mar 2025 12:30:04 +0000

We got rid of all Azure DevOps PAT usage and so should you.

📢 Big shout out to Jesse!

Without his blog post and direct help, I was probably still renewing expired PATs manually. But let's start at the beginning.

Michael, author of some of my favorite Azure DevOps Extensions, reached out on the Azure DevOps Club slack channel for help. He wanted to automatically rotate Personal Access Tokens to integrate with Azure DevOps, which lead to a quest to completely eradicate Personal Access Tokens altogether. After sharing my work to make the Azure DevOps Extension Tasks work without Personal Access Tokens, Michael went to work. I've asked him to share his results in this guest blog, since I feel they're useful for anyone doing automation against Azure DevOps. As you can see he agreed!

Jesse Houwing

PATs, PATs and more PATs - our 5 scenarios we used PATs for

Building Bravo Notes here at Agile Extensions we are extensively using the Azure DevOps REST APIs in many ways.

And I'm not talking about the API calls the Bravo Notes extension makes at runtime e.g. to load work items, publish wiki pages etc.

We use the Azure DevOps REST APIs in these 5 scenarios as well:

publish multiple versions of the Bravo Notes extension for development, staging and production via the marketplace APIs (in Azure Pipelines)
retrieve marketplace event data for installs/uninstalls and more on a schedule (in Azure Functions app)
run integration tests for many Bravo Notes components that need Azure DevOps APIs (in Azure Pipelines)
run e2e tests of the main Bravo Notes app outside of Azure DevOps (in Azure Pipelines)
run integration and e2e tests locally

In the past we used personal access tokens (PATs) for all of those scenarios.

For years our process was:

watch for notifications that a PAT was about to expire
be annoyed that this manual regular chore was on your plate (AGAIN!)
regenerate PAT in Azure DevOps
update pipeline variables and secrets with the new PAT
feel bad and wonder why there wasn't a more elegant solution
be done and push responsibility for improving this to future Michael

PAT REST APIs to the rescue?

A while I ago a new set of APIs got introduced to manage PATs. That sounded intriguing and a few weeks back I reached out to the community to ask for guidance on how to automatically renew and manage PATs and do away with the manual process entirely.

🤔 Me: Is there a good resource online on how to transition from manually rotating PATs to rotating PATs automatically (a pipeline using PAT REST API, KeyVault)?

Thankfully Jesse pointed me to a better (the right) way:

💡 Don't use PATs but use an Azure Service principle with workload Federation.

When you are reading this on his blog you probably already know that Jesse is one of those people who uses the internet for what it was build for:

He shares what he has learned (often the hard way) to make the lives of all of us easier. In this case in form of this blog post on how to publish marketplace extensions without using PATs.

I was convinced already that using PATs was less than ideal.

But at this point I was still hesitant about whether this approach could help us do away with PATs entirely. After all we had 5 different scenarios.

There was only one way to find out so we started by tackling the scenario that Jesse wrote about in his blog post.

Scenario 1: Publish marketplace extensions without a PAT

The blog post didn't disappoint as it contained all the pieces to help us publish marketplace extensions without a PAT - whohoo 🥳!

Below are the steps we needed to take. Please refer to Jesse's blog post for all the datails. Our mileage varied a bit as the ARM service connection now uses an "App registration" in Azure instead of a "Service Principal".

The process is the still the same in general, though.

Create the ARM service connection (the UX changed a bit since Jesse's blog post)
- Choose "Azure Resource Manager" Connection
- Identity Type: "App registration (automatic)"
- Credential: "Workload identity federation"
- the rest is about the same
Add the App registration/service principal as a user in Azure DevOps
Extract the Azure DevOps Identity Id from the Profile API via a pipeline task
Use that Id to add the App registration as a member to the marketplace publisher
Update our pipeline to use the ARM service connection

Note that we could use the newest version of the marketplace extension tasks (v5) that allows you to publish using the ARM service connection directly.

So there is no need to fetch an access token and overwrite the marketplace service connection credentials anymore. After changing our pipelines we were able to delete our old "Visual Studio Marketplace" service connection.

- task: PublishAzureDevOpsExtension@5
  displayName: 'Publish Extension'
  inputs:
    connectTo: 'AzureRM'
    connectedServiceNameAzureRM: 'marketplace-service-connection'
    fileType: vsix
    vsixFile: '$(Pipeline.Workspace)/vsix/production.vsix'
    updateTasksVersion: false

And that was it. The pipeline worked as before and we could revoke the PAT for publishing extensions to the marketplace.

Of course we didn't want to stop here and tackle the next scenario and remove one more PAT.

Scenario 2: Call Azure DevOps APIs from a Azure Function app

We maintain an Azure Function app to query data about install and uninstall events and pull statistics from the marketplace. Authentication works using - what else could it be - a PAT.

In Azure Functions of course we don't have the magic of service connections that we can utilize in Azure Pipelines.

After initially failing to find the code to use for our NodeJS based function app, searching for the magic GUID 499b84ac-1321-427f-aa17-267ca6975798 that represents Azure DevOps led me right to what we needed to acquire an access token inside the Function app.

After finding the right way to acquire the access token, we got this solved in 2 simple steps:

Add the Azure Function app as a user in Azure DevOps
Use the code below to acquire an access token and authorize Azure DevOps REST API calls

const { ManagedIdentityCredential } = require("@azure/identity");

async function getMarketplaceAccessToken() {
    const credential = new ManagedIdentityCredential({
        // `AZURE_CLIENT_ID` points to the App registration created via the ARM service connection
        clientId: process.env.AZURE_CLIENT_ID,
    });
    const tokenResponse = await credential.getToken(
        "499b84ac-1321-427f-aa17-267ca6975798/.default",
        {
            tenantId: process.env.AZURE_TENANT_ID,
        }
    );

    return tokenResponse.token;
}

There was no stopping now - but still 3 PATs to go.

Scenario 3: Run intergration tests in Azure Pipelines

Next up was our integration testing pipeline tasks that run integration tests using NodeJS and jest. Until now a PAT was used from a secret pipeline variable. Another one that we didn't want to renew manually every few months.

As we already acquired an access token in a pipeline via an ARM service connection in scenario 1, this dind't seem to difficult to pull off.

Following the least priviledge principle we didn't want to use the same service principal from scenario 1. This would have given it access to the marketplace AND Azure DevOps resources that were needed for our integration tests.

Luckily you can create as many ARM service connections as you want and so we did. To use the newly created App registration/service principal, the pipeline task from Jesse's blog post to retrieve an access token that we dind't need for scenario 1 now came in handy again:

- task: AzureCLI@2
  displayName: 'Accquire token for integration testing'
  inputs:
    azureSubscription: 'azure-devops-integration-testing-connection'
    scriptType: 'pscore'
    scriptLocation: 'inlineScript'
    useGlobalConfig: true
    inlineScript: |
      $accessToken = az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 --query "accessToken" --output tsv
      write-host "##vso[task.setsecret]$accessToken"
      write-host "##vso[task.setvariable variable=SECRET_INGETRATION_TESTING_TOKEN;issecret=true]$accessToken"

Now we could use the the token for subsequent integration test tasks. Yay only 2 PATs to go!

Scenario 4: Run e2e tests in Azure Pipelines (against external Azure DevOps organization)

While this scenario seemed to be exactly the same es scenario 3 at first, we had an issue here that stopped us in our tracks at first.

The e2e tests called APIs in a separate Azure DevOps org that only existed for testing.

[OrgA Pipeline] ---- REST API call ----> [OrgB Work item API]

The solution was too simple to be true.

We were just able to add the App registration/service principal as a user in OrgB and give it access to the resources needed for the e2e tests.

Another PAT down!

Scenario 5: Running integration and e2e tests locally

This might be my least favorite solution so far, but it was simple and did the trick for now.

We use jest to run our tests and in our npm scripts we just stuck in a Azure Developer CLI call to log in prior to starting the test script.

In our code to set up API authentication we now run some special code to acquire the access token, again with the azure identity SDK.

Modify testing script to log in - azd auth login && npx jest --config jest.integration.config.js"
Retrieve the access token from the Azure Developer Cli:

let token = process.env.SECRET_INGETRATION_TESTING_TOKEN;
if (!token && !process.env.CI) {
    const credential = new AzureDeveloperCliCredential({
        tenantId: process.env.AZURE_TENANT_ID,
    });
    const tokenResult = await credential.getToken(
        '499b84ac-1321-427f-aa17-267ca6975798/.default',
        {
            tenantId: process.env.AZURE_TENANT_ID,
        },
    );
    token = tokenResult.token;
}

(Would love to hear about a better solution to securly get an access token for running scripts locally in a development environment.)

All 5 scenarios done! The champaign 🍾 was almost open when I revoked all PATs that were not needed anymore when I discovered that I had forgotten one last PAT.

Bonus scenario: the last token - the renovate pipline

We use renovate to update dependencies and automatically create pull requests in yet another pipeline. (You can thank Jesse for his work online here as well).

Immideately I had a bad feeling and I thought that renovate would only accept PATs as it only accepts RENOVATE_TOKEN as a means to authenticate. The docs make it look that way for sure.

Looking at the code (yay open source) I could see that renovate automatically detects whether the token is a PAT or a OAuth token and chooses the Authorization header (Basic or Bearer) accordingly.

So we could just use the same AzureCLI pipeline task as in scenarios 3 to acquire and 4 and be done!

P.S.: Talking to Jesse about that he mentioned that Azure DevOps now accepts OAuth tokens when using the Basic authentication scheme as well which is great in my opinion.

Wrapping up

Just reading all the terms involved:

ARM service connection
Open Id Connect
Workload Identity federation
service principal
App registration

made the whole process seem daunting to me.

But in the end we got rid of all our PATs within a couple of hours.

Personal access tokens are great to quickly test an API call, authenticate a throw-away script where you don't have other means of authentication ready.

However for the long term you really don't want to have to create PATs and renew them until the end of time.

Thanks again to Jesse and Joost Voskuil for their help.

In case you are facing your own challenge with Azure DevOps or GitHub, don't hesitate to join the Azure DevOps Club.

GitHub's new Billing - Assigning Cost Centers in Bulk

Jesse Houwing — Thu, 20 Feb 2025 14:10:17 +0000

You can assign resources to cost centers, which will in turn charge any costs associated with that resource to that cost center.

Examples of ressource categories are:

Repositories - All costs associated with a specific repository. Action Minutes, Codespaces, LFS Storage etc.
Users - All costs associated with a specific user. Enterprise Seat, Copilot, Advanced Security.
Organizations - All costs associated with a specific organization. Costs generated by any unassigned repositories in the organization. GitHub Packages storage and network traffic.
Enterprise - All costs associated with the whole enterprise. Costs associated with users who are not explicitly assigned to a cost center.

By linking organizations, users and repositories to a cost center, you can cause their specific costs to be redirected from the Enterprise to the specific Cost Center.

Linking Organizations and Repositories can be done through the Cost Center UI in the GitHub Enterprise Admin portal:

In the Enterprise Admin portal, you can assign Organizations and Repositories to Cost Centers. Members can only be assigned through the API.

If you're lucky there are only a few Organizations that need to be assigned, but in our case, Members were a whole other story. We already have 500+ members and external collaborators in our enterprise and these needed to be assigned to specific cost centers. As you can see, this can't be done in the UI, but requires API calls.

It turns out the API is a bit tricky, there are certain things about the API that aren't (yet) nicely documented:

You can only assign users to a cost center in batches of 50 (confirmed with GitHub Support)
You can only assign a user if it isn't already assigned to another cost center. Changing cost centers requires 2 operations.
The API is eventually consistent. Changes made may take a couple of seconds to show up in subsequent API calls.
While you could assign cost centers on a per-user basis, this is a great way to burn through your API Rate Limits quickly.

To work around these issues, I've written a small wrapper around the API in PowerShell which uses the existing resource assignments stored in the Cost Centers to reduce the number of required calls to the API:

function Update-CostCenterResources {
    param(
        [Parameter(Mandatory=$true)]
        [string[]]$Handles,

        [Parameter(Mandatory=$true)]
        [ValidateSet('Add','Delete')]
        [string]$Action,

        [Parameter(Mandatory=$true)]
        $CostCenter,

        [Parameter(Mandatory=$true)]
        [string]$Enterprise
    )

    switch ($Action)
    {
        'Add' {
            $method = 'POST'
            $Handles = $Handles | Where-Object { 
                $handle = $_
                return (($costCenter.resources | ?{ $_.type -eq "User" } | ?{$_.name -eq $handle }).Count -eq 0)
            }
        }
        'Delete' {
            $method = 'DELETE'
            $Handles = $Handles | Where-Object { 
                $handle = $_
                return (($costCenter.resources | ?{ $_.type -eq "User" } | ?{$_.name -eq $handle }).Count -gt 0)
            }
        }
    }

    # Call fails when processing too many users at once. Thus batching the calls...
    $count = 0
    do {
        $batch = $Handles | Select-Object -Skip $count -First 50
        $count += $batch.Count

        if ($batch.Count -gt 0) {
            $body = @{
                users = [string[]]$batch
            }

            $_ = ($body | ConvertTo-Json) | gh api --method $method /enterprises/$Enterprise/settings/billing/cost-centers/$($CostCenter.id)/resource --input -
        }
    } while ($batch.Count -gt 0)
}

$enterprise = "xebia"
$costCenters = (invoke-gh -fromJson -- api /enterprises/$enterprise/settings/billing/cost-centers).costCenters

$costCenterNL = $costCenters | ?{ $_.name -eq "Netherlands" }

$handles = @("jessehouwing", "jessehouwing-demo")

# First remove the users from their currently assigned cost centers (if any)
$costCenters | 
  ?{ $_.id -ne $costCenterNL.id } | 
  ?{ $_ | ?{ $.resources | ?{ $_.type -eq "User" -and $_.name -in $handles } } | 
  %{ 
    Update-CostCenterResources -handles $handles -action "Delete" -CostCenter $_ -Enterprise $enterprise
  }

# Then assign the users to their new cost center
Update-CostCenterResources -handles $handles -action "Add" -CostCenter $costCenterNL -Enterprise $enterprise

You can find the current cost center for a user in the Cost Center's Resources array, or from the GitHub Enterprise assigned-seats REST API:

$handle = "jessehouwing"
$enterprise = "xebia"

# retrieve from Cost Center API
$costCenters = (gh api /enterprises/$enterprise/settings/billing/cost-centers | ConvertFrom-Json).costCenters
$currentCostCenter = $costCenters | ?{ $_.resources | ?{ $_.type -eq "User" -and $_.name -eq $handle } }

# retrieve from Assigned Seats API:
$enterpriseUsers = gh api https://api.github.com/enterprises/$enterprise/consumed-licenses --jq '.users[]' --paginate | ConvertFrom-Json
$currentCostCenter = ($enterpriseUsers | ?{ $_.github_com_login -eq $handle }).github_com_cost_center

By combining the above logic with some more proprietary magic, we were able to quickly assign all our members to Cost Centers based on their Azure EntraID metadata.

As you can see in the chart below, when we assigned out cost centers on January 9th, all future costs were no longer associated to the enterprise (in green), but to the respective cost centers.

Costs distributed between Enterprise and a number of Cost Centers as of January 9th.

Note: It is currently not possible to retroactively assign a resource to a cost center. Otherwise, we'd have assigned all costs as of the 1st of the month.

Use inline completions for more answers with GitHub Copilot Free

Jesse Houwing — Mon, 10 Feb 2025 16:03:31 +0000

2 months ago, GitHub gave everybody their taste of GitHub Copilot with the limited free tier. You get 2000 completions and 50 chats.

This is awesome! But 50 chats aren't that many.

Did you know you can use your completions for simple chat answers as well? And this is a nice tip for NeoVim users as well who don't have access to Chat in their editor. Inline Suggestions are significantly faster than Chat to boot.

You can use inline completions to get tips, answer questions and suggest improvements to your code as well.

Q&A

// Q: Explain the key usage scenarios for the Singleton pattern
// A: The singleton pattern |

In most cases typing // A: will do the trick, but sometimes you'll need to give Copilot a starting sentence to provide you with an answer:

// Q: Explain the key usage scenarios for the Singleton pattern. A detailed description
// A: The Singleton pattern is used when we want to ensure that a class has only one instance and provide a global point of access to that instance. Some key usage scenarios for the Singleton pattern include:

If you type // - on the next line or open the completions panel (ctrl-enter), you'll get more details:

// 1. Global Access: It provides a single point of access to the instance of the class, allowing other objects to easily access and use its functionality.
// 2. Resource Management: It can be used to manage shared resources, such as database connections, file systems, or configuration settings, ensuring that there is only one instance to avoid conflicts or resource wastage.
// 3. Caching: It can be used to implement a cache that stores frequently used data or objects in memory, reducing the need to recreate or fetch them from external sources.
// 4. Logging: It can be used to create a single logger instance that logs messages from different parts of the application, centralizing the logging functionality.
// 5. Configuration Settings: It can be used to manage application-wide configuration settings, ensuring that changes are reflected consistently across the application.
// 6. State Management: It can be used to maintain a shared state or context across different parts of the application, allowing objects to share and update state information.
// 7. Thread Safety: It can be used to provide a thread-safe way to access and modify shared resources or data, ensuring that concurrent access does

Suggest test cases

You can also ask questions about the contents of the current file:

// Suggest additional test cases to add to this test fixture, signatures only:
// |

Returns the following suggestions:

// public void GetById_WhenFlightDoesNotExist_ReturnsNull()
// public void Add_WhenFlightIsNull_DoesNotAddFlightToRepository()
// public void Add_WhenFlightHasNullFlightNumber_ThrowsException()
// public void Add_WhenFlightHasEmptyFlightNumber_ThrowsException()
// public void Add_WhenFlightHasNullOrigin_ThrowsException()
// public void Add_WhenFlightHasEmptyOrigin_ThrowsException()
// public void Add_WhenFlightHasNullDestination_ThrowsException()
// public void Add_WhenFlightHasEmptyDestination_ThrowsException()
// public void Add_WhenFlightHasDepartureTimeInThePast_ThrowsException()
// public void Add_WhenFlightHasArrivalTimeBeforeDepartureTime_ThrowsException()
// public void Add_WhenFlightHasFuelRangeLessThanZero_ThrowsException()
// public void Add_WhenFlightHasFuelTankLeakTrue_ThrowsException()
// public void Add_WhenFlightHasNullFlightLogSignature_ThrowsException()
// public void Add_WhenFlightHasEmptyFlightLogSignature_ThrowsException()
// public void Add_WhenFlightHasNullAerobaticSequenceSignature_ThrowsException()
// public void Add_WhenFlightHasEmptyAerobaticSequenceSignature_ThrowsException()
// public void Add_WhenFlightHasDuplicateId_ThrowsException()
// public void Add_WhenFlightHasDuplicateFlightNumber_ThrowsException()
// public void Add_WhenFlightHasDuplicateOriginAndDestination_ThrowsException()
// public void Add_WhenFlightHasDuplicateFlightLogSignature_ThrowsException()
// public void Add_WhenFlightHasDuplicateAerobaticSequenceSignature_ThrowsException()
// public void Add_WhenFlightHasDuplicateFlightNumberAndOriginAndDestination_ThrowsException()
// public void Add_WhenFlightHasDuplicateFlightNumberAndFlightLogSignature_ThrowsException()
// public void Add_WhenFlightHasDuplicateFlightNumberAndAerobaticSequenceSignature_ThrowsException()
// public void Add_WhenFlightHasDuplicateFlightNumberAndOriginAndDestinationAndFlightLogSignature_ThrowsException()
// public void Add_WhenFlightHasDuplicateFlightNumberAndOriginAndDestinationAndAerobaticSequenceSignature_ThrowsException()
// public void Add_WhenFlightHasDuplicateFlightNumberAndFlightLogSignatureAndAerobaticSequenceSignature_ThrowsException()

Suggest improvements

You can ask for improvements you can make to the current code file:

// To improve the security of this REST API I can:
// |

And GitHub Copilot will reply with:

// add an authorization filter to this controller
// and require authentication for all actions
// using the [Authorize] attribute

You can tweak the replies by adding // and or //something else I could do is

Conclusion

If you run out of Chat messages, you can continue the conversation using inline suggestions. If you run out of those too, you seem to be getting enough value from GitHub Copilot to warrant a subscription ;).

This also adds a bit of Chat support to NeoVim, Qt Explorer, XCode and other tools that only support inline suggestions for now.

Troubleshooting GitHub Copilot keyboard shortcuts in Jetbrains IDEs

Jesse Houwing — Wed, 22 Jan 2025 18:39:29 +0000

Over the past months I've delivered 100s of GitHub Copilot workshops. Ranging from 2 hour introduction talks, where I quickly walk through the main features of the product, up to day-long workshops with interactive hands-on.

For most people it's pretty easy to get started. In Visual Studio Code you install a plugin, in Visual Studio 2022 it's installed by default, but in other IDEs it can be a bit trickier to get GitHub Copilot to work.

In my experience the different Jetbrains IDEs are among the trickiest. This includes IntelliJ, Rider, Pycharm, Webstorm, RubyMine and more. So far I've found two main problems that break GitHub Copilot.

Keyboard shortcut is already assigned to another command

The Jetbrains IDEs, for many users, are an alternative IDE which at some point in their career replaced another product. Because of that there are many predefined keymaps available. For example, Rider offers the following:

IntelliJ
Emacs
Resharper
Sublime Text
Visual Assist
Visual Studio
Visual Studio 2022
Visual Studio Code

Plugins that install themselves in these IDEs need to find a keyboard shortcut which isn't already in use, and among all of these different layouts that leaves very few easily accessible keys available.

You can see whether an assigned keyboard shortcut is conflicting with other assigned commands by double-clicking the command in the keymap and picking Add keyboard shortcut from the context menu.

See whether an assigned keyboard shortcut is conflicting with other assigned commands by double-clicking the command in the keymap and picking Add keyboard shortcut from the context menu

For example, the default alt+\ keyboard shortcut is used by GitHub Copilot to Show Completions, but in the Visual Studio 2022 keymap, which is my preference, that shortcut is already assigned to 3 other commands:

alt+\ is already assigned to 3 other commands

In order to use GitHub Copilot to its fullest potential, you need to go through each of its keyboard shortcuts and either assign a free key combination or remove the other assigned actions by clicking ok and then choosing to remove the other assigned actions:

Remove other assigned actions to allow GitHub Copilot to use the keyboard shortcut

For now, I haven't found a better solution. On the plus side, this will allow me to assign the same keyboard shortcuts I use in Visual Studio 2022 or Visual Studio Code so that I don't need to juggle even more shortcuts in my head.

Conflict between GitHub Copilot and Jetbrains' bundled line completion

Completing a suggestion using tab might not work when GitHub Copilot and Jetbrains' bundled Full line code completion are enabled at the same time. Work is clearly underway to let these two coexist and work together.

Yet I've encountered issues when GitHub Copilot and Jetbrains' Inline completion are both turned on. Under normal circumstances the two features work it out among eachother, but when connecting to a remote IDE using ssh (using GitHub Codespaces for example) the assigned keyboard shortcuts may fail.

This is either fixed by ensuring GitHub Copilot and Full line code completion are both installed locally as well as remotely and are both up to date. In case tab completion still fails, disabling the Full line code completion plugin both locally and remotely usually solves the problem.

Disable the "Full line completion" plugin in case GitHub Copilot can't tab complete.

The use or uselessness of signed commits

Jesse Houwing — Wed, 05 Jun 2024 12:59:00 +0000

This is why it's possible to sign a commit. Signing a commit adds cryptographic proof of your identity using a public/private key verification. Commits can be signed using GPG, SSH or S/MIME from your workstation and GitHub can sign commits you make on the web "on your behalf" (it basically signs it with GitHub's own key).

I'll use GitHub as an example hosting platform for your git repositories, but conceptually this applies to Git in general and all other Git hosting platforms.

To verify your identity on GitHub, you need to upload your keys, so they can verify you are who you are. If done correctly, you'll see a small badge☑️ / (verified) next to a commit that is signed this way.

The cryptographic signature is part of the commit, so when a repository is transferred from one machine to another or one hosting provider to another, you should still be able to verify the repo is "intact" if you have a copy of all the public keys of the contributors. This is why commit-signing works agnostic to the repository host. This is also why it's so hard to do it right.

"Great!" You might say. And many experts will confirm this on the web, in conference talks and books, including a few of my own colleagues. But if you read through these blogs, you'll see immediately that it isn't simple to set up.

And, yes, it's great... In a way. And it's also not great. Let me explain.

What does the `verified` badge mean?

A commit that shows the verified badge will tell you the following:

✅ The commit was signed

✅ The signature is correct

✅ The signature matches the email used in the commit

✅ The key used to sign is known by GitHub

✅ They key and email are owned by the account that's shown (in this case octocat)

✅ Whether the contents of the commit (still) match the signature

✅ Potentially: That the key is owned/derived/trusted by an organization or corporation.

✅ The commit was likely made by the person you think made the commit

Great!

What does the `verified` badge not mean?

A commit that shows the verified badge doesn't tell you the following:

⛔ Whether the signer inspected and approves the content of the commit

⛔ Whether the contents of the commit were altered between staging and committing

⛔ Whether the signer made the commit in person (or had automation sign on their behalf)

⛔ Whether the account is compromised or not

⛔ Whether the account owner has securely stored their private key

⛔ Whether the account is actually the account you were expecting. This may be difficult for you to distinguish. Is it Octocat or 0ctocat?

⛔ Whether the account is stil owned by the same person last time you encountered their profile (it might have been repo-jacked)

⛔ In essence: Whether the commit was actually made by the person you think made the commit

So, the verified badge isn't a seal of approval, nor a universal signal of trust

What does the `unverified` badge mean?

That leaves us with the unverified badge. Which might be the most useful signal that commit signing may give. it signals that:

🟠 The commit was signed with a key that doesn't belong to the account that made the commit.

🟠 The commit was signed with a key that doesn't match a known email for the account that made the commit

🟠 The commit wasn't signed and the account has enabled vigilant mode

🟠 You cannot trust the commit was made by the person you think made the commit.

How can this all go wrong

Generating and storing your keys

To perform commit signing correctly, users must take several steps.

They need to acquire a keypair.
- Most people will generate a ssh key or a GPG key on their box.
- Some companies will generate a key for you and hand it over to you.
You need to protect the private keys in some way
- Encrypt them using a passphrase
- Protect them with a 2FA token (like a YubiKey)
- Set the file system permissions
- Store them in a software key-vault (for example: windows credential manager or 1password)
- Or put them on a smartcard
- Back up the key somewhere safe
- Delete the unprotected key thoroughly
You need to put the private key on every device, VM, container, user profile you want to use to sign commits. (Or generate a new key for each device, but then, how will people know they should trust your keys?).
You need to upload your keys to GitHub and other platforms where you want your identity to be known.
You need to make sure your keys are "locked" most of the time to prevent malware on your machine to intercept the key during development.

If you search for instructions to start signing your commits, you'll find the generic instructions to generate the key and to setup git to use the key, but these generally won't tell you how to then store the private key securely and how to ensure it is locked most of the time.

I've seen a lot of people that have stored their key material in their user's home directory, without a passphrase. I've made that mistake myself at least once using my career.

Recently vendors like 1Password have taken notice of how hard it is to actually do this right, and they're now offering features to handle most of the key generation and secure storage for you.

Which is great, they're making commit signing easy. Like how Let's Encrypt made it easy for everyone to secure their website.

Of course, key generation and distribution could be handled by your IT department. And they might hand you 2 Yubikeys on your first day at work. Unfortunately, for the vast majority of maintainers and contributors there is no IT department to take care of all the nitty gritty details.

Or if you're contributing to the Linux kernel and submit your patch over email using a signed patch file, it's the only way to have your work integrated into the repository:

Submitting patches: the essential guide to getting your code into the kernel — The Linux Kernel documentation

Verifying the identity of a commit author

It still doesn't solve the issue of distributing your keys to your peers. Your commit may show verified on GitHub, but you won't be able to verify the signature of a commit on your local machine, build agent or in the repo of a different vendor, like GitLab. Unlike the browsers, which are preloaded with trusted root certificates, the git client and hosts don't have a built-in system to share public keys.

So, to provide true protection, not just on GitHub, you'll need:

A way to distribute your public key.
Be able to revoke a key
Be able to rotate a key
A way to receive the keys of the people you work with

In a corporate environment, your IT or security department may be able to set this up for you, but for most contributors in the world, this is something they have to do themselves. And it is something many have no clue how to do well.

What makes this even harder is that identity verification in this case is hard. Many people use an email address from gmail.com or outlook.com, so it would be trivial to create an email address that's similar, but different. For example, I own: jesse.houwing@gmail.com, but nothing blocks you from creating jhouwing@gmail.com or jesse.h0uwing@gmail.com or jesse.houw1ng@gmail.com, generating a key for that and registering it with github. GitHub will show verified in the commit and it's up to you to inspect the identity of the underlying user. The same is true for GitHub profiles. It's not hard to create a new account, mirror a bunch or repos and then create a pull request that will look authentic at first glance.

Another issue is the fact that when a GitHub account is renamed or deleted, it will become available to new users after 90 days. So octocat from last year's excellent pull request may be someone completely different than octocat today. Since the new owner of the account can add new key material, any commit they make will show as verified. Not every repo host currently prevents you from creating highly similar usernames or warns their users a highly similar username was created. GitHub even suggests to slightly change your desired username if the account you want is already taken, and won't prevent you from creating an account that looks a lot like an existing account.

And then there is one another issue: privacy. GitHub and other platforms allow users to make their true email address private. Instead github will use an address in the following format ID+USERNAME@users.noreply.github.com for your commits. Users opting into this privacy feature will be even harder to verify, as you won't be able to see their true email address.

Validating the contents of a commit

Even if the commit was signed by the correct person, there is no guarantee the contents of the commit were written by them.

When you want to add new code to your repository you'd go through a few steps:

# change a set of files
echo "// small change" >> thefile.cs

# stage the changes
git add thefile.cs

# commit & sign
git commit -m "committing my stuff" -s

# push the commit to the remote
git push origin main

The thing is, malware could inject itself between any of these steps. So, a diligent contributor would run git status and do a diff to verify the contents of the staging area before committing and the results of the commit prior to pushing. A slightly more paranoid maintainer will have to type their SSH passphrase to initiate the push. But let's be honest, most of us don't.

A pre-commit hook or a git filter or an executable running in the background could change the contents of the file between git add and git commit and most contributors would be none the wiser. Or it might intercept your passphrase for your private key and exfiltrate it from your machine.

An even simpler attack is possible: a malicious npm package may have slipped onto your machine and has nestled itself into the package-lock.json, for which most diffs are hidden by default anyway:

Contents of generated and large files are hidden in diff view by default.

Hopefully any changes to the code are detected during the pull request or by some tool that runs a pull request check. It wouldn't be the first time such changes slip by undetected.

Unfortunately, very few IDEs and Git clients will show you the contents you are about to sign or the contents you have just signed. And even if they did, if you're committing a large change to a file, you are unlikely to notice any code that was injected.

If a contributor hasn't properly protected their key material, it would be even more trivial for an attacker to take the key and send it somewhere to use it at any later point in time, without you ever noticing.

And even when a user has properly set up a passphrase or 2-factor authentication, they may just leave their YubiKey in their usb port and tap it whenever prompted. Unless a user knows exactly when to expect a prompt, they may just enter their passphrase into every prompt that asks for it.

A lot of the "secure" aspect of commit-signing depends on the contributor being fully aware of how many "obscure" utilities work, and when it's safe to enter their password in a prompt. I suspect that it would be much safer if GitHub would prompt the user through GitHub Mobile when they receive a push, than expecting a user to know which prompts to trust.

The `verified` badge is like the 🛡️ we removed from the browser bar.

Which brings me to the reason I wrote this article. Currently, most contributors and maintainers that have properly set up their signing infrastructure are power users who (hopefully) know how to do this well. But pushing the general population to set up commit-signing will only degrade the trust I have in the verified badge. I currently do not believe that the whole population of GitHub users will be able to set up their environment correctly or will know how to protect their private keys.

In its own way, the verified badge is a lot like the 🛡️ shield we used to show in the URL bar or the browser. Banks would even tell you that the shield was a beacon of trust. Websites would proudly show a badge to boast about their certificates:

The '90s seal of trust

Until https became the standard and now everybody uses it. Bad actors have equipped their phishing sites with the same quality SSL certificates and the value of the shield has completely evaporated. It's only when a site truly tries to impersonate someone, that the browser now shows that you're entering dangerous territory. All major browsers now highlight when a connection is unsafe.

To me there is little value in the verified badge, but I do see a lot of value in the unverified or invalid status. Those are a clear signal that something is wrong. Unfortunately, with the current mess in setting things up and the complete lack of a ubiquitous infrastructure to share public keys and verify identity effectively the presence of the verified badge just won't be enough.

I predict that the verified badge will go the way of the https shield. Once most people are verified, vendors like GitHub will not show it like a mark of trust, but they'll only warn you when something's off.

There are already policies you can enable on your repo to force all commits to be signed and for all commits made from github.com to be signed. These will verify that the commits weren't signed incorrectly or not signed at all. But it cannot raelly verify the commits were made by people you should trust.

Why do we need this anyway

To make things worse, git hosting platforms like GitHub, Azure Repos, GitLab already have better ways to establish who you are. Before I push to GitHub, I'm signed in with my GitHub account, performed SSO with my corporate identity and provided multiple 2nd factors along the way. GitHub knows my email addresses, because I have verified these with GitHub already. Other hosting platforms can already do the same and they have an account identity store for all the users on their platforms. The current way of dealing with signing commits is of course there because Git doesn't need a hosting platform. It's a distributed version control system and it might be used by people who don't ever connect to GitHub or Azure Repos or GitLab. But a large portion of git's users do.

GitHub won't need me to generate any keys myself to verify that I pushed a commit to GitHub. But we need the whole signing infrastructure to ensure you can verify I authored a commit when you're not looking at it on GitHub. And to me that's probably more important information than the commit author's name and email address they had configured in their git client.

I suspect that, like the Git Credential Manager which securely stores your logon credentials on your system, we will have a Git Signing Manager in the future that will be able to sign your commits with your authentication token. GitHub basically already does that when you commit changes from GitHub Codespaces or when you edit files directly on the web. I wouldn't be surprised when DNS will play a central role in public key distribution.

We'll end up in a state where everyone signs their commits, but we won't see this anywhere unless there is something wrong.

What do you think?

I've had several lengthy discussions with colleagues, friends and random people at conferences about this topic. What do you think? What does a verified badge mean to you?

Don't get lost in your plans

Jesse Houwing — Wed, 14 Feb 2024 12:18:15 +0000

Today I got a call from a customer who is trying to understand what's going on in their teams and is getting lost among the many different Features and Product Backlog Items on their boards.

There are 2 things going on at this customer.

They're changing how they use their tools to better fit their needs.
They're changing their delivery process to improve collaboration in and among teams.

They're currently using Azure Boards to visualize their work, but the tool doesn't really matter. If they were using GitHub Projects or Jira, they would have similar issues.

Let me quickly describe their setup. There are 9 teams, they're loosely connected at the moment. Teams are working on their own Product backlog Items day-to-day, but those deliver a set of working functionality across the teams in the form of a Feature. Value is currently delivered at the Feature level.

In the past each team had their own backlog of Epics and Features and Product Backlog Items and this caused a lot of problems around ordering, setting priority, crafting common goals, managing dependencies.

One of the configuration changes made at the tool level is to show all Epics and Features on a single Backlog and a single Board. With this change there is now:

a single view on the common order
a single place to visualize work with high priority

With this change a few other things were also made transparant:

There are only a few "Epics"
There are a LOT of "Features"
But worse, there are a LOT of "Features" in progress
Their laptops have relatively small screens 😉.

That's why I got that call today. The client knows they need to change their processes, but that's taking its sweet time. So, for now, they were looking for ways to not get lost in their work.

6 tips to not get lost in your plans that won't require you to change your process.

Reduce the amount of data you show on your cards - The default settings of tools show quite a bit of data on every card. An issue number, title, who the card is assigned to, the parent item. and on top of that people tend to add more data: estimate of size, estimate of value, subtasks etc.
Adjust the board's settings based on your current needs - The settings of your Backlog and Kanban board are not static, you can tweak the fields based on the activity you're using the board for. For example, in refinement: show the estimates. In a scrum-of-scrums, hide most fields, except the Title. Some tools will let you quickly clone a board to make adjustments (GitHub Projects), in others you will need to manually tweak the settings each time (Azure Boards). Many tools will allow you to hide or collapse columns on your boards.
Naming is important - Instead of "As a grumpy project manager I want to improve the visualization of my work in progress in order to not accidentally miss important visual signals that would help me reduce risks" you could use a much simpler title such as "Reduce visual clutter". Then add all of the details in the card's description field.
Filtering - Every tool I've ever used offers filtering features. You can craft a query or a filter to show a subset of the cards on your board. In order to filter you may need to add some additional metadata to your cards in the form of a custom field or a tag.
Blow up & zoom out - Most tools either offer a built-in full screen feature as well as the ability to zoom in and out. If you're looking at the plan from a browser, hitting F11 will also remove the browser's chrome. While it may only give you a little more space, it's the small things that count.
Use the right visualization - Most tools offer multiple ways to visualize the work. Backlogs, boards, at every level, for a single team or across all teams. It's important to know which visualization is right for the activity you're doing. Doing cross-team-refinement to reduce dependencies? Use a view that visualizes your dependencies (Delivery Plans in Azure Boards for example).

While these tricks to tweak the tools can provide some relief, they simply hide the actual problem: there is too much work on the board at the same time. It's well known that having a lot of Work in progress (WIP) causes work to take longer (longer cycle time). It increases context switching. It reduces collaboration between individuals and teams. Usually the end result is: fewer things get delivered and they all take longer.

6 tips to change your process

Instead of hiding the problem, here are a few tricks to fix things for good.

Don't scale - As my colleague Laurens Bonnema puts it: the first rule of scaling: don't. As my friend Simon Reindl says in his Scrum classes: Nail it before you scale it. Or as I often bluntly put it: If you scale shit, you get heaps of it. Try to form one or a few teams around smaller objectives and coordinate each of these collectives with their own board. Improve your tools, techniques, architecture and collaboration patterns before trying to scale up further, if you even want to.
Improve in-team and cross-team collaboration - Reducing WIP and refining as you go will greatly reduce the number of things your teams are working on. There are examples from organizations practicing Mob Programming (aka Ensemble Programming or Whole Team Programming) where each team works only on a single item at a time. In that case, if you have 9 teams, you only have 9 cards in progress. Possibly those 9 cards all contribute to only 1 or 2 "Bigger items" such as a feature. To achieve this your teams will need to be composed of the "right people" to deliver a piece of valuable work. It may require that those people are capable of doing more than 1 element in your work process.
Reduce WIP - All of the tools offer setting a Work-in-progress limit on each column of your board. If you set a limit and actually stick to that limit, you'll have fewer items on your board and it will be easier to keep an overview of the total work in progress. Having a Kanban board or a Sprint board with 30 cards in a column and a scrollbar is a recipe for reduced transparency. Ensure you do not only reduce WIP at the Product backlog Item or Task level. Reducing WIP works best if also applied at the "Feature" and "Epic" level, if that's what you're using. Or at the "Project", "Theme" or "Saga" level. Try to do as few things in parallel as you can.
Optimize effectiveness, not busy-ness - The goalkeeper in soccer has one important job, and that is to protect their team's goal. Yet, if the ball is on the other side of the field, they don't suddenly start reading their email, coach the kiddy-league or walk over to another field to protect another goal. They remain engaged in the soccer match, even if they aren't actively handling the ball. The same principle should apply to the people on your team. While the team is working on delivering their Sprint goal, if an individual can't immediately contribute, there is really no need to find less important work to keep them busy. Busy-ness is an epidemic in modern organizations. In increases WIP, it reduces collaboration and it reduces motivation and commitment over the long term.
Refine & plan just-in-time - Many organizations practice a Quarterly planning cycle on top of their bi-weekly sprint cycle. As does this customer. If, during this quarterly cycle, work is planned for the whole quarter and most of that work is immediately broken down into actionable Product Backlog Items, then each quarter starts with a completely overloaded board with way too many cards. Instead, try to make a rough selection of "large" items that may need to be delivered that quarter, order them and then start decomposing the top 1, 2, maybe 3 of these into smaller items as you go. This will greatly reduce the clutter on your boards.
Use the right "levels" - If you're using a product Backlog decomposition across multiple levels. I personally prefer a single, short, flat backlog over a complex tree with dependencies between branches. But my customer has chosen to decompose their work across Epics, Features, Product Backlog Items and Tasks. Their biggest problem is currently at the "Feature" level. That's where work from multiple teams comes together and that's where they're working on a lot of things at the same time. The big question of course is, "should this have been a feature"? They have very few Epics they're working on, a LOT of features and Product Backlog Items. There is a possibility that redefining what constitutes a Feature may simplify things. Maybe multiple features can be combined? Maybe some could be relegated to a Product Backlog Item and be moved out of this overcrowded visualization?
Improve the definition of Workflow - The process this customer follows has a few issues where large batches of work may occur: they plan a large batch of items at a quarterly level, decompose work far in advance and it may take them a while to get work accepted and moved to Done. Yet their workflow is relatively simple and only has a few columns with a pretty generic definition like "development". As a result, as soon as work starts on any item (at any level), cards are pulled into "development". And even though the column was defined with a WIP limit of 10, last week they had about 30 cards in that column in various sub-states. There are 2 things one may do in such a case. reinforce the WIP limits , or in order to better understand where work is "stuck", further decompose these columns into 2 or more (sub-)columns. This may help visualize any bottlenecks (and would reduce the need for vertical scrollbars 😉) even if it adds opportunity to set a total higher WIP limit.

2 tips in case of emergency😜

Buy a bigger screen - Instead of trying to squeeze your whole plan into a 13" HD laptop screen, connect an external 8K 80" screen or a decent projector. Heck, a 46" ultra-wide screen can visualize a LOT of columns on your Kanban board.
Buy multiple screens - If a bigger screen isn't an option, maybe using multiple smaller ones may help you better understand what work is in progress,

Conclusion

Knowing you have a problem is half the solution. There is only so much you can tweak with tools. Instead of trying to work around the problem and possibly hiding it for a bit longer, try making the real underlying issues transparant and addressing those.

Forem: Jesse Houwing

Automatic versioning for GitHub Actions authors

What does it do exactly?

What's new

Recommended usage

Improved Dependency Submission for GitHub Actions

Background

Impacted features

Solution

How to

GitHub Billing - Let Organization and Repository admins set the Cost Center in GitHub

Creating the Custom Property

The reason for Inherit

Making it all work using GitHub Actions

GitHub Actions learnings from the recent nx hack

Understanding the vulnerable workflow

Understanding the vulnerable (human) process

Remediation

Change the default permissions for GitHub Actions to read-only

Set workflow permissions at the job level

Enable branch rulesets to require pull requests for all important branches.

Require approval for running fork pull request workflows from contributors

Do not rely on pull_request_target

Do not use workflow_dispatch or repository_dispatch triggers in workflows that have access to critical secrets.

Do not mix pull-request validation and continuous integration workflows

Store critical secrets in environment secrets, instead of repository secrets.

Pass secrets only to the steps that need them

Pass data to script steps using the environment

Pin all actions to their sha.

Enable GitHub Advanced Security Code Scanning for GitHub Actions and add Branch Ruleset.

Enable Dependabot for GitHub Actions to automatically update action references to the latest version.

Consider 3rd party security hardening tools like StepSecurity's Harden Runner.

Conclusion

Dear GitHub,

Authenticate Connect-MgGraph using OIDC in GitHub Actions

Picking the right GitHub Copilot model

GPT-4.5

Pros:

Cons:

GPT-4o

Pros:

Cons:

Recommendation:

Say goodbye to your Personal Access Tokens

PATs, PATs and more PATs - our 5 scenarios we used PATs for

PAT REST APIs to the rescue?

Scenario 1: Publish marketplace extensions without a PAT

Scenario 2: Call Azure DevOps APIs from a Azure Function app

Scenario 3: Run intergration tests in Azure Pipelines

Scenario 4: Run e2e tests in Azure Pipelines (against external Azure DevOps organization)

Scenario 5: Running integration and e2e tests locally

Bonus scenario: the last token - the renovate pipline

Wrapping up

GitHub's new Billing - Assigning Cost Centers in Bulk

Use inline completions for more answers with GitHub Copilot Free

Q&A

Suggest test cases

Suggest improvements

Conclusion

Troubleshooting GitHub Copilot keyboard shortcuts in Jetbrains IDEs

Keyboard shortcut is already assigned to another command

Conflict between GitHub Copilot and Jetbrains' bundled line completion

The use or uselessness of signed commits

What does the verified badge mean?

What does the verified badge not mean?

What does the unverified badge mean?

How can this all go wrong

Generating and storing your keys

Verifying the identity of a commit author

Validating the contents of a commit

The verified badge is like the 🛡️ we removed from the browser bar.

Why do we need this anyway

What do you think?

Don't get lost in your plans

6 tips to not get lost in your plans that won't require you to change your process.

6 tips to change your process

2 tips in case of emergency😜

Conclusion

Do not rely on `pull_request_target`

Do not use `workflow_dispatch` or `repository_dispatch` triggers in workflows that have access to critical secrets.

What does the `verified` badge mean?

What does the `verified` badge not mean?

What does the `unverified` badge mean?

The `verified` badge is like the 🛡️ we removed from the browser bar.