Forem: Craigory Coppola

Superpowered Git Aliases using Scripting

Craigory Coppola — Fri, 20 Sep 2024 01:21:13 +0000

What are Git Aliases

Git aliases work similarly to regular aliases in the shell, but they are specific to Git commands. They allow you to create shortcuts for longer commands or to create new commands that are not available by default.

Aliases run in the same shell environment as other git commands, and are mostly used to simplify common workflows.

Simple Aliases

Simple aliases call a single Git command with a set of arguments. For example, you can create an alias to show the status of the repository by running git status with the s alias:

[alias]
  s = status

You can then run git s to show the status of the repository. Because we configured the alias in ~/.gitconfig, it is available for all repositories on the system.

More Complex Aliases

You can also create git aliases that run an arbitrary shell command. To do so, the alias needs to start with a !. This tells git to execute the alias as if it was not a git subcommand. For example, if you wanted to run two git commands in sequence, you could create an alias that runs a shell command:

[alias]
  my-alias = !git fetch && git rebase origin/master

This alias runs git fetch and git rebase origin/main in sequence when you run git my-alias.

One limitation of git aliases is that they cannot be set to a multiline value. This means that for more complex aliases you will need to minify them.

Additionally, in an INI file a ; character is used to comment out the rest of the line. This means that you cannot use ; in your alias commands.

These two limitations can make it difficult to create more complex aliases using the standard git alias syntax, but it can still be done. For example, an alias using if to branch may look like this:

[alias]
  branch-if = !bash -c "'!f() { if [ -z \"$1\" ]; then echo \"Usage: git branch-if <branch-name>\"; else git checkout -b $1; fi; }; f'"

These limits make it way more complex to create and maintain aliases that have any form of control flow within them. This is where scripting comes in.

Setting up Aliases with Scripts

You can script a gitalias using any programming language you'd like. If you are familiar with bash scripting and would like to use it, you can create a bash script that runs the desired git commands. The truth is that I am much stronger with JavaScript, so that is what I will use.

One other major benefit is that by using a scripting language, your aliases can take and operate on arguments much more easily. Git will forward any arguments you pass on the CLI to your alias by appending them to the end of your command. As such, your script should be able to read them without issue. For example, in Node JS you can access the arguments passed to the script directly on process.argv.

The basic steps to set this up do not change based on the language choosen. You'll need to:

Create a script that runs the desired git commands
Write an alias that runs the script

Case Study: Rebase Main / master

In recent years the default branch name for new repositories has changed from master to main. This means that when you clone a new repository, the default branch may be main instead of master. There is no longer a super consistent name, as the ecosystem is in transition. This is overall a good thing, but it means that our alias above to rebase will not work in all cases.

We need to update our alias to check if the branch is main or master and then rebase the correct branch. This is a perfect use case for a script.

#!/usr/bin/env node

const { execSync } = require('child_process');

// We want to run some commands and not immediately fail if they fail
function tryExec(command) {
  try {
    return {
      status: 0
      stdout: execSync(command);
    }
  } catch (error) {
    return {
      status: error.status,
      stdout: error.stdout,
      stderr: error.stderr,
    }
  }
}

function getOriginRemoteName() {
  const { stdout, code } = tryExec("git remote", true);
  if (code !== 0) {
    throw new Error("Failed to get remote name. \n" + stdout);
  }
  // If there is an upstream remote, use that, otherwise use origin
  return stdout.includes("upstream") ? "upstream" : "origin";
}

// --verify returns code 0 if the branch exists, 1 if it does not
const hasMain = tryExec('git show-ref --verify refs/heads/main').status === 0;

// If main is present, we want to rebase main, otherwise rebase master
const branch = hasMain ? 'main' : 'master';

const remote = getOriginRemoteName()

// Updates the local branch with the latest changes from the remote
execSync(`git fetch ${remote} ${branch}`, {stdio: 'inherit'});
// Rebases the current branch on top of the remote branch
execSync(`git rebase ${remote}/${branch}`, {stdio: 'inherit'});

Currently, to run the script we'd need to run node ~/gitaliases/git-rebase-main.js. This is not ideal, and isn't something you'd ever get in the habit of doing. We can make this easier by creating a git alias that runs the script.

[alias]
  rebase-main = !node ~/gitaliases/git-rebase-main.js

Now you can run git rebase-main to rebase the correct branch, regardless of if it is main or master.

Case Study: Amend

Another alias that I set up on all my machines is to amend the last commit. This is a super common workflow for me, and I like to have it as a single command. This is a great use case for a script, as it is a simple command that I want to run often.

#!/usr/bin/env node

// Usage: git amend [undo]
const tryExec = require('./utils/try-exec');

async function getBranchesPointingAtHead() {
  const { stdout, code } = await tryExec('git branch --points-at HEAD', true);
  if (code !== 0) {
    throw new Error('Failed to get branches pointing at HEAD. \n' + stdout);
  }
  return stdout.split('\n').filter(Boolean);
}

(async () => {
  const branches = await getBranchesPointingAtHead();
  if (branches.length !== 1) {
    console.log(
      'Current commit is relied on by other branches, avoid amending it.'
    );
    process.exit(1);
  }
  if (process.argv[2] === 'undo') {
    await tryExec('git reset --soft HEAD@{1}');
  } else {
    await tryExec('git commit --amend --no-edit');
  }
})();

This script is a bit more complex than the last one, as it has some control flow in it. It will check if the current commit is relied on by other branches, and if it is it will exit with an error. This is to prevent you from amending a commit that is relied on by other branches, as doing so would cause issues when trying to merge whichever branch relies on the commit.

To set up the alias, you can use the same method as before:

[alias]
  amend = !node ~/gitaliases/git-amend.js

Now you can run git amend to amend the last commit, or git amend undo to undo the last amend. This is a script that I initially wrote inline in my gitconfig, but as it grew in complexity I moved it to a script file. This is a great way to manage complexity in your aliases. For comparison, here is the original alias:

[alias]
  amend = !bash -c "'f() { if [ $(git branch --points-at HEAD | wc -l) != 1 ]; then echo Current commit is relied on by other branches, avoid amending it.; exit 1; fi; if [ \"$0\" = "undo" ]; then git reset --soft \"HEAD@{1}\"; else git commit --amend --no-edit; fi }; f'"

This script could have been extracted to a .sh file as well, but keeping things in node lowers the maintenance burden for me personally. In the past, anytime I needed to update this alias I would have to paste it into a bash linter, make my changes, minify it, and then paste it back into my gitconfig. This was a pain, and I would often avoid updating the alias because of it. Now that it is in a script file, I can update it like any other script.

Some Caveats

Setting up aliases as scripts can unlock a whole new level of power in your git aliases. However, there are some things to be aware of when doing this.

When setting up aliases like this, its important to remember that the cwd of the script will be the current working directory of the shell that runs the script. Any relative file paths in the script will be treated as relative to the cwd of the shell, not the location of the script. This is super useful at times, and super painful at others. For our rebase-main script its not an issue though, and the only indication that this is happening is we used ~ in the file path to reference the script location as an absolute path.

Introducing scripting into your git aliases can also make it tempting to add more and more logic to your aliases. This can make them harder to maintain and understand, but also harder to remember. Its not worth maintaining a super complex alias, as you'll be less likely to use it anyways. Additionally, you should be careful to not intoduce anything that may take too long to run into your aliases. If you are running a script that takes a long time to run, you may want to consider if it is the right place for it.

Conclusion

I hope this article has shown you the power of scripting in your git aliases. By using scripts you can create more complex aliases that are easier to maintain and understand. This can make your git workflow more efficient and enjoyable. For more examples of git aliases, you can look at my dotfiles project. It contains a lot of the configuration that I keep on all my machines, including my git aliases.

Unlisted GitHub Repositories

Craigory Coppola — Mon, 16 Sep 2024 16:52:25 +0000

Services such as YouTube provide support for unlisted videos. This allows publishing a video that is publicly accessible, but only via a direct link. This is useful for sharing content with a select group of people. It be used for getting early feedback on a video draft, or for sharing a video with a client before it goes live.

There is no equivalent feature for GitHub repositories, but the need still exists. Some reasons I've needed this feature include:

Sharing issue reproductions
Sharing more complex code snippets like multifile gists

These could both technically be done as a public repository, but it clutters up the profile and makes it harder to find the repositories that are actually meaningful.

Unrelated Branches

A git repository frequently has many branches of code, typically waiting to be merged into the main branch. Branches, however, do not necessarily have to be related to the main branch. As such, you can push any branch to any repository, even if they don't share a common ancestor.

This is the key to our approach of creating unlisted repositories.

Steps to Create an Unlisted Repository

Create a "host" repository that will contain all of your unlisted repositories. This repository will be public, but not contain code on the main branch. It differs from other repositories in that it will contain many branches, each representing a different unlisted repository. I'd recommend adding a README to this repository that explains the purpose of the repository and how to use it.

For example, I use https://github.com/agentender/github-issues as my host repository.

Create or open the repository that will be published as unlisted. Only the checked out branch will be shared.
Run the following command to push the branch to the host repository:

git push $HOST_REPO_REMOTE_URL HEAD:$BRANCH_NAME

You should replace $HOST_REPO_REMOTE_URL with the URL of the host repository and $BRANCH_NAME with the name you want to use when sharing the branch.

Share the URL of the branch with the people you want to have access to it.

The link should look something like this:

https://github.com/{user}/{host-repo}/tree/{branch-name}

Supporting Infra

If you plan to use this semi-frequently, I'd recommend creating a script or git alias to simplify the process. Here's an example script I use:

const REMOTE_URL = 'https://github.com/agentender/github-issues';

const tryExec = require('./utils/try-exec');

function prompt(message) {
  return new Promise((resolve) => {
    const readline = require('readline').createInterface({
      input: process.stdin,
      output: process.stdout,
    });
    readline.question(message, (answer) => {
      readline.close();
      resolve(answer);
    });
    readline.on('SIGINT', () => {
      readline.close();
      process.stdout.write('\n');
      console.log('Aborted.');
      process.exit(1);
    });
  });
}

function isForced() {
  return process.argv.includes('--force') || process.argv.includes('-f');
}

async function checkIfBranchAlreadyOnRemote(branch) {
  const { code } = await tryExec(
    `git ls-remote --exit-code ${REMOTE_URL} ${branch}`,
    true
  );
  return code === 0;
}

(async () => {
  const forced = isForced();
  const currentBranchName = (
    await tryExec('git rev-parse --abbrev-ref HEAD', true)
  ).stdout.trim();
  let branch = currentBranchName;
  if (currentBranchName === 'master' || currentBranchName === 'main') {
    branch = await prompt(
      `What branch would you like to publish to?${
        forced ? ' (Any existing contents will be overwritten)' : ''
      }`
    );
  }
  while (!forced && (await checkIfBranchAlreadyOnRemote(branch))) {
    const next = await prompt(
      `Branch ${branch} already exists on remote. What branch would you like to publish to?`
    );
    if (next === '') {
      console.log('Aborted.');
      process.exit(1);
    }
    if (branch) branch = next;
  }
  await tryExec(`git push ${REMOTE_URL} ${currentBranchName}:${branch}`);
  console.log(`✅ Published branch at ${REMOTE_URL}/tree/${branch}`);
})();

Wrapping the process in a script makes it easier to use, and ensures consistency. This also gives a spot to add additional checks or features in the future.

The example script above for example:

Prevents accidentally overwriting an existing branch
Prompts for branch name if the current branch is master or main, or would otherwise overwrite an existing branch
Allows forcing the push with --force or -f

If you setup a script like this, you can add it as a git alias to make it easier to use. I've covered how to do this in a previous post: Superpowered Git Aliases

Wrapping Up

This isn't quite as good as native support for unlisted repositories, but it's a good workaround. Sharing reproductions will no longer clutter up your profile, and you can share more complex code snippets without needing to create a new repository for each one. Of note, this approach has limitations that native support would not.

Repo's shared with this manner:

Can not really have more than one branch
Can not have issues or pull requests
- This also makes it hard to test things like Github Actions or similar
Can be found by anyone with access to the host repository

With that said, I still think this is a useful tool to have in your toolbox.

Build Monorepos, not Monoliths

Craigory Coppola — Fri, 24 Dec 2021 17:46:46 +0000

This article was written as part of the C# Advent 2021 event. Find more articles and information here: https://csadvent.christmas/

Monolithic Software

C# has a long history with monoliths. They have their own MSDN page, which describes them as below.

A monolithic app has all or most of its functionality within a single process or container and it's componentized in internal layers or libraries

On MSDN, some downsides are immediately noted.

Scaling the individual components is either hard, or impossible in some cases. As an example, you may be able to scale up the database as a whole, but you would not be able to easily bump up the authentication layer of your application.
Changing a single layer of the application requires the whole app to be rebuilt and tested.
The whole app must be deployed at the same time.

Microservices

In response to these issues, many developers and architects have been building microservices. In a microservices architecture, each layer or vertical slice of the application is written and deployed as its own executable or container. Each service communicates through protocols like gRPC or HTTP, rather than directly calling functions.

This eliminates many of the scaling concerns, as each individual host for the services can be scaled independently. Testing can also be slimmed down. As long as the public API for a service is not changed, or the changes are backwards compatible, only that service must be tested.

However, some benefits from monoliths are lost. Sharing code between microservices is much harder. If you have a common "microservice-template" repository, updating it becomes a chore. Existing microservices may not get the benefits from the update, or you may have to manually copy and paste the functionality to each microservice. You may decide to publish a company internal package to nuget, or NPM to alleviate some of that pain, but now testing your changes across the services can result in going back and forth between solutions to check if your changes work.

Additionally, layers of complexity quickly stack up and progress can slow down to a halt if the architecture begins to slip.

Some big cons from microservices that are split across multiple repositories include:

Implementing one feature may have a developer submitting pull requests to several repositories. These PR's may require each other for integration tests to pass.
Deployment and CI/CD work is much more complicated, as integration tests should test the whole system but are harder to orchestrate

Modern Monorepos

The front-end community has began to adopt the concept of monorepos, and many top companies have been using them for years.

In its most basic form, a C# monorepo may be a git repository that contains all of the microservices inside a solution. Even at this basic level (which I refer to as colocation), you start getting some benefits.

A single pull request represents the entire change set for a feature.
System level integration tests and CI checks can be implemented much easier, since you are only dealing with one source repository.

Beyond Co-location

Introducing something like @nrwl_io's Nx build tooling allows for many more benefits to come through, with advanced task orchestration to maximize developer productivity and minimize CI times.

There are some similar tools available, but I'm most familiar with Nx so that is the example I will use here. For a comparison with other tools, see monorepo.tools.

After adopting Nx, and using its commands for your builds you get several things, but we will focus on a few of them.

Local and Remote caching for build / test targets
Project Graph Analysis
Distributed Task Execution

Local and Remote Cache

When using Nx, commands such as build and test have their outputs (both on disk and terminal) cached during execution. What this means, is that when running the same command with the same source files you don't waste computation or developer time.

With the help of Nx Cloud, this time saved gets extended on your CI/CD agents as well. Outputs get cached to the cloud servers, and if an agent runs a command which has already been ran it retrieves it from the cache instead of running the command.

This remote cache can also help developers, as when checking out a PR branch locally the builds will pull from the remote cache that was created during PR checks or on the other developers machine.

Project Graph Analysis

With the help of nx-dotnet, Nx is able to understand how the projects in your repository depend on each other. Nx is then able to look at what has changed on your branch and only build and retest what has been affected by your changes. When using microservices you may not have ProjectReferences in your .csproj files. Nx makes it easy to add an "implicit dependency" between projects to inform the tooling that there is a hidden link there.

Taking advantage of nx affected in CI / CD means that you are not wasting cloud agent time to retest things that haven't changed, but you are not missing tests that may fail in a dependent project.

Distributed Task Execution

Nx can intelligently run targets across a network of CI agents, such that dependent jobs are ran with the inputs they need as soon as they are ready. This can speed up CI/CD times by a large factor, as jobs are parallelized and computational time is not wasted redoing jobs that have already been completed.

There is an existing article describing how this process works from @nrwl_io.