Managing CircleCI secrets via Terraform

Alex Kucksdorf — Fri, 06 Jan 2023 22:37:57 +0000

If you are just interested in the examples, jump straight ahead or have a look at the example code

It sucks, if your CI provider has a security breach. What sucks even more is, if you are the lucky one in charge who has to go, rotate all possibly compromised secrets and update those in your build pipeline environment variables 🤦‍♂️.

Sadly, this is not some dystopian fantasy, but was just announced this week (2023-01-04) by CircleCI. This is no news you want to receive on your first day back to work after the holidays - lucky me 🤪. But, to be fair, I guess you don't want those news on any other day either 🤷‍♂️...

Jokes aside, when one of your clients - and their 10s or more repositories - is affected by this, it can get hairy pretty quickly. Worst case, you have to click your way through the UI and collect all secrets stored in contexts or, even better, in project/repository-specific settings. In the meantime, CircleCI has published a repository to help those unlucky ones: CircleCI-Public/CircleCI-Env-Inspector. Using this, you can get at least a high-level overview of all used secrets in your organization, e.g. name, location and anonymized value.

The best case on the other hand would be, if you could just rotate most - if not all - of your secrets with the click of a button and be done with it. Lucky for us, this is indeed achievable via Terraform and this neat plugin: mrolla/circleci. Chances are, a good chunk of your secrets are already managed alongside your infrastructure via Terraform. Using the plugin, you can update your secrets in CircleCI automatically whenever they are changed in Terraform. (Although this is not achievable for all secrets and passwords, many resources support this directly via Terraform, e.g. Azure Service Principal Passwords)

Set CircleCI Secrets via Terraform

# Configure the plugin
terraform {
  required_providers {
    circleci = {
      source  = "mrolla/circleci" # c.f. https://registry.terraform.io/providers/mrolla/circleci/latest/docs
      version = ">=0.6.1"
    }
  }
  # possibly some more

  required_version = ">= 1.2.5, < 3.0.0"
}

provider "circleci" {
  vcs_type     = "github"
  organization = "my-org"
}

# Create a CircleCI Context
resource "circleci_context" "example" {
  name = "my-terraform-variables"
}
# Populate context with ENV variables
resource "circleci_context_environment_variable" "example" {
  for_each = {
    FOO = <some-value-from-resource-foo> # e.g. secret from Key Vault
    BAR = <some-value-from-resource-bar>
  }

  variable   = each.key
  value      = each.value
  context_id = circleci_context.example.id
}

# OR set project-specific ENV variable instead
resource "circleci_environment_variable" "example" {
  for_each = {
    FOO = <some-value-from-resource-foo>
    BAR = <some-value-from-resource-bar>
  }

  name         = each.key
  value        = each.value
  project      = "my-repo"
  organization = "my-org"
}

Hopefully 🤞, you are not affected by those news and are just reading this out of curiosity (thanks!). If not, maybe this plugin can save you some sweat and tears - at least in the future! 🍀

You can find a more detailed example here on GitHub.

Living with Multiple Git-Personalities without Going Crazy

Alex Kucksdorf — Sun, 06 Oct 2019 19:48:59 +0000

TL;DR

Change your git config based on the location where you clone your repository - jump there
Easily switch between different git accounts via SSH-key during clone - jump there

So... You know you have a problem.
You have known for quite some time.
Up until now you haven't been able to put your finger on it, but something was always bothering you. You have known there had to be others out there, facing the same issues. Although some of them must have come up with a solution by now, you didn't stumble across it yet.

But fear no more. Your search is over. You have finally come to the right place.

Forgive my exaggeration, but I hope you get my point. This is a feeling every one of us developers has once in a while. We continue to learn throughout our careers and often are fortunate enough to work together with other awesome developers from whose experiences we should always try to learn. This is why decided to start sharing those lessons and tricks as I come to learn them.

I dare say everybody of us found themselves banging their head against a wall in the past, struggling with some kind of issue or annoyance that turned out to be easily solvable - once you have the right trick up your sleeve.

One of those annoyances for me was handling git in different contexts. I use it for private projects as well as for work related projects. Additionally, since I work at a consulting company, I also work for varying clients.

Conditional Git Config

Although this is no must, I prefer slightly different appearances in all these contexts. I want my private projects to be associated with my private email address and work projects with my company address respectively. Some of the clients also provide their external developers with dedicated email addresses that I prefer to use in those cases.

Additionally, I find my name relatively long to type and just go with Alex. However, in a professional context I prefer to use my full name.

I know those settings can be adjusted for every repository I work on, but I prefer this to be automated. You might call it lazy, but I try to avoid it since I tend to forget doing it in time and get thrown out of my flow down the road.

This is where the .gitconfig comes into play. By default, git will store your global settings in this file within your home directory. My default private config looks like this:

# ~/.gitconfig
[user]
    name = Alex Kucksdorf
    email = <my_private_mail>

The real game changer there for me was learning about conditional includes. This enables me to do something like the following:

# ~/.gitconfig
[user]
    name = Alex Kucksdorf
    email = <my_private_mail>

[includeIf "gitdir:~/dev/work/"]
    path = .gitconfig-work
[includeIf "gitdir:~/dev/work/<client>/"]
    path = .gitconfig-<client>
...
# ~/.gitconfig-work
[user]
    name = Alexander Kucksdorf
    email = <my_work_mail>
...
# ~/.gitconfig-<client>
[user]
    name = Alexander Kucksdorf@it-economics
    email = <my_<client>_mail>

This way I only need to think about the folder where I clone the repository and my git config is automatically good to go.

Manage Accounts via SSH

Resulting from this, I have already been in the situation where I had multiple Github accounts because of client requests. Due to the requirement of using 2FA as well, I very much prefer to use SSH instead of username/password authentication. However, this means that I have to use different private keys, depending on which account I want the commit to be published with. An easy solution is to define a configuration at ~/.ssh/config like so:

# ~/.ssh/config
Host github.com-work
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_rsa_work

Host github.com-private
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_rsa

After you have set up your SSH like this, all you need to is clone your repositories like this:

git clone git@github.com-work:<organization>/<repo>.git