Forem: Justin Garrison

How I Track My Resume in Git

Justin Garrison — Mon, 23 Oct 2023 06:49:04 +0000

I use Git to track my resume revisions and job applications.
I've been using this method—or variation of it—for over 10 years. You can see my system engineer resume which is from my role-syseng branch.

I made a video that walks through a couple different examples that you can check out here.

How to track your resume

Here's an example repo that can give you an idea for how to track your resume in git. This is filled with dummy data and commit dates so you can understand what the repo looks like over time. This usually isn't important because I'm only ever dealing with HEAD on each branch, but it's cool to look back on all the roles and resumes I've had.

Please make sure you create a private repo for your resume if you are going to have personal information like address, phone number, etc.

The basic idea for putting your resume in Git is so you can easily make changes to the resume without relying on resume_company-job-v3_final.pdf naming. You don't have to track everything I do, but I have found value in tracking roles, companies, and applications in addition to my resume.

Branches

My repo heavily uses branches to keep resume format separate based on roles and applications to companies. I keep track of different application-specific resumes and maintain a main format, which hosts my primary resume.

I don't apply to that many different companies. When I look in my repo there are dozens, not hundreds over 10 years, and the vast majority of them are only 1 or 2 applications.

Whenever I apply to a new company, I create a branch for that company. For instance, I might have one for Pixar or Google.

For each specific role type I apply for, such as software engineering or management, I create a branch named role-. I have less than 10 role branches because even new job titles can usually use the content from one of the other roles with minimal modifications.

Each time I apply to a company for a role, I can take the resume from the relevant role branch and copy it to the company branch. To see an example please watch the video at the top of this post.

Keep files consistant

I keep file names consistant between branches to make it easier to know what I'm looking for. In general I have the following files:

Working copy of my resume (Justin Garrison.md)
Artifact copy of the resume (Justin Garrison.pdf)
readme.txt for information about tools used
justfile for common shortcuts
Copy of the job description (jd.pdf or jd.txt)

All the other context about what company or role I'm working on come from the current Git branch.

Whenever I apply for a job I save a copy of the job description and commit it to the repo with a Apply: ... git commit. This gives me a history of applications and job descriptions with an easy way to search for applications at specific companies based on the branch.

It has been a pain to apply through job portals like LinkedIn that don't let me download the job description. Usually, I save a link to the role in jd.txt, but it's not the best solution. Please let me know if you know a way to save them to a PDF.

Conclusion

That's all there is to it. I started doing this just because I learned how to use Git in 2013 and wanted to use it for something. 10 Years later it has been an invaluable source of remembering just how far I've come in my career and some of the weird places I've applied.

If you use git for tracking your resume and have a workflow you like please let me know by opening an issue on the example git repo.

Mastodon instance with 6 files

Justin Garrison — Wed, 07 Dec 2022 20:14:55 +0000

Mastodon is built on the ActivityPub protocol which is based on Activity Streams which stores data in JSON Linked Data (JSON-LD).
All that means is Mastodon uses a lot of JSON that references other JSON.
A Mastodon instance can serve those JSON documents any way it wants so long as they are UTF-8 encoded.

Why would you do this?

There are more than 17,000 Mastodon instances.
Why would you implement one with static files?

The first reason is security.
Ars Technica has a great article about some of the concerns with running large scale, multi-user social network servers.
I have a lot of my own concerns about Mastodon that I'll save for a future post.

There are scalability challenges on multiple levels.
The size of databases and uploads is what most admins are concerned with, but number of active users and scale of a single user (e.g. celebrity, company, government) is what will really take down a social network.
An instance with 30,000 active users costs nearly $1900 per month.

If Mastodon is going to be adopted by the critical users it needs to grow, instances—many of which are run by volunteers—would be crushed under the operational and financial responsibility.
Governments and companies aren't going to join shared servers; they're going to run their own instances on the domains they already own.
The best way to scale and maintain a server is to not run one.

Create a server

If you want to watch how I created these files check out the video.

So let's create a Mastodon instance using JSON files.
You can see the files on GitHub.

The files are hosted at https://mastodon.jgarr.net so you can test this for yourself by searching for the user @justin@mastodon.jgarr.net

You only need 1 file but to make a more complete user we'll use these 6:

2 files to create a user
2 files to pretend we are popular
2 pictures to make it look pretty

The only required file is the user, but I wanted to show how easy it is to lie in the fediverse.

Here are the files we'll be using.

.
├── .well-known
│  └── webfinger    <- user discovery (optional)
├── banner.png      <- banner image (optional)
├── followers       <- how many followers (optional)
├── following       <- how many following (optional)
├── image.jpg       <- profile image (optional)
└── justin          <- user information

Now let's explain what they do.

User discovery

When you're using Mastodon you can search for a user on any Mastodon instance with @user@domain.
This is a short hand format which relies on webfinger to translate a user at a domain into a standard URL.

When you do this search your Mastodon server will query the external server

GET https://server/.well-known/webfinger?resource=acct:user@domain

You can bypass webfinger if you know how to fetch the user's information directly.
If you search in Mastodon for https://mastodon.jgarr.net/justin you'll get the same user.

Here's the full access log so you can see the request.

{
  "request": {
    "remote_ip": "127.0.0.1",
    "remote_port": "43636",
    "proto": "HTTP/1.1",
    "method": "GET",
    "host": "mastodon.jgarr.net",
    "uri": "/.well-known/webfinger?resource=acct:justin@mastodon.jgarr.net",
    "headers": {
      "Date": [
        "Wed, 30 Nov 2022 06:00:09 GMT"
      ],
      "X-Forwarded-For": [
        "fd7a:115c:a1e0:ab12:4843:cd96:626f:140a"
      ],
      "User-Agent": [
        "http.rb/5.1.0 (Mastodon/4.0.2; +https://mastodon.social/)"
      ],
      "Accept": [
        "application/jrd+json, application/json"
      ],
      "Accept-Encoding": [
        "gzip"
      ]
    }
  },
  "user_id": "",
  "duration": 0.000258163,
  "size": 203,
  "status": 200,
  "resp_headers": {
    "Accept-Ranges": [
      "bytes"
    ],
    "Content-Length": [
      "203"
    ],
    "Server": [
      "Caddy"
    ],
    "Etag": [
      "\"rm5bpw5n\""
    ],
    "Content-Type": [],
    "Last-Modified": [
      "Wed, 30 Nov 2022 05:39:32 GMT"
    ]
  }
}

The GET request technically uses the parameter resource=acct:justin@mastodon.jgarr.net but with this static file example we only have one user on the domain so we'll ignore that part.
If you want to have multiple users on the same domain you will have to handle parameters on the server side.
Meaning, you can't do that with static files.

This request returns the file

{
    "subject": "acct:justin@mastodon.jgarr.net",
    "links": [{
        "rel": "self",
        "type": "application/activity+json",
        "href": "https://mastodon.jgarr.net/justin"
    }]
}

This says where to go fetch the next JSON document at the /justin path.
Your Mastodon server will then go fetch that object.

GET https://server/justin

Here's the full access log so you can see the request.

{
    "request": {
        "remote_ip": "127.0.0.1",
        "remote_port": "43650",
        "proto": "HTTP/1.1",
        "method": "GET",
        "host": "mastodon.jgarr.net",
        "uri": "/justin",
        "headers": {
            "Accept": ["application/activity+json, application/ld+json"],
            "Accept-Encoding": ["gzip"],
            "Date": ["Wed, 30 Nov 2022 06:00:10 GMT"],
            "Signature": ["keyId=\"https://mastodon.social/actor#main-key\",algorithm=\"rsa-sha256\",headers=\"(request-target) host date accept\",signature=\"FIFlf1AqeWuDGqF0lNJy+eRoxsy83dZ44nyhe3O+kEAB4WE8rDNKwhrGrO
                67 GZQin3lbkMZ4BKpj71wAjhNbFW8p7FtdbvGGKPwceRh5gx1hh2iqdd / INw9NZFpRbPG4wq9oHNU4MIMikICcgNDeLzcYYXbUMaDDe9W4eVzExK6SF5ulJDY0tbZchT + kaZKZqGhae25FFLc0gEPEjA3XOiZRhsVU + 7 bGPyX8Lo2g6ebGuIHPynB5WYeOu8u8noEHtbxzx + LIQZJqy1gHDb9zKq09q + f2h6ngaegayxFxZOVLMVEbhpauq1iELxlPCXaWAcwFFmWS7tJZHpqnFBAXKg == \""
            ],
            "X-Forwarded-For": ["fd7a:115c:a1e0:ab12:4843:cd96:626f:140a"],
            "User-Agent": ["http.rb/5.1.0 (Mastodon/4.0.2; +https://mastodon.social/)"]
        }
    },
    "user_id": "",
    "duration": 0.000505261,
    "size": 912,
    "status": 200,
    "resp_headers": {
        "Content-Length": ["912"],
        "Server": ["Caddy"],
        "Etag": ["\"rm5bxppc\""],
        "Content-Type": [],
        "Last-Modified": ["Wed, 30 Nov 2022 05:44:13 GMT"],
        "Accept-Ranges": ["bytes"]
    }
}

If you look at the access log you'll notice the signature in the header.
This uses a keyId https://mastodon.social/actor#main-key which is the instance that searched for the user.
There's a signature which can be used to verify the correct server―or user―is making requests.

If you want to you can skip webfinger by searching for a user by their URL directly.
If you search in Mastodon for https://mastodon.jgarr.net/justin you'll get the same user.

That means we need 1 less file but it doesn't seem as magical as @justin@mastodon.jgarr.net

This returns our actual user document.

{
    "@context": [
        "https://www.w3.org/ns/activitystreams",
        "https://w3id.org/security/v1"
    ],
    "id": "https://mastodon.jgarr.net/justin",
    "type": "Person",
    "following": "https://mastodon.jgarr.net/following",
    "followers": "https://mastodon.jgarr.net/followers",
    "inbox": "https://mastodon.jgarr.net/inbox",
    "preferredUsername": "justin",
    "name": "Justin Garrison",
    "summary": "Static mastodon server example.",
    "url": "https://justingarrison.com",
    "manuallyApprovesFollowers": true,
    "discoverable": true,
    "published": "2000-01-01T00:00:00Z",

    "icon": {
        "type": "Image",
        "mediaType": "image/jpeg",
        "url": "https://mastodon.jgarr.net/icon.jpg"
    },
    "image": {
        "type": "Image",
        "mediaType": "image/jpeg",
        "url": "https://mastodon.jgarr.net/image.png"
    }
}

If you want to see your user's JSON document you can append .json to your user's URL (e.g. https://mastodon.social/@jgarr.json)

Not everything in this example user document is required, but here's the first place we can lie about our account and make it look more legitimate.

You'll noticed the published date 2000-01-01T00:00:00Z which means we created an account long before Mastodon existed.
Not a big deal, but it's completely unverified.

We also add an icon and image to the profile so it doesn't have the default image.
The images are completely optional, but it adds legitemacy to a federated account posing as a real user.

Because we own the domain and can lie about the accounts we can use a commonly misspelled domain or unicode to create fake accounts.
We could easily use any domain to make accounts like @charles@gov.co.uk or @tim@apple.ceo (both of these domains are currently available).

Mastodon puts the zero in zero trust.
In reality, any completely decentralized system—like the internet—only has trust through reputation, but in Mastodon you can fake a repulation.

Here's what the profile looks like.

After the user is requested your Mastodon instance will automatically fetch /followers and /following.
Just like other documents these are reference documents to the actual data documents, but the data isn't verified so we can lie again.

You'll notice this account has 1 million followers and follows 1 account.
Both of which are not possible because even if you click the follow button the instance cannot acknowledge your request and this account has no keys so it cannot follow any accounts.

GET https://mastodon.jgarr.net/followers

Here's the full access log so you can see the request.

{
    "request": {
        "remote_ip": "127.0.0.1",
        "remote_port": "43692",
        "proto": "HTTP/1.1",
        "method": "GET",
        "host": "mastodon.jgarr.net",
        "uri": "/followers",
        "headers": {
            "User-Agent": ["http.rb/5.1.0 (Mastodon/4.0.2; +http
                s: //mastodon.social/)"], "Accept": ["application/activity+json, application/ld+json"], "Accept-Encoding": ["gzip"], "Date": ["Wed, 30 Nov 2022 06:00:12 GMT"], "Signature": ["keyId=\"https://mastodon.social/actor#main-key\",algorithm=\"rsa-sha256\",headers=\"(request-target) host date
                accept\ ",signature=\"wUAArkeEJh4yXkstcC8IgrnSlsRcledOUjo63nqRZrXI0RtoKo369/+j5K7bEFDoJ8psuCnnY9cW+KDgog7Gg2mQjAb1cZa2ffeqFY3PPXqpO+5entfRkAEyYBsrd3CiVn5wz0LEwbOs3XHe1w2wVgoIbSunCE/DN0Ra5tQLriITzBA5YzI26QuQSJzb5sMmMjiTiVocF/i0djqXfLmnjvhyaxsS0i0O8LfPHVPzSSGFHaqzawIL28MZu8J42ha//baJmP
                ozQQquFHKs7lcDcSSGtrvMGjfJYoFy4cMSsSqLH / 8 VRzNR0nXs47ydDwQ9XRpT55LPWL7uRQoeYBAkwA == \""
            ],
            "X-Forwarded-For": ["fd7a:115c:a1e0:ab12:4843:cd96:626f:140a"]
        }
    },
    "user_id": "",
    "duration": 0.000082826,
    "size": 235,
    "status": 200,
    "resp_headers": {
        "Server": ["Caddy"],
        "Etag": ["\"rm1lyn6j\""],
        "Content-Type": [],
        "Last-Modified": ["Mon, 28 Nov 2022 05:30:23 GMT"],
        "Accept-Ranges": ["bytes"],
        "Content-Length": ["235"]
    }
}

GET https://mastodon.jgarr.net/following

Here's the full access log so you can see the request.

{
    "request": {
        "remote_ip": "127.0.0.1",
        "remote_port": "43678",
        "proto": "HTTP/1.1",
        "method": "GET",
        "host": "mastodon.jgarr.net",
        "uri": "/following",
        "headers": {
            "Date": ["Wed, 30 Nov 2022 06:00:11 GMT"],
            "Signatur
            e ": ["
            keyId = \"https://mastodon.social/actor#main-key\",algorithm=\"rsa-sha256\",headers=\"(request-target) host date accept\",signature=\"a3EqAl1mUNhvTQvgWCng44mJpxSNcYo1CnTlUH8qy9i84S3bBSR6tIdHvK1dpoLI2+evgUvyLW0H8l20dG9jzLUIUPoXyTG+TapAKr6Z9i80F20IxoInQzoZVl3ytgkMqGw2EFV0fU2/K18/Z+
            wECJCQoFivt / QcMXPs7ox / EqxikZ + WyKsBX / TprzqFTSfg / ozpEluAxLmfNsN3IxYnb8XAGZlZC2n4Vkg9Ue + LYHH7PhLq3XAdQPgCKSI1IUxZVpeo0WttESxhAmoxVMd5bXSGJTVFInqKSH3J8UyhwbPKcCWm4oFnuVGAeZuL1UwyIQsiFgj53pU6oV + zwzZrJQ == \""],
        "X-Forwarded-For": ["fd7a:115c:a1e0:ab12:4843:cd96:626f:140a"],
        "User-Agent": ["
            http.rb / 5.1 .0(Mastodon / 4.0 .2; + https: //mastodon.social/)"], "Accept": ["application/activity+json, application/ld+json"], "Accept-Encoding": ["gzip"]}}, "user_id": "", "duration": 0.000069315, "size": 230, "status": 200, "resp_headers": {"Last-Modified": ["Mon, 28 Nov 2022 05:30:23 GMT "], "
                Accept - Ranges ": [" bytes "], "
                Content - Length ": [" 230 "], "
                Server ": [" Caddy "], "
                Etag ": ["\"rm1lyn6e\""], "Content-Type": []
        }
    }

/following

{
    "@context": "https://www.w3.org/ns/activitystreams",
    "id": "https://mastodon.jgarr.net/following",
    "type": "OrderedCollection",
    "totalItems": 1,
    "first": "https://mastodon.jgarr.net/following_accts"
}

/followers

{
    "@context": "https://www.w3.org/ns/activitystreams",
    "id": "https://mastodon.jgarr.net/followers",
    "type": "OrderedCollection",
    "totalItems": 1000000,
    "first": "https://mastodon.jgarr.net/follower_accts"
}

Mastodon never validates the data in /follower_accts that we claim holds our 1 million followers so we don't have to create that file.

If you click the follow button your Mastodon instance will send a POST request to /inbox with your user's key signature.

POST https://mastodon.jgar.net/inbox

"request": {
    "remote_ip": "127.0.0.1",
    "remote_port": "42294",
    "proto": "HTTP/1.1",
    "method": "POST",
    "host": "mastodon.jgarr.net",
    "uri": "/inbox",
    "headers": {
        "Content-Type": ["application/activity+json"],
        "Digest": ["SHA-256=8we9H5V74oUdQr8R5vay/dyQEi0I2up5wwI7+9e8T70="],
        "Signature": ["keyId=\"https://mastodon.social/users/jgarr#main-key\",algorithm=\"rsa-sha256\",headers=\"(request-target) host date digest content-type\",signature=\"hx1jRjCGyBfnI/Cak8ujAlfau5G1Ph+9niCFyRdm5J7b9wQGxbk+SbUhG0kV2L7W0h54JBc6htQhR8V+fFqxX+UiLXe1l7jRoBZOYSKq7UKqtogJwLLvS89DeDgWLWDPqbZ6W1FzU9MLUqJLTqFNnhgtOH+m+YhKEfjE35+65d5vmPUNjR8TRDAjXjMugMi3NmeaeA789NV3gs7GaGyfI734kGwvPDHcLp9MyDHqBivdDqmPAzXRP4gyrjqXHQRpxCdX7iinA/aqgnsNf2CoY/uH7M+z+zPWohlTvVDU2L+xeT2N7pXFc6WxREPV4ojZ+VxMzmIuzHkxW8TVpVNwMw==\""],
        "X-Forwarded-For": ["fd7a:115c:a1e0:ab12:4843:cd96:626f:140a"],
        "User-Agent": ["http.rb/5.1.0 (Mastodon/4.0.2; +https://mastodon.social/)"],
        "Content-Length": ["779"],
        "Accept-Encoding": ["gzip"],
        "Date": ["Wed, 30 Nov 2022 07:01:44 GMT"]
    }
}, "user_id": "", "duration": 0.000112712, "size": 0, "status": 404, "resp_headers": {
    "Server": ["Caddy"]
}

Unlike the requests before, this request will use the mastodon.social user's signature and key instead of the instance actor account.
My instance should connect back to the mastodon.social server to verify the user's signature, but you'll notice the status 404 because I didn't implement following or create an inbox file.

Even though the status is 404 the requesting server still shows a follow request is sent.
If you cancel the request it will decrement the followers count.

What doesn't work

Those 6 files is all you need to create a Mastodon user.
Here are some caveats you may have already noticed.

Following doesn't work
Posts don't work
Only 1 user per domain

You can create JSON objects with posts, replies, or anything you'd like, but Mastodon instances don't fetch posts from external users unless someone from that instance follows the user or has reposted one of their posts.
I implemented a single post in the /outbox file so if you want to see how they are structured you can browse the source files.

The instance is supposed to fetch pinned posts, but I couldn't figure out how that is implemented.
If someone knows please reach out and let me know at my real mastodon account @jgarr@mastodon.social.

Next, we'll give this instance some of the functionality that doesn't work.
We'll allow users to follow the account, and then let it create posts.

Reverse engineering a chrome extension

Justin Garrison — Thu, 27 Jan 2022 06:22:35 +0000

I've been using Revue for my 123dev newsletter and wanted an easier way to save URLs to include in future emails.
If you're not familiar with it, Revue has a chrome extension so you can send URLs to a queue which shows up next to the editor.

It's a really handy feature and I wanted to use it without the extension.
Ideally, I could send these URL from my phone via a Siri Shortcut (I haven't figured this part out yet).

The functionality wasn't exposed in their API docs so I'd have to figure out another way.
I learned some new things exploring the extension so I thought I'd share how I did it.

Reverse engineering the extension

The first thing I needed was to figure out what URLs the extension was calling.
I tried watching Chrome dev tools for network calls, watching DNS requests, and tcpdump.

Without having a man in the middle to decode https it wasn't going to work.
Thankfully, someone pointed out the code is available if you have the extension installed.

First, we need to get the extension ID from the installation URL.

The long string in the URL fdnhneinocoonabhfbmelgkcmilaokcg will be in our home folder with the source code.

On my computer it's under $HOME/.config/google-chrome/Default/Extensions/fdnhneinocoonabhfbmelgkcmilaokcg.
I opened the folder in vscode and looked at the main.*.chunk.js file.

It was minified so first I had to unminify it as best as possible.
Formatting the javascript was as good as I could get it.

From there I looked for POST url verbs to see what it was calling.
I found this relevant code which looked like what I needed.
It's calling https://www.getrevue.co/extension/add.

You'll see from the code the only thing it's sending is a POST with a body.
At this point I don't know what the body should be, but I'll try to figure that out later.

Getting session cookie

Now I need to jump over to Chrome to get my session cookie.
Open getrevue.co in a tab and open dev tools.

Go to the Application tab and then find Cookie in the left sidebar.
Copy the value for _revue_session.

Send a curl request

Now we need to send our request and see if it works.

We still don't know what the body data should look like, but looking at the API objects that are documented I'm going to guess it needs a title and url.

export COOKIE="your cookie session here"

curl -X POST -b "_revue_session=$COOKIE" \
    -H "Content-Type: application/json" \
    -d '{"title": "TESTING", "url": "https://justingarrison.com"}' \
    https://www.getrevue.co/extension/add

Sure enough that worked!

Here's a snippet of the response

The response gives us a much better idea of the full body data we can use.
Adding a description will be a minimal amount of information that would be useful.

Now we can send items from the CLI but what about from iOS?

[WIP] Siri Shortcut

Siri shortcuts are very powerful but also very cryptic.

I was able to make a shortcut with the "Get contents of URL" function which is able to make a POST call.

I can put in the URL, change the method to POST, and add a body with the required title and url variables.

Unfortunately, when I try to use this shortcut from the share sheet I don't think it uses my session token so I never get authenticated to the API.

If anyone knows a way to either open a Safari page and perform the action or a way to store an authentication token in the shortcut please reach out on twitter and let me know.

Debug Kubernetes Clusters in 8 Commands

Justin Garrison — Wed, 20 Oct 2021 04:45:44 +0000

I wrote this post for The New Stack and wanted to share it here on dev.to also.

If you’ve lived with any system long enough, you have assuredly had to debug it. Kubernetes is no different. It is a distributed system with a lot of moving parts and sometimes those parts need to be understood by humans. In this post, we’ll look at 8 commands you can run to start debugging any Kubernetes cluster.

There are lots of optional components in a cluster, but we won’t focus on all of them here. We will discuss troubleshooting workloads in a future article. This post will focus on cluster operations. It will help you understand a cluster and make sure the core functionality — running pods — is available.

This post will assume you have admin access to the cluster. We will pretend you’ve been handed a kubeconfig file with access to a cluster and you were told the cluster is broken. Where do you start?

Here are the eight commands to run:

kubectl version --short
kubectl cluster-info
kubectl get componentstatus
kubectl api-resources -o wide --sort-by name
kubectl get events -A
kubectl get nodes -o wide
kubectl get pods -A -o wide
kubectl run a --image alpine --command -- /bin/sleep 1d

Let’s break down each command to understand why it’s important and what you should be looking for. For cluster debugging, we’re going to take a breadth first approach to understand what is in the cluster before we dive deep into workloads.

kubectl version --short

With this command, we’re looking to see what version of the API server is running. This gives us important information later when we’re troubleshooting specific errors and it’s very useful to know if we’re on an older cluster like 1.16.

Knowing the version can also help us when searching for errors and reading changelogs. There may be known issues that require a version upgrade or a newly introduced bug. There are sometimes version compatibility issues between different components and knowing what version is running is the first step.

kubectl cluster-info

Next we should understand where the cluster is running and if CoreDNS is running. You can parse the control plane URL to know if you’re dealing with a hosted cluster or something on-prem.

In this example output, we can tell that we’re running an Amazon Elastic Kubernetes Service (Amazon EKS) cluster in the us-east-2 region. This information is also useful to look up if there is a current outage with your provider. You can look at your provider's service health dashboard to know if the current issues are with your cluster or something outside of it.

This can also give you a clue if additional authentication will be needed for the cluster. There could be an AWS Identity and Access Management (IAM) permission issue or maybe you need to install an authentication plugin like the aws-iam-authenticator.

kubectl get componentstatus

This command will be the easiest way to discover if your scheduler, controller-manager and etcd node(s) are healthy. These are all critical control plane components to run your pods. You should look for any component that doesn’t show an “ok” status and look for any errors.

If you are using a cluster with a managed control plane (e.g. Amazon EKS) you may not have direct access to the scheduler or controller manager. Being able to see their status from this output is likely the easiest way to know whether something is wrong with etcd or another component.

It’s also important to note that the componentstatus command is deprecated in the CLI but not yet removed. There isn’t currently a replacement for this command, but until it is removed from the CLI it is safe to use and very useful. Depending on your cluster it may require multiple commands to get similar output. There are design limitations to this command which is why it has been deprecated.

An alternative option to see other health endpoints (including etcd) is kubectl get --raw '/healthz?verbose'

Although this command does not show scheduler or controller-manager output, it adds a lot of additional checks that might be valuable if things are broken.

kubectl api-resources -o wide --sort-by name

This is the first command with a lot of information. We already know what version and where our cluster is running. At this point, we should know whether the control plane is healthy and now we need to look at some of the resources inside the cluster.

I like to list all the resources sorted by name for consistency. It’s easier for me to scan the resources in alphabetical order. Adding -o wide will show the verbs available on each resource. This can be important because some resources do more than others. Knowing what verbs are available — or aren’t — will help narrow down where you should look for errors.

Using this command will tell you what CRDs have been installed in your cluster and what API version each resource is at. This could give you some insights into looking at logs on controllers or workload definitions. Your workloads might be using an old alpha or beta API version but the cluster may only use v1 or apps/v1.

kubectl get events -A

Now that we have an idea of what’s running in the cluster, we should look at what’s happening. If something broke recently, you can look at the cluster events to see what was happening before and after things broke. If you know there’s only a problem in a specific namespace, you can filter the events to just that namespace and block out some of the additional noise from healthy services.

With this output, you should focus on the type of output, reason and object. With those three pieces of information you can narrow down what errors you’re looking for and which component may be misconfigured.

kubectl get nodes -o wide

Nodes are a first class resource inside Kubernetes and are fundamental for pods to run. Using the -o wide option will tell us additional details like operating system (OS), IP address, and container runtime. The first thing you should look for is the status. If the node doesn’t say “Ready” you might have a problem — but not always.

Look at the age of the nodes to see whether there’s any correlation between status and age. Maybe only new nodes have problems because something changed in the node image. The version will help you know quickly whether you have version skew on the kubelets and possibly have known bugs because of different versions between the kubelet and API server.

The internal IP can be useful if you see IP addresses outside of the subnet. It’s possible a node started with an incorrect static IP address and your CNI cannot route traffic to a workload.

OS image, kernel version and container runtime are all great indicators for possible differences causing a problem. You may only have a problem with a specific OS or runtime. This information will help you quickly zero in on a potential problem and know where to look deeper at logs.

kubectl get pods -A -o wide

Finally, our last information-gathering command. As with listing nodes, you should first look at the status column and look for errors. The ready column will show how many pods are desired and how many are running.

Using -A will list pods in all namespaces and -o wide will show us IP addresses, nodes and where the pods are nominated. Using the information from listing nodes, you can look at which pods are failing on which nodes. Correlating that information with details like OS, kernel, and container runtime might give you the insights you need to fix the cluster.

kubectl run a --image alpine --command -- /bin/sleep 1d

Sometimes, the best way you can debug something is to start with the simplest example. This command doesn’t have any direct output, but you should see a running pod named “a” from it.

If I’m not debugging something with other people, I like to name my debug containers single letters because it’s faster to type and it’s easy to iterate (e.g. b, c, d). I often like to keep old containers around while I’m debugging because sometimes it’s helpful to see what’s different with previous pods and it’s easier to leave them running — or crashing — than to search my terminal output history.

If for some reason you don’t see a running pod from this command, then using kubectl describe po a is your next best option. Look at the events to find errors for what may have gone wrong.

Conclusion

With these commands, you should be able to get started with any cluster and know if it’s healthy enough to run a workload. There are additional things to consider such as CoreDNS scaling, load balancing, volumes, and central logging and metrics. The commands here should work in the cloud or on-prem.

If you need to troubleshoot nodes or external resources (e.g. load balancers), then you should look at your controllers and API server logs for errors. Depending on what is broken you may need to look at kube-proxy, CNI plugins or service mesh sidecar container logs.

Hopefully, these 8 commands will help you narrow down the scope of what could be broken in your cluster. If you don’t know what the problem is then starting with a breadth first search is the best approach. Pay attention to anomalies of mismatched versions and nodes or settings that are different. If you look for what has changed it can point you in the right direction to find and fix the problem quickly. We will look at how to debug workloads in your cluster in a future article.

Kubernetes Cluster Upgrade Patterns

Justin Garrison — Wed, 21 Apr 2021 17:01:08 +0000

I wrote this article for The New Stack and wanted to re-share it here for the dev.to community.

If you’ve been using Kubernetes for any amount of time, you need to plan for regular upgrades. Starting with Kubernetes 1.19, each open source release provides one year of patches. You need to upgrade to the latest available minor or patch release to receive the security and bug fixes. But how can you upgrade a critical part of your infrastructure without downtime? This article will guide you through common patterns to consider when upgrading Kubernetes in any environment.

We won’t dive into all of the tools and considerations to perform an upgrade. If you are using a cluster management tool or hosted Kubernetes service, you should consult your documentation for the best option for your environment. You also need to be aware that some workloads and environments may restrict what upgrade strategy you choose.

We will discuss a few high-level patterns for cluster upgrades:

In place
Blue/Green
Rolling
Canary

These patterns are similar to application upgrade options, with some unique considerations because of their potential blast radius. Upgrading infrastructure can incur a considerable cost, depending on how long your upgrade takes and how large your environment is.

Control Plane Components

The Kubernetes control plane consists of the Kubernetes API server, etcd database, controller manager, scheduler, and any additional controllers such as cloud or ingress that you may have in your environment. Upgrading the API server is the first step when upgrading a cluster. Kubernetes stores state in etcd and with any major application upgrades, you need to make sure you have at least one backup and that you’ve verified the backup can be restored. In some cases, an API server upgrade may also require an etcd upgrade, but we won’t be covering that in this post.

Data Plane Components

The Kubernetes data plane consists of the kubelet, a container runtime, and any network, logging, or storage drivers you use in your cluster workloads. For many clusters this will — at minimum — require a kube-proxy and CNI plugin update. Your data plane components must be equal to, or one minor version behind, your API server version. Ideally, your host OS, container runtime and data plane components can be upgraded independently from each other. Decoupling these components will ensure that you can upgrade quickly when there are bug fixes, new features, or security patches.

Kubernetes Hosted Service

If you’re using a hosted Kubernetes service such as Amazon Elastic Kubernetes Service (EKS), control plane upgrades are handled for you. If you’re using a managed data plane service, such as Managed Node Groups (MNG), your data plane upgrades should also be automatically handled by your provider.

Even with a hosted service, you are still responsible to verify workloads, additional controllers, and third-party plugins (such as CNI) that you have installed in your cluster. Those components should be tested for API compatibility before you upgrade your cluster in a test or development environment. We talked about making sure your workloads and controllers are compatible with different Kubernetes API versions in a previous blog post.

With any Kubernetes upgrade, you should upgrade components in the following order:

Control plane
Data plane and nodes
Add-ons
Workloads

These upgrade patterns will help you decide how to upgrade those components that best suit your cluster and environment.

In-Place Upgrades

When performing an in-place upgrade, you must be extra careful that components remain healthy, because you are performing work on a cluster currently serving production traffic. In-place upgrades can consist of package updates (e.g., yum, apt), config management automation (e.g., Ansible, Chef), or VM/container image changes. Ideally, your upgrade will be scripted and automated — including roll-back — but if this is your first time upgrading, doing it manually in a development or test environment may be helpful.

In-place upgrades mean that all the components will upgrade at roughly the same time. If you change your desired API server version with configuration management and push a new configuration, all of the API servers will upgrade once they receive the new configuration. This is different from rolling upgrades, which we discuss later.

The main benefits of in-place upgrades are:

It is the fastest to perform at any scale.
If done manually, it can allow for more control of the components and upgrade process.
It is easily suited for multiple environments (on-prem or cloud).
It is the cheapest from an infrastructure cost perspective.

Depending on your process, scale and tooling, an in-place upgrade is probably the most straightforward approach to be able to script and roll out. Scripts can be tested locally or in development clusters, without needing to reconfigure resources that cluster administrator teams may not have access to — such as load balancers or DNS.

In-place upgrades also have the following restrictions to consider if you want to use this method for upgrades:

It is possible to cause downtime if all of your API servers or controllers upgrade at the same time.
If you want to move from Kubernetes 1.16 to 1.20, you’ll have to upgrade the entire cluster four times to each minor version.
Validating each step may be a manual process, which can add additional time and opportunity for mistakes.
You should test a rollback plan in case of failure because some upgrades cannot be easily reverted. (e.g., scheme changes).

Blue/Green Upgrades

Blue/green cluster upgrades will require you to create a second cluster with a new version of Kubernetes. You will need to deploy a new control plane and data plane, then copy all of your workloads to the new cluster before you switch traffic from the old cluster to the new one. You can use blue/green to update each component of your cluster, but a holistic cluster upgrade is easier to deploy and rollback.

The good news is, setting up new clusters is generally easier than upgrading a cluster in place. You have multiple options on how you deploy workloads to the new cluster. If your workloads are already part of a GitOps or continuous delivery, you can have deployments go to the new cluster simultaneously to the old cluster, prior to or during your upgrade. If you do not have deployments automated, you use a tool like Velero to back up your existing workloads and deploy them to the new cluster.

Creating a new “green” cluster can give you a lot of confidence that the new version works as you expect and puts you in control over when you switch versions. The new cluster can also be used to validate automation tooling, such as Terraform modules or GitOps repos. You can make the change via DNS or load balancers whenever you are ready, or even during a maintenance window or low utilization time.

Key benefits of a blue/green upgrade are:

Pre-validate that all components are healthy before sending traffic.
You can upgrade multiple versions at a time (e.g., 1.16 directly to 1.20).
You can change other parts of the infrastructure that might be difficult to test (e.g., switch regions, add zones, change instance types).
It is the safest and easiest to roll back.
Downsides for blue/green deployments to consider include:
This is the most expensive strategy in infrastructure cost, because you have to run twice the compute capacity during the migration.
You may not be able to get all of the compute capacity you need to run a complete second cluster if you have thousands of worker nodes.
This strategy is hard to scale to dozens or hundreds of clusters if you have multiple concurrent cluster upgrades.
Blue/green is not easy to do on-prem without virtualization, unless you have spare servers.
Switching all traffic at once may not be easy if you have lots of endpoints to update. Load balancers may need to be pre-scaled and caches warmed. Beware of DNS time to live (TTL) which may or may not work for spreading load.
Switching all cluster traffic at once requires cross-team coordination to migrate to new clusters; as well as engineering cycles to verify that workloads are the correct scale.

Blue/green can be a great strategy when you have a smaller number of clusters or fewer than a couple of hundred worker nodes. It allows you to skip versions and it is safe for rollbacks, but beware how much it may cost you in infrastructure spend and coordination time.

Rolling Upgrades

If you’re familiar with Kubernetes deployment strategies, you’ll be familiar with rolling upgrades. A rolling upgrade will deploy one new copy of a component and then scale down one old copy. It will continue this pattern until all of the old components have been removed. The incremental nature of rolling upgrades has some advantages over in-place and blue/green strategies.

Similar to in-place upgrades, you’ll need to upgrade one minor version of Kubernetes at a time. This can be additional work when needing to upgrade multiple versions, but it’s the only supported option. Depending on the component you’re upgrading, you may use different tools to upgrade each.

For resources like the control plane, you may want to add a new server with an upgraded API server to the control plane and then shut down an old server. If you’re in AWS, you could change an Auto Scaling group launch configuration AMI and replace one instance at a time. Other control plane components (e.g., scheduler) may be running as containers inside your cluster, so you can upgrade those with standard Kubernetes rolling deployment upgrades.

The key difference in a rolling upgrade compared to blue/green is that your external traffic routing (DNS and load balancers) will stay pointed to the same place. You will want to make sure you test all add-ons and workloads in a different cluster or environment before you move forward with a production cluster upgrade.

Note that AWS managed node groups, kOps, Cluster-API and many other Kubernetes cluster management tools use a rolling upgrade strategy. Benefits include:

Safer roll-outs and roll back compared to in-place upgrades.
Costs less than blue/green and less likely to run out of resources.
Can be paused mid-upgrade if something breaks.
Can be adapted to on-prem environments.

Rolling upgrades are the most common for automated tools. They have a good balance between speed and cost, and even though they’re not fully immutable, they’re still immutable in the right areas to reduce manual work and risk.

When upgrading a production cluster, all of your existing workloads will still be deployed; and as long as you’ve tested their compatibility, your upgrades should be automatable.

Further considerations when using rolling upgrades include:

Rolling upgrades can be slow depending on your scale.
You may need to coordinate controller, daemonset, or plugin upgrades during the rollout.
You might not be able to make cluster-wide changes, such as adding an availability zone or changing architecture.

Canary Upgrades

Canary application deployments serve small increments of traffic to a new version of an application at a time. Canary upgrades can be thought of as rolling upgrades with blue/green benefits.

With canary upgrades, you will create a new Kubernetes cluster with the version you want to deploy. Then add a small data plane and deploy your existing applications at smaller scales to the new cluster. Add the new cluster workloads to your existing production traffic via your load balancer configuration, DNS round-robin, or service mesh.

Now you can monitor traffic going to the new cluster, slowly scale up workloads in the new cluster and scale down workloads in the old cluster. You can do this one workload at a time and as slowly or quickly as you’re comfortable with. If any individual workload starts to get errors, you can scale down the individual workload in the new cluster to have it automatically use the old cluster.

The benefits to canary cluster upgrades include:

New clusters are easier to create and validate.
You can skip minor Kubernetes versions during upgrades (e.g., 1.16 to 1.20).
Application deployments can be opt-in on a per-team basis.
Errors are minimally impacting due to incremental traffic usage.
You can make large infrastructure changes during upgrades.
Clusters start small and scale, so infrastructure costs are lower and you can warm caches and load balancers as you scale.

If you want to make large changes (such as changing architecture) or you want to add an additional availability zone, then canary is a great option. By starting the cluster small and growing it per workload, you can make sure that you’re not over-provisioning infrastructure if your new instances are more efficient or workload requests and limits have changed.

As with anything, there are trade-offs. When using canary deployments, you should be aware of some of the following concerns:

Rolling back an application may require manual intervention to change a load balancer or scale down the new cluster.
You may end up with an old cluster longer than you want, as applications slowly deploy and scale-up.
Debugging applications can be harder because you need to know which cluster errors are happening.
If you have dozens or hundreds of clusters, you will likely increase your cluster count by 50% or more as clusters are being upgraded.
Canary is the most complex upgrade strategy, but it benefits from automated deployments, health checks, and performance monitoring.

Conclusion

No matter which upgrade strategy you pick, it’s important to know how they work and any possible concerns as your Kubernetes usage grows. You need to have an upgrade strategy because Kubernetes has frequent releases and (like any software) occasional bugs.

Keeping up to date with new versions can be an important part of your infrastructure security process and will enable applications to take advantage of new features quickly. If you deployed Kubernetes and migrated all of your workloads without considering how you would upgrade, now is the perfect time to start planning.

If you do not have a business need to run your own Kubernetes clusters, I highly recommend you use one of the hosted Kubernetes options available. Opting into a managed control plane and data plane can save you days or weeks of planning and upgrades each year. Each hosted option may perform upgrades differently, but they will all allow you to focus on your workloads and business value instead of control plane high availability or data plane compatibility.

The Document Culture of Amazon

Justin Garrison — Mon, 15 Mar 2021 17:30:29 +0000

In my time at Amazon, I've observed the way we use documents is incredibly unique.
A lot has been written about the six-pager and PR/FAQ so I'm not going focus on document formats, but I wanted to share how our process benefits from document-based meetings.
I also have identified some areas for improvement if you are looking to adopt document-based meetings for your workplace.

I've been wanting to write this blog post for a while and Jaana's tweet finally gave me the inspiration to share my thoughts.

Jaana Dogan ヤナドガン

@rakyll

Meetings starting with silent document reading is the best idea ever. I have no idea why this isn't common practice in the industry.

20:20 PM - 10 Mar 2021

My role is a Sr. Developer Advocate within AWS Container Services so my experience may be different from other areas and roles at Amazon.
A majority of my meetings start with reading a document.
This takes "this meeting could have been..." and flips it upside down.
Most of the time, if there isn't a doc there isn't a meeting.

Depending on the meeting, the document could be a six-pager, a PR/FAQ, a one-pager about an idea, a narrative to help find a solution to a problem, or even a service review full of charts, graphs and bullet points.
The document adapts to fit the audience and purpose of the meeting.

Reading documents is so ingrained in our culture and process that our scheduling tools have checkboxes to automatically create a document.
If I'm catching up on a new service or feature launch, I will find the document rather than emailing or calling the product manager.

The interesting part to me isn't in the format of the document, but how it is used.
Meetings start with reading.
Depending on the length of the document, we'll read anywhere from ten minutes to half an hour.
If the meeting has a long document (six-pagers are the longest) and many attendees, the meeting will be scheduled for enough time to read and discuss.

Reading the doc is part of the scheduled time.
I've worked at plenty of places that I've tried to document everything for a meeting ahead of time.
I've written well thought-out emails, shared links to documents, and written detailed wiki pages.
In all of my meetings outside of Amazon I had one of three outcomes:

No one read the email/document and I had to explain everything in the meeting
Some people read the document but forgot what it said because they read it days or hours before
At least one person had a question that I could have answered via email

Before I started at Amazon these were some of the expected benefits to reading in meetings.

All of the information for the meeting is contained in the doc
People are not expected to find their own time to read
The document is fresh in everyone's mind

Some other benefits I've found while putting it in practice are:

Presenters, including myself, don't have to feel nervous about presenting in front of people.
Documents help eliminate many biases, for or against, the person who wrote the document.
You read everything in your own voice and vocal communication barriers such as accents, vocal ticks, or handicaps (e.g. stuttering or hearing loss) are not present in the document.
There's no "can you see my screen", background noise, or call audio disconnects while understanding the main content for the meeting.
There's a wealth of current and past documents. Want to see what the PR/FAQ looked like for Amazon EKS or AWS Lambda? It exists.
You are not expected to retain document information outside of the meeting. Feedback and discussions happen during the meeting. Comments can be answered asynchronously. If more discussion is needed then the document will be revised and a new meeting will be scheduled to read and discuss it.
If a document is provided early and I have a conflict at the begining of the meeting I can read the document ahead of time and join the meeting 10-20 minutes late without missing anything.
Document reading from home has been a great way to get 10-20 minutes of exercise in.

Justin Garrison

@rothgar

Reading docs at the beginning of meetings is a great way to squeeze in some exercise!

I'm trying to make this a new habit. Even a 10 minute climb is enough to get my heart rate up.

Pay no attention to the cables 🙈

17:30 PM - 15 Jan 2021

There are some limitations to document based meetings.
The first I have noticed is if you're not a very good writer you will be at a disadvantage communicating your ideas.
Amazon has a lot of internal training to help you write good Amazon documents, but even with training it still requires a good foundation.
I'm very thankful for my years of practice writing for How-To Geek and Cloud Native Infrastructure.
Even with that background, my first one-pager review was intimidating.

I've been told "if there's no document, it doesn't get done."
This includes meetings.
If there's no document, we cancel the meeting.

While documents can create a more level playing field it's also a high barrier to entry.
Even for small ideas, features, or iterations the first thing you will likely need is a document.
My perspective may be skewed because in my current role I don't write production service code, but I do affect features, design, and positioning of the services.
I have a lot of feedback and ideas, but I don't implement them.

The available wealth of historical documents can be enlightening to read through, but it also can be confusing to trace the lineage of a service with a plethora of documents and comments.
I have caught myself more than once reading a document which I thought sounded similar to an existing service only to realize 20 minutes later the document is for the service and I didn't look at the date of the document or I didn't know the product code name.

I have the benefit of working with the service teams and providing direct feedback about new features and services.
Most documents I read are very interesting.
Many teams at Amazon don't have direct product input.
However, they're still required to write documents to communicate plans with their team.
Not all docs are interesting, but they are crucial to the decision making process.

I've said multiple times that Amazon has the foundation for a great remote-first company.
The document-based process provides employees in multiple timezones the same context as someone in Seattle.
The discussions in meetings enable faster feedback loops, but the downside is they can limit the potential for asynchronus engagements if notes and questions aren't captured in the document.

I've never had the benefit of attending an Amazon meeting in person.
From what I've heard document-centered meetings have been extremely successful in transitioning to remote requirements.
But the content isn't the only thing that matters.

Amazon, like many large organizations, uses a lot of tools to communicate.
It's often difficult to find documents spread across so many tools without an ability to cross search or centrally organize them.
I could see how a tool like Command E could help a lot.
This can be additionally difficult with the multitude of open source and social platforms I engage with on a weekly basis.
I currently have a user in 53 Slack workspaces. 💀

Even with areas that could use improvement I can't think of a place I have worked that wouldn't benefit from document-based meetings.
I'm sure with more practice I will get better at writing with a more Amazonian style.

I thoroughly enjoy not having to prep for meetings, because I'm confident the document will give me the context I need to participate.
I know the person who called the meeting invested their time so they don't waste mine.
I recipricate the same thoughtfulness with meetings I create and documents I write.

Thank you everyone who reviewed this document. ☺️
Banner image credit pixbay

Getting started with signal

Justin Garrison — Fri, 29 Jan 2021 07:08:04 +0000

You've discovered that Signal can provide you with secure communication from your phone that is cross platform and free!

First of all you may be asking "who makes Signal?"
Signal Messenger LLC. is the company that makes the Signal app.
They are funded by the Signal Foundation which is a non-profit organization that relies on your donations to make the app free and not have ulterior motives with your data.

The foundation's mission is

To develop open source privacy technology that protects free expression and enables secure global communication.

The Signal app is all open source on GitHub and their protocols are well documented and available on their website.

But that's not why you're here.
Let's show you some tips to get the most out of your new secure messaging app.

Important settings

Signal is great, but there are a few things you may want to change depending on how secure or private you want your communications.

Notifications.

You can disable what is shown in notifications to help with privacy.
Turn off whatever you don't want shown.

You may also want to disable the notification you get when one of your contacts join Signal.
Especially ad the rapid pace people are joining.

Privacy

You can set the Signal app to lock when your phone locks under privacy.
It may also be a good idea to enable incognito keyboard which will ask your keyboard to remove personalization while typing.

You can turn off read receipts, typing indicators, and automatic link previews under the communication settings.
Link previews only work on sites that use https://

Chats and media

In chats you can change when to auto-download files.
Some of these can be quite large so it may be a good idea to only automatically download on Wi-Fi.

At the very bottom of Chats and media is a message backup option.
The backup is encrypted and stored locally on your phone.
If you're using a backup service for your phone they will also be backed up to the cloud which you may or may not want (even if it's encrypted).

Linked devices

You can use Signal from another device including tablets and laptops.
Install the Signal app on the other device and it will show a QR code.
Open the linked devices setting and click the + button at the bottom.
Scan the QR code and you're all set.

If you tap on an existing linked device you have the option to remove it from being able to send or receive messages.
Be aware that any existing messages on the device may still be available and cached on the device.
This setting is not a "secure wipe."

Chats and calls

Signal chats work similarly to traditional SMS messaging apps.
You can send text, emoji, and stickers, or attach pictures, gifs (searched from giphy), files, contacts, or locations.
You can record voice messages by holding the microphone.

You can also make audio or video calls when you select a contact and click the icons in the top banner.
All calls are encrypted and can optionally go through a relay server so you do not expose your IP address to the recipient.
You can enable the relay server in settings -> privacy -> communication.

A relay is a server that Signal controls that your call will go through.
Similar to a VPN in the way it would expose the server IP to the recipient without exposing your IP.
If you're interested in the Signal server architecture you can check out this post.

A single hallow check box next to a message you sent means your message was sent.
Two check marks means the message was delivered.
Once the recipient reads the message the check marks will fill in.

If you see a little person icon next to the person's name in chat it means that person is in your contacts.
If you want their name to automatically be filled out you can go into your phone's contacts and add the number to a contact.

If you open a chat you can look in settings and enable disappearing messages.
This is super helpful to prevent messages persisting.

You can—and should—verify a contact to make sure you're really talking to them.
This verifies the encryption keys used at both ends of the conversation.
You can do that manually or automatically by scanning their QR code.

Ideally you will do this verification out-of-band from Signal either in person or through some other video/image sharing service.
Facetime, zoom, etc. are great options if you can't meet in person.

Once you verify a contact you'll see "Verified" text under their name.
If you manually mark the contact as verified it will show a message in the chat that you marked them verified.

Group messages

You can create a group message from the main screen clicking the pencil and then "New group."

Add contacts to the group and you're ready to go.

You have similar settings to 1:1 chats.
You can have a group video call with up to 8 people and a group chat with up to 150 people—RIP notifications!
If you want to have a group call without video you need to start a video call and disable your camera before you join.

You can change who can invite people to the group or make a group link easier for someone to share if they're not in your contacts.

The links are public but an admin needs to approve members joining the group by default.

Group video calls work the same way as 1:1 calls but groups don't have audio only calls—although you can disable your video in a video call.

Other things to consider

If you want to use signal without giving out your real phone number you can use a service like twilio that will provide you a forwarding number.

Check out this great guide on how to set it up.

Your safety number has changed

If you see this seemingly scary message don't be alarmed.
It likely just means the person on the other end of the conversation got a new phone or re-installed Signal.

You can still verify the persons safety number to make sure there is no man-in-the-middle attacks.
This is distinctly different than how iMessage and Telegram work because in both of those apps they store your private key.

Pins

Pins are important because they will enable Signal to get rid of phone number requirements.

The support page says it best

Your Signal PIN is a code used to support features like non-phone number based identifiers. This means that your PIN can recover your profile, settings, contacts, and who you’ve blocked if you ever lose or switch devices. A PIN can also serve as an optional registration lock to prevent others from registering your number on your behalf.

It hasn't rolled out yet but has been in the works for a while.

If you want to stay up to date on signal news I highly recommend you follow the Signal blog.
They have lots of great technical and non-technical information on new features and news.