Forem: Caleb Lemoine

How I use "AI" to entertain my cat

Caleb Lemoine — Fri, 03 Nov 2023 17:52:14 +0000

Background

It's been a while since I've made a dev.to post and I wanted to share my latest "dumb" project. My cat, Max, really enjoys watching wildlife in my backyard through the back door. There's birds, squirrels, opossums, and even a family of tree climbing rats. It's his favorite thing to do, but there's one "problem". The wildlife isn't always present and he gets pretty pissed about it. He often comes into my home office yelling at me to summon birds, but little does he know, I'm no Disney princess. What's frustrating for me is that even when wildlife is present, he'll often miss them because he's off doing other cat things. I thought, if only I could automate letting him know that he's missing some good bird watchin'!

Solution

I had the idea of pointing an IP camera out the back door and figure out how to detect animals then train him to go peek when a familiar sound is played. This was more difficult than I thought. After hours of reading, googling, browsing GitHub, there was no open source solution available to perform this simple idea out of the box. Welp, guess I'll build it.

Hardware

After doing some research on IP cameras that support an easy access video stream via RTSP, I got a cheap TP Link Tapo C100 camera for $20. I set it up on my home network using the app and placed it at my back door for a similar viewing angle of Max's.

Here's the setup, not super elegant, but it'll do.

And then here's the view from the camera itself.

Perfect view of the back patio to see all the animals.

Software

Next, I needed to figure out, how can I access the stream, recognize an animal, then let Max know? There are tons of examples of recognizing an object via camera frames, but I ultimately found this python library called ultralytics that supports RTSP streams and classifying objects in the video frames using pre-built models. The docs looked like it would be pretty low effort, so after some experimentation, I was successful in having the ultralytics library recognize objects from my cheap camera!

The gist that was needed to get it working:

from ultralytics import YOLO

model = YOLO('yolov8s.pt')
results = model.predict('rtsp://<username>:<password>@192.168.1.101/stream1', conf=0.5, stream=True, verbose=False)

for result in results:
    if result.boxes:
        box = result.boxes[-1]
        class_id = int(box.cls)
        detected_object_name = model.names[class_id]
        detected_object_confidence = round(float(box.conf), 2)
        print(f"Object detected: {detected_object_name}")
        print(f"confidence: {detected_object_confidence}")

OK, now I needed to recognize animals. Since the underlying model used by ultralytics is based on the widely popular COCO model, (Common Objects in Context), I found a list of the objects the model could detect. In that list, there were a few things that stuck out to me:

bird (of course)
cat
dog
mouse

After some testing of the previously mentioned code, I noticed that squirrels were often detected as dogs or cats based on the size of them. So I modified the code a bit to filter based on those objects supported by the model like so:

from ultralytics import YOLO

model = YOLO('yolov8s.pt')
results = model.predict('rtsp://<username>:<password>@192.168.1.101/stream1', conf=0.5, stream=True, verbose=False)

filters = ['bird', 'dog', 'cat', 'mouse']

for result in results:
    if result.boxes:
        box = result.boxes[-1]
        class_id = int(box.cls)
        detected_object_name = model.names[class_id]
        detected_object_confidence = round(float(box.conf), 2)
        if detected_object_name in filters:
            print(f"Object detected: {detected_object_name}")
            print(f"confidence: {detected_object_confidence}")

And boom! We recognize animals!

Next, I needed to wire this up to my home speakers and play a sound familiar to Max. In the before times of not watching live animals outside, Max liked it when I'd play some bird videos on YouTube for him and they would all start with the same "chirp" sound. He knew this sound meant bird watching time. So I downloaded the video, extracted the audio, then split the chirp out into a custom 4 second .mp3 and stored it on my local Home Assistant instance which was already integrated with my Google Nest speakers. Luckily, Home Assistant's API is pretty friendly, but the docs definitely suck. Once I added the .mp3 file onto my Raspberry Pi where Home Assistant is hosted, I was able to trigger the sound to play on my speakers with this simple request to its REST API:

import requests

request_opts = {
    'url': 'http://192.168.1.100:8123/api/services/media_player/play_media',
    'method': 'POST',
    'headers': {
     'Content-Type': 'application/json',
     'Authorization': 'Bearer <Home Assistant API token>'
    },
    'data': {
        "entity_id": "<Nest Speaker ID>",
        "media_content_id": "/local/chirp.mp3",
        "media_content_type": "audio/mp3"
    },
    'timeout': 10
}
response = requests.request(**request_opts)
# Raise an error in the event of a non 2XX response code
response.raise_for_status()

Open source all the things

Then it dawned on me. This should be a configurable open source program that can be configured to do a thing, when a camera sees a thing, but also let a user pick how frequently they want to be annoyed with bird chirp sounds in their house. I wanted to make the program very configurable because there's no way I was going to write all of this again for my next computer vision related project. So I created cv-notifier. A program that's super configurable and allows anyone to replicate what I've done so far.

TLDR, input a config to cv-notifier and it'll handle the rest:

config:
  source: 'rtsp://$STREAM_USER:$STREAM_PASSWORD@192.168.1.101/stream1'
  schedule:
    startTime: '07:00'
    endTime: '18:00'
  webhooks:
    - url: http://localhost:8080/someDumbFutureAPI
      notifyInterval: 900
      objects:
        - bird
        - cat
        - dog
        - mouse
      method: 'POST'
      headers:
        Content-Type: application/json
        Authorization: Bearer $API_TOKEN
      body: >
        {
          "someKey": "$object_name detected with confidence score of $object_confidence"
        }

Results

Lots of techy jargon up until this point, but the question remains... what did Max think? Well at first it confused him. The sound that he previously knew which only came from the TV was now playing in the kitchen and he didn't know how to respond. So when the chirp would play, I'd pick him up, take him to the back door, and show him that it meant animals. After doing that 4-5 times, he started to get it. Then when the chirp sound would play, he'd scurry across the house to go see what animals were out there!

The app has been running every day for a few months now. There lies another question, does it still work for Max? Eh, not really. He's gotten a bit numb to it since he's learned it goes off frequently enough that he can go look at birds pretty much whenever he pleases. He'll still go look occasionally, but he's not excited about it as he once was. Nonetheless, it was a fun project, I learned some cool things about object detection, and now I have a product I can use when I want to detect more objects in the future and call some random API.

Fin

By this point, you may be thinking to yourself, "What does Max look like?". Well here's a picture of the little guy along with a video of Max bird watching!

Max	More Max

How to share a volume between cloud servers using DigitalOcean Spaces

Caleb Lemoine — Sat, 22 Jan 2022 19:51:33 +0000

Overview

So you need to share files across multiple servers in the cloud, but then you come to find out you can only attach a volume to one host! What do you do?!

Well, you have a few options:

Create an NFS mount using another server
- You could create a Network Filesystem using another server, but this introduces a few challenges:
- Your storage capacity is bound to one underlying server, creating a single point of failure.
- You need some decent linux chops to get this working and automate it.
You can create a filesystem mount using s3fs and DigitalOcean Spaces
- Since DigitalOcean Spaces uses Amazon's S3 protocol, we don't actually have to use AWS S3 to use s3fs, we just need storage that implements the protocol.

Why

Why would you use object storage for this filesystem to share between servers?

It's cloud native.
It's highly available.
It's performant.
It's cheap. $5/month for 250G!
You don't have to maintain and secure a separate server for storage.
You can use the object storage for other applications outside of your servers via HTTP.

How to

OK, so this sounds pretty great right? You can have some amazing object storage power your storage needs to easily share files across your servers. Today, I'll show you how using Terraform.

Create a DigitalOcean Spaces access key

First you'll want to login to the DigitalOcean console and create a new access key + access key secret. This will be used to authenticate with your DigitalOcean Spaces bucket to ensure only you can access the storage.

When you click "Generate New Key", you'll need to type your key name into the text box, then click the blue check mark. After you click the blue check mark establishing your key name, you'll see these 2 new fields(save these for later):

DigitalOcean API Key

Now that you have your access key and secret for the spaces bucket, you'll still need an API key to use with Terraform to create a few resources such as DigitalOcean droplets and a Spaces bucket. This can also be done in the "API" section of the console.

Terraform Code

OK, so now we have our Spaces Access Key + Secret as well as our DigitalOcean API key. We can now move on to actually creating some droplets, a bucket, and share files across the two using s3fs.

A quick overview of what is happening below. We're creating a new bucket and 2 droplets that will share files back and forth. This is done by taking some example input such as a region, mount point (filesystem path), and bucket name which will use cloud-init to mount the bucket to the droplets when they first boot.

First let's make a terraform.tfvars file that takes in our configuration. It looks likes this:

# Spaces Access Key ID
spaces_access_key_id = "XXX"
# Spaces Access Key Secret
spaces_access_key_secret = "XXX"
# DigitalOcean API Token
do_token = "XXX"
# SSH Key ID to be able to get into our new droplets (can leave this empty if no need to ssh)
ssh_key_id = ""

Now we need to create a file called main.tf with the following content below. This will create our bucket and droplets and will configure s3fs on our droplets to be able to read and write files to the same bucket.

Please refer to the comments for the walkthrough of each component:

# Needed for terraform to initialize and
# install the digitalocean terraform provider
terraform {
  required_providers {
    digitalocean = {
      source = "digitalocean/digitalocean"
    }
  }
}

# Expected input, DigitalOcean Spaces Access Key ID
variable "spaces_access_key_id" {
  type      = string
  sensitive = true
}

# Expected input, DigitalOcean Spaces Access Key Secret
variable "spaces_access_key_secret" {
  type      = string
  sensitive = true
}

# Expected input, DigitalOcean API Token
variable "do_token" {
  type      = string
  sensitive = true
}

# SSH key in DigitalOcean that will allow us to get into our hosts
# (Not Necessarily Needed)
variable "ssh_key_id" {
  type      = string
  sensitive = true
  default   = ""
}

# DigitalOcean region to create our droplets and spaces bucket in
# Let's just go with nyc3
variable "region" {
  type    = string
  default = "nyc3"
}

# Name of our DigitalOcean Spaces bucket
variable "bucket_name" {
  type    = string
  default = "s3fs-bucket"
}

# Where to mount our bucket on the filesystem on the DigitalOcean droplets
# Let's just default to /tmp/mount for demo purposes
variable "mount_point" {
  type    = string
  default = "/tmp/mount"
}

# Configure the DigitalOcean provider to create our resources
provider "digitalocean" {
  token             = var.do_token
  spaces_access_id  = var.spaces_access_key_id
  spaces_secret_key = var.spaces_access_key_secret
}

# Create our DigitalOcean spaces bucket to store files
# that will be accessed by our droplets
resource "digitalocean_spaces_bucket" "s3fs_bucket" {
  name   = var.bucket_name
  region = var.region
}

# Let's create a sample file in the bucket called "index.html"
resource "digitalocean_spaces_bucket_object" "index" {
  region       = digitalocean_spaces_bucket.s3fs_bucket.region
  bucket       = digitalocean_spaces_bucket.s3fs_bucket.name
  key          = "index.html"
  content      = "<html><body><p>This page is empty.</p></body></html>"
  content_type = "text/html"
}

# Configure our DigitalOcean droplets via cloud-init
# Install the s3fs package
# Create a system-wide credentials file for s3fs to be able to access the bucket
# Create a the mount point directory (/tmp/mount)
# Call s3fs to mount the bucket
locals {
  cloud_init_config = yamlencode({
    packages = [
      "s3fs"
    ],
    write_files = [{
      owner       = "root:root"
      path        = "/etc/passwd-s3fs"
      permissions = "0600"
      content     = "${var.spaces_access_key_id}:${var.spaces_access_key_secret}"
    }],
    runcmd = [
      "mkdir -p ${var.mount_point}",
      "s3fs ${var.bucket_name} ${var.mount_point} -o url=https://${var.region}.digitaloceanspaces.com"
    ]
  })
}

# Convert our cloud-init config to userdata
# Userdata runs at first boot when the droplets are created
data "cloudinit_config" "server_config" {
  gzip          = false
  base64_encode = false
  part {
    content_type = "text/cloud-config"
    content      = local.cloud_init_config
  }
}

# Create 2 DigitalOcean droplets that will both mount the same spaces bucket
# These 2 hosts will share files back and forth
resource "digitalocean_droplet" "s3fs_droplet" {
  count     = 2
  image     = "ubuntu-20-04-x64"
  name      = "s3fs-droplet-${count.index}"
  region    = var.region
  size      = "s-1vcpu-1gb"
  ssh_keys  = var.ssh_key_id != "" ? [var.ssh_key_id] : []
  user_data = data.cloudinit_config.server_config.rendered
}

# Output our ip addresses to the console so that we can easily copy/pasta to ssh in
output "s3fs_droplet_ipv4_addresses" {
  value = digitalocean_droplet.s3fs_droplet[*].ipv4_address
}

Terraform Output

Now that we have our configuration defined above, we simple need to run terraform init && terraform apply -auto-approve to create our things!

❯ terraform apply -auto-approve

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # digitalocean_droplet.s3fs_droplet[0] will be created
  + resource "digitalocean_droplet" "s3fs_droplet" {
      + backups              = false
      + created_at           = (known after apply)
      + disk                 = (known after apply)
      + graceful_shutdown    = false
      + id                   = (known after apply)
      + image                = "ubuntu-20-04-x64"
      + ipv4_address         = (known after apply)
      + ipv4_address_private = (known after apply)
      + ipv6                 = false
      + ipv6_address         = (known after apply)
      + locked               = (known after apply)
      + memory               = (known after apply)
      + monitoring           = false
      + name                 = "s3fs-droplet-0"
      + price_hourly         = (known after apply)
      + price_monthly        = (known after apply)
      + private_networking   = (known after apply)
      + region               = "nyc3"
      + resize_disk          = true
      + size                 = "s-1vcpu-1gb"
      + ssh_keys             = (sensitive)
      + status               = (known after apply)
      + urn                  = (known after apply)
      + user_data            = "dc35535cfb286b2994e31baa83c32ef808b9bdff"
      + vcpus                = (known after apply)
      + volume_ids           = (known after apply)
      + vpc_uuid             = (known after apply)
    }

  # digitalocean_droplet.s3fs_droplet[1] will be created
  + resource "digitalocean_droplet" "s3fs_droplet" {
      + backups              = false
      + created_at           = (known after apply)
      + disk                 = (known after apply)
      + graceful_shutdown    = false
      + id                   = (known after apply)
      + image                = "ubuntu-20-04-x64"
      + ipv4_address         = (known after apply)
      + ipv4_address_private = (known after apply)
      + ipv6                 = false
      + ipv6_address         = (known after apply)
      + locked               = (known after apply)
      + memory               = (known after apply)
      + monitoring           = false
      + name                 = "s3fs-droplet-1"
      + price_hourly         = (known after apply)
      + price_monthly        = (known after apply)
      + private_networking   = (known after apply)
      + region               = "nyc3"
      + resize_disk          = true
      + size                 = "s-1vcpu-1gb"
      + ssh_keys             = (sensitive)
      + status               = (known after apply)
      + urn                  = (known after apply)
      + user_data            = "dc35535cfb286b2994e31baa83c32ef808b9bdff"
      + vcpus                = (known after apply)
      + volume_ids           = (known after apply)
      + vpc_uuid             = (known after apply)
    }

  # digitalocean_spaces_bucket.s3fs_bucket will be created
  + resource "digitalocean_spaces_bucket" "s3fs_bucket" {
      + acl                = "private"
      + bucket_domain_name = (known after apply)
      + force_destroy      = false
      + id                 = (known after apply)
      + name               = "s3fs-bucket"
      + region             = "nyc3"
      + urn                = (known after apply)
    }

  # digitalocean_spaces_bucket_object.index will be created
  + resource "digitalocean_spaces_bucket_object" "index" {
      + acl           = "private"
      + bucket        = "s3fs-bucket"
      + content       = "<html><body><p>This page is empty.</p></body></html>"
      + content_type  = "text/html"
      + etag          = (known after apply)
      + force_destroy = false
      + id            = (known after apply)
      + key           = "index.html"
      + region        = "nyc3"
      + version_id    = (known after apply)
    }

Plan: 4 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + s3fs_droplet_ipv4_addresses = [
      + (known after apply),
      + (known after apply),
    ]
digitalocean_spaces_bucket.s3fs_bucket: Creating...
digitalocean_droplet.s3fs_droplet[1]: Creating...
digitalocean_droplet.s3fs_droplet[0]: Creating...
digitalocean_spaces_bucket.s3fs_bucket: Still creating... [10s elapsed]
digitalocean_droplet.s3fs_droplet[1]: Still creating... [10s elapsed]
digitalocean_droplet.s3fs_droplet[0]: Still creating... [10s elapsed]
digitalocean_droplet.s3fs_droplet[1]: Still creating... [20s elapsed]
digitalocean_droplet.s3fs_droplet[0]: Still creating... [20s elapsed]
digitalocean_spaces_bucket.s3fs_bucket: Still creating... [20s elapsed]
digitalocean_spaces_bucket.s3fs_bucket: Creation complete after 28s [id=s3fs-bucket]
digitalocean_spaces_bucket_object.index: Creating...
digitalocean_spaces_bucket_object.index: Creation complete after 0s [id=index.html]
digitalocean_droplet.s3fs_droplet[0]: Still creating... [30s elapsed]
digitalocean_droplet.s3fs_droplet[1]: Still creating... [30s elapsed]
digitalocean_droplet.s3fs_droplet[0]: Still creating... [40s elapsed]
digitalocean_droplet.s3fs_droplet[1]: Still creating... [40s elapsed]
digitalocean_droplet.s3fs_droplet[1]: Creation complete after 43s [id=283287872]
digitalocean_droplet.s3fs_droplet[0]: Creation complete after 43s [id=283287873]

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

s3fs_droplet_ipv4_addresses = [
  "165.227.106.47",
  "45.55.60.230",
]

Sharing files

Cool! Now have our bucket and some droplets already configured, let's ssh to both and checkout that /tmp/mount path that we set up in our Terraform configuration above.

Let's do a recap of what's happening above.

On both s3fs-droplet-0 and s3fs-droplet-1, I ran df -h | grep s3fs which gives us our disk usage for all of the mounted volumes, but I filtered specifically for the term s3fs to shorten the list. This shows us that our bucket is mounted and available at /tmp/mount! Hooray!

root@s3fs-droplet-0:/tmp/mount# df -h | grep s3fs
s3fs            256T     0  256T   0% /tmp/mount

root@s3fs-droplet-1:/tmp/mount# df -h | grep s3fs
s3fs            256T     0  256T   0% /tmp/mount

Next, I ran ll /tmp/mount on both hosts so that we can see that the contents of the bucket and we can see the index.html file that I created in the bucket in the Terraform code is there and is viewable by both droplets. Awesooooome!

root@s3fs-droplet-0:/tmp/mount# ll /tmp/mount/
total 5
drwx------  1 root root    0 Jan  1  1970 ./
drwxrwxrwt 12 root root 4096 Jan 22 18:54 ../
-rw-r-----  1 root root   52 Jan 22 18:48 index.html

root@s3fs-droplet-1:/tmp/mount# ll /tmp/mount/
total 5
drwx------  1 root root    0 Jan  1  1970 ./
drwxrwxrwt 12 root root 4096 Jan 22 18:54 ../
-rw-r-----  1 root root   52 Jan 22 18:48 index.html

OK, so next I ran a touch command on s3fs-droplet-0 which created a file in /tmp/mount:

root@s3fs-droplet-0:/tmp/mount# touch file_from_$(hostname)

I used $(hostname) to substitute the name of the droplet in the file name so that we can see said file on s3fs-droplet-1. Let's have a look and see if that file is viewable on the other server.

root@s3fs-droplet-1:/tmp/mount# ll /tmp/mount/
total 6
drwx------  1 root root    0 Jan  1  1970 ./
drwxrwxrwt 12 root root 4096 Jan 22 18:54 ../
-rw-r--r--  1 root root    0 Jan 22 19:00 file_from_s3fs-droplet-0
-rw-r-----  1 root root   52 Jan 22 18:48 index.html

It's there! We successfully shared files between our 2 droplets. Now let's go look at our spaces bucket in the DigitalOcean console:

WOOT WOOT! Since we're using the spaces bucket, we can access these files from anywhere and in any application! NFS is looking pretty gross at this point. Yay for cloud object storage and thanks to DigitalOcean for providing us such a cool service!

Fin.

The time I had night terrors about us-east-1 outages

Caleb Lemoine — Tue, 28 Dec 2021 15:33:28 +0000

Background

I normally never write about personal things, only technical things. However this is one of those times that I feel this is too good not to write down and I need to write this all down now before I forget.

As the title suggests, I just woke up in a cold sweat from an incredibly awkward nightmare thanks to all the AWS us-east-1 outages as of late, here's the story.

The nightmare

It started as me being back in college going to classes, but for some reason, these college classes were at my high school. Because dreams always make sense. I was in the middle of class and I fall out of my desk very suddenly and I remember feeling very disoriented. I can't get up. People are staring. Finally some kind soul helps me out of the classroom and it just hits me. I'm behaving so strangely because obviously I'm in a simulation. It was obvious to me in the dream that it couldn't be real because well 1. I was in college at my high school and 2. I never just collapse on my own. I think this is what is called lucid dreaming.

Next I'm sitting outside of the classroom in the hallway, still disoriented, but then it goes next level. I start losing my shit. I start taking off my clothes and a neighboring classroom has a teacher giving some standardized test. So naturally she notices me wigging out and brings me a copy of the standardized test to do (because dreams make sense). I'll be damned if I do another one of those tests, so I yeet on out of there half-naked.

As I'm wandering through the halls of school half-dressed, the worst thing that could possibly happen... it happens. The bell rings. All of my peers then walk past me looking at me like I'm a freak show. Then it clicks. The simulation that I'm in is running in us-east-1. I had to tell someone. So what do I do? Grab a bystander of course and yell in their face "EBS is experiencing degraded performance!" I mean it made sense right? Why else would I be looking, feeling, and acting this way.

It keeps going. I'm still wandering the halls trying to find my next class. I can't. I can't for the life of me find where this stupid classroom is. I knew my theory was correct because I went to this school for 4 years. I knew where everything was. I would only make sense that my memory is shit because EBS (cloud storage) isn't working.

I make it to my next class eventually and sit down. And wouldn't you know it, my old college professor Dr. Russell is there teaching... in my high school.. where I was taking college classes. Yeah. Anyway, I realize I'm still way too disoriented to sit through this class, so back to the halls I go... still half-dressed.

Then I see my girlfriend (now wife). I thought if anybody could help me or set me straight, it's her. So I run up to her and couldn't even call her by her name. I call her "mom" thanks to the degraded cloud storage powering my horrid simulation brain. She then responds to me and calls me "Mr..." something.

It was at this point that I knew everything was absolutely as bad as it could be. My own girlfriend didn't know who I was. I didn't even remember her name. I'm in the hallway half-naked. It was then that my virtual world started crashing down around me. I remember falling backwards into the abyss much like Alice in Wonderland. All thanks to us-east-1 being down and not being able to power my simulation.

Thanks AWS.

If it's any consolation, I did just go see The Matrix Resurrections

Ephemeral Jenkins Users + API Tokens using Hashicorp Vault

Caleb Lemoine — Mon, 27 Dec 2021 21:24:18 +0000

New year, new credentials! (well almost)

The Jenkins rabbit hole

Not long ago, I (like most people), went down the rabbit hole of working around some Jenkins limitations. I was experiencing some issues of hitting identity provider API rate limits due to heavy API usage of Jenkins. After some experimenting, I found that using Jenkins API tokens for machine to machine communication instead of user credentials doesn't require identity provider authentication which significantly cuts down on hitting said rate limits. "This is great!", I thought.

SDK's... those darn SDK's...

Next up, I was looking into the the gojenkins client library to see if there were some built-in methods to allow me to do my own API token management and rotation to solve the issue above. And of course, it didn't exist. It was at this point I sat back and thought, "Really? No one has yet to implement a decent solution for managing Jenkins tokens/users yet? This is the most widely used CI server out there. Surely it has to exist..."

Sure, I was already aware of the vault-jenkins-plugin which allowed Jenkins to fetch secrets from Vault, but what about accessing Jenkins itself?! Nope, it didn't exist. I then began to think about all the work needed to solve this problem in a non-crufty way. First of which included some PR's to the gojenkins client library to add some API/User management methods to make the overall solution cleaner, as well as give back to the community so that they can automate all of their problems away as well.

Time to do the needful

I've built other plugins for products within the Hashicorp ecosystem before, such as Terraform, but never a Vault plugin. I thought this would be a good opportunity to learn more about the inner workings of Vault and fill this gap in the community.

Low and behold, I created the vault-plugin-secrets-jenkins project!

Yes. That is a glorious chicken sandwich and I'm leaving it.

So what does this thingy do?

I'm glad you asked!

Once installed, the vault plugin takes in a configuration of a "root" Jenkins user. Then on behalf of this user, Vault can create short lived users as well as short lived API tokens for the configured user!

But why?

Dude, it's almost 2022. Hardcoding credentials as environment variables is not the best way to handle secrets these days. Now you can just request/rotate your own dynamic credentials by integrating with Vault.

So what does it look like to use this thing?

First detailed guidance/examples, you'll want to checkout the project README here, but while I have you, I'll show you some examples.

Enable the plugin

vault secrets enable -path=jenkins vault-plugin-secrets-jenkins
Success! Enabled the vault-plugin-secrets-jenkins secrets engine at: jenkins/

Configure the plugin

vault write jenkins/config url=http://localhost:8080 username=admin password=admin
Success! Data written to: jenkins/config

Bonus: The plugin validates your config before moving on 😁

Create a short lived API token

vault read jenkins/tokens/mytoken
Key                Value
---                -----
lease_id           jenkins/tokens/mytoken/fJ57afQZMyXDcJnm74BgLLt8
lease_duration     5m
lease_renewable    true
token              1184cb7b22c404efa1c293e9841b66f345
token_id           1c2864f3-4108-4417-807a-358357bc8432
token_name         mytoken

Create a short lived user

vault write jenkins/users/myuser password=password fullname="Jenkins the Butler" email=email@example.com
Key                Value
---                -----
lease_id           jenkins/users/myuser/hTGbJhDFbAQpALv1FjJyJ4vz
lease_duration     5m
lease_renewable    true
email              email@example.com
fullname           Jenkins the Butler
username           myuser

So what now?

Woohoo! We now have automatically expiring users and tokens we can manage from our app and can give our pals over in information security some piece of mind that even if our credentials do leak, they'll be useless within just a couple of minutes. Pretty slick right?!

Fin

Thanks for taking the time to checkout my article!

(gifs of more chicken sandwiches are encouraged in the comments section)

Geofence your self-hosted web server and API's

Caleb Lemoine — Sat, 27 Nov 2021 19:45:00 +0000

Background

Not long ago, I made a post about my home-hosted interactive Halloween system. It's basically a custom built REST API to interact with a bunch of smart things, but needed to be hosted at home to more easily interact with devices on my local network. I then exposed my system to the world, but the intent was to only have folks on my street be able to interact with my Halloween decorations.
It's no fun when folks from across my neighborhood hear about it, go to the site and trigger a bunch effects, and no one be there to enjoy. The other scenario is that I have friends on the other side of the country mess with me by having my lights flash red and blue 🚨 on repeat (looking at you Traci).

Geofencing

What did I do to solve this? I implemented something called IP geofencing on my self-hosted API.

So what is geofencing?

IP geofencing is a security measure that restricts IP availability by geography, regardless of a user's access permissions. Think of it as a virtual perimeter around a given location.

But how?

I wrote a library called go-geofence that uses ipbase.com behind these scenes because the service gives you 150 requests per month for free and 65,000 per month for $10.

How does it work?

I wanted something free and simple. Here's some example usage of the Go library I wrote below. All you need is an API key from ipbase.com.

When the client is instantiated, it uses ipbase.com to determine the GPS coordinates of the client running the software. Then when the IsIPAddressNear() method is called, the library then looks up the GPS coordinates of the IP address specified, then compares the coordinates and computes distance in kilometers and determines if the IP address' location is close to yours.

package main

import (
    "fmt"
    "log"
    "time"

    "github.com/circa10a/go-geofence"
)

func main() {
    geofence, err := geofence.New(&geofence.Config{
        // Empty string to geofence your current public IP address, or you can monitor a remote address by supplying it as the first parameter
        IPAddress: "",
        // ipbase.com API token
        Token: "YOUR_IPBASE_API_TOKEN",
        // Maximum radius of the geofence in kilometers, only clients less than or equal to this distance will return true with isAddressNearby
        // 1 kilometer
        Radius: 1.0,
        CacheTTL:    7 * (24 * time.Hour), // 1 week
    })
    if err != nil {
        log.Fatal(err)
    }
    isAddressNearby, err := geofence.IsIPAddressNear("8.8.8.8")
    if err != nil {
        log.Fatal(err)
    }
    // Address nearby: false
    fmt.Println("Address nearby: ", isAddressNearby)
}

How was it implemented?

Since I'm using echo as the web framework to control my decorations, I was able to implement some pretty simple middleware that rejects POST requests from IP addresses that aren't within close proximity to mine with a 403 status code.

I want to implement the same thing for my home web server/API, but don't use Go

Well, boy do I have the solution for you!

I was surprised with the lack of open source geofencing solutions for this when I was originally looking into it. So to make the implementation more accessible, I built a module for the open source web server called Caddy.

Caddy

Caddy is an awesome web server alternative to nginx and apache (httpd). Caddy is written in Go, is a much more performant and extensible web server (in my opinion). With Caddy, you can host basic files or reverse proxy your API's, which is how I use it. I use Caddy because of its automatic TLS functionality so I don't have to worry about manual creation of certificates and keys.

caddy-geofence

To integrate with Caddy, I wrote a geofencing module than can be used with a wealth of configuration options thanks to Caddy's versatile abilities. You can configure the module per route, request method, pretty much any condition you can think of.

To implement the same geofencing functionality that I did to restrict who can use your HTTP services, all you need is an API key from ipbase.com and create a new web server with a Caddyfile to specify your token and configuration preferences.

If you're new to Caddy, one thing to note about Caddy and installing modules is that modules are compiled with Caddy, thus resulting in a single binary to execute.

Here's an example Caddyfile you would use to configure Caddy with geofencing functionality:

{
    order geofence before respond
}

:80

route /* {
    geofence {
        # cache_ttl is the duration to store ip addresses and if they are within proximity or not to increase performance
        # Cache for 7 days, valid time units are "ms", "s", "m", "h"
        # Not specifying a TTL sets no expiration on cached items and will live until restart
        cache_ttl 168h

        # ipbase.com API token, this example reads from an environment variable
        ipbase_api_token {$IPBASE_API_TOKEN}

        # radius is the the distance of the geofence, only clients within the distance will be allowed.
        # If not supplied, will default to 0.0 kilometers
        radius 1.0

        # allow_private_ip_addresses is a boolean for whether or not to allow private ip ranges
        # such as 192.X, 172.X, 10.X, [::1] (localhost)
        # false by default
        # Some cellular networks doing NATing with 172.X addresses, in which case, you may not want to allow
        allow_private_ip_addresses true

        # allowlist is a list of IP addresses that will not be checked for proximity and will be allowed to access the server
        allowlist 206.189.205.251 206.189.205.252

        # status_code is the HTTP response code that is returned if IP address is not within proximity. Default is 403
        status_code 403
    }
}

log {
    output stdout
}

Then to run Caddy with the module installed:

docker run --net host -v /your/Caddyfile:/etc/caddy/Caddyfile -e IPBASE_API_TOKEN -p 80:80 -p 443:443 circa10a/caddy-geofence

Or if if you would rather compile Caddy yourself with the module and not use docker, you can use xcaddy like so:

xcaddy build --with github.com/circa10a/caddy-geofence

For more info on implementing a geofenced web server/reverse proxy with caddy, see the caddy-geofence repo

Fin

And there you have it, expose your web server or API's and limit who can use them based on proximity to your physical location.

Pull Requests need more cute animals

Caleb Lemoine — Tue, 09 Nov 2021 04:14:43 +0000

My Workflow

I created a new GitHub action called animal-action

When a pull request is opened to a repository, it will comment with a cute picture of either a cat, dog, or fox!

The types of animals and comment supplied are completely configurable.

Current animals supported:

cats 🐈
dogs 🐕
foxes 🦊

Example

Submission Category:

Wacky Wildcards

Yaml File or Link to Code

circa10a / animal-action

A github action to add smiles to pull requests

animal-action

A github action to add smiles to pull requests

Inputs

`github_token`

Required A GitHub token

`animals`

A comma-delimated string of types of animals pictures to comment with.

Default: "cats,dogs,foxes".

`pull_request_comment`

Comment to post along with animal picture.

Default: ':tada: Thank you for your contribution! While we review, please enjoy this cute animal picture'.

Outputs

None

Example usage

name: comment
on:
  pull_request:
    types: [opened]
jobs:
  comment:
    runs-on: ubuntu-latest
    steps:
      - uses: circa10a/animal-action@main
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          animals: 'cats,dogs' # cats,dogs,foxes currently supported
          pull_request_comment: '🎉 Thank you for the contribution! Here's a cute animal picture to say thank you!'

View on GitHub

Additional Resources / Info

Why?

More smiles in Open Source is a definite win. 😄

Interactive Halloween decorations with raspberry pi's 🎃

Caleb Lemoine — Sun, 31 Oct 2021 20:04:46 +0000

Video Demo

Link to demo video

Background

Last year (2020), I decided to do something fun for my neighborhood due to the dark and shitty times that is COVID. I was convinced that kids wouldn't have a normal Halloween for trick or treating. So I built a thing. The idea was to have outdoor lighting with a mobile friendly web app that folks could use and change the colors of my outdoor Halloween lighting to the color/effect of their choice.

I ended up calling the project/site witchonstephendrive.com since I had this pretty slick witch silhouette curtain in the front bedroom window of my house and my street is Stephen Drive of course.

Anyways, here's what it looks like outside of my house with blue/green colors set.

Mobile app

I put a sign in my front yard to tell folks where to go to on their phones to do the needful. Here's a preview of what the site looks like.

What all does it do?

Lights

I have 5 philips hue lights in the front that are controlled via the app. There are several colors to choose from individually or you can select a rainbow effect or the "flash" effect which turns all of the lights off then back on a few times for a spooky effect.

Sounds

When a user selects presses a button to change the lights, a spooky sound gets played via a google assistant connected speaker.

How does it work?

Backend

I wrote a Go based REST API with lights/sounds endpoints that my front end interacts with.

Lights

The /colors/:color endpoint interacts with the hue bridge to set color/brightness state using a 3rd party philips hue sdk.

Sounds

The /sounds/:sound endpoint interacts with the google assistant speaker using home assistant.

Sounds are processed using a configurable in-memory queue to ensure sounds aren't interrupted and play all the way through. It would suck to have kids smashing buttons and you would never get to hear a full mp3 play.

Front end

The front end web app for this site is just a vanilla html/css/js that interacts with the REST API, but here's the cool part. Since Go 1.16 started supporting embedding files into the compiled binary, the front end and backend are both served via a single binary which makes deployment of this application super simple.

Hosting

This app is just hosted on a simple raspberry pi 4 in my TV stand running in a docker container. It's ideal to host this on your local network to auto discover the hue bridge. No cloud or kubernetes needed.

All that was needed was to forward port 80/443 from my router to my raspberry pi's local ip address.

Notable Features

Schedules
- Turning lights on/off at configurable times.
- Setting individual lights back to default colors at configurable times.
- Bridge IP address renewal at configurable interval to ensure DHCP doesn't change the address
Quiet time
- Sounds supports configurable start and end hours to not play sounds. Don't want to be a 🍆 to the neighbors by having sounds blaring at all hours of the night.
Monitoring
- Prometheus metrics built-in to monitor color selection usage frequency.
Rate limiting
- I'm using caddy for my web server due to it's awesome automatic certificate functionality using Let's Encrypt behind the scenes. Caddy supports rate limiting via this plugin so I don't have to worry about folks killing my API.
IP Geofencing
Terraform module to interact with lights/sounds ...because why not.

What's that cool pumpkin under the tree that watches you in the video?

That's my pumpkin-pi project that I also created using another raspberry pi! It uses 2 motion sensors and a servo motor to detect people walking on the sideway and turns the pumpkin towards them to simulate it "watching" you. Check out the repo to see how to build your own!

Here is a video demo close up

Source code

Here's a link to the project if you'd like to deploy your own next year!

Monitoring GitHub Pull Requests with Prometheus

Caleb Lemoine — Tue, 10 Aug 2021 23:13:06 +0000

The problem

Have you ever wanted to track your open source contributions? Or perhaps monitor contributions made by multiple users for some sort of event? Well I had the exact the same problem 😊.
With Hacktoberfest just around the corner, I wanted a way to automatically track open source contributions to be able to incentivize participation via prizes or similar. It's fairly difficult to track who opens pull requests to what projects on GitHub at scale within a large organization, but boy do I have the solution for you.

The solution

I built a prometheus exporter that takes in a config file with a list of users and will use the GitHub search API to find pull requests created by said users. This exporter exposes some useful data such as user, created_at, status and link.

What is a prometheus exporter?

Prometheus is an open source time series database that uses a "pull" model where it reaches out to configured clients (basically plugins) called exporters. Then ingests the data from the exporters via configured interval which is typically 15 seconds.

I want to see some users' pull requests, what now?

If you're familiar with prometheus, you can view the github-pr-exporter docs here.

Maybe you're not familiar with prometheus, follow along for how to get started!

How does it work?

The exporter is written in Go and utilizes a GitHub client library to execute searches using the GitHub search API. The exporter is basically a CLI that takes in configuration options via arguments then runs a web server in a loop that looks for new pull request data periodically.

Searches

The exporter uses the search API and runs this search for every user: "type:pr author:<user> created:>=<calculated timeframe>"

The search is executed once per user instead of a bulk search due to the 256 character limit of the search API, but because the search API has a rate limit of 10 searches per minute for unauthenticated clients, there is a hard 6 second wait between collecting pull request data for each user to avoid said rate limit. This isn't a huge deal since this would process around 1000 users per 90 minutes which completely fine for this kind of data.

Configuration Options

--config YAML config file with usernames
--days-ago How many days prior to the current to get pull requests from. Defaults to 90
--ignore-user-repos Ignore pull requests that the user made to their own repositories (no cheating!). Defaults to false.
--interval How long (in seconds) to sleep between checking for new data. Defaults to 21600 (6 hours)
--port Which port to run the web server on. Defaults to 8080

Deploying github-pr-exporter, prometheus, and grafana

There are 3 components needed to get up and running to make use of the data.

You will also need docker, docker-compose, and git installed. This will run the 3 components above in the form of containers and connect them together via a docker network.

Start by cloning the repo and cd'ing into the directory

git clone https://github.com/circa10a/github-pr-exporter.git
cd github-pr-exporter

Start the containers

docker-compose up -d

Profit!

You should now be able to access grafana at http://localhost:3000 and the preconfigured dashboard at http://localhost:3000/d/h_PRluMnk/pull-requests?orgId=1

The default admin login is username: admin password: admin

Monitoring your own user or others

To change the default configuration, simply update examples/config.yaml to include your GitHub username or others then recreate the containers by running:

docker-compose down
docker-compose up -d

Then check out the new details on the dashboard! Please note that it can take around 2 minutes for the dashboard to reflect the new data so just try refreshing or turn on auto-refresh.

How to use feature toggles with Terraform

Caleb Lemoine — Sat, 23 Jan 2021 19:11:45 +0000

Reinventing the wheel for the better

I've seen several articles around this subject including the official Hashicorp doc, but since there wasn't one on dev.to, I took the typical "Well I need to rewrite this whole thing because it sucks" software engineer approach 😄.

What are feature toggles?

Feature toggles allow you to either enable or disable software functionality via a boolean value (true/false).

Why would I use a feature toggle?

The two biggest reasons in my opinion are:

Safety
Have options

By having new functionality toggleable, you can dark launch new features without impacting existing software. I work at a large SaaS company and we promote this practice extensively. By having features that can turned on/off selectively, we rollout experimental features to beta customers without impacting anyone else while using the same codebase. Another element to this is for cost optimization, especially when it comes to infrastructure software like Terraform. Maybe I want to enable load balancing or clustering for customers who pay for it, this way I can offer tiered services and tailor deployments to customers needs.

Diving in

This remainder of the article assumes you have prior knowledge of or experience with Terraform

Here's the example repo we'll be following along with

Let's say we have a module to provision nginx web servers on digitalocean, it would be nice to have a toggle to enable/disable load balancing to control costs in certain environments. What would that look like?

Well, ideally I would like to have a module that has the ability create a load balancer or not like this:

module "web_servers" {
  source                 = "./modules/web_servers"
  instance_count         = var.instance_count
  load_balancing_enabled = var.load_balancing_enabled
}

A load_balancing_enabled flag would be pretty useful to give the consumer options.

How to create the toggle

There's a few key components to load balancing, we need multiple servers and we need a load balancer to be aware of all of the servers provisioned.

Let's create a digitalocean_droplet resource that has a variable for how many instances to create and a load balancer with all of the instances behind it.

resource "digitalocean_droplet" "web" {
  count     = var.instance_count
  name      = "web-${count.index}"
  size      = var.instance_size
  image     = var.image
  region    = var.region
  user_data = var.user_data == "" ? file("${path.module}/files/cloud-init.yaml") : var.user_data
}

resource "digitalocean_loadbalancer" "public" {
  count  = var.load_balancing_enabled ? 1 : 0
  name   = "web-servers-loadbalancer"
  region = var.region

  forwarding_rule {
    entry_port     = 80
    entry_protocol = "http"

    target_port     = 80
    target_protocol = "http"
  }

  healthcheck {
    port     = 22
    protocol = "tcp"
  }

  droplet_ids = digitalocean_droplet.web[*].id
}

Count

When we tell the terraform module that we have multiple instances, how does it name them? Terraform supports a cool meta-argument called count. A meta-argument is simply a language feature that can be applied to any Terraform resource independent of the provider(DO, AWS, GCP, Azure, etc). The count meta-argument will create the resources as if it were performing a for loop over an array with the number of count as the iterator.

resource "digitalocean_droplet" "web" {
  count = var.instance_count
  name  = "web-${count.index}"

So if instance_count were 2, 2 resources(servers) would be created and named like so:

web-0
web-1

Any time the count meta-argument is supplied, Terraform will store these resources as an array in Terraform state named like so:

digitalocean_droplet.web[0] or digitalocean_droplet.web.0
digitalocean_droplet.web[1] or digitalocean_droplet.web.1

This is an important concept when it comes to feature toggling in Terraform because if we want to selectively turn things off and on, we need use the count meta-argument on everything so that we can set it to either 1 or 0, e.g. create a thing or not.

Let's look at the next use of count:

resource "digitalocean_loadbalancer" "public" {
  count  = var.load_balancing_enabled ? 1 : 0

Here we are using a ternary operator as a conditional expression in Terraform. The above code reads as:

if (load_balancing_enabled) {
    create()
} else {
    doNothing()
}

Creating the resources

Let's play with some of these parameters and see how Terraform responds.

To follow along, you'll need to have a digitalocean account to create an API token.

# Set digitalocean token for authentication
export DIGITALOCEAN_TOKEN=XXXXX
# Clone the repo
git clone https://github.com/circa10a/terraform-feature-toggle-example.git/
# Change directory into the repo
cd terraform-feature-toggle-example/
# Install web_servers module and digitalocean provider
terraform init

If you look at the files in the repo, we have a default.auto.tfvars file which makes it easy to change configurations.

Here's the default:

instance_count = 1
load_balancing_enabled = false

This will create 1 droplet and no load balancer. Here's the output of terraform apply:

❯ terraform apply -auto-approve

module.web_servers.digitalocean_droplet.web[0]: Creating...
module.web_servers.digitalocean_droplet.web[0]: Still creating... [10s elapsed]
module.web_servers.digitalocean_droplet.web[0]: Still creating... [20s elapsed]
module.web_servers.digitalocean_droplet.web[0]: Still creating... [30s elapsed]
module.web_servers.digitalocean_droplet.web[0]: Creation complete after 34s [id=227920620]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

droplet_ips = [
  "167.172.28.220",
]
load_balancer_ip = ""

Outputs

Now we have Terraform making decisions about what to create using our toggle variable. We told our web_servers module to create 1 instance and no load balancer, let's change that and enable load balancing by modifying our default.auto.tfvars file and have load_balancing_enabled = true:

instance_count = 1
load_balancing_enabled = true

And now run terraform apply again:

❯ terraform apply -auto-approve

module.web_servers.digitalocean_droplet.web[0]: Refreshing state... [id=227920620]
module.web_servers.digitalocean_loadbalancer.public[0]: Creating...
module.web_servers.digitalocean_loadbalancer.public[0]: Still creating... [10s elapsed]
module.web_servers.digitalocean_loadbalancer.public[0]: Still creating... [20s elapsed]
module.web_servers.digitalocean_loadbalancer.public[0]: Still creating... [30s elapsed]
module.web_servers.digitalocean_loadbalancer.public[0]: Still creating... [40s elapsed]
module.web_servers.digitalocean_loadbalancer.public[0]: Still creating... [50s elapsed]
module.web_servers.digitalocean_loadbalancer.public[0]: Still creating... [1m0s elapsed]
module.web_servers.digitalocean_loadbalancer.public[0]: Still creating... [1m10s elapsed]
module.web_servers.digitalocean_loadbalancer.public[0]: Creation complete after 1m19s [id=fc36d0bf-12e5-4d7c-a9c2-06f3859588c5]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

droplet_ips = [
  "167.172.28.220",
]
load_balancer_ip = "144.126.248.10"

Awesome! 🎉 🎉 🎉

Our load balancer is now being created because we set load_balancing_enabled to true, but wait, outputs are different.

When load_balancing_enabled was false:

Outputs:

droplet_ips = [
  "167.172.28.220",
]
load_balancer_ip = ""

And when load_balancing_enabled was true:

Changes to Outputs:
Outputs:

droplet_ips = [
  "167.172.28.220",
]
load_balancer_ip = "144.126.248.10"

We now have a load_balancer_ip output. Well that's because of 2 reasons.

1. If we look at modules/web_server/outputs.tf

output "droplet_ips" {
  value = digitalocean_droplet.web[*].ipv4_address
}

output "load_balancer_ip" {
  value = var.load_balancing_enabled ? digitalocean_loadbalancer.public[0].ip : ""
}

Our module is conditionally outputting the load balancer's ip based on var.load_balancing_enabled variable. If it's true, give the value of digitalocean_loadbalancer.public[0].ip else "".

Back to count

So why does the output resource value have that 0? If you recall our overview of the count meta-argument, any resource that has count set will output its resources as an array, so in this case we're forced to use 0 to reference the first (and only) ip attribute from the digitalocean_loadbalancer.public resource.

2. Our primary outputs.tf in the root of the project, outputs the values above from the module like so:

output "droplet_ips" {
  value = module.web_servers.droplet_ips
}

output "load_balancer_ip" {
  value = module.web_servers.load_balancer_ip
}

Recap

Thanks to count and ternary operators in Terraform, we can make module configuration in Terraform pretty intuitive.

Let's not forget about all the other added benefits of feature toggling in Terraform::

Decouple deploy from release
Enable customization for operators as well as consumers
Save costs on resources you may or may not need

Don't forget to run terraform destroy to remove the resources we created! Otherwise you'll see the costs on your bill!

References

GitHub Repo

Additional Resources

Spacelift has an excellent article for more Terraform language features and their usage. Check out Terraform Functions, Expressions, and Loops by Spacelift!

Notifications for free developer swag 🎉

Caleb Lemoine — Mon, 18 Jan 2021 01:27:37 +0000

Backstory

I love building things and recently placed as a runner up in the digitalocean hackathon and am pretty stoked to be getting moar hoodies! After my subtle brag about getting all the swag, a friend of mine then told me "why don't you make an app to automate getting more hoodies?".

What a fantastic idea.

What I built

I originally had a reddit bot written in node.js and thought "I have so many uses for bots, I don't want to deploy a new one for each idea. That would be a pain."

For my use case, I went against the whole microservices trend and kept all the scheduling and configuration for my three bots inside one long running process called all-the-things-bot.

For the sake of staying on topic, you can browse the source here.

Free Dev Shit Bot

A bot that notifies you of potential opportunities for free dev shit also called "swag". Because we always need more laptop stickers, shirts, and hoodies!

After refactoring my app to support multiple bots, schedules, and configs. I implemented the "Free Dev Shit Bot". The bot is available on two platforms at the time of this writing.

Twitter

Simply follow the bot for updates!

Discord

Create a channel called swag in your discord server and add the bot to your server. The bot will check daily for free swag opportunities and notify you in the channel!

What it does

The bot will scan dev.to for the terms "free swag" and will tweet the article with a link and post to swag channels on all subscribed discord servers.

How it works

At the start of the application, all bots are initialized and scheduled with node-schedule. The Free Dev Shit bot is scheduled to post to twitter and discord servers at 11:55 PM CST. The logic behind the scenes searches the dev.to feed_content API by filtering the last 60 articles which is ordered newest. The bot then processes those 60 and finds any that are less than 24 hours old and has more than 1 public reaction(for credibility). If we have any results, simply post to all the things. The dev.to API is definitely the MVP here! 🥇

Resources

Monitoring the digitalocean app platform with ...the digitalocean app platform?

Caleb Lemoine — Sat, 09 Jan 2021 18:47:56 +0000

What I built

I created a pull request to add existing functionality to a popular digitalocean monitoring app to monitor resources on the digitalocean app platform and tied it into my first hackathon entry. Along with adding functionality to monitor the app platform, I added a deploy template + button to allow simple deployment for consumers.

Category Submission:

Built for Business

I contemplated whether or not this should be submitted for the Random Roulette category or not. While this app is not suited to turn a profit, I think it can really help commercial entities monitor and track their infrastructure on digitalocean to monitor costs, availability, and just get general insight into their resources on the cloud platform such as deployments and builds.

App Link

Screenshots

Description

This is actually my second hackathon entry which builds off of my first app.

For this entry, I went back to our hacktoberfest roots and took an open source approach. For this entry, I made a pull request to add additional features to the digitalocean prometheus exporter. This allows any user to deploy the prometheus exporter on the app platform and monitor the status of their apps for things like builds, deployments, regions and tiers.

Link to Source Code

Permissive License

MIT

Background

I've previously worked on the application and this is my 3rd featured added. Given the new capabilities of the platform, it felt like a no-brainer for this functionality to be enabled for the community.

How I built it

I wrote some Go code utilizing digitalocean's godo client to pull details from the app platform and converted them to prometheus metrics via a new collector. I also added a digitalocean application deployment template to make it simple for users to deploy.

Once the new features were ready, I deployed the exporter on the platform and added the endpoint to my existing prometheus configuration to scrape. Once data was being ingested, I created a new grafana dashboard to showcase available metrics and ideal usage of the data in a real world setting.

It was lots of fun to make the app platform data more accessible... while also using the platform!

Additional Resources/Info

While the pull request hasn't been merged yet, I'm confident that it will be. I even added new functionality to scrape digitalocean managed database info during the hackathon... because why not.

The mcbroken dashboard - digitalocean hackathon entry

Caleb Lemoine — Mon, 21 Dec 2020 21:58:51 +0000

What I built

I built a digitalocean app template that deploys grafana, prometheus, and a custom prometheus exporter which monitors/displays the availability information of all the broken Mcdonald's ice cream machines in the United States.

Category Submission:

Random Roulette

App Link

Click here to go to the mcbroken dashboard

Screenshots

Description

The mcbroken dashboard is an app template of 3 applications(listed below) which pre-configure a dashboard that is powered by mcbroken.com. Its purpose is to provide availability information of all the broken Mcdonald's ice cream machines in the United States.

Stats include:

Current broken percentage of mcdonald's ice cream machines in the US
City with the most broken machines and it's outage percentage
Outage percentage of most major US cities

Link to Source Code

https://github.com/circa10a/mcbroken-dashboard

Permissive License

MIT

Background

I remember seeing mcbroken.com circle the news a little while ago and thought this was absolutely hilarious. What's the point of being an engineer if you can't over engineer things for a good laugh?

Not only would it be hilarious, but thought it would also be a good learning opportunity to learn how to wire up multiple components for a single application on digitalocean's new platform.

How I built it

I utilized digitalocean's new app platform to deploy an application that consists of a pretty cloud native architecture. Wiring up all the components on the platform was much simpler than I thought. I love that it's essentially kubernetes under the hood so all the existing functional knowledge is applicable, yet there's so much more simplicity when it comes to getting your application to "just work".