Forem: Jacob See

Short Stuff: Let Me Paste Passwords!

Jacob See — Sun, 18 Apr 2021 00:00:00 +0000

I have to be honest - I thought we were done with this phase of Internet "security" a long time ago. Don't we all use password managers these days? Does anybody actually know their password to any major website anymore? I don't. That's why I was so surprised to run across a website (cough Costco cough) that was still disabling paste on password inputs... at least, on their registration page. ^{yes it took me this long to make a Costco account...}

It was at this point I decided that instead of opening my password manager in a new window and just typing the very long generated password into this registration page twice, I would spend significantly more time to fix the problem itself.

Obviously, we can't edit anything server-side... but for this, we don't really need to. The code executing this betrayal is just JavaScript running in our own browser. A-la "We've traced the call. It's coming from inside the house." And we can do whatever we want inside of our own house.

One convenient way to inject our own JavaScript in the browser is to run a plugin such as Tampermonkey (Firefox, Chrome). Tampermonkey provides an environment for you to write your own scripts (or use public scripts published by others), and specify the URLs on which those scripts should activate and run.

Luckily for us - this is a very simple problem to solve with a script! As seen in the screenshot above, they simply attach an event handler to the paste event and then return false, effectively canceling the paste.

To fix pasting, we need to intercept the event before it reaches this handler and do something else. Click on the Tampermonkey icon on your taskbar, go to the dashboard, and create a new script with the following:

// ==UserScript==
// @name         Allow Pasting
// @namespace    https://jacobsee.com
// @version      0.1
// @description  Allow pasting passwords on sites that try to disable it
// @author       Jacob See
// @match        https://www.costco.com/*
// @icon         https://www.google.com/s2/favicons?domain=costco.com
// @grant        none
// ==/UserScript==

(function() {
    var youllNeverGetMyPaste = function(e){
        e.stopImmediatePropagation();
        return true;
    };
    document.addEventListener('paste', youllNeverGetMyPaste, true);
})();

This script adds its own "capturing" event listener for the paste event across the entire document, with a handler that prevents propagation of that event to other handlers, and "accepts" the paste. At this point, you can save the script and exit Tampermonkey.

Open a new tab and navigate to Costco, and the Tampermonkey icon should illuminate, indicating that a script is active! The original evil paste event handler still exists on the page, but it doesn't matter because the handler defined in our script takes care of it first!

Now we can paste our massive, inconvenient passwords to our heart's content. 😎

Gotcha: Configuring Containers with Ansible

Jacob See — Thu, 08 Apr 2021 00:15:00 +0000

If you're running serious container infrastructure these days, chances are strong that you're using an orchestration system like Kubernetes or OpenShift. But, what if your workloads are a little less... clustered? A little more traditional? Maybe you have containers as a part of your workload, but don't need to migrate the entire system to them? Running one-off containers on normal Linux hosts is still very much a thing, it's just more on you to manage them - perhaps using an automation tool like Ansible. I was doing this recently, and hit a bit of a snag, so I thought I'd write about it here!

To kick things off - running a container on a target machine in Ansible is a pretty easy task. Depending on your target container type, you can use the docker_container or podman_container modules in a task like so:

- name: Run the container 
  docker_container: 
    name: my_container 
    image: "{{ my_exporter_image }}:{{ my_exporter_image_version }}" 
    state: "started"

Fair enough.

Reading through the documentation for those modules above, we see that we can also mount files from the host into the container as volumes - an important thing to do when a container takes some kind of a configuration file in order to run. Since it's likely that we're also creating that config file with Ansible, let's take a look at what writing that file and then mounting it into the container would look like:

- name: write the config file
  template:
    src: config.yml.j2
    dest: "/home/{{ ansible_user }}/my_config.yml"
          # ^ we're writing the config file here

- name: Run the container
  docker_container:
    name: my_container
    image: "{{ my_exporter_image }}:{{ my_exporter_image_version }}"
    state: started
    volumes: 
      - "/home/{{ ansible_user }}/my_config.yml:/config.yml"
        # ^ and mounting it into the container here

And there we go! The my_config.yml file in the home directory of whichever user is running this task is now mounted into the container at /config.yml.

This works nicely... once. This is where the "gotcha" is!

If we were to run this task a second time with different values for the config file, we would notice some interesting behavior. The file on the host (at /home/{{ ansible_user }}/my_config.yml) would appear to have our new configuration in it, but the application in the container would not be performing as expected! In fact, if we were to ssh into the remote machine and run docker exec my_container cat /config.yml to see the contents of the config file from the container's perspective, we would see that it has retained the old config data from before we ran the task a second time! Why have the file on the host and the file in the container diverged?

The answer lies in a combination of how Ansible writes files, and how Docker mounts them. Files in a UNIX-based filesystem are represented by inodes, which are records in a table that contain metadata about those files (including where they physically exist on a disk). If you're familiar with hard links and symbolic links - congratulations! You're already manipulating inodes! Hard links are paths on the filesystem that point at an underlying inode, and symbolic links are paths on the filesystem that act as shortcuts to other paths.

Two things are important to know at this point, and they are that:

Containers mount a file by referencing that file's inode at the container's creation time, and:
When creating files, Ansible first writes the data to a temporary file, and then moves that temporary file into its final location

This is important because, on the filesystem level, Ansible is actually creating a new inode every time it writes this data - even though, intuitively, it seems like we're writing to the same path every time. It then updates the hard link at that path to point at this newly created inode. This makes sense as it is actually a safer way for Ansible to write data remotely, but it means that any already-running container with that path (and therefore the old inode) mounted do not get updated with our newly-written config data.

There are two easy fixes for this.

First, we could force a container restart every time we make a change to the config file. The Ansible module has a restart flag to make this easy:

- name: write the config file
  template:
    src: config.yml.j2
    dest: "/home/{{ ansible_user }}/my_config.yml"
          # ^ we're writing the config file here

- name: Run the container
  docker_container:
    name: my_container
    image: "{{ my_exporter_image }}:{{ my_exporter_image_version }}"
    state: started
    restart: yes
    volumes: 
      - "/home/{{ ansible_user }}/my_config.yml:/config.yml"
        # ^ and mounting it into the container here

However, if our application can live-reload its config and we don't want to force a restart every time we make a change to the config file, we have to get slightly more creative. Notice how earlier we said that moving a file changes the inode referenced by a certain path. Copying a file does not have this same effect. We can use this to our advantage!

- name: write the config file (temporary)
  template:
    src: config.yml.j2
    dest: "/home/{{ ansible_user }}/my_config.yml.tmp"

- name: copy the config file to the correct path (so that the inode stays intact)
  command: "cp /home/{{ ansible_user }}/my_config.yml.tmp /home/{{ ansible_user }}/config.yml"

- name: remove the temporary config file
  file:
    path: "/home/{{ ansible_user }}/my_config.yml.tmp"
    state: absent

By doing this, we have ensured that the inode at the path mounted into the container is not modified, and our new configuration is ready to use in the container immediately!

Happy automating!

You can't CNAME your root domain (but also, you can!)

Jacob See — Sun, 04 Apr 2021 00:00:00 +0000

I ran across an interesting situation recently where I wanted to take a friend's domain (that they had given me access to on their Google Domains account) and host a landing page on the same server that the rest of my sites run on - with one small catch:

I didn't want to have to maintain a separate A record for their domain pointing to my server's IP address. I already maintain an A record for my own domain. I thought that I should be able to CNAME their domain to mine, and have the IP address of my own A record simply set the course for this new domain as well.

It turns out that this breaks some rules under normal DNS conventions.

Specifically, under the authoritative RFCs 1912 & 2181:

CNAME records, when defined, must be the only record for a given hostname.
SOA and NS records must exist at the domain root.

These two constraints would contradict each other - so CNAME records are not technically allowed at the domain root!

The thing is - This is not a technical limitation. This is merely a limitation in the specification, written during a time when the modern Internet (and the infrastructure that we use to drive it) simply did not look anything like what it does today.

This leaves us with an interesting situation. Google Domains has not implemented any sort of workaround for this, so using Google as the nameserver for my friend's domain was not an option. However, some other providers have implemented what they call ALIAS records, ANAME records, or virtual CNAME records!

These pseudo-records are a fantastic piece of DNS server trickery. Instead of replying to a DNS request for the root hostname with a CNAME record (which, remember, breaks the rules) - they have the DNS server perform its own lookup of the ALIASed domain on-the-fly and return that IP to the requesting client as a normal A record, indistinguishable from any other totally normal A record on a domain root!

Out of the public DNS providers that support this "virtual CNAME" service, CloudFlare is the one that I decided to use (since I use CloudFlare for other things as well). They call this trickery "CNAME Flattening", and they have written their own detailed post about it here. In the CloudFlare DNS panel, once you enable CNAME flattening, you can treat CNAME records as if they can exist at the root. Create the CNAME record at the root, and it will automatically handle the flattening (i.e. conversion to an A record) for you!

This made for an elegant implementation of my desired DNS configuration. I used the Google Domains panel to point the NS records for my friend's domain at CloudFlare, and from there was able to virtually CNAME that domain to point at my main domain (and thus be hosted by my same web server), without breaking any DNS rules!

Microservice Coordination at a High Level

Jacob See — Wed, 05 Dec 2018 02:03:41 +0000

Photo by Campaign Creators / Unsplash

Why

I recently had the privilege of working on a team tasked with creating a system using a composable architecture. We needed a system that operated as a sort of pipeline - with information flowing into one end, undergoing a number of transformations, and finally getting returned to the client as it leaves the pipeline out of the other end. Not all data required the same transformations to be applied to it, or in the same order, so we needed a way to adjust the behavior of this data pipeline on-the-fly in the cleanest, most scalable manner possible.

We approached this by first deciding that each transformation should be its own microservice within our cluster. When it came time to string the microservices together to form a whole pipeline, we had a group discussion to consider the possible strategies we could use - namely orchestration v.s. choreography. This article brings some points from that discussion to the Internet.

Orchestration

Photo by Manuel Nägeli / Unsplash

When you go to a concert house to listen to your local symphony orchestra, arguably the most important role played in the performance is that of the conductor. Each member of the orchestra acts in a synchronized manner because they're all receiving instructions from the same point of reference. Not only does the conductor synchronize the sounds of all the musicians while you're listening, the conductor shaped the music itself in rehearsals before you ever arrived. Adjusting timings here and volumes there, what you are hearing is tuned by the conductor to bring you their interpretation of the best possible performance.

Orchestration in the context of microservices is very similar! Just as a violinist wouldn't look to the woodwinds to learn what they should be doing, one service does not need to have any knowledge of the others to play its part in the grand scheme of the application. All microservices pay attention to one system (the orchestrator - generally another service on the same cluster) that is charged with determining what needs to happen, and directing the microservices under its control to perform whatever tasks are necessary in order to reach its desired outcome. The orchestrator is the centralized authority for everything that happens within its scope of control. Other services can simply be thought of as its workers.

No services communicate without using the orchestrator to do so. Oblivious to what's going on around them, they perform tasks and return results.

This strategy brings some benefits and some pitfalls to the service composability table. In the spirit of DRY - the orchestrator is a clean place to put workflow logic. Since the orchestrator has the high-level understanding of what should be happening within the system, it can provide advanced progress monitoring and error handling right out of the box. It's not all good news for the orchestration pattern though... Orchestration inherently sets up a structure with a single point of failure. If the orchestrator becomes unable to delegate tasks for any reason - it's as if the conductor of the symphony orchestra dropped dead in the middle of a performance. All progress within the system stops, and unless the fault is of a type which can be automatically detected and healed within your environment, your on-call engineer is not going to have a good night.

For further reading on orchestration in practice, or to try it out yourself, you may want to check out Uber's Cadence or Netflix's Conductor orchestration engines. Conductor, the system I'm more familiar with, defines workflows using a JSON based DSL that looks like this:

{
  "name": "encode_and_deploy",
  "description": "Encodes a file and deploys to CDN",
  "version": 1,
  "tasks": [
    {
      "name": "encode",
      "taskReferenceName": "encode",
      "type": "SIMPLE",
      "inputParameters": {
        "fileLocation": "${workflow.input.fileLocation}"
      }
    },
    {
      "name": "deploy",
      "taskReferenceName": "d1",
      "type": "SIMPLE",
      "inputParameters": {
        "fileLocation": "${encode.output.encodeLocation}"
      }

    }
  ],
  "outputParameters": {
    "cdn_url": "${d1.output.location}"
  },
  "schemaVersion": 2
}

Choreography

Photo by Michael Afonso / Unsplash

To continue with the performing arts analogies, and because the names of these strategies align so well with these examples, imagine you're attending a ballet performed by skilled dancers. There is no conductor leading them - dancing is a visual art and that would block the audience's view! The dancers are synchronized and move fluidly around each other because they've gone through rigorous training to be able to do exactly that. The absence of an orchestrator during their performance is supplanted by the fact that they each know what should be happening and when, and maintain an awareness of their surroundings and other dancers to make sure that the performance goes as planned.

This is also a valid strategy in microservice coordination. Choreography is a design pattern in which there is no orchestrator present. Each service must know not only how to do its job, but also when and how to interact with other services in the same system. All components are autonomous, and typically agree to adhere to a contract, but otherwise are fully self-managed and do not receive direction from an external source.

No orchestrator is present. Services do not receive instructions to perform tasks, rather, they understand the result they are given, and know what to do with it next.

Compared to orchestration, choreography is (in my opinion) quicker to get started with for a new project. It does not require the upfront time investment of learning to use someone else's orchestration engine, and lets you jump right into what you know best - code. If designed correctly, it can even be faster than an orchestrated system, as you do not incur the overhead of having to wait on the orchestrator between each logical step. Be careful though - some of the benefits that the orchestrator provides have no equivalent here. With services directly interacting with each other, error handling within the workflow is entirely your responsibility. There is no mediator to intervene and prevent cascading failures. Communication within your development team becomes especially critical, since the integrity of the workflow as a whole is dependent on each service obeying its contract, which itself is dependent on a shared understanding between developers of how both individual services and the entire system itself should operate.

Conclusion

Orchestration and choreography are both well-accepted methods of coordinating work within complex systems, and it's entirely up to you to decide which you prefer for solving any given problem. It is also worth mentioning that you do not have to pick only one! Your system may be decomposable further into subcomponents, each of which lending itself more towards orchestration or choreography individually. As for us - we were operating on a limited timeline and chose choreography for our system out of simplicity, with workflow composability accomplished using a routing slip pattern. I'm planning another article on that pattern soon, so stay tuned!