Forem: John Pham

Building a Reverse Proxy with Cloudflare Workers

John Pham — Thu, 11 Nov 2021 06:10:39 +0000

Background

One of the features we have on Highlight is the ability to create a comment on a session. The cool thing about these comments is they have a spatial property. A comment is created at a specific location on the screen. This allows users to call out things in the session with the comment's location rather than writing something like "the blue button in the top right corner below the red button...".

While creating a comment, you can also tag your individual team members or Slack channels. When you do this, Highlight will send a preview of your comment's text along with a screenshot of the screen you commented on.

The Problem

Under the hood, we use html2canvas to get the screenshot. I go over more on why in the Alternative Solutions section.

At a high level, html2canvas creates an image by recreating the DOM in a <canvas>. We then get a base64 representation of the <canvas> to use.

If the screen we're creating a screenshot of has external resources like images, html2canvas might not be able to load them due to CORS restrictions. When external resources are blocked, the places where the external resources are on the screen are blank in the screenshot.

For Highlight, this problem is pretty common because our customer's sessions are recorded off the Highlight origin. Most of the external resources Highlight tries loads will probably be blocked by the browser due to CORS.

Reverse Proxy to the Rescue

So our problem is the CORS restrictions. We can use a reverse proxy to get around the problem.

Instead of making a direct request from the Highlight app to the cross-origin resource, we'll make a request to a Highlight proxy which will make the request to the cross-origin resource, then return the response. To the browser, the requested resource is on the same origin so the cross-origin resource is loaded successfully!

To implement the reverse proxy, we chose to go with Cloudflare Workers for the following reasons:

Independent scaling from our main app API
We don't have to worry about infrastructure
Fun opportunity to try the new shiny toy on a non-mission critical code path

Code

// Boilerplate for workers.
addEventListener("fetch", (event) => {
  event.respondWith(
    handleRequest(event.request).catch(
      (err) => new Response(err.stack, { status: 500 })
    )
  );
});

async function handleRequest(request) {
  const { pathname, searchParams } = new URL(request.url);
  // Gets the URL that will be proxied from `url` search parameter.
  const resourceToProxy = searchParams.get("url");

  // Respond to the requesting caller with the response of the proxied resource.
  return fetch(resourceToProxy);
}

Now when you call html2canvas, you can pass the URL to the proxy server.

html2canvas(document.querySelector('#player'), {
    proxy: "https://path_to_proxy.com", // This is the address to your Cloudflare Worker.
}).then((canvas) => {
  // Do stuff with the canvas.
});

Alternative Solutions

Using `<canvas>`'s `drawImage()` on a `<video>`

For technical reasons, the video you see isn't actually a video. If you inspect the page on Highlight, you won't find a <video> tag. Instead, you'll find an <iframe>.

At a high level, the video you see is a reconstructed DOM that has changes applied to it as the video plays. Because the video isn't an actual <video>, we couldn't go with this approach.

Using a headless browser with `getDisplayMedia()`

We could spin up a headless browser that takes the screenshot asynchronously. This would be a costlier project in terms of engineering effort and maintenance. In the long term, this will probably be what we end up doing.

There are some performance implications when using html2canvas on deep DOM trees. In the ideal world, we offload this work from the client to the server.

Sending Slack Messages with Images using Go

John Pham — Fri, 24 Sep 2021 23:05:40 +0000

Background

I was building a feature on Highlight that allows you to create a comment on a video. This comment is special because it has not only the author and the text but also the video's x-coordinate, y-coordinate, and current time. You can create a comment by clicking anywhere on the video at any point during playback.

When writing a comment, you can tag individuals or Slack channels. When you create a comment that has tags, then the things that are tagged will get a notification in Slack.

The first version of notifications we shipped only showed the comment's author and text. This wasn't ideal because since the comment is related to the video's coordinates and time, any context in the comment text is lost. By adding a screenshot, we bring the whole context of the comment to the notification. Now the recipient doesn't have to go into Highlight to get the full context, they'll have all of it with the notification.

Here's what I ended up with:

The Slack Go SDK is a community SDK and not officially maintained by Slack. This means it doesn't get the same care or attention in terms of documentation and code examples.

It took me a bit to figure out how to send a Slack message with an image so I'm hoping this blog will save you time.

The Code

From the browser, I'm sending the image as a base64 image. This isn't required. I'm using base64 instead of a file because of other reasons.

Here's a simplified version of the code I ended up using:

import (
    "os"
    "encoding/base64"
    "errors"
    "github.com/slack-go/slack"
)

func SendSlackAlert(taggedSlackUsers []string, commentText string, base64Image string) error {
    slackClient := slack.New(os.Getenv("SLACK_ACCESS_TOKEN"))

    // For every tagged user, join the channel and send the message.
    for _, slackUser := range taggedSlackUsers {
        if slackUser.WebhookChannelID != nil {
            // The Slack API handles:
            // 1. Joining a channel the bot is already a member of
            // 2. Joining a Slack user
            // Because of this, we can skip checking for this in our application code.
            _, _, _, err := slackClient.JoinConversation(*slackUser.WebhookChannelID)
            if err != nil {
                log.Error(e.Wrap(err, "failed to join slack channel"))
            }

            _, _, err = slackClient.PostMessage(*slackUser.WebhookChannelID, slack.MsgOptionBlocks())
            if err != nil {
                return e.Wrap(err, "error posting slack message via slack bot")
            }
        }
    }

    // We need to write the base64 image as a png on disk to upload to Slack.
    // We create a unique file name for the image.
    uploadedFileKey := fmt.Sprintf("slack-image-%d.png",time.Now().UnixNano())

    dec, err := base64.StdEncoding.DecodeString(*base64Image)
    if err != nil {
        log.Error(e.Wrap(err, "Failed to decode base64 image"))
    }
    f, err := os.Create(uploadedFileKey)
    if err != nil {
        log.Error(e.Wrap(err, "Failed to create file on disk"))
    }
    defer f.Close()
    if _, err := f.Write(dec); err != nil {
        log.Error(e.Wrap(err, "Failed to write file on disk"))
    }
    if err := f.Sync(); err != nil {
        log.Error("Failed to sync file on disk")
    }

    // We need to write the base64 image to disk, read the file, then upload it to Slack.
    // We can't send Slack a base64 string.
    fileUploadParams := slack.FileUploadParameters{
        Filetype: "image/png",
        Filename: "Upload.png",
        // These are the channels that will have access to the uploaded file.
        Channels: channels,
        File:     uploadedFileKey,
    }
    _, err = slackClient.UploadFile(fileUploadParams)

    if err != nil {
        log.Error(e.Wrap(err, "failed to upload file to Slack"))
    }

    if uploadedFileKey != "" {
        if err := os.Remove(uploadedFileKey); err != nil {
            log.Error(e.Wrap(err, "Failed to remove temporary session screenshot"))
        }
    }
}

Considerations

The above code will result in 2 messages being sent:

For the comment text
For the uploaded image

Ideally, we only send 1 message with the image attached. The Slack API doesn't allow you to do this unless you are attaching the image with a URL. This means the image has to exist somewhere on the internet already.

In the above code, we send the messages then we upload the images. What if we upload the images first then use the URLs for the uploaded images to attach to each message?

That will work but will lead to this behavior:

The image is posted in the channel
The comment is posted in the channel with the attached image

This isn't the experience I wanted. The code as-is is closer to the desired experience of showing the comment, then the image to provide context.

The Slack API doesn't provide a way to upload a file "silently". Each upload to a channel will result in a message with a preview of the uploaded content.

Forming habits by taking advantage of the consistency and commitment biases

John Pham — Fri, 17 Sep 2021 02:56:21 +0000

Forming Habits

There's lots of research and studies that show creating a habit can take anywhere between 21 days to 8 months. I'm too lazy to cite them, you can find them yourselves (if you find studies that prove the counterpoint, please share!).

For the past couple of years, I've been working on a few daily habits:

Adding and reviewing my Anki cards
Practicing Chinese
Walking 11,000 steps
Intermittent fasting
Lifting weights

Most habit trackers are made for you to check off the habit after you've completed it. This means, unless you have the habit scheduled, you are unlikely to do it. I've found a way to use the consistency and commitment biases to help me do my daily habits.

The Consistency Bias

Humans have a bias to staying consistent. If you've told someone or even yourself you'll do something, the odds of you doing it is a lot higher if you hadn't told it. Even when it acts against our best interest, we tend to be consistent with our prior commitments, ideas, thoughts, words, and actions.

There are many reasons for this: a big one for me is to appear "right". If I tell someone I'm going to do something, otherwise, I'll be in an awkward situation. Being or appearing "right" is better than appearing "wrong".

The Commitment Bias

The commitment bias is very similar to the consistency bias. The difference is that the commitment bias is a subset of the consistency bias. The commitment bias applies to things you've committed to while the consistency bias applies to your beliefs, ideas, and actions.

How to take advantage of the biases for your habits

Okay so how do we use the consistency and commitment bias to form habits?

Instead of checking off a habit after you've completed it, you should check off the habit at the start of the day before you've completed it.

By doing this, you're committing yourself to do the habit. If you don't do the habit by the end of the day, you will have to face yourself being inconsistent by un-checking the completed habit.

Enabling brotli compression in Go and Chi

John Pham — Tue, 07 Sep 2021 23:17:48 +0000

Why Brotli?

Brotli is supported on most modern browsers. For browsers that don't support Brotli, the compression method will fall back to gzip or deflate based on the request's Accept-Encoding header. Enabling Brotli compression is an easy performance win.

Brotli is usually more effective at compressing compared to gzip and deflate. When I enabled Brotli compression for Highlight, I saw response sizes decrease around 40% with no latency increases (for the most part requests were faster!).

The Code Changes

Installing Dependencies

# Install the Chi, older versions of Chi don't support Brotli
go get -u github.com/go-chi/chi

# Install the package that will do the Brotli compression
go get -u gopkg.in/kothar/brotli-go.v0

Enabling Brotli Compression

r := chi.NewMux()
// /* means to compress all content types that can be compressed.
compressor := middleware.NewCompressor(5, "/*")
compressor.SetEncoder("br", func(w io.Writer, level int) io.Writer {
    params := brotli_enc.NewBrotliParams()
    params.SetQuality(level)
    return brotli_enc.NewBrotliWriter(params, w)
})
r.Use(compressor.Handler)

The "/*" means to compress all content types that can be compressed. These are the supported types:

var defaultCompressibleContentTypes = []string{
    "text/html",
    "text/css",
    "text/plain",
    "text/javascript",
    "application/javascript",
    "application/x-javascript",
    "application/json",
    "application/atom+xml",
    "application/rss+xml",
    "image/svg+xml",
}

That's it, congrats on the easy performance win for you and your users!

How we create animated product stories

John Pham — Fri, 13 Aug 2021 18:18:33 +0000

At Highlight, we make it our goal to add sparkles to our product updates when we can. One way we do this is by sharing product updates with dynamic animations rather than with static images. This allows us to tell the story of a product update in its entirety; something that we wouldn't be able to with just an image. Here are some examples:

The full images can be seen here, here, and here.

We use these animated stories when sharing to Twitter, LinkedIn, and directly to customers.

These animated stories are not a new concept. We were heavily inspired by others like GitHub and Retool.

We'd like to share how we create our animated stories.

Tools

A screen recorder (We use QuickTime Player on Mac)
A video editor that supports keyframes (We use Adobe After Effects)
- After Effects is overkill for creating these. Here are some alternatives (we haven't tried all of these):
  - descript
  - Blender
  - Shotcut

1. "The Script"

Plan out the story you want to show. You should be able to record the story in less than 15 seconds. We shoot for animated stories that are 5 seconds. We usually get a 5 second animated story from a 15 second recording after cuts and speed ramps (speeding up the video).

Tips

Record the beginning and ending so you can create a looping video (no real reason other than it's oddly satisfying)
Use a mouse instead of a trackpad (we're more accurate and have smoother motion with a mouse)
Make sure you're not recording any sensitive information

2. Editing

Import your recording into your video editor
Set the video canvas resolution (we use 500x500 if we're lazy)
- This depends on where you want to show the animated story. Different platforms UIs are optimized to render media at different sizes. You can read this guide for sizes.
Optional: Set your recording layer to 3D mode if you want to do any 3D effects.
Add speed ramps
- Speed ramps are used to speed up portions of your video. I usually speed up the video by 200% (because we're slow at recording mouse movements and keyboard input). We also will sometimes go up to 400% depending on what's on-screen.
Add keyframes
- We use scale, position, opacity, and rotation often.
Export to .gif and .mp4.
- Why both? We prefer using .mp4 where possible and .gif as a fallback. Most platforms give you analytics like view count for .mp4s but not .gifs. .mp4s are generally a lot smaller size-wise compared to .gifs too.

Example

Here's what our video editor looks like after adding keyframes.

Demo

Here's a recording of how we edited an animated story.

Tip

There's no "right" way to edit your video. Every edit is different depending on what's recorded. Some "plots" for the stories will be pretty stale. In those cases, we will add "sparkles" by playing with perspective. A simple rotation will take a boring pan and add a little more wow.

Here's an example: the left is without a 30° tilt and the right is with a 30° tilt.

Aaand that's pretty much it! Hope this helps you add some sparkles to your product updates.

If you end up creating animated stories, I'd love to see them! You can reach me on Twitter @JohnPhamous.

Until next time, ✨

Thanks to jsjoeio for reviewing the draft!

Hiding Microsoft Teams status icons

John Pham — Tue, 23 Jun 2020 04:20:12 +0000

Microsoft Teams shows your coworkers current status from available, busy, to away. I can see the utility in knowing this but for myself, I'd rather not know.

From working remotely, I've found asynchronous communication to be the best way to do most of my communication. The status icons changes the way I chat. For example, if I send a message to someone that is available I could sit there waiting for an instant response. Take the status away and my expectation that the person will respond soon goes away.

Removing the status icons

Teams makes an API call to get the status of your coworkers. The URL that it hits is http://presence.teams.microsoft.com. If we can block those requests, we'll stop seeing the status icons.

We can use Fiddler to block requests to a specific URL. Because Fiddler works on all OS', we can do the following steps across all the devices we use.

Install Fiddler
Open the AutoResponder tab
Check "Enable rules"
Check "Unmatched requests passthrough"
Click "Add rule"
In the "Request URL Pattern" input, enter http://presence.teams.microsoft.com
In the "Local file to return or *Action to execute" input, choose "*drop"

With that done, Teams will no longer show the status icons.

Extending this to other applications

If you'd like to do this for other applications, you can use Fiddler to see all the network requests an application makes. Once you find the API endpoint the application makes, you can make a rule to drop the request.

Debugging Checklist

John Pham — Tue, 28 Apr 2020 18:55:00 +0000

We've all been there. You're working on fixing a bug and have been at it for hours. You're jumping around files, looking at call stacks, and randomly changing code hoping it'll fix the bug. I've been down this rabbit hole many times and it didn't help that I didn't have a framework for debugging.

Recently I read Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems by Dave Agans. The book is a short read that teaches you the rules through stories. The 9 rules are:

Understand the system
- Does this function handle edge cases?
- Are dates suppose to be in a specific format or timezone?
- Will data ever be null?
Make it fail
- Find reproduction steps to make it fail
- The shorter the reproduction steps are, the better
- Can you write a test case to cause the failure? If so add it to your test suite
Quit thinking and look
- Don't just think/guess about what could be causing the bug, find it!
- Your guesses can help you focus your search
Divide and conquer
- Just like in algorithms, divide and conquer algorithms are faster than linear algorithms
- Is the bug from the backend? Database? Frontend? A library?
Change one thing at a time
- Don't change multiple things then observe if the bug persists
Keep an audit trail
- Write down what changes you've tried and what the effects are if any
- This is helpful if you need to pull in other people because you can easily reference what you've already tried
Check the plug
- Validate your assumptions
- Is there data in the database? Can it ever be null?
- 99.999% uptime is still not 100%
- Is your development/test environment the same as production?
Get a fresh view
- Reach out for help
- Write/talk out loud your problem, rubber duck debugging helps
- Take a break and come back with fresh eyes
If you didn't fix it, it ain't fixed
- If the bug "disappears" without any fixes applied, it isn't fixed
- The bug can only occur when the stars are aligned

As you read through the rules, you might think they're obvious things you already do while debugging. The best things are often times obvious in retrospect.

What the COBOL?

John Pham — Sat, 11 Apr 2020 23:06:42 +0000

The COmmon Business-Oriented Language (COBOL) is a compiled programming language first introduced in 1959. COBOL is not a language you would reach for when building new software however it is still used by many systems today. These are huge systems that keep the world running. Think of systems that handle banking transactions for example. Banking itself has such complex business logic that rewriting the COBOL code to a "modern" language would be a huge undertaking.

So why learn COBOL? COBOL isn't exactly the language taught or sought after today. If you look at today's landscape, you'll find C++, Javascript, Rust, and many other "modern" languages. The number of COBOL developers is decreasing but there are systems that still need to be maintained. Being a COBOL developer could be a very lucrative job.

Let's now take a look at COBOL at a high level.

COBOL versions

The COBOL language has 3 main versions:

COBOL-85
COBOL-2002
COBOL-2014

As a COBOL developer, you can expect most of your development will be done in COBOL-85.

GnuCOBOL

GnuCOBOL is one of the more popular COBOL compilers.

Installing on Windows

Follow the instructions on GnuCOBOL's SourceForge page.

Installing on Ubuntu

sudo apt-install gnucobol

Installing on MacOS

We will be using Homebrew to install GnuCOBOL.

brew install gnu-cobol

File extensions

COBOL programs can have the following file extensions:

.cobc
.cbl
.cob
.cpy

The extensions used will depend on the coding practices set by the project.

Hello world!

Let's create a new file called hello-world.cbl.

IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO-WORLD.
PROCEDURE DIVISION.
DISPLAY 'Hello, world'.
STOP RUN.

We've written our first COBOL program! Let's compile and run it now. In a terminal, do the following:

cobc -x -F hello-world.cbl # Creates an executable called hello-world
./hello-world # Runs the executable
> Hello, world

That's it for now. In future posts we'll take a deeper look in how to write COBOL.

Build a better email verification experience

John Pham — Thu, 26 Dec 2019 23:44:38 +0000

You find a new application and you're excited to use it. You go through the account creation process but are blocked from using the application until you verify your email. This is a pain point in the user journey and can be the point where potential users stop using your application.

Let's avoid our user from doing that. In this article we'll be focusing on how we can change the experience around email verification in on-boarding experiences.

Interrupting the on-boarding experience

The on-boarding experience is one of the most crucial experiences your users will have. If the experience is painful, then users are more likely to exit. Our goal as creators should be to create experiences that delights and brings value to the user.

Most on-boarding experiences need you to verify your email before you use the application. You should consider allowing the user to use the smallest set of features that will entice and keep the user engaged. Requiring the user to step outside of your application, to their email client, find the email, and then verify their account adds friction. If the user isn't enticed during the previous experience then what's keeping them from moving forward?

There are security and resource implications with allowing the user to use your application before verifying their account. This is where you need to come up with policies such as sandboxing, account cleanups, and rate limiting to protect you and your other users.

Reducing the friction

So we need the user to verify their account at the end of the day. We can tell the user to go to their inbox and verify from there. That's a lot of steps and your competitors are most likely requiring the user to do the same thing. This is your chance to differentiate yourself from the competition.

Here's the general experience:

1. Go through account creation journey
2. Prompt to verify account by clicking link in email
3. Open new tab
4. Go to email provider
5. Find email
6. Click the verification link

Let's look at how we can improve the experience. From the initial account creation step, we know their email address. What if we can provide a link to their email provider from our experience? Here's how the new experience would look like:

1. Go through account creation journey
2. Prompt to verify account by clicking link in email, provide a link to email provider
-3. Open new tab
-4. Go to email provider
5. Find email
6. Click the verification link

2 steps less may not seem like a lot but it could be the difference between successfully or unsuccessfully on-boarding a new user.

The technicals

Format of an email address

How can we achieve this? Let's look at a few scenarios.

Scenario 1

User signs up with ada@gmail.com
We want to provide the user with a link to Gmail

Scenario 2

User signs up with turing@outlook.com
We want to provide the user with a link to Outlook

Okay, this problem looks pretty simple. We can look at the domain of an email to determine which email provider to link to. We'll end up with a mapping like this:

Domain	Email Provider
outlook.com	https://outlook.office.com
gmail.com	https://gmail.com

But wait! What about edge cases?

Scenario 3

User signs up with grace@hopper.com
We want to provide the user with a link to ???

We haven't considered the case where the user is using a custom email domain. We can't determine which email provider the user is using from only looking at their email address' domain.

For this example, let's assume hopper.com uses Gmail as their email provider. How can we determine that? We can use MX records!

Mail exchanger records (MX records) tell the internet which server handles sending and receiving emails for a domain. For our example, hopper.com would have a MX record telling the internet that Gmail handles it's emails.

In our backend, we can leverage existing tools to look up the MX records for a domain. If you're on a Nix* OS, you can test this out by using host.

[I] ➜ host hopper.com
hopper.com has address 198.179.227.105
hopper.com mail is handled by 5 aspmx.l.google.com.
hopper.com mail is handled by 10 alt1.aspmx.l.google.com.
hopper.com mail is handled by 15 alt2.aspmx.l.google.com.

By looking up the MX records, we now cover all cases. If you're looking to add this to your project, here are a few ways to do it: Node, Ruby, and Python.

Wrapping up

We've learned that we can improve our email verification process by:

Allowing users to use our application in a non-destructive way before they verify their email
Reduce the friction for users going to their email to click on the verification link

A problem for you

There's another pain point with email verifications. You are asking your user to go to their inbox and find an email from you.

Scenario 1

User gets prompted to verify email
User immediately goes to their inbox
Your email is most likely at the top of their inbox

Scenario 2

User gets prompted to verify email
User doesn't immediately go to their inbox
Your email is most likely n emails down in their inbox

In both scenarios, your user could get distracted by other emails and forget about verifying their account.

Is there a way to solve this problem? Here's a hint: query parameters.

Fast track yourself through hackathons with APIs

John Pham — Wed, 25 Dec 2019 06:07:13 +0000

You're at a hackathon, have a stellar team, and a really cool idea but only 24-36 hours to work on it. Your team could implement everything from scratch. If your goal is to learn then do that! If you want to test an idea then you should leverage APIs and focus on building what makes your idea unique.

What's an API?

An API is an Application Programming Interface. Think of this as a function you can call from your application to do work such as: identifying puppies in an image and storing the location of your favorite coffee shops in a database.

It doesn't matter if you're building a web application, mobile app, or virtual reality game. APIs are platform agnostic so you can use them no matter what you're building.

Most APIs are either completely free or paid once you past a certain usage. For most hackathon projects, you'll stay in the free tiers. If your project becomes an overnight success then you'll have to start looking at the paid tiers.

There are pretty much APIs that do anything you could want! You can find APIs in this list, the GitHub Student Developer Pack, and from your hackathon's sponsors.

Here's a general workflow you can use during a hackathon:

Identify a feature
Find an API to help implement that feature
Find documentation and code examples for the API
Add the API to your application

Now let's walk through an example:

I want to build an application to show Pokemon
I've found the PokéAPI that can provide me the data
Found the documentation
I can add the API by creating my own client or using a wrapper library

Let's take a deeper look at steps 3 and 4.

The documentation

Before you choose to use an API, make sure it fulfills your requirements. Next you'll want to look at the documentation and ask:

Are there a lot of code examples?
Are all the parameters defined?
How much can I use this API before I have to start paying?
Is there a community around the API? Slack, Reddit, Twitter, StackOverflow, etc.

These are all signs of a well-maintained API. A well-maintained API means that if you run into a problem, you can usually find someone to help.

Using the API

If you don't know how REST and Request Methods works, please read Understanding REST and REST APIs.

Request Methods are how your application will communicate with APIs. Everyone programming language has it's own way of using Request Methods.

Going back to our example, we want to get show Pokemons in our application. From reading the documentation, there is the /api/v2/pokemon endpoint.

The documentation shows which Request Method to use (GET), what parameters we can use ({id or name}), and what the expected response is.

Before using this API directly in our application, let's test the API using a GUI tool. The benefits of using GUI tools includes iterating quickly without having to rebuild your application and having code exported from the GUI which you can use directly in your application.

For this example we will be using Postman, but you can also use Insomia and Postwoman.

With Postman, we want to make a GET request to https://pokeapi.co/api/v2/pokemon endpoint. Clicking send with make the request, we'll then see Pokemon data below.

What if we want to lookup a specific Pokemon? Say Charizard? From the documentation, all we have to do is add the name of the Pokemon we want to lookup to the endpoint: /pokemon/charizard.

We've successfully made API calls to get all the Pokemon and a specific Pokemon! Let's add this to our application now. A handy feature of Postman is it allows you to export your request to code.

You can access this feature by clicking Code then select the language you want to export to.

Wrapping up

You've learned how to leverage APIs during hackathons, how quickly iterate using a GUI tool, and export code. With this, you can focus on building what makes your project unique during the hackathon.

Notes on working at NASA JPL vs. Microsoft

John Pham — Thu, 31 Oct 2019 00:21:57 +0000

I wanted to sum up what it's like working at a research facility vs. a big tech company. Note that my experience isn't the same for everyone at both organizations. There will always be nuances based on factors such as team, role, and seniority.

What it's like working at NASA Jet Propulsion Laboratory

John Pham ・ May 24 '19 ・ 4 min read

#career

This is going to be a series of small posts covering the following areas:

Development Workflow
Work Life Balance
Resources/Benefits
Community/Culture

Happy to talk about other areas folks would like to hear about!

The Development Workflow

Some questions to answer:

How does the team function?
How are requirements formed? Who does it?
How is the tech stack decided?
How is testing done?
What's the scope of your job?

At NASA JPL

Team

The way I define team here are the people I worked with on a daily basis for the project I was on. The team I was only had about 9 people:

2 front-end focused software engineers
4 back-end focused software engineers
1 program manager
3 testers

Requirements

The program manager converts conversations with our end users (scientists, operations, and engineers) to requirements. Requirements manifest as tickets in Jira. The tickets would be assigned to a person based on the type of work (frontend/backend). We'd then have a 2 week sprint to finish the work. Work was measured with points based on the Fibonacci sequence.

Tech Stack

Backend mostly Java with some in Node with Typescript
Frontend is Angular
Data contracts were written in JSON Schema which allowed us to export type definitions for Java and Typescript
Database was MongoDB
Code is versioned with GitHub

Testing

Engineers were not expected to write their own tests. Once developers felt their code was ready to be tested, a ticket would be created and assigned to a tester to test the code. Testers would write their test cases and if cases failed, they would let the developer know. The developer would then fix their code and pass it along to the tester again. This loop would continue until no more bugs were found.

This was a pattern I saw more among the senior engineers. It's a pattern that didn't really sit well with many and would cause headaches for engineers, testers, and program managers. Engineers could work on their feature for a month+ before handing it off to a tester for a first pass and the tester could be given little context on what it's suppose to do.

Other engineers were open to writing their own tests to supplement those written by testers.

The testing requirements were mandated by the US Government but the implementation was up to the testers. Some groups were fine with just unit tests and others required a bigger test suite that included integration, smoke, and E2E tests.

Job Scope

When you join JPL, you join a team that focuses on a specific area
You join project(s) which compose of people across teams
Your time is split between projects based on a %

Because of these 2 points, the type of work you do can either be narrowly or widely scoped.

Say you join a frontend team. You would then be assigned to projects that need frontend help. If you wanted to get experience with backend, you could start working on backend tasks for that project. Down the line you could join a project as a backend dev or switch to a backend focused team.

Now off to Washington

At Microsoft

Team

The way I define team here are the people that are in the same organization.

61 software engineers
26 program managers

The difference here is that software engineers are expected to work across the stack. Individuals do pick up preferences for frontend/backend/infrastructure.

Requirements

Our organization works on a breath of products/services (projects) used internally and externally. Program managers are assigned project(s) and create requirements from market research, customer engagements, and logs. Requirements manifest themselves as features in Azure DevOps.

Software engineers also join projects and can be on multiple projects.

Big picture goals are decided on a quarterly basis. At the start of each sprint, engineering managers, engineers, and program managers go through the backlog together and pull features in. Features are weighted based on how many engineering hours.

Tech Stack

Backend is .NET
Frontend is Angular (maintaining) and React (new development)
Databases are Azure managed databases (MySQL, Cosmos, etc.)
Code is versioned and deployed with Azure DevOps

Testing

Software engineers are expected to test their own code. Tests are ran locally and also as part of the CI/CD process on Azure DevOps.

Job Scope

When you join Microsoft, you join a team that focuses on a specific area
Your team works on different projects, you're expected to be able to move between all of them

Most teams are full stack. They also manage the operations, telemetry, and support for their projects.

If you prefer to have a specialization, you can communicate that with your manager and they'll be happy to get you work in your area of choice.

A lot of work leverages work done by other teams. For example, the platform that ingests application logs is managed by a different team. Our team doesn't have to standup our own instance or recreate the wheel.

Checklist for Starting a New Job in a New State

John Pham — Thu, 13 Jun 2019 05:52:18 +0000

Starting a new job is exciting. Moving to another state is exciting too but can also be daunting. I hope you find this checklist useful for when you find yourself in this situation.

This is written under the assumption that you're flying to your new place and packing light

Before you move

Find a place to live
- Crash a friend's, AirBnb, or Corporate Housing while you search. This should take about a week to do. (Pick the best place after you've seen 37% of the places you want to visit, it's the most optimal choice
Get copies of your medical records
- Visit your doctor, dentist, and anyone else who you've seen in the past and get physical or digital records. The records will be useful for your new health care providers.
Cancel utilities/services/memberships you won't be using
Request for mail forwarding with USPS
- If you're expecting any mail at your old address, you can get it forwarded to your new address
Change the billing address with your banks
Change the mailing address with the IRS with a Form 8822
- Ensures you receive all correspondence and tax refunds as soon as possible

Once you get there

Register to vote
- Some States let you do it online or when you apply for a driver license
Get a driver license or an Enhanced Driver License
- The latter can be used as proof of citizenship and act as a limited passport
Sign up for utilities
- Includes: phone, internet, gas, water, electricity
- Check with the place you're staying, your employer, and any organizations you belong to if they have deals
Establish Legal Residency (Legal Domicile) in your new state allows you to:
- Avoid paying income taxes for 2 states
- California has income taxes, Washington doesn't, so you won't be subject to income taxes
- Qualify for in-state tuition for universities
- Power to vote for the local and state government
Buy used when you can
- You can find great pieces of furniture for the fraction of the retail price on the used market
- If you live near a university or large workplaces, check if they have garage sales
- Buying used is not only a good economically, but also good for the environment
Plan your commute during the hours you will be commuting. Google Maps and CityMapper are some tools you can use.

After your first week at work

Sign up for your employer's 401(k) and Health Savings Account
- If your employee has 401(k) matching, this is basically free money
Checkout the perks/discounts your employer offers
Find new health care providers

Once you're settled in

Sign up for a gym/fitness club
Participate in your companie's Employee Stock Purchase Plan
Contribute to your individual retirement account (Set contributions to occur on an automated schedule)
Explore your new place!
- Hikes, classes, food scene, and new hobbies!

Thanks to Sam and Bill for some advice

Forem: John Pham

Building a Reverse Proxy with Cloudflare Workers

Background

The Problem

Reverse Proxy to the Rescue

Code

Alternative Solutions

Using <canvas>'s drawImage() on a <video>

Using a headless browser with getDisplayMedia()

Sending Slack Messages with Images using Go

Background

The Code

Considerations

Forming habits by taking advantage of the consistency and commitment biases

Forming Habits

The Consistency Bias

The Commitment Bias

How to take advantage of the biases for your habits

Enabling brotli compression in Go and Chi

Why Brotli?

The Code Changes

Installing Dependencies

Enabling Brotli Compression

How we create animated product stories

Tools

1. "The Script"

Tips

2. Editing

Example

Demo

Tip

Hiding Microsoft Teams status icons

Removing the status icons

Extending this to other applications

Debugging Checklist

What the COBOL?

COBOL versions

GnuCOBOL

Installing on Windows

Installing on Ubuntu

Installing on MacOS

File extensions

Hello world!

Build a better email verification experience

Interrupting the on-boarding experience

Reducing the friction

The technicals

Scenario 1

Scenario 2

Scenario 3

Wrapping up

A problem for you

Scenario 1

Scenario 2

Fast track yourself through hackathons with APIs

What's an API?

The documentation

Using the API

Wrapping up

Notes on working at NASA JPL vs. Microsoft

What it's like working at NASA Jet Propulsion Laboratory

John Pham ・ May 24 '19 ・ 4 min read

The Development Workflow

At NASA JPL

Team

Requirements

Tech Stack

Testing

Job Scope

Now off to Washington

At Microsoft

Team

Requirements

Tech Stack

Testing

Job Scope

Checklist for Starting a New Job in a New State

Before you move

Once you get there

After your first week at work

Using `<canvas>`'s `drawImage()` on a `<video>`

Using a headless browser with `getDisplayMedia()`