Forem: Youngho Andrew Chaa

The Secret Language of Data: Vectors and Cosine Similarity

Youngho Andrew Chaa — Wed, 10 Dec 2025 16:13:16 +0000

Ever wonder how Netflix knows what movies you'll like, or how Google finds exactly what you're searching for even if you just type a few words? It's math, unfortunately 😂. And at the heart of it lies a powerful concept called Cosine Similarity, built upon the idea of vectors and the humble cosine function.

Part 1: What is a Vector? (Your Data's Direction and Distance)

Imagine you're giving directions to a friend: "Walk 5 blocks North." This simple instruction contains two crucial pieces of information:

Direction: North
Magnitude (or Distance): 5 blocks

In the world of math and data science, anything that has both a direction and a magnitude is called a vector.

Vectors in Real Life:

A plane flying 500 mph East.
The wind blowing 15 mph from the Northwest.
The force you use to kick a soccer ball.

We draw vectors as arrows, where the arrow's length shows its magnitude, and its point shows its direction. In computers, we represent them as lists of numbers: [3, 4] might mean "go 3 steps right, then 4 steps up."

Part 2: The Cosine Function (The Shadow Ruler)

Next, let's talk about Cosine. This is one of three main tools (Sine, Cosine, Tangent) we use to measure right-angled triangles (triangles with a perfect 90-degree corner).

Imagine a ladder leaning against a wall. The ladder itself is the Hypotenuse (the longest side). The ground it touches is the Adjacent side (it's "next to" the angle the ladder makes with the ground). The wall it reaches is the Opposite side (it's "opposite" the angle).

The Cosine rule (CAH) tells us:

Think of it like this: The cosine tells you how much of the ladder's length is stretched out along the ground (its "shadow").

If the ladder is almost flat (small angle), its shadow is long ($\cos(\theta)$ is close to 1).
If the ladder is almost straight up (large angle), its shadow is short ($\cos(\theta)$ is close to 0).

Part 3: Putting It Together – Cosine Similarity for Documents!

Now, for the exciting part: how do vectors and cosine help us compare documents or anything else?

Imagine every document, sentence, or even a user's movie preferences, is turned into a vector.

📝 Document Vectors Explained:

Let's take two sentences:

Sentence A: "Apple is sweet."
Sentence B: "Banana is sweet."

Vocabulary: First, we make a list of all unique, important words across all our sentences: [Apple, Banana, sweet]. This list defines our dimensions. (Our vector will have 3 dimensions, not because each sentence has 3 words, but because our vocabulary has 3 unique words!)
Vector Creation (Term Frequency): We then count how often each word appears in each sentence:
- Vector A for "Apple is sweet": [1, 0, 1] (1 Apple, 0 Banana, 1 sweet)
- Vector B for "Banana is sweet": [0, 1, 1] (0 Apple, 1 Banana, 1 sweet)

The "Shadow" of Document Similarity:

Now, we want to know how similar these two vectors (sentences) are. Cosine Similarity asks: "Are these two vectors pointing in roughly the same direction?"

The formula is:

Let's break down the parts:

A . B (The Dot Product): This calculates the "overlap" or "alignment" between the two vectors. It multiplies the counts for each word and adds them up.
- For our example: (1*0) + (0*1) + (1*1) = 0 + 0 + 1 = 1.
- The dot product effectively finds out how much of Document A's "shadow" falls directly onto Document B's "path." If both documents frequently use a common word, this number goes up!
||A|| ||B|| (The Magnitudes): These are the "lengths" of our document vectors. A longer document will have a larger magnitude.
- For our example: ||A|| = sqrt(1^2 + 0^2 + 1^2) = sqrt(2) and ||B|| = sqrt(0^2 + 1^2 + 1^2) = sqrt(2).
Putting it all together:

What does 0.5 mean?

A score of 0.5 tells us that "Apple is sweet" and "Banana is sweet" have a moderate similarity. They share the concept of "sweet" but differ on the main subject.

A score of 1 means they are identical in direction (talking about the exact same thing).
A score of 0 means they are completely unrelated in direction (talking about totally different things).

The Power of Cosine Similarity

The real genius of Cosine Similarity is that it focuses purely on the direction (topic) of the vectors, not their magnitude (length). A short document and a long document can be highly similar if they both discuss the same topic in the same proportions. This is incredibly useful for:

Search Engines: Finding documents that are topically relevant to your query.
Recommendation Systems: Suggesting movies, products, or articles similar to what you already like.
Plagiarism Detection: Identifying documents with similar content.

So, the next time you get a perfect search result or a spot-on recommendation, remember the silent power of vectors and their angular dance, measured by the humble cosine!

Decoding the 'Invalid Color Space' Error: A Dive into an FFmpeg 7 Bug

Youngho Andrew Chaa — Wed, 08 Oct 2025 09:08:42 +0000

It's a scenario every developer knows: a critical process, running smoothly for months, suddenly starts failing in production. No recent code changes on our side, no infrastructure tweaks. This is the story of how we chased down a mysterious ffmpeg error and implemented a robust fix.

The Unexpected Error

Recently, our video processing pipeline began throwing a peculiar error for some user-uploaded videos:

Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, reserved/reserved/smpte170m, progressive), 398x720, 1060 kb/s, 27.37 fps, 29.83 tbr, 90k tbn (default)
      Metadata:
        creation_time   : 2025-10-08T08:31:00.000000Z
        handler_name    : VideoHandle
        vendor_id       : [0][0][0][0]
[graph -1 input from stream 0:0 @ 0x7f505c004880] Invalid color space

The key phrase here is Invalid color space. This was puzzling because our ffmpeg commands for image extraction hadn't changed. The error suggested that ffmpeg couldn't determine the correct color mapping for the video, but why now?

The Root Cause: A Bug in FFmpeg 7

After some investigation, I traced the problem to a recent update of ffmpeg on our production servers to version 7. A search through the ffmpeg bug tracker led us to this ticket: #11020 - Invalid color space error.

The bug report confirmed our suspicions. A change in ffmpeg version 7 meant that it no longer defaulted to a standard color space (like BT.709) when a video's metadata was missing or ambiguous. While technically a move towards stricter compliance, this change broke compatibility with many existing videos that don't explicitly declare their color space.

The Fix: Explicitly Setting Metadata

Since we couldn't ask all our users to re-encode their videos, we needed to adapt our code. The solution was to explicitly tell ffmpeg which color space to assume for these videos.

I achieved this by using ffmpeg's bitstream filter (-bsf:v) to inject the necessary metadata during processing. Specifically, I added the argument h264_metadata=matrix_coefficients=1. This tells ffmpeg to treat the H.264 video stream as having color matrix coefficients corresponding to the BT.709 standard, a common choice for HD video, resolving the ambiguity.

However, there's a catch.

Making the Fix Robust: A Codec-Specific Solution

The h264_metadata filter is, as the name implies, only for H.264 videos. Applying it to a video encoded with a different codec (like VP9 or HEVC) would cause ffmpeg to fail with a different error. To prevent this, we first need to identify the video's codec.

We use ffprobe, a companion tool to ffmpeg, to inspect the video file and retrieve the codec name before constructing our ffmpeg command.

Here's the ffprobe command we run:

ffprobe -v error -select_streams v:0 -show_entries stream=codec_name -of csv=p=0 /path/to/video.mp4

-v error: Suppresses all logging except for errors.
-select_streams v:0: Selects the first video stream.
-show_entries stream=codec_name: Instructs ffprobe to only output the codec_name.
-of csv=p=0: Formats the output as plain text with no headers.

This command reliably gives us the codec name, for example, h264.

Putting It All Together in Code

Here is the final implementation logic in our Python service.

First, we get the codec_name using ffprobe:

ffprobe_result = await run(
    "ffprobe",
    "-v", "error",
    "-select_streams", "v:0",
    "-show_entries", "stream=codec_name",
    "-of", "csv=p=0",
    video_path.as_posix(),
)

# Error handling omitted for brevity...

ffprobe_output = ffprobe_result.stdout.decode("utf-8").strip()
codec_name = ffprobe_output.split("\n")[-1]

Next, we conditionally build our ffmpeg arguments. If the video is H.264, we add our fix. Otherwise, we proceed as normal.

ffmpeg_args = []

# Conditionally add the metadata fix for H.264 videos
if codec_name == "h24":
    ffmpeg_args.extend(["-bsf:v", "h264_metadata=matrix_coefficients=1"])

# Add the rest of the arguments for thumbnail extraction
ffmpeg_args.extend([
    "-ss", str(safe_timestamp_s),
    "-noaccurate_seek",
    "-i", video_path.as_posix(),
    "-frames:v", "1",
    image_path.as_posix(),
])

# Execute the command
result = await run("ffmpeg", *ffmpeg_args)

This approach ensures that we fix the color space issue for problematic H.264 files without breaking processing for any other video formats.

Key Takeaway

This experience was a powerful reminder that dependencies can and do change. A seemingly minor update in a tool like ffmpeg can have significant ripple effects. By being explicit in our commands and building robust, conditional logic, we can create more resilient systems that are less susceptible to upstream changes. Happy coding!

The Pitfall of ORDER BY Timestamp (And How to Fix It)

Youngho Andrew Chaa — Tue, 30 Sep 2025 09:30:03 +0000

As developers, we often need to fetch the "latest" record from a database. It could be the most recent status update, the last comment on a post, or the newest entry in a history table. The SQL seems obvious, right? You just ORDER BY a timestamp column and grab the top one.

-- Find the latest status for a model
SELECT model_status
  FROM model_status_history
 WHERE model_uid = 'some-uid'
 ORDER BY created_at DESC
 LIMIT 1;

This query looks perfectly logical. But lurking beneath its simplicity is a subtle bug that can cause inconsistent results, especially in high-traffic applications. The "latest" record you get back might not actually be the one you expect.

The Problem: The Tie

The issue arises when your system records multiple events within the same time resolution. For example, if your created_at column stores time down to the millisecond, what happens when two records are inserted in the exact same millisecond?

When the database sees two rows with identical created_at values, it considers them tied. Without a clear "tie-breaker" rule, the database makes no guarantee about which row will come first. In one query, it might return Row A. A moment later, under slightly different conditions, it might return Row B. This is called a non-deterministic sort, and it's a recipe for unpredictable behaviour.

The Solution: Creating a Deterministic Sort

To fix this, we need to give the database a tie-breaker—a second rule to apply when the first one isn't enough. The perfect candidate is a column that is guaranteed to be unique and sequential: the primary key id.

Method 1: The Tie-Breaker

Since a primary key is unique, it can break any tie. We can add it as a second condition to our ORDER BY clause.

-- First, sort by time. If there's a tie, sort by ID.
ORDER BY created_at DESC, id DESC

This tells the database:

First, sort all records by created_at in descending order.
If any group of records has the exact same created_at value, sort that specific group by id in descending order.

Because id is unique, a tie is now impossible. The row with the highest id (the one inserted last) will always be chosen. Your query is now deterministic—it will return the same, correct result every single time.

Method 2: The Simpler, Faster Way

There's an even more direct approach. If your id is an auto-incrementing integer, it already serves as a perfect timeline of events. A higher id always means the record was created later.

This means you don't even need the timestamp for ordering. You can rely solely on the id.

-- The most reliable way to get the last inserted row
ORDER BY id DESC

This approach is not only simpler but often more performant. Sorting on an indexed, integer primary key is typically faster for a database than sorting on a more complex timestamp or datetime column.

Takeaway

Relying on timestamps alone to find the "latest" record is a risk. Due to the possibility of ties, you can get inconsistent results.

The best practice is simple: when you need the most recent entry, always sort by a unique, sequential column like your primary key. Whether you use it as a tie-breaker or as the sole ordering key, it guarantees your queries are robust, reliable, and predictable.

Securely Send SMS with Twilio, AWS Lambda, and Terraform

Youngho Andrew Chaa — Tue, 30 Sep 2025 07:23:39 +0000

In modern applications, automated notifications are essential for user engagement. Sending an SMS is a direct and effective way to reach your customers. This guide will walk you through a secure, automated, and serverless solution for sending SMS messages using Twilio, running the code on AWS Lambda, and managing the infrastructure with Terraform.

We'll stick to a key security principle: never hardcode your credentials. We'll see how to pass sensitive information like your Twilio accountSid and authToken securely from GitHub Secrets all the way to your Lambda function's environment.

The AWS Lambda Function

First, let's look at the Node.js/TypeScript code that will run on AWS Lambda. This function is responsible for the actual sending of the SMS.

The magic happens in this simple script which uses the official twilio SDK.

import twilio from 'twilio';

const accountSid = process.env.TWILIO_ACCOUNT_SID;
const authToken = process.env.TWILIO_AUTH_TOKEN;
const from = process.env.TWILIO_FROM; // Your Twilio phone number

// A crucial check to ensure the function has the credentials it needs
if (!accountSid || !authToken || !from) {
  throw new Error('Twilio configuration is missing in environment variables.');
}

const client = twilio(accountSid, authToken);

export async function sendTextMessage(recipient: string, message: string) {
  try {
    const response = await client.messages.create({
      body: message,
      from: from,
      to: recipient,
    });
    console.log('Message sent successfully:', response.sid);
    return response;

  } catch (error) {
    console.error('Failed to send message:', error);
    throw error;
  }
}

Takeaways:

No Hardcoded Secrets: Notice how accountSid, authToken, and the from number are retrieved from process.env. This is the standard way to access environment variables in Node.js. It keeps your sensitive data out of your source code.
Error Handling: The code gracefully handles potential failures by wrapping the API call in a try...catch block.
Initialization: It performs a startup check to ensure the necessary environment variables are present. If not, the function will fail fast, which is better than failing silently later.

The Infrastructure: Defining the Lambda with Terraform

Now, how do we get those environment variables into our Lambda function? We define them using Infrastructure as Code (IaC) with Terraform. This allows us to version control our cloud setup.

The Terraform configuration below defines the Lambda function and, most importantly, injects our variables into its runtime environment.

module "lambda_function_send_notification" {
  source = "./lambda-function"

  function_name                = "${var.component}_${var.env}_send_notification"
  role_arn                     = aws_iam_role.iam_lambda_role.arn
  env_vars                     = local.env_vars # This is where the magic happens!
  # ... other configurations
}

locals {
  env_vars = {
    TWILIO_ACCOUNT_SID = var.TWILIO_ACCOUNT_SID # Mapping Terraform var to Lambda env var
    TWILIO_AUTH_TOKEN  = var.TWILIO_AUTH_TOKEN  # Mapping Terraform var to Lambda env var
    TWILIO_FROM        = var.TWILIO_FROM      # Add your Twilio 'from' number here
    # ... other variables
  }
}

# Variable definitions to receive values from the CI/CD pipeline
variable "TWILIO_ACCOUNT_SID" {
  type = string
}

variable "TWILIO_AUTH_TOKEN" {
  type = string
}

variable "TWILIO_FROM" {
  type = string
}

Takeaways:

env_vars Block: The locals.env_vars map is the crucial link. It defines the key-value pairs that will become the environment variables inside the AWS Lambda function.
Variable Mapping: We map the incoming Terraform variables (e.g., var.TWILIO_ACCOUNT_SID) to the names our JavaScript code expects (e.g., TWILIO_ACCOUNT_SID). This keeps the code clean and decouples it from the naming conventions used in your infrastructure or CI/CD system.

The Automation: CI/CD with GitHub Actions

Finally, let's tie it all together. The GitHub Actions workflow automates the deployment. This is where we securely pull the actual secret values from GitHub Secrets and pass them to our Terraform plan.

name: Deploy

on:
  push:
  workflow_dispatch:

jobs:
  deploy:
    name: deploy
    runs-on: ubuntu-latest
    env:
      # These TF_VAR_ prefixes tell Terraform to populate variables
      TF_VAR_TWILIO_ACCOUNT_SID: ${{ secrets.TWILIO_ACCOUNT_SID }}
      TF_VAR_TWILIO_AUTH_TOKEN: ${{ secrets.TWILIO_AUTH_TOKEN }}
      TF_VAR_TWILIO_FROM: ${{ secrets.NAVIEN_TWILIO_FROM }} # Added the from number
      # ... other environment variables
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      # ... build and test steps ...

      - name: Set up Terraform
        uses: hashicorp/setup-terraform@v3

      - name: Terraform Init
        working-directory: ./terraforms
        run: terraform init

      - name: Terraform Plan
        working-directory: ./terraforms
        run: terraform plan -input=false

      - name: Terraform Apply
        working-directory: ./terraforms
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve -input=false

Takeaways:

GitHub Secrets: The actual credentials (${{ secrets.TWILIO_ACCOUNT_SID }}) are stored securely in your GitHub repository's settings. They are never printed in logs.
TF_VAR_ Prefix: The TF_VAR_ prefix is a special convention used by Terraform. When GitHub Actions sets an environment variable like TF_VAR_TWILIO_ACCOUNT_SID, Terraform automatically uses its value for the TWILIO_ACCOUNT_SID input variable in your .tf files.

The Complete Secure Flow

Here’s the full journey of your secret credentials, from storage to execution, without ever being exposed in your code:

Storage: Your Twilio Account SID, Auth Token, and phone number are stored as GitHub Secrets.
CI/CD Pipeline: The GitHub Actions workflow reads the secrets and exports them as environment variables prefixed with TF_VAR_.
Infrastructure as Code: Terraform reads the TF_VAR_ variables and uses them to populate the variable blocks in its configuration.
Lambda Environment: Terraform then passes these values into the environment variables section of the AWS Lambda function resource.
Execution: When the Lambda function runs, the Node.js code accesses these variables via process.env to securely authenticate with the Twilio API.

By following this pattern, you build a system that is not only automated and scalable but also secure by design. ✨

That Cryptic sbt Error? My JDK Was Too New

Youngho Andrew Chaa — Tue, 23 Sep 2025 16:27:50 +0000

I was following Scala course on Coursera. I've downloaded the code for the given assignments, I've got the tests, and I was ready to go. I opened the terminal, navigate to the project directory, and type the magic words: sbt test.

Then, it happens. A wall of red text fills your screen.

error:
  bad constant pool index: 0 at pos: 48454
    while compiling: <no file>
...
Exception in thread "sbt-parser-init-thread" java.lang.ExceptionInInitializerError
Caused by: scala.reflect.internal.FatalError: 
  bad constant pool index: 0 at pos: 48454
...

My first thought was, "What did I break?" I scanned the code, but I haven't even changed anything yet. This wasn't a simple typo or a missing semicolon. It was a deep, scary-looking error from the internals of the compiler.

The Real Culprit: A Generational Gap

This error was a tell-tale sign that my Java version (JDK) is too new for your sbt version.

When I hit this problem, the log showed I was running sbt 1.8.2 with Java 21. Here’s what’s happening under the hood: The sbt build tool is itself a Scala program, and older versions of it use an older Scala compiler (version 2.12). This older compiler simply doesn’t know how to handle the modern bytecode produced by a very new JDK like Java 21.

Think of it like trying to play a vintage video game on a 4K TV. The TV is too advanced and doesn't understand the old signal format, so the screen just glitches out.

The Solution: SDKMAN! to the Rescue

I could hunt down old Java versions and manually manage my JAVA_HOME path for every project, but who has time for that? The best tool for this job was SDKMAN, a command-line tool for managing parallel versions of multiple Software Development Kits.

With SDKMAN!, you can install multiple Java versions and switch between them with a single command.

Here’s how I solved the "bad constant pool index" error in under five minutes:

Step 1: Install SDKMAN

If you don't have it, just follow the simple instructions on the official SDKMAN website. It's usually a single curl command.

Step 2: Find and Install a Compatible JDK

The sbt 1.x series works beautifully with JDK 11 or JDK 17. I opted for 17. You can list all available versions and then install your choice.

# List available Java versions
sdk list java

# Install a compatible version (I chose Temurin)
sdk install java 17.0.12-tem

Step 3: Switch Your Java Version

This is the magic step. For your current terminal session, just tell SDKMAN which Java version to use.

# Switch to the newly installed JDK
sdk use java 17.0.12-tem

SDKMAN will reconfigure your shell's PATH on the fly. You can verify the change with java -version.

Step 4: Run sbt Again

Now, back in your project folder, run the command that started all this trouble.

sbt test

Success! The cryptic errors vanish. The build kicks off, dependencies download, tests run, and you see the beautiful green text of a successful test suite.

Running Your Expo App on a Real Device for Testing

Youngho Andrew Chaa — Sat, 06 Sep 2025 22:02:37 +0000

As an Expo developer, you’ve likely spent countless hours running your app on a simulator or emulator. While that's great for most development, nothing beats the feel of your app on a real device. It's the only way to truly test performance, native features, and user experience.

There are two primary paths for running your Expo app on a physical device: using the Expo Go app or building a custom development client. The right choice depends on your project's dependencies.

Method 1: The Fast Path with Expo Go

If your project relies only on the standard APIs that come with the Expo SDK—meaning no custom native modules or libraries outside of what Expo Go supports—this is your go-to method. It’s quick and requires minimal setup.

Prerequisites

Your Device: An iPhone or Android phone.
Expo Go: Make sure the free Expo Go app is installed on your device from the App Store or Google Play Store.
Shared Network: This is the most critical step. Your computer and your mobile device must be connected to the exact same Wi-Fi network. Issues with corporate or university networks that have client isolation are the most common cause of connection failures.

Step-by-Step Guide

Start Your Development Server: In your project's root directory, open your terminal and run the command npx expo start.
Scan the QR Code: The terminal will display a QR code. Use your device to scan it.
- On iOS: Open the built-in Camera app and point it at the QR code. A notification will pop up. Tap it to open your app in Expo Go.
- On Android: Open the Expo Go app and tap the "Scan QR Code" button to use the in-app scanner.
Run the App: Expo Go will connect to your computer, download the JavaScript bundle, and run your app. Now, whenever you save changes in your code, they'll instantly update on your device.

Accessing the Developer Menu

Once the app is running, you can access the developer menu for debugging tools by shaking your device firmly. This menu allows you to open a remote debugger, reload the app, or view a performance monitor.

Method 2: The Professional Path with a Development Build

Once you add custom native modules—such as a specific video player, a payment library, or Firebase—the Expo Go app won't work anymore. Its core is immutable and doesn't contain the native code your new library needs. The solution is to create a custom development build, which is a personalized version of Expo Go with your project's unique native code compiled in.

Prerequisites

Expo Account: You'll need a free Expo account.
EAS CLI: Install the Expo Application Services (EAS) command-line interface by running npm install -g eas-cli and then log in with eas login.

Step-by-Step Guide

Add the Development Client Library: This library allows your custom-built app to communicate with your development server. Run npx expo install expo-dev-client.
Configure the Build Profile: Make sure your eas.json file is configured with a development build profile that has "developmentClient": true. If you don't have this file, you can generate it by running eas build:configure.
Create the Development Build: This is the process of building the native .apk (Android) or .ipa (iOS) file. It's done on Expo's cloud servers and can take 10-20 minutes.
- For Android: eas build --profile development --platform android
- For iOS: eas build --profile development --platform ios (Note: This requires a paid Apple Developer account.)
Install the Build: Once the build is complete, EAS will provide a link and a QR code. Scan the QR code to download and install the app on your device. On iOS, you'll likely use TestFlight for this.
Run Your App:
- Start your server with npx expo start --dev-client.
- Open the new custom app you just installed on your phone. It will automatically connect to your development server.
- Just like with Expo Go, you can shake your device to access the developer menu and debugging tools.

When to Rebuild Your App

With a development build, you don't need to rebuild every time you make a change. Your JavaScript code is loaded dynamically from the development server, providing a fast and iterative experience.

You must create a new build only when you change the native code. This includes:

Adding or removing a native module npm package.
Upgrading or downgrading an existing native module.
Changing native configuration files like app.json, Info.plist, or AndroidManifest.xml (e.g., updating your app icon or package name).
Upgrading your project to a new Expo SDK version.

For everything else—like editing your JavaScript/TypeScript files or changing CSS styles—the Metro server will instantly push the updates to your running app.

This two-path approach makes Expo incredibly flexible, allowing you to choose the right workflow for your project at every stage of development.

Fix for the Android Black Screen Bug on Camera Re-entry

Youngho Andrew Chaa — Thu, 04 Sep 2025 12:19:14 +0000

Have you ever opened a barcode scanner in an app, clicked the back button, and returned to find a black screen where the camera view should be? This is a frustratingly common bug on Android devices, and it's a symptom of a deeper issue with how mobile apps handle camera resources.

Understanding the Root Cause: The "Stale Camera" Problem

The black screen bug on Android is essentially a timing issue. Here's a breakdown of what's happening behind the scenes:

1. Android's Aggressive Resource Management

Unlike iOS, Android is designed to aggressively manage resources to optimize memory. When a user navigates away from the camera screen (e.g., by pressing the "back" button), the Android operating system may suspend or even release the camera hardware session to free up resources.

When the user returns to the app, the camera component in memory is "stale." It still exists, but the underlying native camera session has been terminated. This leaves a black, empty view.

2. The Race Condition

The core of the problem lies in a race condition between the app's component lifecycle and the native camera's hardware initialization.

User Navigates Back: The screen comes back into focus.
React Component Renders: The React Native CameraView component re-renders almost instantly.
Black Screen Appears: The native camera hardware is still in the process of starting up.
Premature Scanning: In a flawed implementation, the app might try to enable barcode scanning even though the camera isn't ready, leading to crashes or no results.

The result is a frustrating black screen for the user until the camera finally reinitializes, if it ever does.

The Solution: State-Driven Camera Management

My solution tackles this issue head-on by creating a system that ensures the app never tries to scan until the camera is 100% ready. Here's a look at the key changes we implemented:

1. Camera State Management

I introduced a new state variable, isCameraReady, to track the camera's status. Barcode scanning is only enabled when this variable is true.

<CameraView
  key={cameraKey}
  style={StyleSheet.absoluteFillObject}
  // onBarcodeScanned is only active when isCameraReady is true
  onBarcodeScanned={isCameraReady ? handleBarCodeScanned : undefined}
  onCameraReady={handleCameraReady}
  barcodeScannerSettings={{
    barcodeTypes: ['code128', 'code39', 'ean13', 'ean8', 'upc_a', 'upc_e'],
  }}
  autofocus='on'
>
  <ScanningOverlay />
</CameraView>

By assigning undefined to onBarcodeScanned when the camera isn't ready, we completely disable the native barcode detection process. This is more efficient than constantly scanning and checking the state inside the callback function. It saves battery and CPU resources.

2. Enhanced Focus Effect and Forced Remount

We use useFocusEffect to handle navigation events. When the screen comes into focus, we first reset the camera state:

useFocusEffect(
  React.useCallback(() => {
    // Reset camera state when screen comes into focus
    setIsCameraReady(false)
    setCameraKey((prevKey) => prevKey + 1) // Force camera remount
    setIsScanning(true) // Reset scanning state

    // Small delay to ensure camera is properly reinitialized on Android
    const timer = setTimeout(() => {
      setIsCameraReady(true)
    }, 100)

    return () => {
      clearTimeout(timer)
      setIsCameraReady(false)
    }
  }, [])
)

Incrementing the cameraKey forces the CameraView component to completely remount. This ensures it's not a "stale" component but a fresh instance ready for initialization.

3. Explicit Camera Ready Handler

The critical part of the fix is waiting for the native camera to tell us it's ready.

// Handle camera ready state (important for Android)
const handleCameraReady = () => {
  setIsCameraReady(true)
}

<CameraView
  key={cameraKey}
  onCameraReady={handleCameraReady} // This callback is key
/>

The onCameraReady callback is a native-level event that fires only after the camera hardware has fully initialized and the preview stream has started. When this callback fires, we can confidently set isCameraReady to true, and the app can begin scanning.

We also added a small delay as a fallback, which helps in a small number of edge cases where the onCameraReady event might be delayed.

Summary of Benefits

This comprehensive solution completely resolves the Android black screen issue by:

Fixing Black Screens: By forcing a proper camera re-initialization on navigation.
Preventing Premature Scanning: Barcode detection is only enabled when the camera is confirmed ready.
Improving Performance: The app avoids unnecessary CPU usage by disabling scanning when the camera isn't active.
Preventing Memory Leaks: We've included a cleanup function that resets state and clears timers when the user leaves the screen.

The black screen bug is a classic example of the challenges of working with native hardware in a cross-platform environment. Our fix provides a robust and reliable solution, ensuring a smooth and frustration-free experience for users on both Android and iOS.

Know your videos: Inspecting & Transcoding with FFmpeg and Homebrew

Youngho Andrew Chaa — Tue, 12 Aug 2025 11:52:12 +0000

Have you ever wondered what's inside a video file? Or maybe you need to convert a video to a different format for better compatibility or smaller file size. The command line tools FFmpeg and ffprobe are your secret weapons for these tasks. We'll walk you through how to install them and then use them to inspect and transcode a video.

Installing FFmpeg with Homebrew

If you're on a macOS, the easiest way to install FFmpeg is with Homebrew, the "missing package manager for macOS."

If you don't have Homebrew installed, you'll need to do that first by following the instructions on their official website. Once Homebrew is ready, installing FFmpeg is a breeze. Open your Terminal and type this command:

brew install ffmpeg

After the installation is complete, it's a good idea to confirm that everything is working. You can do this by checking the version of FFmpeg that was installed:

ffmpeg -version

If you see the version information displayed, you're all set!

Checking a Video's Codec with ffprobe

Before you change a video, it's smart to know what you're starting with. That's where ffprobe comes in. It's a powerful command-line tool that's part of the FFmpeg suite. It can gather a ton of information about your multimedia files, including the video and audio codecs.

To find out the codec of a video file named input.mp4, simply run this command in your Terminal:

ffprobe ./input.mp4

This will print out a detailed report about the video, including its codec, resolution, framerate, and more. Look for the "Stream #0:0(eng): Video:" section to find the codec information.

Transcoding to H.264 with ffmpeg

Now for the fun part: changing the video! Transcoding is the process of converting a video file from one codec or format to another. The H.264 codec is a widely used and highly compatible standard, making it a great choice for web videos, mobile devices, and more.

The ffmpeg command has many options, and it can seem a bit intimidating at first. Let's break down a common command used to transcode a video to H.264:

ffmpeg -i input.mp4 -c:v libx264 -crf 23 -preset medium -c:a aac -b:a 128k output.mp4

Let's dissect this command:

-i input.mp4: This is the input file. It tells ffmpeg which video to process.
-c:v libx264: This is the video codec. We're telling ffmpeg to use the libx264 encoder, which is the H.264 implementation.
-crf 23: This stands for Constant Rate Factor. It's a quality setting for the video. A lower number means higher quality and a larger file size, while a higher number means lower quality and a smaller file size. A value of 23 is a good starting point for a balanced output.
-preset medium: This option controls the encoding speed. Faster presets (like ultrafast) result in lower quality and larger file sizes, while slower presets (like veryslow) result in higher quality and smaller file sizes. medium is a great balance.
-c:a aac: This is the audio codec. We're specifying the AAC (Advanced Audio Coding) codec, which is a common and efficient choice.
-b:a 128k: This sets the audio bitrate to 128k, which is a good quality for most purposes.
output.mp4: This is the output file name. ffmpeg will save the transcoded video to this file.

After running this command, you'll have a new video file named output.mp4 that's ready for its new purpose!

Classic TDD: Less Mocking, More Confidence

Youngho Andrew Chaa — Wed, 16 Apr 2025 17:33:01 +0000

Introduction

Test-Driven Development (TDD) is a powerful practice, but it's easy to get bogged down in excessive mocking, leading to brittle tests and a false sense of security. This post explores the principles of "Classic TDD" and how focusing on testing behavior over implementation details can lead to more robust and maintainable code.

The Problem with Over-Mocking

Many developers, even experienced ones, find testing a chore. Why? Often, it's because we end up writing more test code than implementation code. A single change in the implementation can break a large number of tests, not because the behavior changed, but because the tests were tightly coupled to internal details. This leads to frustration and a reluctance to refactor.

The Classic TDD Approach

Classic TDD, as explained by Kent Beck in "TDD By Example", offers a different approach. The core idea is that internal code changes should not break existing tests. To achieve this, we need to shift our focus:

Test Behavior, Not Implementation: Instead of focusing on testing individual methods or classes, we should test the public interface of our system. This could be an API endpoint, a queue message, or any other point of interaction with the system.
The Trigger for Tests: Don't write a test just because you added a new method. The trigger for writing a test should be the need to implement a requirement.
Mock Sparingly: Mocks should be used as little as possible. Ideally, only mock 3rd-party libraries or external dependencies. Avoid mocking your own implementation code.

Benefits of Classic TDD

By following these principles, we can reap significant benefits:

Increased Test Coverage: Testing behavior naturally leads to better coverage of the system's functionality.
Realistic Tests: Tests become more realistic and reflect how the system is actually used.
Easier Refactoring: Because tests are focused on behavior, you can refactor your code with confidence, knowing that you won't break tests unless you change the external behavior.
Less Brittle Tests: Fewer mocks mean less brittle tests that are less likely to break with implementation changes.
Less Code: By avoiding excessive mocking, you write less test code overall.
More Confidence: Ultimately, Classic TDD leads to greater confidence in your code and your ability to maintain and evolve it.

The System Under Test (SUT)

It's crucial to remember that the System Under Test (SUT) is not a single class, but the public interface of a system. Tests should be written against this stable contract, focusing on the output of the implementation code.

Conclusion

Classic TDD, with its emphasis on testing behavior and minimizing mocks, offers a path to more robust, maintainable, and reliable software. By embracing these principles, we can move away from brittle tests and towards a development process that fosters confidence and enables continuous improvement.

Set up a new Mac

Youngho Andrew Chaa — Mon, 14 Apr 2025 17:36:49 +0000

I bought a new MacBook, as I thought the new US tariff would increase the price of Apple products. This is how I set up a new Mac.

Install oh my zsh: sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
Install brew

Then, I install my favourite apps

Chrome: brew install --cask google-chrome
Rectangle: brew install --cask rectangle
Warp: brew install --cask warp
Whats App Desktop: brew install --cask whatsapp
Obsidian: brew install --cask obsidian
Slack: brew install --cask slack
Perplexity AI

Adjust the settings

Delete all the apps from the Dock: defaults delete com.apple.dock persistent-apps && killall Dock
Turn on Auto hiding on Dock.
Enable 3 finger drag and drop.
Add a new language of your choice. Mine is Korean. Go to Settings > Keyboard > Text Input > Edit.
Use F1, F2 as function key: Settings > Keyboard > Keyboard Shortcuts > Function Keys
Enable Keyboard navigation: Settings > Keyboard > Keyboard navigation
Display application switcher on every screen in multi-monitor setup: defaults write com.apple.dock appswitcher-all-displays -bool true Then killall Dock

Install developer tools

XCode
Github CLI: brew install gh
VS Code: brew install --cask visual-studio-code
MongoDB Compass: brew install --cask mongodb-compass
Postman: brew install --cask postman
Docker
Minikube: brew install minikube
Terraform: brew install terraform
AWS CLI
Fork: brew install --cask fork

Set up Git

ssh-keygen -t ed25519 -C "your_email@example.com": to generate a new SSH key
eval "$(ssh-agent -s)": to start the ssh agent
Update ~/.ssh/config to automatically load keys into the ssh-agent

Host github.com
  AddKeysToAgent yes
  UseKeychain yes
  IdentityFile ~/.ssh/id_ed25519

ssh-add --apple-use-keychain ~/.ssh/id_ed25519: add the SSH private key to the ssh-agent
pbcopy < ~/.ssh/id_ed25519.pub: copy the public key
Go to Github settings and add the key
git config --global user.name "Your Name": set your username
git config --global user.email "your.email@example.com": set your email
git config pull.rebase false: set the pull action to be merge

Set up VS Code

Shell Command: Install 'code' command in PATH

Set up Node.js

curl https://get.volta.sh | bash: to install Volta:
volta install node: to install the latest LTS node
npm i -g eas-cli: to install eas

Set up Ruby

brew install ruby@3.2
Add export PATH="/opt/homebrew/opt/ruby@3.2/bin:$PATH" to ~/.zshrc
source ~/.zshrc: to reload the settings

Set up XCode

Check if XCode is in the correct path: xcode-select -p
If it's in a different place like this, /Library/Developer/CommandLineTools, reset the path: sudo xcode-select -s /Applications/Xcode.app/Contents/Developer. This is necessary to build a React Native project successfully locally

Set up Python

brew install pyenv: to install python version manager
Update ~/.zshrc

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
echo '[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(pyenv init - zsh)"' >> ~/.zshrc

source ~/.zshrc

pyenv install 3.12: to install python 3.12
pyenv global 3.12: to set the global version
Install Python extension from Microsoft in VS Code
Install Jupyter extension in VS Code

Set up AWS

aws configure to set region, aws_access_key_id, and aws_secret_access_key

Python `try` Blocks: A Breath of Fresh Air for Variable Scope (Unlike Java & C#!)

Youngho Andrew Chaa — Mon, 10 Mar 2025 13:57:06 +0000

Python's try...except error handling is fantastic for writing robust code. But beyond just catching errors, Python offers a subtle yet incredibly convenient feature related to variable scope within try blocks. If you're coming from languages like Java or C#, you might be pleasantly surprised – and relieved!

Have you ever defined a variable inside a try block, hoping to use it later, only to find it's inaccessible outside in other languages? Python elegantly sidesteps this frustration. In Python, variables defined within a try block are readily available outside of it, as long as the try block completes successfully without raising an exception. This is a real time-saver and makes your code cleaner and more intuitive.

Python's Convenient Scoping: No Need for Extra Boilerplate!

Python's variable scoping rules are inherently flexible. Variables declared in blocks like if, for, while, and yes, try and except, are generally accessible in the surrounding scope. This "forgiving" approach means you don't have to jump through hoops to use variables created within a try block – they are simply there for you if the try block runs smoothly.

This is in stark contrast to languages like Java and C#. In those languages, variables declared inside a try block are typically scoped only to that block. If you want to use them outside, you often need to declare the variable before the try block and then potentially assign it within, just to make it accessible later. This adds extra lines of code and can feel unnecessarily verbose.

Example: Python's Simplicity in Action

Let's see how Python's approach shines in practice:

try:
    value = "Hello, World!"  # Variable defined inside try
except Exception as e:
    print(f"An error occurred: {e}")

#  `value` is immediately accessible here! How convenient!
print(value)  # Output: Hello, World!

See how clean and direct that is? We define value inside the try block, and if no error occurs, we can use value directly afterwards without any extra steps. Python just "gets it" and makes it easy.

Imagine the Java/C# equivalent (simplified):

```java // Or similar C#
String value; // Need to declare it outside
try {
value = "Hello, World!"; // Assign inside try
} catch (Exception e) {
System.out.println("An error occurred: " + e);
}

// Now value is accessible because we declared it outside!
System.out.println(value);




Notice the extra line of code in Java/C# to declare `value` *before* the `try` block?  Python eliminates this extra step, making your code more concise and readable.

### What About Exceptions?  Python Still Handles It Gracefully

Of course, error handling is still crucial.  If an exception *does* occur within the `try` block, Python jumps to the `except` block as expected.  And just like before, if an exception is raised *before* a variable is defined within the `try` block, that variable won't be available later.

However, even in error scenarios, Python's approach can be more convenient.  You can still initialize variables *before* the `try` block if you need them to have a default value in case of errors, but you're not *forced* to do so in every successful case just to make the variable accessible outside.

**Example highlighting error handling and convenience:**



```python
result = None # Initialize outside if needed for error cases, but not always necessary!
try:
    # Potentially error-prone operation
    result = calculate_important_value()
except ValueError as ve:
    print(f"ValueError during calculation: {ve}")
    result = "Default Value" # Or handle error differently

# `result` is always accessible here - either calculated value or default
print(f"The result is: {result}")

Here, we initialize result to None as a safety net. But even if calculate_important_value() succeeds, we didn't need to declare result outside the try block just to use it later. Python's flexibility gives you options without adding unnecessary boilerplate.

Summary: Python's `try` Block Scoping is Just… Easier!

Python is Convenient: Variables defined within a try block are readily accessible outside if the try block completes successfully. This saves you from extra steps and makes your code cleaner.
Contrast with Java/C#: Languages like Java and C# often require you to declare variables outside the try block to use them later, adding extra lines of code. Python avoids this.
Still Handles Errors Well: Python's try...except blocks still provide robust error handling. You can initialize variables outside if you need default values in error scenarios, but you're not forced to for successful paths.

Python's approach to variable scope in try blocks is a testament to its design philosophy: making things easier and more intuitive for the developer. It's a small detail, but it contributes significantly to the overall pleasantness and efficiency of writing Python code, especially when compared to more restrictive languages. Embrace this convenience and enjoy writing cleaner, more Pythonic code!

Solving the Jest Native Module Error in a React Native Expo Project

Youngho Andrew Chaa — Sun, 28 Jul 2024 11:28:57 +0000

While working on a React Native Expo project, I encountered a perplexing error message during my Jest tests. The error halted my progress and left me searching for a solution. Here's a detailed account of the error and how I resolved it

The error message I encountered was as follows:

Cannot find native module 'ExpoClipboard'

   7 |   Box,
   8 | } from 'native-base'
>  9 | import * as Clipboard from 'expo-clipboard'
     | ^
  10 |
  11 | import { Job, JobStatus, Service, Photo } from '../../types/JobModel'
  12 | import { NativeStackNavigationProp } from '@react-navigation/native-stack'

  at Object.requireNativeModule (node_modules/expo-modules-core/src/requireNativeModule.ts:17:11)
  at requireNativeModule (node_modules/jest-expo/src/preset/setup.js:257:73)
  at Object.<anonymous> (node_modules/expo-clipboard/src/ExpoClipboard.ts:3:35)
  at Object.<anonymous> (node_modules/expo-clipboard/src/Clipboard.ts:11:1)
  at Object.<anonymous> (src/screens/JobEdit/index.tsx:9:1)
  at Object.<anonymous> (src/screens/JobEdit/index.test.tsx:4:1)

The error message clearly indicated that Jest was unable to find the native module ExpoClipboard. This issue arises because Jest tests run in a Node environment, which does not support native modules used in a React Native application.

After some research, I found a solution to mock the expo-clipboard module in the Jest environment. By mocking the module, I could bypass the need for the native environment and ensure my tests ran smoothly. Here’s the code that solved the issue:

jest.mock('expo-clipboard', () => ({
  getStringAsync: jest.fn(),
  setString: jest.fn(),
}));

This snippet creates a mock version of the expo-clipboard module, providing mock functions for getStringAsync and setString. This mock setup ensures that any calls to these functions within the tests will use the mocked versions instead of attempting to access the real native module.

To apply this solution, you need to add the mocking code to your Jest setup file. If you don't have a Jest setup file, you can create one, typically named jestSetupFile.js. Here's how to include the mock:

Create or open your Jest setup file (e.g., jestSetupFile.js).
Add the mock code to the file:

   jest.mock('expo-clipboard', () => ({
     getStringAsync: jest.fn(),
     setString: jest.fn(),
   }));

Ensure that your Jest configuration is set to use this setup file. You can do this in your jest.config.js or package.json under the Jest configuration section:

   "jest": {
     "setupFiles": ["<rootDir>/jestSetupFile.js"]
   }

With the mock in place, you can now run your tests again. The error should be resolved, and Jest will use the mocked functions instead of trying to access the native ExpoClipboard module.

Encountering issues like the native module error in Jest can be frustratin. By mocking the expo-clipboard module, I was able to run my tests without errors, allowing me to continue developing my React Native Expo project smoothly.

If you encounter similar issues with other native modules, consider mocking them in the Jest environment using the same approach. Happy testing!

Forem: Youngho Andrew Chaa

The Secret Language of Data: Vectors and Cosine Similarity

Part 1: What is a Vector? (Your Data's Direction and Distance)

Vectors in Real Life:

Part 2: The Cosine Function (The Shadow Ruler)

Part 3: Putting It Together – Cosine Similarity for Documents!

📝 Document Vectors Explained:

The "Shadow" of Document Similarity:

What does 0.5 mean?

The Power of Cosine Similarity

Decoding the 'Invalid Color Space' Error: A Dive into an FFmpeg 7 Bug

The Unexpected Error

The Root Cause: A Bug in FFmpeg 7

The Fix: Explicitly Setting Metadata

Making the Fix Robust: A Codec-Specific Solution

Putting It All Together in Code

Key Takeaway

The Pitfall of ORDER BY Timestamp (And How to Fix It)

The Problem: The Tie

The Solution: Creating a Deterministic Sort

Method 1: The Tie-Breaker

Method 2: The Simpler, Faster Way

Takeaway

Securely Send SMS with Twilio, AWS Lambda, and Terraform

The AWS Lambda Function

Takeaways:

The Infrastructure: Defining the Lambda with Terraform

Takeaways:

The Automation: CI/CD with GitHub Actions

Takeaways:

The Complete Secure Flow

That Cryptic sbt Error? My JDK Was Too New

The Real Culprit: A Generational Gap

The Solution: SDKMAN! to the Rescue

Running Your Expo App on a Real Device for Testing

Method 1: The Fast Path with Expo Go

Prerequisites

Step-by-Step Guide

Accessing the Developer Menu

Method 2: The Professional Path with a Development Build

Prerequisites

Step-by-Step Guide

When to Rebuild Your App

Fix for the Android Black Screen Bug on Camera Re-entry

Understanding the Root Cause: The "Stale Camera" Problem

1. Android's Aggressive Resource Management

2. The Race Condition

The Solution: State-Driven Camera Management

1. Camera State Management

2. Enhanced Focus Effect and Forced Remount

3. Explicit Camera Ready Handler

Summary of Benefits

Know your videos: Inspecting & Transcoding with FFmpeg and Homebrew

Installing FFmpeg with Homebrew

Checking a Video's Codec with ffprobe

Transcoding to H.264 with ffmpeg

Classic TDD: Less Mocking, More Confidence

Set up a new Mac

Python `try` Blocks: A Breath of Fresh Air for Variable Scope (Unlike Java & C#!)

Python's Convenient Scoping: No Need for Extra Boilerplate!

Example: Python's Simplicity in Action

Summary: Python's try Block Scoping is Just… Easier!

Solving the Jest Native Module Error in a React Native Expo Project

Summary: Python's `try` Block Scoping is Just… Easier!