Forem: Amir Hossein Shekari

Why I Ditched SQS for Step Functions (and Ended Up Contributing to SST)

Amir Hossein Shekari — Mon, 11 May 2026 15:31:26 +0000

Every developer knows the "happy path"—that glorious moment when your code works perfectly on a small sample size. But when the "happy path" meets the reality of 100,000+ emails and strict API rate limits, things tend to break.

This is the story of how a bottleneck in my AI startup, Glim, led me to build and contribute the Step Functions component to the SST (Serverless Stack) ecosystem.

The Problem: The Gmail "Wall"

I was building Glim, an AI assistant designed to help users manage their chaos-ridden inboxes. We offered real-time filtering and a powerful bulk-cleanup tool that could archive or delete thousands of emails based on specific criteria like sender, date, or domain.

To make this work, I had to implement a full inbox sync. According to the Google API documentation, this is a two-step process:

Fetch a list of all email IDs.
Iterate through that list and perform a messages.get request for each individual email's full data.

The Math of the 429

Google’s rate limits are notoriously strict and follow a specific quota system. The limit consists of two primary factors:

Per-second Unit Limit: You are allowed 100 quota units per second.
Request Weight: Each type of request has a "weight." A messages.get request costs 20 units.

This means you can only fetch 5 emails per second (or roughly 300 per minute). If you violate this, Google hits you with a 429 Too Many Requests error and an exponential block that gets longer every time you retry too early.

Attempt One: A SQS Disaster

My first architecture was a classic serverless pattern: Lambda + SQS.

I bundled 20 email IDs into each SQS message. To respect the rate limits, I throttled the message publishing to one message per second. I set a 60-second visibility timeout, figuring that if a batch hit a 429, SQS would simply retry it a minute later based on standard AWS rules.

It worked... until it didn't.

For users with fewer than 40,000 emails, it was fine. But for power users with massive inboxes, the math fell apart. Multiple SQS batches would hit the rate limit simultaneously. The 60-second window wasn't long enough to clear the Google block, leading to more retries, more 429s, and a cascading failure that increased the block time exponentially. I realized I didn't just need a queue; I needed a State Machine.

The Solution: The "Scatter-Gather" Step Function

I needed a way to strictly control the flow of execution. If "Batch 1" hit a rate limit, the entire process needed to pause until that block cleared before "Batch 2" even started.

I decided to move to a Scatter-Gather pattern using AWS Step Functions. This allowed me to:

Fetch IDs: Fetch 500 IDs at a time (even fetching the IDs can hit limits once you pass 40k emails).
Strict Flow Control: Process the batch, respect the rate limit, then move to the next 500.
Software-Level Backoff: Implement an exponential wait step within the flow so that retries don't stack on top of each other at the API gateway level.

The Hurdle: SST Didn't Speak Step Functions

At the time, Glim was built entirely on SST (Serverless Stack). I loved the developer experience of SST, but there was a major missing piece: it didn't have a built-in component for Step Functions.

I had two choices: manually write raw Pulumi code (which would feel out of place in my clean SST codebase) or build a native SST component myself.

I chose the latter. I designed the component using a linked-node architecture. I visualized the state machine as a circular tree where each node connects the steps of the SF. I wrote the logic to compile this tree into the exact JSON format required for Pulumi (the backend of SST) and pushed it to AWS.

Giving Back to the Community

Once I had a working version, I didn't want to keep it in a silo. I forked the SST repository, integrated my new StepFunctions component, and used it to successfully fix the Glim inbox sync. It handled 100,000+ emails without breaking a sweat.

Seeing it work in production, I opened a Pull Request.

After several rounds of refinement, the code was merged. Today, that component, born out of a desperate need to sync 100,000 emails without hitting a Google rate limit, is available for everyone in the SST ecosystem to use.

Lessons Learned

Queues aren't Orchestrators: SQS is great for decoupling, but when you need to "stop the world" to wait for a rate limit, Step Functions are superior.
Understand the Quota: If I hadn't dug into the 20-unit weight of a messages.get request, I would have kept guessing why my Lambda was failing.
Scratch your own itch: The best open-source contributions come from solving real-world production problems.

Now, whether you're syncing a massive inbox or orchestrating a complex AI pipeline, you can do it natively within SST. Happy coding!

Boosting VS Code Productivity - Custom Labels for React Component Files

Amir Hossein Shekari — Fri, 05 Jul 2024 11:33:48 +0000

Ever felt like you're drowning in a sea of index.ts files? Trust me, I've been there. Working on a massive React project, I found myself constantly lost in a maze of identical-looking files. But then I stumbled upon a game-changer: VS Code's custom labels feature. Let me tell you how it transformed my coding life.

The Problem with `index.ts` Overload

Picture this: You're deep in the zone, juggling multiple components, when suddenly you need to find that one specific index.ts file. You open the search bar, and bam! A flood of indistinguishable results. Frustrating, right? That's exactly where I was until I discovered the magic of custom labels.

Enter Custom Labels

So, what's the deal with these custom labels? In a nutshell, they're VS Code's way of letting you personalize how file names appear in your editor tabs and search bar. For us React devs drowning in index.ts files, it's like throwing us a lifeline.

Setting Up Custom Labels Step by Step

1. Enable Custom Labels

First, you'll need to enable custom labels in your VS Code settings. Just add this line to your settings.json:

"workbench.editor.customLabels.enabled": true

2. Define Patterns for `index.ts` Files

Now, here's where the real magic happens. Add this bit to tell VS Code to replace those generic index.ts names with their parent directory names:

"workbench.editor.customLabels.patterns": {
  "**/index.{ts,js,tsx,jsx}": "${dirname}"
}

3. Customize for Specific Directories

If you're like me and have a specific folder where all your components live, you can target just that directory:

"workbench.editor.customLabels.patterns": {
  "components/*/index.{ts,js,tsx,jsx}": "${dirname}"
}

The Benefits of Custom Labels

Increased Readability: Instead of a sea of index.tsx tabs, you'll see "Button", "Header", and so on. It's like your editor suddenly learned to speak your language!
Efficient Searching: Searching for specific components became a breeze.
Reduced Mental Load: No more second-guessing which index.ts I was looking at.

So, fellow coders, if you're tired of playing "Where's Waldo?" with your index.ts files, give custom labels a shot. It's a small change that packs a big punch in your daily coding life. Trust me, your future self will thank you!

Building Safe and Dynamic URLs with TypeScript

Amir Hossein Shekari — Fri, 13 Oct 2023 19:58:13 +0000

When developing applications, one of the common tasks is constructing URLs, especially when they contain dynamic parts, like IDs. Ensuring the correctness of these URLs can be tedious, leading to potential runtime errors if not done right. This is where TypeScript can lend a hand.

The Need for urlConverter

Using string-based routes can be error-prone. Miss a single character or forget to replace a dynamic segment, and the whole URL can break. By using TypeScript, we can create a utility that helps in generating these URLs while ensuring their correctness.

Building the urlConverter Step by Step

1. Organizing URLs

One of the first steps is to have all URLs organized in one place. Enums in TypeScript provide an excellent way to do this:

export enum UserUrls {
  getUserUrl = "/users/[userId]",
  getMyUserURL = "/users/me",
  // other URLs can go here...
}

2. Extracting Dynamic Parameters

For URLs with dynamic parts, we need a way to recognize and replace them. A custom TypeScript type can help identify these parts:


type IsParameter<Part> = Part extends `[${infer ParamName}]`
  ? ParamName
  : never;
type FilteredParts<Path> = Path extends `${infer PartA}/${infer PartB}`
  ? IsParameter<PartA> | FilteredParts<PartB>
  : IsParameter<Path>;
/**
 * A very special type that parse a template string to it's object
 * source: https://lihautan.com/extract-parameters-type-from-string-literal-types-with-typescript/
 * type ParamObject = Params<'/purchase/[shopId]/[itemId]'>;
 * type ParamObject = {
 *      shopId: string;
 *      itemId: string;
 *    }
 */
export type Params<Path> = {
  [Key in FilteredParts<Path>]: string | number;
};

This type works to find any segment enclosed within [].

3. Confirming the Presence of Parameters

To ensure that if a URL requires parameters, they're provided, we can use another type:

type HasParams<Path> = Path extends `${string}[${string}]${string}`
  ? true
  : false;

This will check if a route needs parameters.

4. Creating the urlConverter Function

With all the above in place, we can now create our utility:

type AllEnums = UserUrls;

export const urlConverter = <Route extends AllEnums>(
  routeKey: Route,
  ...paramsArr: HasParams<typeof routeKey> extends true ? [Params<Route>] : []
) => {
  let route = routeKey as string;

  if (!route) {
    throw Error("Invalid route key");
  }

  const params = paramsArr[0];
  if (params) {
    Object.keys(params).forEach((paramKey) => {
      const value = params[paramKey]?.toString();
      route = route.replace(`[${paramKey}]`, value);
    });
  }
  return route;
};

Usage Example

Generating a reset URL with a given token can be done easily:


const getUserUrl = urlConverter(UserUrls.getUserUrl, { userId: 12 });
console.log(getUserUrl); // Outputs: `/users/12`

Conclusion

The urlConverter utility ensures that constructing dynamic URLs is straightforward and less error-prone. Leveraging TypeScript allows for catching potential mistakes at compile-time rather than runtime, resulting in more robust applications.

I hope this approach helps in your development journey, ensuring cleaner and safer URL generation with TypeScript.

Protect Production Database from Wrong Migrations

Amir Hossein Shekari — Tue, 25 Apr 2023 15:03:58 +0000

As a .NET developer, I had a close call when I applied a migration to my production database without changing the database connection string. I was mortified when I realized my mistake, but I quickly got to work finding a solution to prevent it from happening again.

That's when I came up with the idea to add a console message that displays the migration details before applying it. This simple addition gives me the chance to review the migration details and ensure that I am working with the correct database before making any changes.

With this new safeguard in place, I can breathe easy knowing that my production database is safe from accidental alterations. It just goes to show that sometimes the simplest solutions can be the most effective.

public class ApplicationDbContextFactory : IDesignTimeDbContextFactory<ApplicationDbContext>
{
    public ApplicationDbContext CreateDbContext(string[] args)
    {
        // ... Some code to generate dbContextBuilder
        var context = new ApplicationDbContext(dbContextBuilder.Options);

        // This is where magic happends
        var pendingMigrations = context.Database.GetPendingMigrations();
        Console.WriteLine("*********************************************\n");

        Console.WriteLine("This command is going to apply migrations with following details");

        Console.Write("ConnectionString: ");
        Console.ForegroundColor = ConsoleColor.Yellow;
        Console.WriteLine(connectionString);
        Console.ResetColor();

        Console.Write("Migrations:\n\t");
        Console.ForegroundColor = ConsoleColor.Yellow;
        Console.WriteLine(string.Join("\n\t", value: pendingMigrations.ToArray()));
        Console.ResetColor();
        Console.WriteLine();
        Console.WriteLine("*********************************************");

        Console.WriteLine("Do You confirm? (Y/N)");
        var userInput = (Console.ReadLine());

        if (userInput is "Y" or "y")
            return context;

        Console.ForegroundColor = ConsoleColor.Red;
        Console.WriteLine("Aborted!");
        Environment.Exit(1);
        return null;
    }
}

More info about the ApplicationDbContextFactory

Toggle your VPN connection with a shortcut, in Manjaro

Amir Hossein Shekari — Mon, 31 May 2021 15:55:55 +0000

This is a little script that allows you to toggle your VPN connection by running a bash script.
I mostly use it by assigning it to a shortcut (meta + v).