Forem: Software Jutsu

AWS X-Ray

Software Jutsu — Tue, 03 Mar 2026 09:23:34 +0000

In the modern landscape of cloud application development, the monolithic architecture has largely given way to microservices.

This approach, which involves breaking down applications into a collection of smaller, loosely coupled, and independently deployable services, has revolutionized the industry by offering unparalleled benefits in scalability, fault tolerance, and development speed.However, this architectural shift introduces a new, and critical, challenge: observability.

In a microservices ecosystem, a single user request can traverse a complex and often invisible web of services, databases, and third party APIs. When something goes wrong, be it a sudden spike in latency, an unusual error, or a performance bottleneck it can be incredibly difficult to answer the question,
"Where is the problem coming from?"

Traditional monitoring tools, while adept at monitoring individual service health, fall short in this environment. They provide a siloed view of the world, making it almost impossible to understand how an issue in one service might be impacting another.

This is where AWS X-Ray steps in.

Solution to the microservices observability problem.

It’s a powerful distributed tracing service that allows developers to "see" inside their applications, trace the path of a request as it moves from one service to another, and visualize the entire application architecture in real-time. By providing this end-to-end visibility, X-Ray empowers development teams to move beyond mere monitoring and embrace true observability.

It transforms your application from a collection of "black boxes" into a transparent, understandable, and ultimately, a more reliable system. This article will serve as your comprehensive guide to understanding, deploying, and maximizing the value of AWS X-Ray in your environment.

The Problem of Visibility in a Microservices World

Before we dive into how X-Ray works, it’s important to fully appreciate the problem it solves. To illustrate, imagine a simple e-commerce application. In a monolithic architecture, a user request to "place an order" is a single, self-contained unit of work. If it's slow, you can examine the application logs for that specific request and pinpoint the slow database query or the inefficient loop within your code.

Now, consider the same application in a microservices environment. A user’s "place order" request might touch:

An API Gateway (the entry point)
An Authentication Service (to verify the user)
An Inventory Service (to check stock)
A Payment Processing Service (to handle the transaction)
A Shipping Service (to generate a label)
A Database (used by the Inventory Service)
An External Third-Party API (used by the Payment Service)

When that request takes 10 seconds to complete, where is the bottleneck?

Is the Authentication Service taking too long to connect to its user database?
Did the Payment Service hang while waiting for a response from the external API?
Is the Inventory Service running an inefficient SQL query?
Was the network latency between the Shipping Service and its database unusually high?

Without a way to trace the request, answering these questions is a nightmare. It requires you to log into each individual service, manually correlate the logs based on time (which is notoriously difficult), and "hope" you can piece the puzzle together.
This approach is slow, error-prone, and unsustainable.

X-Ray provides the "missing link"

Understanding the Key Concepts of AWS X-Ray

X-Ray is built on a few core concepts that work together to provide this distributed visibility.

1. Segments
A segment represents a logical unit of work performed by a single service in response to a request. When a request hits your application, the first service it encounters creates a parent segment. This segment records metadata about the service itself (its name, host, etc.) and information about the incoming request (the HTTP method, URL, and client IP).
As that service performs its internal processing, the segment grows. If the service then makes a call to another component—like a database, an S3 bucket, or another microservice, it creates a subsegment within that segment.
Essentially, a segment tells you what an individual service did and how long it took.

2. Subsegments
Subsegments provide a more granular view of the operations within a service. They are created when a service interacts with other downstream resources. They are crucial for breaking down the total time spent in a service.

For example, a subsegment might represent:

A database query: Recording the query type, the table name, and the time the query took to execute.
An HTTP client call: Recording the URL being called and the status code returned.
An AWS SDK call: Recording the specific AWS service and action (e.g., s3.PutObject).
A custom code block: You can even create subsegments for specific, long-running functions within your own code to measure their performance.

Subsegments can be nested within each other, creating a detailed, hierarchical "flame graph" of activity.

3. Traces
A trace is the most important concept in X-Ray. It is the end-to-end "story" of a request. A trace is a collection of all the segments and subsegments from all the different services that participated in fulfilling that single request.
Each trace has a unique Trace ID, which is generated by the first service that receives the request. This Trace ID is then passed down to all subsequent services in an HTTP header (the X-Amzn-Trace-Id header).

This correlation mechanism is what stitches all the segments together into a single, cohesive view.
A trace allows you to follow the request’s complete journey, from the moment it arrived at your application to the final response.

4. Annotations and Metadata
To make your traces even more powerful, you can add custom information to them. X-Ray provides two ways to do this:

Annotations: These are key-value pairs with indexed values. You can use annotations to add business-relevant context to a trace, such as the customer ID, the product ID, or a unique transaction ID. Because they are indexed, you can search and filter your traces in the X-Ray console using these values. (e.g., "Show me all traces for CustomerID=123").
Metadata: These are also key-value pairs, but they are not indexed. They can contain larger, more complex data structures like JSON objects. While you can't search on them, they are invaluable for storing detailed context about a trace that you might want to review when inspecting a specific request. (e.g., a copy of a large request payload).

Annotations are for filtering; Metadata is for context.

How AWS X-Ray Work: The Flow of Data

To understand how X-Ray gets its data, we need to look at the X-Ray architecture. The process is designed to be highly efficient and asynchronous, ensuring that it adds minimal overhead to your application's performance.

The X-Ray SDK
The foundation of the entire system is the X-Ray SDK. This SDK must be included as a library within your application’s code. It's available for all major programming languages, including Java, Python, Node.js, Ruby, and .NET.

The SDK does the heavy lifting of:

Generating the initial Trace ID.Creating the initial segment.
Automatically creating subsegments for supported libraries (like database drivers and AWS SDKs).
Propagating the Trace ID to downstream services in the X-Amzn-Trace-Id header.
Buffering the collected segments and subsegments.

The X-Ray Daemon
The X-Ray SDK doesn't send data directly to the X-Ray API. Instead, it sends the buffered data (via UDP) to a local process called the X-Ray Daemon.

The daemon is designed to run close to your application. It can run as a background process on an EC2 instance, as a sidecar container in an ECS task or Kubernetes pod, or as a component within a Lambda function.

The daemon acts as a data aggregator and forwarder. It collects data from multiple SDK instances, batches it, and securely transmits it to the X-Ray API over HTTPS.
This asynchronous, buffered design ensures that your application's request-processing code is not blocked by network calls to the X-Ray service.

For serverless environments like AWS Lambda, you don't need to manually install the daemon. Instead, you simply "tick a box" in the Lambda configuration, and AWS manages the daemon for you.

Putting it into Action: Instrumenting Your Application

Getting started with X-Ray involves a process called "instrumentation," which is the process of adding the X-Ray SDK to your application.

While this sounds complex, AWS has worked hard to make it as simple and automatic as possible.There are three main levels of instrumentation:
1. Zero-Code Instrumentation (For Containerized/EC2 Workloads)
This is the easiest path. You can use an X-Ray-compatible agent or the AWS Distro for OpenTelemetry (ADOT) to automatically instrument your application.

ADOT is a highly recommended approach. It’s an AWS-supported, production-ready distribution of the OpenTelemetry project. By using the ADOT Collector and language SDKs, you can collect traces from your application with almost zero code changes. You simply configure your container to include the ADOT agent, and it automatically handles the collection and propagation of trace data to X-Ray.

This is a fantastic way to quickly add observability to existing applications without rewriting them.

2. SDK-Based Auto-Instrumentation
For more control, or when auto-instrumentation agents aren't supported, you can use the X-Ray SDK and its language-specific auto-instrumentation libraries.
Most X-Ray SDKs provide integrations for popular frameworks and libraries. For example:
Node.js: You can use aws-xray-sdk-express to automatically trace incoming Express requests.
Python: The aws-xray-sdk can be configured to automatically instrument the boto3 library (the AWS SDK) and common database drivers like SQLAlchemy or psycopg2.This is the most common approach for new applications.

3. Custom Instrumentation (For Manual Tracing)
For granular control, you can use the SDK’s API directly in your code. This allows you to:

Manually create subsegments around critical, custom business logic.
Add annotations and metadata to enrich your traces.
Capture custom exceptions and errors.

This approach is powerful but requires code changes and is typically used to supplement auto-instrumentation.

Beyond Basic Tracing: Advanced Insights

X-Ray isn't just about showing you a list of traces. Its value is magnified by the advanced features that help you analyze and act on that data.

The Service Map: A Visual Health Dashboard

The X-Ray Service Map is one of its most powerful features. It’s a dynamic, visual graph that is automatically generated from the trace data it collects.

The service map provides a bird's-eye view of your entire application architecture. Each node on the map represents a service, and the connections between them show the flow of requests. The map updates in real-time, giving you an immediate sense of the health and relationships between your components.

Root Cause Analysis: When a node in the map turns red (due to errors) or orange (due to faults), you can instantly see where the problem is originating.

Bottleneck Detection: The thickness of the connections between services represents the volume of traffic, while the color can indicate latency. This makes it easy to spot over-utilized services or network bottlenecks.

CloudWatch ServiceLens

While X-Ray is powerful on its own, its true potential is unlocked when it’s integrated with other AWS services. This is the core concept behind CloudWatch ServiceLens.

ServiceLens is an observability hub that integrates
traces (from X-Ray),
metrics (from CloudWatch Metrics),
and logs (from CloudWatch Logs)
into a single, unified view.

Imagine you see an error spike on the Service Map.
You click on the error node to see a list of the specific traces that failed.
You select a trace to see its detailed end-to-end timeline and find the failing segment.

ServiceLens can then show you the actual CloudWatch logs for that specific request from that specific service, correlated by the Trace ID.

This gives you ability to seamlessly traverse from metric to trace to log, save time and frustration during incident response.

X-Ray Insights: Proactive Issue Detection

X-Ray Insights is a powerful analysis feature that proactively detects issues in your applications. Instead of waiting for you to find a problem, Insights analyzes your trace data to identify anomalies, such as unexpected spikes in fault rates or significant changes in latency.

When it detects an issue, it creates an "insight" event that highlights the impacted service and the potential root cause. It also allows you to compare the current trace data with past performance to understand the scope of the change.

This proactive detection is crucial for identifying problems before they impact users.

Conclusion

In the world of microservices, observability is no longer an optional "nice-to-have." It is a fundamental requirement for building, deploying, and operating reliable applications.

AWS X-Ray provides the tools you need to overcome the visibility challenges inherent in distributed systems. By offering end-to-end tracing, a visual service map, and powerful integrations with the broader CloudWatch ecosystem, X-Ray transforms your application from a confusing black box into a clear, understandable, and manageable system.

While implementing distributed tracing requires an investment in instrumentation, the returns—in terms of faster mean-time-to-resolution (MTTR), improved performance, and a better understanding of your application—are invaluable. In a maze of microservices, AWS X-Ray is the map and compass that every modern development team needs.

Token Pruning and Prompt Compression in Modern AI

Software Jutsu — Sat, 14 Feb 2026 22:10:55 +0000

In the landscape of Large Language Models (LLMs), "context is king," but context is also expensive.
As we move toward models with million-token windows, the efficiency of how we fill that space has become a critical engineering hurdle.

Token pruning and prompt compression have emerged as the primary solutions to the "bloated prompt" problem.

What is Token Pruning?

Token pruning is the process of identifying and removing redundant, less informative, or "noisy" tokens from a prompt before or during the inference phase.

Unlike simple summarization, which rewrites text to be shorter, pruning often involves algorithmic selection—deciding which specific tokens the model's attention mechanism can afford to ignore without losing the semantic "gist" of the message.

How it works

Importance Scoring: The system assigns a "budget" to the prompt. It uses a smaller, faster model (like a BERT-base or a lightweight GPT) to calculate the perplexity or attention weight of each token.
Filtering: Tokens with low importance scores (e.g., "the," "actually," redundant adjectives, or repetitive background info) are discarded.
Re-alignment: The remaining tokens are concatenated into a condensed string that is then passed to the primary, high-performance LLM (like GPT-4o or Gemini 1.5 Pro).

Usage Scenarios

1. The RAG "Needle in a Haystack"

In Retrieval-Augmented Generation (RAG), a system might pull 10 different PDFs to answer one question. This creates a massive prompt where 90% of the text is irrelevant.

Pruning allows the model to focus its "attention heads" specifically on the relevant paragraphs.

It mitigates the "Lost in the Middle" phenomenon, where models perform worse when key information is buried in the center of a long prompt.

2. Infinite-Turn Conversations

Customer service bots or roleplay AI often accumulate massive chat histories. If you don't prune, the cost per message increases linearly until the model hits its limit and starts "forgetting" the beginning of the interaction.

By pruning 30-50% of the middle-history tokens, you can maintain factual consistency of a conversation for thousands of turns at a fixed cost.

Scenario Example: Legal Document Analysis

The Setup:
A law firm needs to use an LLM to find conflicting clauses across five different 50-page contracts (approx. 60,000 tokens).

Without Token Pruning:
Cost: Processing 60,000 tokens per query costs roughly $0.15 - $0.60 (depending on the model).
Latency: The model takes 30-40 seconds to process the "prefill" (reading the prompt).
Risk: The model might hallucinate or miss a conflict because the relevant clauses are buried under 40 pages of standard "boilerplate" definitions.

With Token Pruning (using a tool like LLMLingua):
Preprocessing: A small model scans the 60,000 tokens and identifies that 45,000 tokens are "filler" that appears in every contract.
Compression: The prompt is compressed by 4x, leaving only 15,000 tokens containing the unique, substantive clauses.
Result: * Cost drops by 75%.
Latency improves significantly as the "prefill" time is slashed.
Accuracy increases because the LLM's attention is focused solely on the unique variables of the contracts.

Optimizing Gemini with Token Pruning

When using LLM in your projects, it does not prune tokens automatically, you should implement one of the following strategies in your application layer, lets say using Gemini

Strategy A: The "Semantic Filter"

Use a lightweight tool like LLMLingua or a smaller model (like Gemini 1.5 Flash-Lite) to "compress" your prompt before sending it to the more expensive Gemini 1.5 Pro.

Implementation Steps:

Scoring: Calculate the "importance" of each sentence or paragraph relative to the user's query.
Pruning: Remove segments with low importance scores.
Inference: Send the compressed prompt to the Gemini API.

Strategy B: Hard Truncation & Summarization

For chat history, don't just send the last 100 messages.
Pruning Logic: Keep the last 5 messages in full. For messages 6-50, provide a one-sentence summary of each exchange rather than the full transcript.

Tip: Use the count_tokens method in the Gemini SDK before sending a request to monitor how much your pruning is actually saving you.

Pruning vs. Summarization

Pruning is different from summarizing.

Feature	Token Pruning / Compression	Standard Summarization
Logic	Removes low-entropy tokens algorithmically.	Rewrites text into a shorter version.
Speed	Very fast (minimal compute).	Slower (requires an LLM pass).
Integrity	Preserves original phrasing of key info.	May lose specific technical terminology.
Primary Goal	Cost and Latency reduction.	Human readability.

Example

1. The Original Prompt (Input)

"The system administrator reported that the primary database server, located in the Northern Virginia (us-east-1) region, experienced a critical hardware failure at exactly 14:02:45 UTC. The error log specifically identified a 'TimeoutException' on the NVMe storage controller (Serial: 99-XJ-22). We need to determine if this is a recurring issue across the cluster."

2. The Pruned Version (Extractive)

Logic: Removes "glue" words and low-information adjectives while keeping high-entropy identifiers.

"administrator reported primary database server us-east-1 experienced critical hardware failure 14:02:45 UTC. error log identified 'TimeoutException' NVMe storage controller Serial: 99-XJ-22. determine recurring issue cluster."

Tokens Saved: ~40% reduction.
Human Readability: Poor (looks like a telegram).
The Gemini model still sees the exact timestamp, the exact error code, the exact serial number, and the exact region. It has all the "signal" it needs to solve the problem.

3. The Summarized Version (Generative)

Logic: Rewrites the meaning into a new, shorter sentence.

"The admin noted a hardware crash in the Virginia region at 2:02 PM due to a storage error. We need to check if other servers are affected."

Tokens Saved: ~60% reduction.
Human Readability: Excellent.
The exact timestamp (14:02:45) is gone.
- Identifier Loss: The specific Serial Number (99-XJ-22) is gone.
- Technical Loss: "TimeoutException" was replaced with "storage error," which is too vague for troubleshooting.

If you need the model to have 100% of the information but want it to be cheaper/faster, you can try explore Context Caching.

Amazon Virtual Private Cloud (VPC)

Software Jutsu — Tue, 10 Feb 2026 05:14:36 +0000

An Amazon Virtual Private Cloud (VPC) is a logically isolated section of the Amazon Web Services (AWS) Cloud where you can launch AWS resources in a virtual network that you define.

It gives you complete control over your virtual networking environment, including the selection of your own IP address range, creation of subnets, and configuration of route tables and network gateways.

In production deployment it is always best practice to implement VPC for the AWS stacks.

Why VPC?
Creating resources directly on the public cloud without a VPC would be equivalent to placing your laptop on a public sidewalk with no password and a "Free Access" sign. While technically possible in the early days of cloud computing, it poses extreme security and operational risks.

Security and Isolation: Without a VPC, every resource (like a database or a web server) would be assigned a Public IP address by default. This makes them visible and targetable by anyone on the internet.
Connectivity between Services
With VPC, your resource can talk to each other using Private IPs as part of the isolation, on top of that because it communicates via private IPs, it significantly save cost over expensive public IP data transfers.
Network Customization: If you created resources directly, you would have no control over the internal networking. In a VPC, you define your own IP address range (e.g., 10.0.0.0/16).
Traffic Routing and Filtering: A VPC allows you to define the "rules of the road" through Route Tables.

Core Components

IPv4 and IPv6 Address Blocks: You define a Private IP address range using CIDR notation (e.g., 10.0.0.0/16).
Subnets: A range of IP addresses in your VPC. You can launch AWS resources, like EC2 instances, into a specific subnet.

-- Public Subnets: Connected to the internet via an Internet Gateway (without gateway you can't connect, its like the physical "door" to the outside world.)

-- Private Subnets: Not reachable from the public internet; typically used for databases or application servers.

Route Tables: A set of rules (routes) used to determine where network traffic from your subnet or gateway is directed.
Internet Gateway (IGW): A horizontally scaled, redundant, and highly available VPC component that allows communication between your VPC and the internet.
NAT Gateway: Allows instances in a private subnet to connect to the internet (e.g., for software updates) but prevents the internet from initiating a connection with those instances.

Security Layers

AWS VPC provides two features to increase security:

Network Access Control Lists (NACL): An optional layer of security for your VPC that acts as a firewall for controlling traffic in and out of one or more subnets (Stateless).

Security Groups: Act as a virtual firewall for your EC2 instances to control inbound and outbound traffic at the instance level (Stateful).

Connectivity Options

VPC Peering: A networking connection between two VPCs that enables you to route traffic between them using private IPv4 or IPv6 addresses.

AWS VPN: Establishes a secure connection between your on-premises network and your Amazon VPC.

AWS Direct Connect: A cloud service solution that makes it easy to establish a dedicated network connection from your premises to AWS, bypassing the public internet.

To setup VPC you can follow this
https://www.youtube.com/watch?v=7_NNlnH7sAg

Pricing

Creating a VPC itself is completely free. You are not charged for defining the network, creating subnets, or setting up Route Tables, Internet Gateways (IGW), or Security Groups.

However, you pay for the traffic that moves through the VPC and any managed services you attach to it.

Components That Are Free

VPC Creation: No setup or monthly fee.
Subnets & Route Tables: No limit on how many you create for free.
Internet Gateway (IGW): Attaching and using an IGW costs $0.
Network ACLs & Security Groups: Virtual firewalls are provided at no cost.

Components That Cost Money 2026 Jan (Maybe outdated)

Public IPv4 Address: $0.005 / hour (~$3.60/mo)

NAT Gateway: $0.045 / hour (~$32/mo), $0.045 per GB processed.

VPC Interface Endpoint: $0.01 / hour (~$7/mo), $0.01 per GB processed.

Transit Gateway: $0.05 / hour (per attachment), $0.02 per GB processed.

Site-to-Site VPN,$0.05 / hour,Standard data transfer rates apply.

Data Transfer Costs

This is where most VPC bills grow. AWS charges based on where the data is going.

Inbound Data: Always Free (Data coming from the internet into your VPC).

Outbound to Internet: Roughly $0.09 per GB (after the first 100GB/month which is free).

Inter-Availability Zone (AZ): If you send data between two EC2 instances in different AZs (e.g., US-East-1a to US-East-1b), you pay $0.01 per GB in each direction.

Intra-Availability Zone: Sending data between instances in the same AZ using Private IPs is Free.

Pro Tip: If you use a Public IP to talk to another instance in the same AZ, you will be charged the $0.01/GB rate even though they are in the same building. Always use Private IPs for internal talk.

Requirement to software Delivery in midsize comp CHEATSHEET

Software Jutsu — Mon, 12 Jan 2026 03:08:53 +0000

In a modern mid-size startup (roughly 50–200 employees), the software development process is designed to balance the speed of a small team with the predictability required by investors and stakeholders. Unlike a "garage startup" that might code on a whim, a mid-size company uses a structured

Discovery → Planning → Execution loop.

The Software development phase (product based delivery) typically involves few phases as following

Initiation: "The Why and What"

Initiation is rarely about a single person’s idea; it’s a response to data, customer feedback, or strategic goals.

The Catalyst: Usually comes from Product Management (PM) (based on user feedback/analytics), Leadership (strategic pivot), or Sales/CS (high-value client requests).

The Discovery Phase: Before a single line of code is written, a "Discovery" squad (PM, Lead Designer, and Tech Lead) validates the idea. They perform user interviews and technical feasibility spikes.

Key Output: The PRD (Product Requirements Document)

In modern startups, this is a "living" document in Notion or Confluence.

Content: Problem statement, target user, user stories, success metrics (KPIs), and "out of scope" items to prevent scope creep.

Our focus will be on engineering side, so i skip the details of PRD, you will only need to know key parts, normally engineer leads will work very close with Product manager to clarify the PRD contents.

Planning: "The How and When"

Once a project is "greenlit" via the PRD, the planning shifts to technical architecture and resource allocation.
UI designers may involved depending on requirement

Designers receives the PRD analyze the requirement, coming up with solution and provide Key output: high-fidelity mockups (usually in Figma).

The Tech Lead Discuss PRD and UI Mockup with engineers, analyze requirements and unveiling details, edge cases. Then engineering team writes a document outlining the system architecture, database changes, and API designs, key user metric gather plan, releases plan.

It is shared with the wider engineering team for peer review to catch flaws early.
Time allocation of this work referenced by SPIKE Ticket within sprint.

Key Output: RFC (Request for Comments) / TDD (Technical Design document)

Upon receiving feedback, passing reviews, Technical design is approved and proceed to

Backlog Grooming: The PM and Engineering Lead break the PRD down into "tickets" (tasks) in a tool like Jira. Each ticket is assigned "Story Points" to estimate effort.

The Flow: Step-by-Step

Startups typically run on Scrum or Kanban frameworks, operating in 2-week "Sprints."

| Phase        | Key Activity                           | Output            |
|--------------|----------------------------------------|-------------------|
| Estimation   | Pointing session with developers       | Sprint Backlog    |
| Execution    | Coding, Code Reviews (PRs), CI/CD      | Feature Branch    |
| QA/UAT       | Automated testing & stakeholder review | Release Candidate |
| Launch       | Feature flags & phased rollout         | Live Feature      |

Key Documents & Tools

Modern startups favor "Asynchronous Documentation" writing things down so anyone can catch up without a meeting.

Product Roadmap: A high-level timeline (usually quarterly) showing "Now, Next, Later."

RACI Matrix: A chart defining who is Responsible, Accountable, Consulted, and Informed for the project.

Service Level Agreements (SLAs): Documentation on expected uptime and performance for the new feature.

Tech Stack: * Project Tracking: Linear, Jira, or Asana.

Docs: Notion, Confluence, or GitHub Wiki.

Design: Figma.

Communication: Slack (integrated with GitHub/Jira).

Timelines

While timelines vary by feature size, a standard "medium-sized" project (e.g., building a new dashboard) usually follows this cadence:

Week 1-2 (Discovery): Research, PRD drafting, and stakeholder alignment.

Week 3 (Technical Design): RFC/TDD writing, peer review, and Figma prototyping.

Week 4-7 (Development): 2 or 3 Sprints of active coding. Includes daily "Stand-ups" (15-min syncs).

Week 8 (Testing & Polish): Bug smashing and User Acceptance Testing (UAT).

Week 9 (Deployment): Usually a "Dark Launch" (deploying code behind a toggle) followed by a full release.

That concludes the overview process from requirement to production delivery, but its also crucial to have post-launch process

Product team

needs to gather the user feedback, monitor analytics like Amplitude, Mixpanel, or Google Analytic to analyze the impact according to initial PRD plan.
Tech team provides analytics and its metric setup and configurations beforehand.

Tech team

Observability & Monitoring
Developers watch dashboards (e.g., Datadog, New Relic, or Sentry) for a "spike" in error rates. If the new code causes the server to crash or slow down, they must hotfix it immediately.

Feature Flag Cleanup
Most startups use LaunchDarkly or Statsig to toggle features on. Once a feature is 100% rolled out and stable, the "tech debt" task is to go back into the code and remove the "if/else" logic for that toggle.

"Nothing is more permanent than temporary solution"
Addressing Technical Debt
Often, to meet a deadline, developers take "shortcuts." Post-launch is the time to go back and refactor that code to make it more maintainable. The team must create clear timeline when to refactor or completely ignore it, otherwise tech debt will be under radar forever and can hurt maintainability in long term.

Performance Tuning
Now that real user data is hitting the database, the Tech Lead checks if the queries are performing as expected or if they need to add new indexes to the database.

The "Post-Mortem" / Retrospective

This is a formal meeting and document created 1–2 weeks after launch. It is the most important document for team growth.

The Document Structure:

What went well? (e.g., "The RFC process caught a major security flaw early.")

What went wrong? (e.g., "The Figma designs were missing a mobile view, causing a 2-day delay.")

Timeline Reality vs. Estimate: Did it take 3 sprints instead of 2? Why?

Action Items: Concrete steps to prevent the same mistakes in the next project.

I thought Vercel's backend was just like another backend...

Software Jutsu — Sat, 10 Jan 2026 02:29:29 +0000

Lesson learned about Vercel vs. Traditional Backend

On my journey to learn Next.js in-depth through projects, i tried to architect my Next.js backend server like normal golang's backend (dependency injection). Turns out its not possible.

I treat Vercel’s /api routes like a "mini Express server." This is a fundamental architectural error. While both run Node.js, their underlying infrastructure (Serverless vs. Long-Running) dictates completely different rules.
Then i dig further to understand why service model like Vercel are different from backend in common...

Other services that work similar are Netlify, Cloudflare Pages / Workers, AWS Amplify.

1. Process Lifecycle: Ephemeral vs. Persistent

Traditional Backend (Long-Running)
The Concept: The process starts and stays alive until you manually stop it or it crashes.
The "Global Root" is permanent. Memory is allocated once and remains available across thousands of different requests.

Implication: You can rely on the server "remembering" things in its internal RAM for days or weeks.

Vercel (Serverless)
The Concept: The process is Ephemeral. It is "born" when a request arrives and "executed" (terminated or frozen) the millisecond the response is sent.
The "Global Root" is destroyed frequently.

Implication: Any internal state is wiped. You must treat every request as if the server just performed a "Cold Start."

2. State Management: The Global Variable Trap

Traditional Backend (Long-Running)
Memory: Global variables are Shared. If User A updates a global counter, User B sees the updated value because they are hitting the same process.

Garbage Collection: As long as the process lives, GC ignores global variables because they are "reachable." This allows for in-memory caching or simple counters.

Vercel (Serverless)
Global variables are Isolated. If 10 users hit your API, Vercel may spin up 10 separate "Execution Environment" (lets call this instance).

The Trap: User A updates a counter in Instance #1. User B hits Instance #2, which has a completely different Global Root where the counter is still 0.

(This is bit more nuance to this,
Global variable can still persist in its warm state, but reuse only happens sequentially, not simultaneously. if its Idle, it will return to its cold state, all global variable will be destroyed too.)

You cannot use in memory data to keep state you want to persist on different request. You must use an external store like Redis or PostgreSQL.

3. Concurrency: Thread Pooling vs. Horizontal Scaling

Traditional Backend (Long-Running)
Mechanism: One server handles multiple concurrent requests using a single thread with an Event Loop (Node.js). It manages a "pool" of connections.
High traffic is handled by the server working harder. If it gets overwhelmed, requests queue up.

Vercel (Serverless)
Mechanism: Vercel scales by Multiplication. If traffic spikes, it doesn't make one function work harder; it creates 1,000 copies of that function.
This creates a "Connection Exhaustion" problem. If 1,000 functions spin up, they all try to open a new connection to your database at once, potentially crashing your DB. You must use a Connection Pooler (like Prisma Accelerate or Supabase Pooling).

4. Execution Timing: Synchronous vs. Background Tasks

Traditional Backend (Long-Running)
You can send a response to the user and continue running code in the background (e.g., res.send(); fireAndForgetEmail();).

The process stays alive, so the email function finishes its work comfortably in the background.

Vercel (Serverless)
The Flow: The moment res.send() is called, the runtime environment is paused or killed.

The Disaster: If you try to run a background task after the response, Vercel will likely "cut the power" mid-execution. Your email might be half-sent or never sent at all.

Every task must be finished before the response, or delegated to a queue service (like Upstash Workflow).

5. Persistent Connections: WebSockets vs. HTTP

Traditional Backend (Long-Running)
Connectivity: Supports Stateful connections.
A client can keep a TCP socket open (WebSockets) for real-time chat or live updates. The server process is always there to "hold the other end of the string."

Vercel (Serverless)
Connectivity: Supports Stateless HTTP requests only.

Since the function dies after the response, it cannot "hold" a connection open. If you try to use socket.io, it will fail because the "server" disappears every few seconds.
For real-time features, you must use a third-party "Realtime-as-a-Service" like Pusher or Ably.

This article made with AI assistance, and note to myself.

🔧 Typescript Generics - Practical Guide and examples

Software Jutsu — Thu, 19 Jun 2025 08:01:27 +0000

Generics, reminds me to early days of studying typescript, when i see those T, U, V characters inside < brackets > that scared the hell out of me and i try to avoid them.
once i understand them, they are actually not that scary, they are powerful.

I collected examples that will help you quickly understand why, and how to use generics, without have to read through the complex documentation and confusing explanations. 😭

What are Generics?

Generics let you pass a type as a parameter to functions, interfaces, or classes.

Think of it as a function parameter, but for types.

function doSomething<T>(input: T): T {
  return input;
}

Here, T is a type placeholder. When you use the function, T gets replaced by the actual type you pass.

T is just convention, you can actually name it anything you like,

function doSomething<TTypeBeingPassedHere>(input: TTypeBeingPassedHere): TTypeBeingPassedHere {
  return input;
}

To give distinct identifier, commonly its starts with T followed by the name, here is another example

function doSomething<TInput>(inputOne: TInput): TInput {
  // ... some logic here
  return input;
}

doSomething<number>(1) // returns number
doSomething<string>("1") // returns string

This code defines the function must always return TInput, and it passed by the of the caller,
in this case TInput passed as number hence it will return number

🧐 Why Use Generics?
Avoid using any (which removes type safety)
Avoid code duplication
Write reusable and type-safe utilities

to easier understand how generic works and its benefits, take a look of more examples below.

Common use case scenarios

Reusing Logic Without Losing Types

Scenario: You want to write a utility function that works with any type but still keeps the original type intact.

function wrapInArray<T>(value: T): T[] {
  return [value];
}

const numberArray = wrapInArray(123);     // number[]
const stringArray = wrapInArray('hello'); // string[]

🧠 Why use generics?

If you used any, you’d lose type safety.
If you hardcoded string, you couldn’t reuse it for number.

Handling API Responses

Scenario: You fetch data from different endpoints. All responses follow the same format: data + success. But the data can vary.

interface ApiResponse<T> {
  data: T;
  success: boolean;
}

const userResponse: ApiResponse<{ id: number; name: string }> = {
  data: { id: 1, name: 'Alice' },
  success: true,
};

const postsResponse: ApiResponse<string[]> = {
  data: ['post 1', 'post 2'],
  success: true,
};

🧠 Why use generics?

You avoid duplicating the structure. ApiResponse can be re-used for different response shape.
You get full type safety no matter what data is.

Creating Flexible List Utilities

Scenario: You want a function like getFirstItem, and you want it to work for any array type.

function getFirstItem<T>(list: T[]): T | undefined {
  return list[0];
}

const firstName = getFirstItem(['Rick', 'Morty']); // string
const firstNumber = getFirstItem([10, 20, 30]);     // number

🧠 Why use generics?

Without generics, you’d either duplicate the function or use any.

Extending Built-in Types Safely

Scenario: You want to write a function that only accepts objects that have a length, like strings or arrays — not numbers or booleans.

function logLength<T extends { length: number }>(value: T): void {
  console.log(value.length);
}

logLength('hello');     // valid
logLength([1, 2, 3]);   // valid
logLength(100);         // ❌ Error: number doesn't have length

🧠 Why use generics with extends?

Combined with use of Extends, you can selectively accepts types of Generics you allow, in this case logLength only accepts anything that compatible with { length: number}
You define a flexible type that still enforces structure.

React Hook That Works with Any Type

Scenario: You want a useLocalStorage hook that works with any type of value — string, number, object, etc.

function useLocalStorage<T>(key: string, initialValue: T): [T, (val: T) => void] {
  const [value, setValue] = React.useState<T>(() => {
    const json = localStorage.getItem(key);
    return json ? JSON.parse(json) : initialValue;
  });

  React.useEffect(() => {
    localStorage.setItem(key, JSON.stringify(value));
  }, [key, value]);

  return [value, setValue];
}

🧠 Why use generics?

So this hook works with boolean, string, { id: number }, or anything else, with full autocomplete and safety.

Generic Class for Storing Any Type

Scenario: You want a class that stores data of any type.

class Store<T> {
  private items: T[] = [];

  add(item: T) {
    this.items.push(item);
  }

  getAll(): T[] {
    return this.items;
  }
}

const userStore = new Store<{ id: number; name: string }>();
userStore.add({ id: 1, name: 'Alice' });

const numberStore = new Store<number>();
numberStore.add(10);

🧠 Why use generics?

One class works for both objects, strings, and numbers - without rewriting the logic.

official reference https://www.typescriptlang.org/docs/handbook/2/generics.html

Understanding TypeScript’s Comments: @ts-ignore, @ts-expect-error, and Friends

Software Jutsu — Sat, 14 Jun 2025 08:22:59 +0000

When working with TypeScript, you've probably seen some strange-looking comments like // @ts-ignore or // @ts-expect-error. These are TypeScript directives, often called “magic comments” — and they can change how the compiler behaves for specific lines or files.
what exactly is it? what are the difference?

Let's walk through the 4 most important directives you should know.

1. // @ts-expect-error

This directive tells TypeScript:

“I expect the next line to throw a type error. If it doesn't, fail the build.”

// @ts-expect-error
const name: number = "John"

If the error is later fixed (intentionally or by accident), the compiler throws an error:

Unused '@ts-expect-error' directive.

✅ Why it's good:
• Enforces that you're suppressing an error on purpose.
• Helps with testing or gradually migrating code.

🛠 Use when:
• You're writing a test that expects an error.
• You're temporarily keeping known type errors but want safety.

2. // @ts-ignore

This tells TypeScript:

“Ignore all errors on the next line. I know what I'm doing.”

// @ts-ignore
const user: string = 123

Unlike @ts-expect-error, this will never alert you even if the error is gone later. It's silent and dangerous if overused.

⚠️ Why it's risky:
• Suppresses all type checking on that line.
• Can hide bugs and technical debt.

🛠 Use only when:
• There's no other option.
• You're integrating with legacy code, third-party libs, or non-typed APIs.

I would avoid this at all cost.

3. // @ts-check (for .js files)

This enables TypeScript type-checking for plain JavaScript files.

// @ts-check

/**
 * @param {number} x
 * @returns {number}
 */
function square(x) {
  return x * x
}

🔍 Why it's useful:
• Allows you to catch type bugs in JS without migrating to .ts.
• You can annotate with JSDoc for full type checking.

🛠 Use when:
• Gradually migrating a JS codebase to TS.
• You want lightweight type safety in JS.

4. // @ts-nocheck

Disables all TypeScript checking for a file even in .ts or .tsx.

// @ts-nocheck

const user = getUser()
user.doSomethingThatDoesNotExist()

💀 Why it's dangerous:
• Completely disables the TypeScript compiler for that file.
• Silently ignores all type and syntax errors.

🛠 Use only when:
• You're working with untyped legacy code.
• You're debugging or temporarily disabling TS during migration.

Overall, my take:
• Prefer @ts-expect-error over @ts-ignore for long-term maintainability.
• Use ESLint rules like @typescript-eslint/ban-ts-comment to restrict or require justification for using these comments.
• In a real project, flag every @ts-ignore for later cleanup.

| Directive          | Scope       | Description                                      | Safe? |
|-------------------|-------------|--------------------------------------------------|--------|
| `@ts-expect-error` | One line    | Expect a type error. Fails if no error.         | ✅     |
| `@ts-ignore`       | One line    | Ignore any TS error on the next line.           | ⚠️     |
| `@ts-check`        | Whole file  | Enable TS checking in JS files.                 | ✅     |
| `@ts-nocheck`      | Whole file  | Disable all TS checking.                        | ❌     |

Use them wisely to reduce friction, not to hide problems.

This article is part of my personal note, as reminder of problems and learnings i encounter on my daily work. I did use AI to help build the article.
If you interested, follow.

Author: @rickvian

How to Check existing SSH public key in Windows

Software Jutsu — Sat, 14 Jun 2025 08:18:58 +0000

To check your existing SSH public key in Windows, you can follow these steps:

Check Existing SSH Public Key in Windows (PowerShell, for Bitbucket use)

Open PowerShell
Navigate to your SSH key directory (replace YourUsername if needed):

cd $env:USERPROFILE\.ssh

Display your public key:

Get-Content id_rsa.pub

Note: If id_rsa.pub does not exist, generate a new key:

ssh-keygen