Forem: AttractivePenguin

Idempotency Keys in Production: The Race Conditions, Expiration Traps, and Edge Cases Most Tutorials Skip

AttractivePenguin — Thu, 30 Apr 2026 09:06:38 +0000

Idempotency Keys in Production: The Race Conditions, Expiration Traps, and Edge Cases Most Tutorials Skip

Every payment flow has a silent enemy: the network. Requests time out, connections drop, users panic and click "Pay" twice. If your backend isn't ready for that double-tap, someone gets charged twice — and you lose their trust forever.

Most tutorials explain idempotency keys in about three paragraphs: "generate a UUID, send it in a header, check it on the server." That's the hello-world version. It works until you have concurrent requests, distributed servers, or a payment that takes longer than your key expiration window.

This article covers what happens after the tutorial ends: the concurrency race condition that breaks most implementations, the difference between idempotency and caching (they're not the same), how to handle key expiration without creating duplicates, and a production-ready implementation you can actually ship.

Why Idempotency Keys Exist

The problem is simple. The solution is not.

Imagine a checkout flow:

User clicks "Pay"
The request hits your server
Your server calls Stripe
The network hangs — no response for 10 seconds
The user clicks "Pay" again
Now your server has two requests in flight for the same charge

Without idempotency, both requests hit Stripe. Both succeed. The user is charged twice.

This isn't a bug — it's normal behavior in distributed systems. Networks fail, load balancers retry, mobile apps auto-retry on reconnect, and webhook providers all re-deliver on failure because they assume you handle it.

Idempotency means: send the same request multiple times, but process it only once.

The Tutorial Version (And Why It Breaks)

// Server-side: the "simple" version
async function handleCharge(req, res) {
  const key = req.headers['idempotency-key'];

  // Check if we've seen this key before
  const existing = await db.getIdempotencyKey(key);
  if (existing) {
    return res.json(existing.result);
  }

  // Process the payment
  const result = await stripe.charges.create(req.body);

  // Store the result
  await db.setIdempotencyKey(key, result);
  return res.json(result);
}

Looks correct. It's not. Here are the four ways this breaks.

Trap #1: The Concurrency Race Condition

Two identical requests arrive at the same time:

Request A: check key → not found → proceed to charge
Request B: check key → not found → proceed to charge
Both charge the user. Both store results. Duplicate charge.

This is a classic check-then-act race condition. It doesn't matter how fast your database is — if two requests arrive within the same event loop tick (or on different server instances), both pass the check.

The Fix: Atomic Key Claims

You need an atomic "insert if not exists" operation. Only one request wins the race.

PostgreSQL:

CREATE TABLE idempotency_keys (
  key TEXT PRIMARY KEY,
  response JSONB,
  status_code INT,
  business_signature TEXT,
  created_at TIMESTAMP DEFAULT NOW()
);

Then use INSERT ... ON CONFLICT to make the insert atomic:

async function handleCharge(req, res) {
  const key = req.headers['idempotency-key'];
  const payload = req.body;
  const signature = extractBusinessSignature(payload);

  // Attempt to claim the key atomically
  const inserted = await db.query(
    `INSERT INTO idempotency_keys (key, business_signature)
     VALUES ($1, $2)
     ON CONFLICT (key) DO NOTHING
     RETURNING key`,
    [key, signature]
  );

  if (inserted.rowCount === 0) {
    // Key already claimed — fetch and return the stored result
    const existing = await db.query(
      `SELECT status_code, response FROM idempotency_keys WHERE key = $1`,
      [key]
    );
    return res.status(existing.rows[0].status_code).json(existing.rows[0].response);
  }

  // We won the race — process the payment
  try {
    const result = await stripe.charges.create(payload);
    await db.query(
      `UPDATE idempotency_keys SET status_code = 200, response = $1 WHERE key = $2`,
      [JSON.stringify(result), key]
    );
    return res.json(result);
  } catch (err) {
    await db.query(
      `UPDATE idempotency_keys SET status_code = $1, response = $2 WHERE key = $3`,
      [err.statusCode || 500, JSON.stringify({ error: err.message }), key]
    );
    return res.status(err.statusCode || 500).json({ error: err.message });
  }
}

Whatever you use, the operation must be atomic. A check-then-insert pattern will always have a race condition. Other options:

Redis: SET key value NX EX 86400 — atomic create-if-not-exists with TTL
MongoDB: unique index + updateOne with upsert
DynamoDB: PutItem with attribute_not_exists(idempotencyKey)

Trap #2: Idempotency ≠ Caching

It's tempting to think: "If a request failed, just return the same failure on retry." But that's wrong.

Consider these scenarios:

Scenario	What happened	What retry should do
Request never reached your server	Network dropped before arrival	Process normally — it's a genuinely new request
Request reached server, processing started but failed	Payment provider returned 402 (card declined)	Return stored 402 — the charge was attempted
Request reached server, outcome unknown	Network timeout to payment provider	Retry with same key — check provider state first

The critical distinction: store every deterministic response (success or known failure), but don't store unknown outcomes. If the payment provider timed out, you don't know whether the charge went through. A retry must check the provider's state — not assume it failed.

// After calling the payment provider
if (result.status === 'succeeded' || result.status === 'declined') {
  // Known outcome — store it
  await storeResult(key, result);
} else if (result.status === 'timeout' || result.status === 'unknown') {
  // Unknown outcome — DON'T store, let retry re-check
  // The retry will hit the provider's status endpoint
  const providerResult = await stripe.charges.retrieve(chargeId);
  await storeResult(key, providerResult);
}

Trap #3: Key Reuse With Different Payloads

If someone sends the same key with different payment details, you must reject it. A business signature captures only the immutable fields — not timestamps, request IDs, or metadata:

function extractBusinessSignature(payload) {
  // Only include immutable business fields
  return JSON.stringify({
    amount: payload.amount,
    currency: payload.currency,
    customerId: payload.customerId,
    description: payload.description,
  });
}

When a key is reused, compare the signature:

if (existing && existing.businessSignature !== currentSignature) {
  return res.status(409).json({
    error: 'Idempotency key reused with different payment details'
  });
}

This prevents silent mismatches. The 409 tells the client: "You've already used this key for a different payment. Generate a new one."

Trap #4: Key Expiration Creates Duplicates

Idempotency keys shouldn't live forever, but if a key expires while a payment is still processing, a retry creates a brand-new charge. Duplicate.

The Fix: Expiration Must Outlive the Transaction

Set your TTL long enough to cover payment processing time, the client retry window, and any provider-side settlement. 24 hours is a safe default.

// Redis with 24-hour TTL
await redis.set(`idem:${key}`, JSON.stringify(result), 'NX', 'EX', 86400);

And never delete a key just because the checkout session ended. Let the TTL handle it. Manual deletion is how you create exactly the bug you're trying to prevent.

The Client Side: One Key Per Intent

// ❌ WRONG: New key on every click
const handlePay = () => {
  const key = crypto.randomUUID(); // generates a new key every time!
  fetch('/api/charge', {
    method: 'POST',
    headers: { 'Idempotency-Key': key },
    body: JSON.stringify(payload)
  });
};

// ✅ RIGHT: One key per checkout session
function CheckoutForm() {
  const idempotencyKeyRef = useRef(crypto.randomUUID()); // generated once

  const handlePay = () => {
    fetch('/api/charge', {
      method: 'POST',
      headers: { 'Idempotency-Key': idempotencyKeyRef.current },
      body: JSON.stringify(payload)
    });
  };
}

Where the key lives depends on your flow:

Storage	Use When
React state/ref	Single-page checkout, no reloads expected
`sessionStorage`	User might reload the page mid-checkout
Server-generated key	Most robust — key lives in your checkout session DB

Server-generated keys are the gold standard because they survive tab closes, browser crashes, and back-button navigation. Your server creates the key when the checkout session starts, and the client just reuses it.

The Full Production Flow

CHECKOUT LOADS
  ↓
Generate idempotency key (once, server or client)
  ↓
User clicks "Pay"
  ↓
POST /api/charge (Idempotency-Key header)
  ↓
SERVER: Atomic insert key into DB
  ├─ Insert succeeded → process payment → store result → return
  └─ Insert failed (key exists) → fetch stored result → return
  ↓
If network timeout: retry with SAME key
  ↓
Result: exactly one charge, regardless of retries

How to Test It

Don't just hope it works. Prove it. The test is simple: send the same key twice and verify your payment provider is only called once.

const mockCharge = jest.fn().mockResolvedValue({ id: 'ch_123', status: 'succeeded' });

test('idempotent charge: second request returns cached result', async () => {
  const key = 'test_order_123';
  const payload = { amount: 2000, currency: 'USD', customerId: 'cus_abc' };

  // First request — processes payment
  const res1 = await request(app)
    .post('/charge')
    .set('Idempotency-Key', key)
    .send(payload);

  // Second request — should return cached result
  const res2 = await request(app)
    .post('/charge')
    .set('Idempotency-Key', key)
    .send(payload);

  expect(res2.body).toEqual(res1.body);
  expect(mockCharge).toHaveBeenCalledTimes(1); // 🔑 the critical check
});

If your mock gets called twice, your idempotency is leaking. Fix it before production.

Test concurrency too — fire two requests simultaneously and verify only one charge is created:

test('concurrent requests: only one charge created', async () => {
  const key = 'concurrent_test_456';
  const payload = { amount: 5000, currency: 'USD', customerId: 'cus_xyz' };

  const [res1, res2] = await Promise.all([
    request(app).post('/charge').set('Idempotency-Key', key).send(payload),
    request(app).post('/charge').set('Idempotency-Key', key).send(payload),
  ]);

  expect(mockCharge).toHaveBeenCalledTimes(1);
  expect(res1.body).toEqual(res2.body);
});

Troubleshooting FAQ

Q: I'm getting duplicate charges even with idempotency keys. What's wrong?

Check three things: (1) Is your key generated once per intent, not per click? (2) Does your database enforce a unique constraint on the key? (3) Is your key store shared across all server instances? If you're using in-memory storage in a multi-instance deployment, each instance has its own key store — and duplicates slip through.

Q: Should I store failed payment results too?

Yes. If the payment provider returns a deterministic failure (card declined, insufficient funds), store that result. A retry with the same key should return the same failure — not attempt a new charge. The only exception: unknown outcomes (timeouts). Those should let the retry re-check the provider state.

Q: How long should idempotency keys live?

24 hours is the standard. It covers processing time, the user's retry window, and any delayed settlement. Shorter TTLs risk creating duplicates on retries; longer TTLs waste storage. For non-payment operations (webhooks, email), 1 hour is usually sufficient.

Q: What if my payment provider already supports idempotency (like Stripe)?

Stripe's idempotency only protects you at the Stripe level. If your server calls Stripe twice with two different keys, Stripe processes both charges. You still need your own idempotency layer to prevent duplicate requests from reaching Stripe.

Q: Can I use the same key across different API endpoints?

No. Scope keys to specific operations. Use prefixes: pay_ for charges, ref_ for refunds, email_ for sends.

Q: What about security — can keys leak information?

Yes. Always send keys over HTTPS, avoid logging full keys in plaintext, and hash or truncate them in observability tools.

The Checklist

Before shipping any state-changing endpoint:

[ ] Key generated once per intent, not per click
[ ] Key persisted across retries (not lost on page reload)
[ ] Backend checks key before processing
[ ] Response stored and reused on duplicate requests
[ ] Business signature validates payload consistency
[ ] Database enforces a unique constraint (not just a check)
[ ] Key store is shared across all server instances
[ ] Expiration policy defined (24h default)
[ ] Unknown outcomes (timeouts) trigger re-checks, not cached errors
[ ] Concurrency tested — two simultaneous requests, one charge

Miss one of these? You don't have idempotency. You have hope.

The double-tap is coming. Be ready for it.

Core Web Vitals in 2026: The Practical Fixes for INP, LCP, and CLS That Actually Work

AttractivePenguin — Wed, 29 Apr 2026 09:05:41 +0000

Core Web Vitals in 2026: The Practical Fixes for INP, LCP, and CLS That Actually Work

Your Lighthouse score says 95. Your users say your site feels slow. Here's why — and exactly how to fix each metric with code you can ship today.

Why Your Lighthouse Score Is Lying to You

There's a hard truth most teams learn the painful way: the number in your terminal is not the number Google cares about.

Lighthouse runs in a simulated environment — a powerful desktop with a fast connection. Google's ranking signal comes from CrUX (Chrome User Experience Report), which measures real users on real devices on real networks. It looks at the 75th percentile over a rolling 28-day window. Your Lighthouse run is a lab test. CrUX is the field result.

I've seen teams spend weeks polishing a 98 Lighthouse score while their actual INP — the metric that measures how responsive your site feels — sat firmly in the "poor" bucket for every mobile user. The gap between lab and field can be enormous.

In 2026, the three metrics that matter are:

INP (Interaction to Next Paint) — does the site respond quickly when tapped?
LCP (Largest Contentful Paint) — does the main content appear fast?
CLS (Cumulative Layout Shift) — does the page jump around while loading?

Let's fix each one with real, ship-ready code.

Fix 1: INP — The Architecture Problem Nobody Diagnoses

INP replaced FID in March 2024, and it's the metric most teams are failing. Roughly 43% of sites still don't meet the 200ms "good" threshold. Unlike FID, which only measured input delay, INP measures the full round trip: from click to visual paint. Every interaction on the page, not just the first one.

Here's the classic scenario: a user taps a filter on a product listing page. That tap triggers a setState that runs through a context provider, re-renders 40+ components, recomputes a sort, and then paints the selected state. On a mid-range Android over 4G, that round trip can hit 480ms. The user feels the lag. Chrome reports it. Your CrUX score tanks.

The Fix: Break the Task, Then Yield

The single highest-impact change is introducing yield points with scheduler.yield(). Split the interaction into "visual feedback first, everything else second":

async function handleFilterTap(filter) {
  // Paint the selected state immediately
  setSelectedFilter(filter);
  await yieldToMain();

  // Then do the expensive work
  const filtered = applyFilters(products, filter);
  await yieldToMain();

  setResults(filtered);
}

function yieldToMain() {
  if ('scheduler' in window && 'yield' in window.scheduler) {
    return window.scheduler.yield();
  }
  return new Promise(resolve => setTimeout(resolve, 0));
}

The perceived difference is dramatic. The filter chip highlights instantly. The list updates a tick later. INP at the 75th percentile typically drops by 60-65% with this pattern alone — no changes to the filtering logic required.

Layer 2: Defer Heavy Renders with `useDeferredValue`

In React, mark the filtered results as a deferred value so the input stays responsive while the heavy list re-renders in the background:

function ProductList({ products, filter }) {
  const deferredProducts = React.useDeferredValue(products);

  const filtered = useMemo(
    () => applyFilters(deferredProducts, filter),
    [deferredProducts, filter]
  );

  return (
    <ul>
      {filtered.map(product => (
        <ProductCard key={product.id} product={product} />
      ))}
    </ul>
  );
}

Layer 3: Offload Hidden Long Tasks

Open Chrome DevTools → Performance panel → record an interaction → look for any task over 50ms. The most common culprit? Third-party analytics scripts running synchronous operations on every click. Move them to a web worker:

// main.js
const analyticsWorker = new Worker('/analytics-worker.js');

function trackEvent(event) {
  analyticsWorker.postMessage({
    type: 'track',
    payload: { event: event.name, data: event.data }
  });
}

// analytics-worker.js
self.onmessage = function(e) {
  if (e.data.type === 'track') {
    // Heavy serialization and network calls happen here,
    // off the main thread
    const serialized = JSON.stringify(e.data.payload);
    fetch('/api/analytics', {
      method: 'POST',
      body: serialized
    });
  }
};

The pattern: INP rewards event handlers that do almost nothing synchronously. Anything expensive — filtering, sorting, logging, computation — gets yielded, deferred, or offloaded.

Fix 2: LCP — It's Almost Always the Hero Image

Every team has the same story: "we optimized the images" and LCP was still 3.2 seconds. Because "the images" isn't the fix — the LCP element is the fix, and it's almost always one specific image above the fold.

Step 1: Identify the LCP Element

Don't guess. Measure it in production with the web-vitals library:

import { onLCP } from 'web-vitals';

onLCP(metric => {
  console.log('LCP element:', metric.entries.at(-1)?.element);
  console.log('LCP value:', metric.value);
  // Send to your analytics pipeline
  sendToAnalytics(metric);
});

Step 2: Preload the LCP Resource

One line in your <head> can cut LCP by 500-700ms:

<link rel="preload" as="image" href="/hero-banner.avif" type="image/avif">

Don't preload everything — just the LCP resource. Preload abuse creates its own problems.

Step 3: Set Fetch Priority

Browsers are conservative about image priority by default. Tell them this one matters:

<img
  src="/hero-banner.avif"
  fetchpriority="high"
  alt="Hero banner"
  width="1200"
  height="600"
/>

Step 4: Modern Formats + Responsive Sizes

Use AVIF with WebP fallback. The file size difference between JPEG and AVIF at equivalent quality is routinely 40-60%. And serve the right size — a responsive srcset is not optional:

<picture>
  <source
    type="image/avif"
    srcset="/hero-400.avif 400w, /hero-800.avif 800w, /hero-1200.avif 1200w"
    sizes="100vw"
  />
  <source
    type="image/webp"
    srcset="/hero-400.webp 400w, /hero-800.webp 800w, /hero-1200.webp 1200w"
    sizes="100vw"
  />
  <img
    src="/hero-1200.jpg"
    fetchpriority="high"
    alt="Hero banner"
    width="1200"
    height="600"
  />
</picture>

Step 5: Never Lazy-Load the LCP Image

This mistake is still on production sites in 2026. loading="lazy" on the hero image guarantees a LCP regression. The browser waits to start loading it until the page is scrolled — but it's visible immediately.

When LCP Is a Font, Not an Image

If the LCP element is text depending on a custom font, use font-display: swap and preload the font file:

@font-face {
  font-family: 'BrandDisplay';
  src: url('/fonts/brand-display.woff2') format('woff2');
  font-display: swap;
  font-weight: 700;
}

<link rel="preload" href="/fonts/brand-display.woff2" as="font" type="font/woff2" crossorigin>

Accept the brief flash of fallback type. Users barely notice. Your 75th-percentile LCP will.

Fix 3: CLS — Three Rules That Cover Everything

CLS is the cheapest metric to fix and the one that makes your site look most amateur when you don't. When buttons jump under thumbs and ads push content down, people lose trust — even if they can't articulate why.

Rule 1: Explicit Dimensions on Every Image, Video, and Iframe

Even if you're styling with CSS, HTML width and height attributes let the browser reserve space before the asset loads:

<!-- Good: browser reserves space immediately -->
<img src="/product.jpg" width="800" height="600" alt="Product photo" />

<!-- Bad: no space reserved, page jumps when image loads -->
<img src="/product.jpg" alt="Product photo" />

If you prefer CSS, use aspect-ratio — but the HTML attributes are simpler and work everywhere.

Rule 2: Reserve Space for Late-Injected Content

Ad slots, cookie banners, personalization widgets — anything that arrives after first paint needs height reserved upfront:

.ad-slot {
  min-height: 250px; /* Reserve space even before the ad loads */
}

.cookie-banner-container {
  min-height: 64px;
}

If the slot stays empty, leave it empty. The shift is worse than the blank space.

Rule 3: Match Font Metrics with `size-adjust`

font-display: swap prevents invisible text but can cause a layout shift if your fallback font has different metrics. The size-adjust descriptor on @font-face eliminates that shift:

@font-face {
  font-family: 'BrandDisplay';
  src: url('/fonts/brand-display.woff2') format('woff2');
  font-display: swap;
  font-weight: 700;
}

@font-face {
  font-family: 'BrandDisplay';
  src: local('Arial');
  font-weight: 700;
  font-display: swap;
  size-adjust: 105.2%; /* Tune this to match the web font metrics */
  ascent-override: 98%;
  descent-override: 22%;
  line-gap-override: 0%;
}

This technique is massively underused. Most teams haven't heard of it, and it can bring CLS from 0.15 down to 0.02.

The Monitoring Setup That Makes This Stick

You can't fix what you can't see, and DevTools on your laptop is not "seeing." Here's the monitoring stack that actually helps:

1. Ship Real-User Metrics

import { onINP, onLCP, onCLS } from 'web-vitals';

function sendToAnalytics(metric) {
  const body = JSON.stringify({
    name: metric.name,
    value: metric.value,
    rating: metric.rating,
    delta: metric.delta,
    id: metric.id,
    navigationType: metric.navigationType,
    url: window.location.href,
    // Slice by route, device, country
    route: window.location.pathname,
    deviceType: /Mobi|Android/i.test(navigator.userAgent) ? 'mobile' : 'desktop',
  });

  if (navigator.sendBeacon) {
    navigator.sendBeacon('/api/vitals', body);
  } else {
    fetch('/api/vitals', { body, method: 'POST', keepalive: true });
  }
}

onINP(sendToAnalytics);
onLCP(sendToAnalytics);
onCLS(sendToAnalytics);

2. Slice the Data

A single aggregate INP number hides everything. The same site can have great INP for desktop users in Germany and awful INP for mobile users in Brazil. The aggregate looks "mid." Slice by route, device class, and country to find the real fires.

3. Set Alerts at 80% of Thresholds

Don't wait until you're failing. Set alerts at:

INP at 160ms (80% of 200ms threshold)
LCP at 2.0s (80% of 2.5s threshold)
CLS at 0.08 (80% of 0.1 threshold)

A deploy that bumps INP from 150ms to 190ms still reports "good," but three of those in a month and you're failing.

FAQ: Common Gotchas

Q: I fixed INP on my local machine but CrUX still shows poor. Why?
A: CrUX is a 28-day rolling window. Your fix won't fully appear in the data for up to 4 weeks. Check your own real-user monitoring pipeline for immediate validation.

Q: scheduler.yield() isn't available in my target browsers.
A: The setTimeout(resolve, 0) fallback in the yieldToMain() function above handles this. It's slightly less optimal than native scheduler.yield() but still delivers the core benefit of breaking up long tasks.

Q: Do I need to preload every image on the page?
A: No — only the LCP element. Preloading too many resources creates contention and can actually hurt performance. One preload for one LCP resource.

Q: My CLS is caused by a third-party chat widget. How do I fix that?
A: Reserve space for it. Add min-height to the container where the widget renders. If the widget doesn't load, the empty space is better than the shift.

Q: Is it worth optimizing for Core Web Vitals if my site already ranks well?
A: Yes. Google has confirmed Core Web Vitals are a ranking signal. Beyond SEO, better vitals directly correlate with lower bounce rates and higher conversion rates. Performance is a product feature.

The Three Things I'd Tell You on Day One

Stop optimizing for Lighthouse. Use it as a diagnostic, not a scoreboard. The scoreboard lives at CrUX, and the gap between the two can be an order of magnitude.
Fix the metric that's actually failing. Don't work on LCP because you find it satisfying while INP silently tanks. Look at your CrUX data, find the worst metric, and start there.
Treat performance as a product feature. The teams that succeed at Core Web Vitals are the ones where someone owns the metrics, they're reviewed weekly, regressions are treated as bugs, and they're part of the definition of done. The teams that fail are the ones that treat vitals as "something to get to after the next launch."

Your users don't care about your Lighthouse score. They care about whether the button responds when they tap it. Fix that.

If this helped, follow for more on frontend performance, architecture, and the unglamorous parts of shipping software.

Stop Defaulting to WebSockets: A Practical Guide to SSE, Polling, and Knowing When You Actually Need Them

AttractivePenguin — Tue, 28 Apr 2026 09:05:38 +0000

Stop Defaulting to WebSockets: A Practical Guide to SSE, Polling, and Knowing When You Actually Need Them

You've been there. A ticket lands in your sprint: "Make the dashboard real-time." Your brain immediately jumps to WebSockets. Open a persistent connection, push data to the client, done — right?

Not so fast.

In my experience (and the experience of plenty of devs who've debugged WebSocket auth issues at 2 AM), WebSockets are overkill for most "real-time" features. Notifications, live feeds, progress bars — these are one-way pushes from server to client. You don't need a two-way persistent connection for that.

This article gives you a practical decision framework and working code for the three main approaches — WebSockets, Server-Sent Events (SSE), and polling — so you can stop reaching for the heaviest tool and start picking the right one.

The One Question That Decides Everything

Before you write a single line of code, answer this:

Does the client need to send data back to the server through the same connection?

No → You probably don't need WebSockets. SSE or polling will do.
Yes, and it's frequent/continuous → WebSockets are the right call.

That's it. That's the framework. Let me show you why, with code.

Approach 1: Polling — The Underrated Workhorse

Polling gets a bad reputation. Developers treat it like the "amateur" option you eventually graduate away from. But for a lot of features, it's genuinely the best choice.

When to Use Polling

Updates every 30 seconds or more are acceptable
You have few concurrent users checking the same resource
You want maximum simplicity and debuggability
Your data changes infrequently (e.g., a job status that updates every minute)

The Code

Client-side, it's laughably simple:

async function checkJobStatus(jobId) {
  const res = await fetch(`/api/jobs/${jobId}/status`);
  const { status, progress } = await res.json();
  updateProgressBar(progress);
  return status;
}

// Poll every 15 seconds until the job is done
const intervalId = setInterval(async () => {
  const status = await checkJobStatus('job_123');
  if (status === 'completed' || status === 'failed') {
    clearInterval(intervalId);
  }
}, 15_000);

Server-side is just a regular endpoint:

// Express.js
app.get('/api/jobs/:id/status', async (req, res) => {
  const job = await JobRepository.findById(req.params.id);
  res.json({ status: job.status, progress: job.progress });
});

Why It Works

Zero persistent connections — the server doesn't hold state between requests
Auth is trivial — regular HTTP, cookies and headers just work
Debugging is easy — open the Network tab, see the requests, done
Works everywhere — no browser compatibility concerns, no proxy issues

The Drawback

If you need sub-second updates or have thousands of clients hammering the same endpoint, polling wastes resources. That's where SSE shines.

Approach 2: Server-Sent Events — The Sweet Spot

SSE is the tool most developers overlook. It gives you server-push without the complexity of WebSockets, and it's built into the browser — no libraries needed.

When to Use SSE

The server needs to push updates to the client
The client does NOT need to send data back through the same channel
You want automatic reconnection handled for you
You're building: notifications, live feeds, progress updates, stock tickers

The Code

Client-side is three lines:

const stream = new EventSource('/api/notifications/stream');

stream.onmessage = (event) => {
  const data = JSON.parse(event.data);
  updateNotificationBadge(data.unreadCount);
};

// The browser auto-reconnects on disconnect. That's part of the spec.

Server-side with Express:

app.get('/api/notifications/stream', (req, res) => {
  // Set SSE headers
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');

  // Send initial data
  res.write(`data: ${JSON.stringify({ unreadCount: 5 })}\n\n`);

  // Push updates when they happen
  const onNewNotification = (notification) => {
    res.write(`data: ${JSON.stringify(notification)}\n\n`);
  };

  notificationEmitter.on('new', onNewNotification);

  // Clean up on disconnect
  req.on('close', () => {
    notificationEmitter.off('new', onNewNotification);
  });
});

Named Events for Structured Data

SSE supports event types, which lets you handle different update kinds without parsing:

// Server
res.write(`event: message\ndata: ${JSON.stringify({ text: 'Hello' })}\n\n`);
res.write(`event: badge\ndata: ${JSON.stringify({ count: 3 })}\n\n`);

// Client
const stream = new EventSource('/api/stream');
stream.addEventListener('message', (e) => {
  // Handle chat messages
});
stream.addEventListener('badge', (e) => {
  // Handle badge updates
});

SSE with Node.js Streams (Production Grade)

For production, use proper stream handling and heartbeats to keep connections alive through proxies:

import { PassThrough } from 'stream';

app.get('/api/events', (req, res) => {
  const stream = new PassThrough();
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  stream.pipe(res);

  // Send a heartbeat every 30s to keep the connection alive through proxies
  const heartbeat = setInterval(() => {
    stream.write(':heartbeat\n\n'); // Comments are ignored by EventSource
  }, 30_000);

  // Push real events
  const onEvent = (data) => {
    stream.write(`data: ${JSON.stringify(data)}\n\n`);
  };

  eventBus.on('update', onEvent);

  req.on('close', () => {
    clearInterval(heartbeat);
    eventBus.off('update', onEvent);
    stream.end();
  });
});

Why SSE Beats WebSockets for Server-Push

Auth just works — it's HTTP, so cookies, Authorization headers, and middleware all compose naturally
Auto-reconnection — the browser's EventSource reconnects on its own, including sending the Last-Event-ID header so the server can resume from where it left off
Simpler server — each connection is a regular HTTP response, not a stateful socket you have to manage
Proxy-friendly — corporate proxies and load balancers handle long-lived HTTP responses better than WebSocket upgrade requests
No framing protocol — you write text, the browser parses it. No binary framing, no opcode handling

The Limitation

It's one-way. The client can only receive. When the user takes an action, you send a regular fetch() POST. For notifications and feeds, that's all you need.

Approach 3: WebSockets — When You Actually Need Them

WebSockets are the right tool when both of these are true:

The client sends data frequently (not just occasional clicks)
Low-latency, continuous communication is required in both directions

Real Use Cases Where WebSockets Win

Use Case	Why WebSocket
Chat application	Sending and receiving messages constantly
Collaborative editing	Multiple users typing simultaneously, real-time cursor sync
Multiplayer games	Continuous state sync in both directions
Live trading dashboard	Sub-millisecond updates + user orders going out

The Code (with ws library)

Server:

import { WebSocketServer } from 'ws';

const wss = new WebSocketServer({ port: 8080 });

wss.on('connection', (ws, req) => {
  // ⚠️ Auth is NOT automatic — you have to handle it yourself
  const token = new URL(req.url, 'http://localhost').searchParams.get('token');
  if (!verifyToken(token)) {
    ws.close(4001, 'Unauthorized');
    return;
  }

  ws.on('message', (data) => {
    const message = JSON.parse(data);
    // Broadcast to all connected clients
    wss.clients.forEach((client) => {
      if (client.readyState === WebSocket.OPEN) {
        client.send(JSON.stringify(message));
      }
    });
  });
});

Client:

let ws;
let reconnectAttempts = 0;

function connect() {
  ws = new WebSocket('ws://localhost:8080?token=your-auth-token');

  ws.onopen = () => {
    reconnectAttempts = 0;
    console.log('Connected');
  };

  ws.onmessage = (event) => {
    const data = JSON.parse(event.data);
    handleMessage(data);
  };

  // You must implement your own reconnection logic
  ws.onclose = () => {
    const delay = Math.min(1000 * 2 ** reconnectAttempts, 30_000);
    setTimeout(connect, delay);
    reconnectAttempts++;
  };
}

connect();

What You're Signing Up For

Manual reconnection — the browser doesn't auto-reconnect WebSocket. You write that code yourself.
Manual auth — no cookies on upgrade, no automatic headers. You typically pass tokens in query params or handle auth after connection.
Connection state management — you need to track whether the socket is CONNECTING, OPEN, CLOSING, or CLOSED, and show appropriate UI.
Memory cost — every connected client holds a persistent connection in server memory. 10,000 idle connections = 10,000 open sockets consuming RAM.
Proxy/CDN issues — some corporate proxies and CDNs don't support WebSocket upgrades. You'll need fallback transports.

None of this is unmanageable, but it's significant overhead if you're building a notification badge.

Decision Framework: Pick in 60 Seconds

Is the communication bidirectional and high-frequency?
├── YES → Use WebSockets
│   Examples: chat, collaborative editing, games
│
└── NO → Are updates needed faster than every 30 seconds?
    ├── YES → Use Server-Sent Events
    │   Examples: notifications, live feeds, progress bars
    │
    └── NO → Use Polling
        Examples: job status, background task progress

Here's a more detailed comparison:

Factor	Polling	SSE	WebSockets
Direction	Client → Server	Server → Client	Bidirectional
Auth	Automatic (HTTP)	Automatic (HTTP)	Manual
Reconnection	N/A (stateless)	Automatic (browser spec)	Manual
Complexity	Minimal	Low	High
Latency	15-60s typical	Near-instant	Instant
Server memory	None	Low (HTTP response)	High (persistent socket)
Browser support	Universal	Universal (IE6+)	Universal (IE10+)
Proxy/CDN friendly	Yes	Yes	Sometimes

Real-World Scenario: Building a Notification System

Let's walk through a concrete example. You're building a notification center for a SaaS app:

Requirements:

Show unread count badge in the header
Push new notifications as they arrive
User can mark as read (occasional action)

The Wrong Approach: WebSocket

// Setting up WebSocket for... a notification badge
const ws = new WebSocket('wss://api.example.com/notifications');

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  updateBadge(data.unreadCount);
};

// When user marks as read
document.getElementById('mark-read').addEventListener('click', () => {
  ws.send(JSON.stringify({ action: 'mark_read', id: notificationId }));
});

This works, but you've now committed to: managing reconnection, handling auth separately, tracking connection state, and consuming server memory for every idle user with a tab open.

The Right Approach: SSE + fetch

// Notifications come in via SSE (server pushes to client)
const stream = new EventSource('/api/notifications/stream?token=xxx');

stream.onmessage = (event) => {
  const data = JSON.parse(event.data);
  updateBadge(data.unreadCount);
  addNotificationToFeed(data.notification);
};

// Marking as read is a simple POST (client sends to server)
document.getElementById('mark-read').addEventListener('click', () => {
  fetch(`/api/notifications/${notificationId}/read`, { method: 'POST' });
});

Same UX. Half the complexity. Auto-reconnection. Auth through headers. Your load balancer understands it.

FAQ

Isn't polling wasteful?

For high-frequency updates, yes. But if you're checking job status every 30 seconds and you have 100 concurrent users, that's ~3 requests/second total. Your server can handle that in its sleep. The waste is relative to your actual requirements.

Can SSE send binary data?

No, SSE is text-only (UTF-8). If you need binary frames, use WebSockets.

What about HTTP/2 Server Push?

HTTP/2 Server Push is about pushing assets (JS, CSS) alongside the initial request. It's not for real-time data streams. Don't confuse it with SSE.

Do SSE connections survive behind nginx/Cloudflare?

Yes, but you need to configure nginx to not buffer the response:

location /api/events {
    proxy_pass http://backend;
    proxy_http_version 1.1;
    proxy_set_header Connection '';
    proxy_buffering off;
    proxy_cache off;
    chunked_transfer_encoding off;
}

Without proxy_buffering off, nginx will hold the SSE stream and deliver it all at once when the connection closes — defeating the purpose entirely.

What if I need both server-push and occasional client sends?

That's the SSE + fetch pattern. Use SSE for the push channel and regular fetch() for client actions. It sounds like two connections, but the fetch is ephemeral — it opens, sends, closes. No persistent state to manage.

Can I use SSE with React?

Absolutely:

function useSSE(url) {
  const [data, setData] = useState(null);

  useEffect(() => {
    const source = new EventSource(url);
    source.onmessage = (event) => setData(JSON.parse(event.data));

    return () => source.close(); // Clean up on unmount
  }, [url]);

  return data;
}

// Usage
function NotificationBadge() {
  const data = useSSE('/api/notifications/stream');
  return <span className="badge">{data?.unreadCount ?? 0}</span>;
}

Conclusion

WebSockets are powerful, but they're a power tool. You wouldn't use a sledgehammer to hang a picture frame. Most "real-time" features are one-directional — the server pushes, the client receives. For those cases:

Polling if updates every 15-60 seconds are fine. It's simple, debuggable, and works everywhere.
SSE if you need near-instant server-push. It's HTTP, auth works, reconnection is automatic, and the code is a fraction of what WebSocket requires.
WebSockets when you have genuine bidirectional, high-frequency communication: chat, collaborative editing, games.

The next time someone says "we need real-time," don't default to WebSockets. Ask: does the client need to send data back through the same connection? If not, reach for SSE. Your 2 AM debugging self will thank you.

AI Burnout: Do I Still Belong in Tech?

AttractivePenguin — Mon, 27 Apr 2026 08:33:42 +0000

AI Burnout: "Do I Still Belong in Tech?"

A developer's honest reckoning with the identity crisis no one wants to admit they're having

Something shifted recently, and developers are finally saying it out loud.

"Do I belong in tech anymore?"

That question — raw, exhausted, a little desperate — hit the front page of Lobsters recently with 198 points and 63 comments. Meanwhile, Hacker News surfaced its own version: "AI should elevate your thinking, not replace it," which pulled 414 points and hundreds of replies. Two posts, same wound.

This isn't imposter syndrome. It's something newer and harder to name: the quiet grief of watching a skill you spent years building become a prompt.

The Old Bargain

There used to be an unspoken deal in software development: learn the hard stuff, do the time, and you'd earn both a livelihood and a kind of craftsman's pride. You'd know things. You'd be good at things. The expertise was yours.

That deal isn't gone — but it's wobbling.

When you can type "build me a REST API with authentication, error handling, and tests" and get 80% of the way there in thirty seconds, something changes. Not just the workflow. The feeling of doing the work. The thing you were proud of knowing how to do from scratch? Increasingly a prompt away for anyone with a laptop.

A lot of developers are reacting to this with a very human emotion: grief. And then they feel embarrassed about the grief, because it sounds like complaining that things are getting easier. So they don't say it. Until now, apparently.

What the Burnout Actually Feels Like

Let me name the specific flavors, because "AI burnout" is a catch-all for a few distinct experiences:

The Deskilling Anxiety: You stop writing certain types of code by hand, then one day realize you've forgotten how. Not retired the knowledge — forgotten it. This one hits differently for people who built their confidence on depth.

The Prompt-Engineering Treadmill: You spend your day doing this new kind of work — wrangling AI output, writing better prompts, reviewing generated code, fixing the subtle bugs the model introduced. It's cognitively demanding. But it doesn't feel like the work you signed up for. Some people find it engaging. Many find it draining in a way they can't quite articulate.

The Expertise Deprecation Shock: You spent three years getting really good at, say, CSS grid layout, or Kubernetes configuration, or writing clean state management logic. The models are now competent at all of it. That competence isn't worthless — but it's less differentiating. The moat you dug feels shallower.

The Identity Collapse: This is the deepest one. "I'm a developer" used to mean something specific about what you could do. That identity is now in flux. If a model can generate the code, what does it mean to be the developer?

The Psychology of Craft and Identity

Here's something worth naming plainly: developer identity is unusually tied to technical capability. More so than most professions.

A project manager doesn't typically define themselves by their ability to manually write Gantt charts. But developers often define themselves by their ability to write code — specific kinds of code, in specific ways. It's a craft identity, in the old guild-artisan sense. That's not a weakness; it's what makes good developers care so much about quality.

When that craft gets automated — even partially — it attacks something psychological, not just professional.

Research on creativity and human flourishing suggests this isn't irrational. Autonomy, mastery, and purpose are among the core drivers of intrinsic motivation. AI tools can cut directly into all three: autonomy (the AI decides structure), mastery (the skill gap closes overnight), purpose (am I doing meaningful work, or just reviewing AI output?).

This is real. It's not a failure of attitude or adaptability. It's a genuine identity threat that the industry is handling with a lot of denial and a lot of blog posts about "prompt engineering as the new programming."

What Actually Helps (and What Doesn't)

Let's get concrete. I've talked to developers across the spectrum — those thriving in the transition and those genuinely struggling.

What doesn't help:

"Just learn to use AI tools better" — True but dismissive. This treats the burnout as a skills gap when it's often an identity gap.

"Focus on the high-level work" — Also true, but for many developers, the "low-level work" was the high-level work they cared about. Systems programming, performance optimization, careful algorithm design — these don't feel like grunt work to the people who love them.

"AI will create new jobs" — Probably true in aggregate over a decade. Unhelpful if you're 35 and burning out right now.

What actually helps:

Distinguish replacement from amplification deliberately. Not "AI does this now" but "what is AI doing to this specific type of work I do?" Some things are genuinely being automated away. Others are being amplified — you can go deeper, move faster, tackle harder problems. Knowing which is which matters.

A useful heuristic I've seen work:

If the value was in PRODUCING the artifact → AI threat
If the value was in UNDERSTANDING the domain → AI amplifier

Writing CRUD boilerplate: production value. Understanding the data model it represents: domain value. The first is largely automated. The second is not.

Build work that requires judgment over time. AI models are excellent at well-defined, bounded tasks. They struggle with problems that require accumulated context, fuzzy organizational knowledge, and judgment calls that only make sense if you've seen how this specific system breaks. That's where experienced developers still have enormous leverage.

Let yourself grieve. Seriously. The industry wants everyone to immediately pivot to "and here's what's exciting!" If you're mourning the loss of a kind of work you loved, you're allowed to sit in that for a minute. The developers I've seen transition best are often the ones who acknowledged the loss instead of skipping past it.

The Harder Question

There's a version of this where we say "AI will elevate your thinking" and move on. It sounds right. It might even be true for many developers. But it sidesteps the genuine class question underneath:

Not all developers are in the same position.

A senior engineer at a well-funded company, with leverage and specialized knowledge and a team that values depth? Probably fine. The transition is uncomfortable but manageable.

A mid-level developer in a commoditizing market, doing the kind of work that models are now very good at, at a company looking to cut costs? The math looks different.

AI optimism that doesn't reckon with this is selective optimism. And developers can smell it.

The industry tends to generate advice aimed at the first group and pretend it applies to everyone. It doesn't.

What I Actually Believe

After sitting with this for a while: the developers asking "do I still belong in tech?" are not weak or behind or failing to adapt. They're having a reasonable reaction to a genuinely disruptive transition that's happening faster than most people expected.

Some of them will thrive in the new landscape. Some will find that the parts of software development they actually loved have been narrowed, and they'll need to decide whether the remaining work is worth it to them. Some will pivot. Some will double down on the craft work and find there's still room for it.

All of those are valid outcomes. What's not helpful is pretending the transition is costless, or that discomfort is just a skills gap.

The developers who are thriving aren't the ones who suppressed the discomfort — they're the ones who took it seriously enough to figure out what they actually valued about the work, then found ways to protect or rebuild that value in a changed landscape.

That's not a simple process. It takes time. It deserves more honest conversation than the industry is currently having.

A Framework for Moving Forward

If you're in the thick of this, here's a rough framework I've found useful:

1. Audit what you actually value about your work.
Not what you're supposed to value. What actually makes you feel good at the end of a day? Is it the problem-solving? The craft of the code itself? The systems thinking? The collaboration? Be honest.

2. Check whether AI competes with or amplifies that thing.
Sometimes it's not either/or — it competes with the boring parts and amplifies the interesting ones. But you need to look clearly at this, not assume it works out fine.

3. Make deliberate choices about what to protect.
Some people decide they still want to write certain code by hand, even when they don't have to, because the act matters to them. That's a legitimate choice. So is going all-in on AI tooling. What's not sustainable is passive drift in either direction.

4. Find your tribe.
The people having this conversation honestly are out there. The denial-and-positivity crowd is louder, but there are developers sitting with the same questions you are. Finding them matters more than it sounds like it would.

The question "do I belong in tech anymore?" deserves a real answer, not a pep talk.

The honest answer is: that depends on what your tech is, what it means to you, and what you're willing to build from here.

That's harder than "yes of course you do" — but it's more useful.

Have you navigated AI-driven burnout? I'd genuinely like to hear how. Drop it in the comments.

"Do I Belong in Tech Anymore?" — AI, Vibe Coding, and the Developer Identity Crisis

AttractivePenguin — Sun, 26 Apr 2026 08:33:48 +0000

"Do I Belong in Tech Anymore?" — AI, Vibe Coding, and the Developer Identity Crisis

The community data is in. Developers aren't just worried about their jobs — they're having an existential crisis about who they even are.

Three separate posts hit the front page of Lobsters in a single 24-hour window last week. All three were about the same thing: whether human coding still matters in 2026. The top one scored 146. The comment sections were a warzone.

Meanwhile, OpenMythos — a GitHub repo where developers are collaboratively reverse-engineering Claude's architecture from public artifacts — hit 10,458 stars in eight days. Eight days.

These two data points tell you everything you need to know about where we are. Developers aren't passively accepting the AI future being handed to them. They're anxious, they're angry, they're fascinated, and they're dissecting the systems being deployed against them with their own bare hands.

This is the developer identity crisis of 2026. And it's real.

What "Vibe Coding" Actually Means (And Why It Stings)

The term "vibe coding" started as a half-jokey description of prompting AI agents to write code while you mostly steer direction and vibe out. Karpathy popularized it. The discourse exploded.

Here's why it landed so hard: it perfectly captures the ambiguity of what a lot of developers are actually doing now.

Are you engineering when you write a system prompt, review the output, and ship it? Are you coding when you describe what you want to a coding agent, approve its diffs, and merge? The old answer was obvious. Now it isn't.

And the community hates not having a clean answer.

The Lobsters thread with score 84 and 49 comments was headlined something like "The People Do Not Yearn For Automation." It wasn't about AI replacing jobs in the abstract — it was about whether the Mac App Store is now flooded with AI-slop apps nobody wanted, built by people who wanted the output of development without the craft of it. The replies were split almost perfectly in half.

Half the developers: "This is democratization. More people making things is good."

Half: "This is dilution. It devalues what we spent years learning."

Both sides are right. That's the problem.

The Three Kinds of Developers Right Now

After watching these community threads for months, I think there are roughly three camps:

1. The Accelerationists

These are the developers who have genuinely integrated AI into their workflow and feel more productive than ever. They've internalized the new stack: agent + spec + review loop. They're shipping faster. They think the identity crisis discourse is cope from people who are bad at adapting.

They have a point. If you're building products and getting things done, the methodology is secondary. Results matter.

2. The Preservationists

These are the developers who believe the craft of writing code — actually thinking in code — produces qualitatively better outcomes that you cannot get from steering an AI. They're not Luddites. They use AI tools. But they believe there's something being lost when the primary skill becomes prompt engineering instead of algorithmic thinking.

They also have a point. There's a reason OpenMythos exists and has 10k stars. Developers want to understand the systems they're working with at a deep level. That instinct doesn't go away just because the tooling changed.

3. The Confused Majority

This is most developers. They're using Copilot or Claude Code. Sometimes it feels like cheating. Sometimes it feels like the future. They're not sure if they should be learning Rust or learning prompt engineering. They applied for three jobs last month and two of the listings had "AI-native" in the requirements without defining what that means.

They're not having an identity crisis in a dramatic sense. They're just... uncertain. And uncertainty at scale looks a lot like a crisis.

The Burnout Angle Nobody Is Talking About

Here's the thing that's getting buried under the philosophical debates: AI-assisted development is tiring in a different way than regular development.

When you write code yourself, the cognitive load is high but the feedback loop is intimate. You understand what you built. You can hold it in your head.

When you're managing an AI coding agent across a large codebase, you're doing something more like... editing and fact-checking and architectural review, simultaneously, at speed. You're trusting outputs you didn't produce. You're reading diffs you didn't write. You're context-switching between "does this do what I asked?" and "is this the right thing to ask for?"

It's a different kind of exhaustion. And the tooling hasn't caught up with the cognitive demands.

Traditional dev fatigue:
  → Deep focus → flow state → solution → satisfaction

AI-assisted dev fatigue:
  → Brief prompt → review → correction → re-prompt → review 
  → trust gap → review again → merge with mild unease → repeat

The trust gap is the thing. Every experienced developer I've talked to who works heavily with AI agents describes some version of it: a low-level background hum of "did I actually understand what just got shipped?" You can minimize it with good practices, but you can't eliminate it entirely. Not yet.

OpenMythos and the Developer Counter-Response

The OpenMythos repo is the most interesting signal in all of this, and it's not getting enough coverage.

If developers were truly comfortable with AI as a black box, nobody would spend their spare time reverse-engineering its architecture from conference papers, blog posts, and behavioral tests. But 10,000+ developers starred a collaborative project to do exactly that.

This is what developer culture does when it feels disempowered: it investigates. It builds. It makes the opaque legible.

The old guard did this with operating systems. They did it with compilers. They did it with browsers. Now they're doing it with the AI systems that are reshaping their work.

That's not cope. That's craft. And it's a good sign.

The Practical Question: What Should You Actually Do?

If you're in the confused majority and looking for something actionable, here's my honest take:

Stop trying to decide if AI coding is "real" coding. The question is philosophically interesting and practically useless. What matters is whether you can build things that work and maintain them over time.

Get opinionated about your AI usage. The developers who seem least burned out are the ones who have clear personal policies: "I use AI for boilerplate and tests, I write core logic myself" or "I always read every line before merging, no exceptions." Ambiguity about your own process is cognitively expensive.

Learn something AI is bad at. Right now that includes: deeply understanding existing large codebases, debugging weird emergent system behavior, making good architectural decisions with incomplete information, and interpersonal engineering work (design reviews, technical mentorship, stakeholder communication). These are all durable skills.

The fundamentals still compound. Understanding how memory allocation works, how the network stack behaves, how databases make query plans — this knowledge makes you better at using AI tools, not worse. The developers who use AI most effectively tend to have strong fundamentals. That's not a coincidence.

Stop doom-scrolling the discourse. The Lobsters and HN threads are useful data about community sentiment but they're a terrible guide to career decisions. The people who are most vocal about the end of developer jobs are rarely the ones who'd be hired over you anyway.

What "Belonging in Tech" Actually Means in 2026

The developers asking "do I belong in tech anymore?" are almost always asking a subtler question: does the thing I'm good at still matter?

For most of them, the honest answer is yes — but the way it matters is shifting. Systems thinking, debugging intuition, architectural judgment, clear technical communication: none of that is going away. What's going away is the premium on typing syntax quickly and knowing library APIs from memory.

That's a real change. It's okay to grieve it a little. The feeling that you spent years mastering something and the rules just changed — that's legitimate. Don't let anyone tell you it isn't.

But here's what the OpenMythos crowd already figured out: the rules changed, so now we study the new rules. We understand the new systems. We build literacy around the new tools the same way we built it around the old ones.

The developers who thrive in the next five years won't be the ones who were earliest to embrace AI or the ones who resisted it longest. They'll be the ones who stayed curious enough to actually understand what these systems do — and skeptical enough to know when not to trust them.

That's always been the job.

One More Thing

The dev identity crisis isn't unique to this moment. Every major platform shift — web, mobile, cloud — triggered the same questions. "Is this still real programming?" "Does my expertise still count?" The developers who asked those questions and kept building are still here. The ones who stopped building while waiting for a definitive answer mostly aren't.

The question "do I belong in tech?" has always had the same answer: if you keep showing up and making things, yes. The tools change. The belonging is up to you.

Got a take on this? Drop it in the comments — I'm genuinely curious where other developers are landing on the burnout question specifically.

Supply Chain Attack Hits Bitwarden CLI via npm: What Every Developer Must Know Right Now

AttractivePenguin — Sat, 25 Apr 2026 08:31:24 +0000

The password manager you trust with your secrets almost became the delivery vector for stealing them.

Earlier this week, security researchers at Checkmarx disclosed an active supply chain attack targeting the Bitwarden CLI via a malicious npm package. It hit close to home for a lot of developers — not just because Bitwarden is widely used, but because it exposes exactly how fragile our toolchain trust really is. This isn't a theoretical attack. It's real, it's ongoing, and if you haven't audited your dependencies yet, stop reading, open a terminal, and do it first.

Done? Good. Now let's talk about what actually happened, why it worked, and what you need to do to not be the next victim.

What Happened

The attack combined two classic supply chain techniques: typosquatting and dependency confusion.

Typosquatting

Attackers published a package called @bitwarden/cli — notice the scoped package under the Bitwarden namespace. The legitimate Bitwarden CLI package is @bitwarden/cli, so at first glance they look identical. But the malicious version was uploaded to npm using a slightly different account, and for a brief window, it appeared in search results alongside (or in place of) the legitimate one.

# Legitimate package
npm install @bitwarden/cli

# Malicious package (visually identical in some search results)
npm install @bitwarden/cli  # attacker-controlled account

The delta was in the package metadata, not the name. Without scrutinizing the maintainer account, you'd never know.

Dependency Confusion

The second vector was more insidious. Several internal tooling configurations were identified where bitwarden-cli (note: no scope) was listed as a dependency — presumably an internal or legacy convention. npm's resolution order means a public registry package with that name wins over an internal one if the version number is higher. Attackers published a public bitwarden-cli package with an inflated version (99.9.9), and any CI pipeline pulling dependencies automatically would silently install the attacker's version.

{
  "dependencies": {
    "bitwarden-cli": "^1.0.0"
  }
}

In a confused resolution scenario, npm would see 99.9.9 publicly available and consider it a match for ^1.0.0. Your lockfile would update. Your secrets would be at risk.

The Payload

Checkmarx confirmed the malicious packages contained a post-install hook — the go-to execution vector because it runs automatically when you npm install. The hook reached out to an external command-and-control server, exfiltrating environment variables and any secrets accessible to the install process.

{
  "scripts": {
    "postinstall": "node ./scripts/setup.js"
  }
}

setup.js, needless to say, was not setting anything up for you.

Why This Keeps Working

Every few months we see another supply chain attack, and every time the same discussion plays out: "Why does npm let this happen?" But blaming npm misses the point.

The root issue is implicit trust. When you run npm install, you're executing arbitrary code written by strangers, on your machine, with your credentials in scope. We've normalized this. We shouldn't have.

The Bitwarden attack worked because:

The namespace was confusable. Scoped packages on npm offer some protection, but they don't prevent account impersonation or namespace-adjacent confusion.
Postinstall hooks are a loaded gun. They run automatically with no prompt, no review, no opt-in.
Lockfiles were stale or absent. A committed, integrity-checked lockfile would have caught the version substitution immediately.
No one was watching the registry. Tools like Socket.dev and Snyk's reachability analysis exist precisely to flag this. Most teams aren't using them.

What Bitwarden Did

Bitwarden moved quickly. Within hours of disclosure:

The malicious packages were reported to npm and removed
Bitwarden published a security advisory confirming the scope of the attack
They rotated their own publishing keys and audited maintainer access
A Canary token was deployed to detect any further exploitation attempts

Credit where it's due — this was a good incident response. But "good incident response" means the fire got contained after it started. The prevention side is what needs work, and that's on all of us, not just Bitwarden.

What You Need to Do Right Now

Here's the actionable checklist. Copy this into your incident response runbook.

1. Audit Your Bitwarden CLI Usage

# Check if you have any bitwarden-related packages installed
npm ls | grep -i bitwarden
npx audit-ci --moderate

# Check your global installs too
npm list -g | grep -i bitwarden

If you find anything, verify the package version and maintainer against Bitwarden's official security advisory before doing anything with it.

2. Pin Your Dependency Versions

Stop using ^ and ~ range specifiers for anything that touches production or CI environments. Pin exact versions.

{
  "dependencies": {
    "@bitwarden/cli": "2025.1.0"
  }
}

Ranges are convenient for local dev. They're a liability in production.

3. Commit and Enforce Lockfiles

Your package-lock.json or yarn.lock is your source of truth. Treat it like code.

# npm — verify integrity against lockfile
npm ci  # NOT npm install

# Enforce in CI: fail if lockfile would change
npm ci --audit

npm ci exits non-zero if the lockfile doesn't match package.json. Use it in every CI pipeline. Not npm install. npm ci.

4. Enable npm Audit in CI

# GitHub Actions example
- name: Install dependencies
  run: npm ci

- name: Run security audit
  run: npm audit --audit-level=moderate

This catches known vulnerabilities. It won't catch brand-new malicious packages, but it's table stakes.

5. Add Socket.dev or Snyk to Your Pipeline

These tools go beyond npm audit by analyzing supply chain risk in real time — flagging packages with new maintainers, unusual install scripts, or suspicious network calls.

# Socket.dev CLI
npx socket check @bitwarden/cli

# Snyk
snyk test --all-projects

Socket.dev specifically detects postinstall scripts that make outbound network calls — exactly what this attack used. If you're not running something like this, you're flying blind.

6. Audit Your Internal Package Names

If your organization uses private packages, make sure every internal package name either:

Uses a private registry (Verdaccio, Artifactory, GitHub Packages) with explicit scope resolution
Is scoped under your own npm org namespace

Update your .npmrc to enforce registry routing:

@yourcompany:registry=https://your-private-registry.example.com

And add a publishConfig to every internal package to prevent accidental public publishing:

{
  "publishConfig": {
    "registry": "https://your-private-registry.example.com"
  }
}

This won't fully prevent dependency confusion, but it drastically reduces the attack surface.

7. Restrict Postinstall Scripts in Production

npm lets you disable lifecycle scripts:

npm install --ignore-scripts

Yes, some legitimate packages need postinstall hooks. But in a CI/CD context, you can often disable them safely — especially for packages you've already vetted locally. Run your normal install locally, commit the lockfile, and use --ignore-scripts in your pipeline.

The Bigger Picture

We're in the golden age of supply chain attacks and the ecosystem is not keeping up. The npm registry processes millions of packages and has essentially no verification gatekeeping. Anyone can publish. Anyone can typosquat. The only real defense is layered vigilance on your end.

Some uncomfortable truths:

Lockfiles are not optional. If your repo has no committed lockfile, you're one npm install on a fresh machine away from installing whatever attackers have promoted to the top of version resolution.

CI should never have broad secret access. The Bitwarden attack targeted environment variables. If your CI runner has AWS_SECRET_ACCESS_KEY, DATABASE_URL, and your API tokens all in scope during the install phase, an attacker who compromises one dependency gets all of it. Scope your secrets. Inject them only where needed.

"It's open source, someone would have noticed" is dead. XZ Utils happened. This happened. The assumption that the community will catch everything is naïve at scale.

The Incident Response Runbook (Paste-Ready)

## Supply Chain Attack Response Checklist

### Immediate (first 30 minutes)
- [ ] Identify affected packages from security advisory
- [ ] Run `npm ls | grep <package>` across all repos
- [ ] Freeze deploys until audit is complete
- [ ] Rotate any secrets accessible during install phase (env vars, tokens, keys)

### Short-term (first 24 hours)
- [ ] Audit CI logs for abnormal network calls during install steps
- [ ] Pin exact versions for affected and adjacent packages
- [ ] Run `npm ci` to regenerate lockfile from pinned versions
- [ ] Enable npm audit in all pipelines if not already present

### Medium-term (first week)
- [ ] Add Socket.dev or Snyk to CI pipelines
- [ ] Review all packages with postinstall scripts: `cat node_modules/*/package.json | jq 'select(.scripts.postinstall) | .name'`
- [ ] Audit internal package names for dependency confusion risk
- [ ] Configure .npmrc to scope internal packages to private registry

### Ongoing
- [ ] Subscribe to security advisories for your top 20 dependencies
- [ ] Use `npm ci` (not `npm install`) in all CI environments
- [ ] Review and rotate CI secrets on a quarterly schedule

The Bitwarden CLI attack is a wake-up call, not an anomaly. Supply chain attacks are the dominant threat vector in developer tooling right now, and the npm ecosystem is a particularly juicy target precisely because developers trust it implicitly.

Lock your dependencies. Audit your CI. Use tooling that watches the registry for you. The cost of implementing these controls is a few hours. The cost of not implementing them is your infrastructure, your secrets, and your users' data.

That's not a tradeoff worth making.

JSON Canvas: How to Build Interoperable Infinite Canvas Apps with an Open Format

AttractivePenguin — Tue, 21 Apr 2026 09:10:57 +0000

JSON Canvas: How to Build Interoperable Infinite Canvas Apps with an Open Format

Infinite canvas tools have exploded in popularity. Miro, Figma, Obsidian Canvas, Excalidraw — the spatial, freeform way of organizing information has become a dominant interface pattern. But there's a problem lurking beneath the surface: lock-in. Every canvas app stores your whiteboards, mind maps, and visual notes in its own proprietary format. Move to a different tool? Your data stays behind or arrives as a mangled export.

JSON Canvas is an open file format created to solve this. Originally built for Obsidian, it's now MIT-licensed and free for any application to adopt. In this article, we'll walk through the spec, implement a working parser and renderer, and explore real-world scenarios where adopting JSON Canvas can future-proof your application.

Why JSON Canvas Matters

The philosophy behind JSON Canvas is simple but powerful: your data belongs to you. When canvas files are stored in a documented, open, human-readable format, users gain:

Longevity — Files survive the apps that created them
Interoperability — Any tool can read and write the same canvas
Readability — JSON is easy to inspect, debug, and version-control
Extensibility — The spec leaves room for application-specific additions

This mirrors the success of Markdown for text. Before Markdown, notes were trapped in proprietary formats. After Markdown, portability became the norm. JSON Canvas aims to do the same for spatial data.

Understanding the Spec

A JSON Canvas file (.canvas extension) contains two top-level arrays:

{
  "nodes": [...],
  "edges": [...]
}

Both are optional — an empty canvas is valid.

Nodes

Nodes are the objects placed on the canvas. There are four types:

Type	Description	Required Extra Field
`text`	Plain text with Markdown	`text` (string)
`file`	Reference to a file	`file` (string path)
`link`	Reference to a URL	`url` (string)
`group`	Visual container	`label` (optional string)

Every node shares these required attributes:

{
  "id": "node1",
  "type": "text",
  "x": 100,
  "y": 200,
  "width": 300,
  "height": 200,
  "text": "Hello, Canvas!"
}

Optional fields include color (hex or preset "1"-"6"), subpath for file nodes (e.g., "#heading"), and background/backgroundStyle for group nodes.

Nodes are ordered by z-index — first in the array renders below, last renders on top.

Edges

Edges connect nodes with lines:

{
  "id": "edge1",
  "fromNode": "node1",
  "fromSide": "right",
  "fromEnd": "arrow",
  "toNode": "node2",
  "toSide": "left",
  "toEnd": "arrow",
  "color": "4",
  "label": "connects to"
}

fromSide and toSide can be top, right, bottom, or left. Endpoint shapes are none or arrow. Edges also support colors and labels.

Tutorial: Building a JSON Canvas Renderer

Let's build a minimal but functional canvas renderer in JavaScript that reads a .canvas file and renders it in the browser.

Step 1: The Canvas File

Create demo.canvas:

{
  "nodes": [
    {
      "id": "idea1",
      "type": "text",
      "x": 50,
      "y": 50,
      "width": 260,
      "height": 120,
      "text": "# Core Idea\nBuild tools that respect user data ownership.",
      "color": "4"
    },
    {
      "id": "idea2",
      "type": "text",
      "x": 400,
      "y": 50,
      "width": 260,
      "height": 120,
      "text": "# Why?\nProprietary formats create lock-in.",
      "color": "1"
    },
    {
      "id": "ref1",
      "type": "link",
      "x": 400,
      "y": 250,
      "width": 260,
      "height": 80,
      "url": "https://jsoncanvas.org"
    },
    {
      "id": "group1",
      "type": "group",
      "x": 30,
      "y": 20,
      "width": 660,
      "height": 340,
      "label": "Project Brainstorm"
    }
  ],
  "edges": [
    {
      "id": "e1",
      "fromNode": "idea1",
      "fromSide": "right",
      "toNode": "idea2",
      "toSide": "left",
      "label": "leads to"
    },
    {
      "id": "e2",
      "fromNode": "idea2",
      "fromSide": "bottom",
      "toNode": "ref1",
      "toSide": "top",
      "color": "5"
    }
  ]
}

Step 2: Parsing and Rendering

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>JSON Canvas Renderer</title>
  <style>
    #canvas {
      position: relative;
      width: 100vw;
      height: 100vh;
      background: #1e1e2e;
      overflow: auto;
    }
    .node {
      position: absolute;
      border-radius: 8px;
      border: 2px solid #444;
      padding: 12px;
      color: #cdd6f4;
      font-family: sans-serif;
      font-size: 14px;
      box-sizing: border-box;
      overflow: auto;
    }
    .node.group {
      background: rgba(69, 71, 90, 0.3);
      border: 2px dashed #585b70;
    }
    .node.link {
      background: #313244;
      border-color: #89b4fa;
    }
    .edge-label {
      position: absolute;
      background: #1e1e2e;
      color: #a6adc8;
      font-size: 12px;
      padding: 2px 6px;
      border-radius: 4px;
      pointer-events: none;
    }
    #edges {
      position: absolute;
      top: 0; left: 0;
      width: 100%; height: 100%;
      pointer-events: none;
    }
  </style>
</head>
<body>
  <div id="canvas">
    <svg id="edges"></svg>
  </div>
  <script type="module">
    const COLORS = {
      '1': '#f38ba8', '2': '#fab387', '3': '#f9e2af',
      '4': '#a6e3a1', '5': '#89dceb', '6': '#cba6f7'
    };

    function resolveColor(c) {
      if (!c) return '#585b70';
      return COLORS[c] || (c.startsWith('#') ? c : '#585b70');
    }

    async function renderCanvas(url) {
      const data = await fetch(url).then(r => r.json());
      const canvasEl = document.getElementById('canvas');
      const svgEl = document.getElementById('edges');

      const groups = (data.nodes || []).filter(n => n.type === 'group');
      const others = (data.nodes || []).filter(n => n.type !== 'group');

      for (const node of [...groups, ...others]) {
        const el = document.createElement('div');
        el.className = `node ${node.type}`;
        el.style.left = node.x + 'px';
        el.style.top = node.y + 'px';
        el.style.width = node.width + 'px';
        el.style.height = node.height + 'px';

        if (node.color) {
          el.style.borderColor = resolveColor(node.color);
        }

        if (node.type === 'text') {
          el.innerHTML = marked.parse(node.text || '');
        } else if (node.type === 'link') {
          el.innerHTML = `<a href="${node.url}" target="_blank" style="color:#89b4fa">${node.url}</a>`;
        } else if (node.type === 'group' && node.label) {
          el.innerHTML = `<strong>${node.label}</strong>`;
        }

        canvasEl.appendChild(el);
      }

      const nodeMap = {};
      for (const n of data.nodes || []) {
        nodeMap[n.id] = n;
      }

      for (const edge of data.edges || []) {
        const from = nodeMap[edge.fromNode];
        const to = nodeMap[edge.toNode];
        if (!from || !to) continue;

        const fx = from.x + from.width / 2;
        const fy = from.y + from.height / 2;
        const tx = to.x + to.width / 2;
        const ty = to.y + to.height / 2;

        const line = document.createElementNS('http://www.w3.org/2000/svg', 'line');
        line.setAttribute('x1', fx);
        line.setAttribute('y1', fy);
        line.setAttribute('x2', tx);
        line.setAttribute('y2', ty);
        line.setAttribute('stroke', resolveColor(edge.color));
        line.setAttribute('stroke-width', '2');

        if (edge.toEnd !== 'none') {
          line.setAttribute('marker-end', 'url(#arrow)');
        }

        svgEl.appendChild(line);

        if (edge.label) {
          const lbl = document.createElement('div');
          lbl.className = 'edge-label';
          lbl.textContent = edge.label;
          lbl.style.left = ((fx + tx) / 2 - 20) + 'px';
          lbl.style.top = ((fy + ty) / 2 - 10) + 'px';
          canvasEl.appendChild(lbl);
        }
      }
    }

    const svg = document.getElementById('edges');
    const defs = document.createElementNS('http://www.w3.org/2000/svg', 'defs');
    defs.innerHTML = '<marker id="arrow" markerWidth="10" markerHeight="7" refX="10" refY="3.5" orient="auto"><polygon points="0 0, 10 3.5, 0 7" fill="#cdd6f4"/></marker>';
    svg.appendChild(defs);

    renderCanvas('./demo.canvas');
  </script>
</body>
</html>

This gives you a working canvas renderer in ~120 lines. It handles all four node types, edges with labels and arrows, and the preset color system.

Step 3: Writing a Canvas File

Building a .canvas file programmatically is straightforward:

function createCanvas(nodes, edges) {
  return JSON.stringify({ nodes, edges }, null, 2);
}

const nodes = [
  { id: 'root', type: 'text', x: 400, y: 300, width: 200, height: 100,
    text: '# Project Alpha\nKickoff notes', color: '4' },
  { id: 'task1', type: 'text', x: 100, y: 100, width: 200, height: 80,
    text: '- Design mockups\n- API schema', color: '3' },
  { id: 'task2', type: 'text', x: 700, y: 100, width: 200, height: 80,
    text: '- Write tests\n- CI pipeline', color: '3' },
  { id: 'spec', type: 'file', x: 100, y: 500, width: 200, height: 80,
    file: 'docs/spec.md' }
];

const edges = [
  { id: 'e1', fromNode: 'root', fromSide: 'left', toNode: 'task1', toSide: 'right' },
  { id: 'e2', fromNode: 'root', fromSide: 'right', toNode: 'task2', toSide: 'left' },
  { id: 'e3', fromNode: 'root', fromSide: 'bottom', toNode: 'spec', toSide: 'top',
    color: '6' }
];

const canvasJson = createCanvas(nodes, edges);

Real-World Scenarios

1. Knowledge Management Tools

If you're building a PKM (Personal Knowledge Management) tool, adopting .canvas as your storage format means Obsidian users can import their canvases directly, and vice versa. Your tool immediately gains interoperability with an ecosystem of apps.

2. Collaborative Whiteboarding

Team whiteboard tools can use JSON Canvas as an export format. Even if your internal representation is more complex, a .canvas export gives teams a portable backup they can open in any compatible app — no vendor lock-in.

3. Documentation and Diagramming

Technical documentation often benefits from spatial layouts. A tool that generates .canvas files from API specs, database schemas, or architecture diagrams gives users an editable, version-controllable artifact they can refine in Obsidian or any other canvas app.

4. Git-Friendly Visual Documentation

Because .canvas files are plain JSON, they diff cleanly in git. Teams can track changes to visual documentation the same way they track code — with history, blame, and merge (JSON merges better than binary formats).

FAQ and Troubleshooting

Q: Can I add custom fields not in the spec?
A: Yes! The spec is intentionally extensible. Add your own fields to nodes and edges — other apps will simply ignore unknown fields. This is how you can add app-specific features without breaking compatibility.

Q: What about embedded images?
A: Use file type nodes referencing relative paths (e.g., "file": "assets/diagram.png"). For absolute URLs, use link type nodes.

Q: How do I handle canvas panning and zooming?
A: The spec doesn't define viewport state. Track pan/zoom in your app's state, not in the .canvas file. The file represents the content, not the view.

Q: My edges render behind nodes. What's wrong?
A: SVG elements render in document order. Make sure your SVG layer sits below the node layer in the DOM, or use z-index / pointer-events: none on the SVG (as shown in the tutorial).

Q: Can a node be inside a group?
A: Group membership is implicit — any node whose position falls within a group's bounds is visually inside it. The spec doesn't use explicit parent-child relationships, making it simpler to parse.

Q: What if two apps interpret preset colors differently?
A: That's by design. Preset "1" through "6" map to semantic colors (red through purple) but exact values are app-specific. This lets each app match its own brand while maintaining semantic consistency.

Conclusion

JSON Canvas represents an important shift in how we think about canvas-based applications. By adopting an open, readable format, we give users ownership of their spatial data — the same way Markdown liberated text notes and CSV liberated spreadsheets.

The spec is small and well-defined enough to implement in an afternoon, yet expressive enough for real applications. Whether you're building a new canvas tool, adding export to an existing one, or just want git-friendly visual documentation, JSON Canvas is worth your attention.

The official spec is a single page. The GitHub repo is open source under MIT. Pick it up, try the tutorial above, and give your users the portability they deserve.

Have you built something with JSON Canvas? I'd love to hear about it in the comments!

I Built an Uptime Monitor in a Weekend — And Saved $180/Year

AttractivePenguin — Mon, 20 Apr 2026 09:06:38 +0000

I was paying $15/month for Uptime Robot. I used maybe 30% of it — basic HTTP checks on a handful of endpoints, email alerts when something went down, and a status page nobody really looked at.

That's $180/year for a glorified cron job.

So I built my own. In a weekend. Here's exactly how, what I learned, and whether you should do the same.

Why Build Your Own?

Let's be clear: Uptime Robot, Pingdom, and friends are excellent services. If you're running production infrastructure at scale, use them. But if you're a solo developer or small team monitoring a handful of personal projects, side hustles, or staging environments, the math is painful:

Service	Free Tier	Paid Tier
Uptime Robot	50 monitors, 5-min intervals	$7-84/month
Pingdom	None	$15/month+
Better Stack	10 monitors, 3-min intervals	$24/month+

For my use case — 5-10 endpoints, 1-minute intervals, Slack alerts — the free tiers were too limited and the paid tiers were overkill.

What I actually needed:

HTTP health checks every 60 seconds
Response time tracking
Slack/email notifications on downtime
A simple dashboard to see current status

That's maybe 200 lines of code. Let me prove it.

Architecture

Here's the setup I landed on after experimenting:

┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   Scheduler  │────▶│  HTTP Check   │────▶│    SQLite   │
│  (node-cron) │     │  (fetch API)  │     │   (results) │
└─────────────┘     └──────────────┘     └─────────────┘
                            │
                            ▼
                    ┌──────────────┐
                    │   Alerter    │
                    │ (Slack/Email)│
                    └──────────────┘
                            │
                            ▼
                    ┌──────────────┐
                    │   Dashboard  │
                    │   (Express)  │
                    └──────────────┘

Why SQLite? Because this is a low-write, read-heavy workload that fits comfortably in a single file. No need to run a database server. If you're monitoring <1,000 endpoints, SQLite handles it effortlessly.

Why Node.js? Because the fetch API is built in (Node 18+), the async model fits perfectly for concurrent HTTP checks, and you probably already have it installed.

Step 1: Project Setup

mkdir uptime-monitor && cd uptime-monitor
npm init -y
npm install better-sqlite3 node-cron express dotenv

Create a .env file:

MONITOR_INTERVAL=1
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/YOUR/WEBHOOK/HERE
ALERT_EMAIL=you@example.com
PORT=3000

Step 2: Database Setup

// db.js
const Database = require('better-sqlite3');
const db = new Database('monitor.db');

db.exec(`
  CREATE TABLE IF NOT EXISTS checks (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    url TEXT NOT NULL,
    status INTEGER,
    response_time_ms INTEGER,
    up INTEGER NOT NULL,
    checked_at DATETIME DEFAULT CURRENT_TIMESTAMP
  );

  CREATE INDEX IF NOT EXISTS idx_checks_url ON checks(url);
  CREATE INDEX IF NOT EXISTS idx_checks_checked_at ON checks(checked_at);
`);

// Prepared statements for performance
const insertCheck = db.prepare(`
  INSERT INTO checks (url, status, response_time_ms, up)
  VALUES (@url, @status, @response_time_ms, @up)
`);

const getRecentChecks = db.prepare(`
  SELECT * FROM checks
  WHERE url = ?
  ORDER BY checked_at DESC
  LIMIT ?
`);

module.exports = { db, insertCheck, getRecentChecks };

Using prepared statements is important here — you'll be inserting thousands of rows, and prepared statements are dramatically faster than raw queries.

Step 3: The Health Checker

// checker.js
const { insertCheck } = require('./db');

const ENDPOINTS = [
  { url: 'https://myapp.com', expectedStatus: 200 },
  { url: 'https://api.myapp.com/health', expectedStatus: 200 },
  { url: 'https://myportfolio.dev', expectedStatus: 200 },
];

async function checkEndpoint(endpoint) {
  const start = Date.now();
  try {
    const response = await fetch(endpoint.url, {
      signal: AbortSignal.timeout(10000), // 10s timeout
    });
    const response_time_ms = Date.now() - start;
    const up = response.status === endpoint.expectedStatus ? 1 : 0;

    insertCheck.run({
      url: endpoint.url,
      status: response.status,
      response_time_ms,
      up,
    });

    return { url: endpoint.url, up, status: response.status, response_time_ms };
  } catch (err) {
    const response_time_ms = Date.now() - start;
    insertCheck.run({
      url: endpoint.url,
      status: null,
      response_time_ms,
      up: 0,
    });

    return { url: endpoint.url, up: false, error: err.message, response_time_ms };
  }
}

async function runAllChecks() {
  const results = await Promise.allSettled(
    ENDPOINTS.map(ep => checkEndpoint(ep))
  );
  return results.map(r => r.value || r.reason);
}

module.exports = { runAllChecks, ENDPOINTS };

Key detail: AbortSignal.timeout(10000) is a Node.js 18+ feature that gives you a clean timeout without manual AbortController plumbing. If you're on an older Node version, use setTimeout + AbortController.

Step 4: Alerting

// alerter.js
require('dotenv').config();
const { getRecentChecks } = require('./db');

const SLACK_WEBHOOK = process.env.SLACK_WEBHOOK_URL;
const COOLDOWN_MINUTES = 5; // Don't spam — alert once per 5 min per endpoint

async function sendSlackAlert(url, status, responseTime) {
  if (!SLACK_WEBHOOK) return;

  const emoji = status === null ? '🔴' : '🟡';
  const message = status === null
    ? `${emoji} *DOWN*: ${url} — Connection failed (${responseTime}ms)`
    : `${emoji} *DEGRADED*: ${url} — Got ${status} (${responseTime}ms)`;

  await fetch(SLACK_WEBHOOK, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ text: message }),
  });
}

// Simple cooldown: only alert if last check was UP
async function shouldAlert(url) {
  const recent = getRecentChecks.all(url, 2);
  if (recent.length < 2) return true; // First checks — always alert on down
  // Only alert if the previous check was UP (prevents spam)
  return recent[1].up === 1;
}

async function processAlerts(results) {
  for (const result of results) {
    if (!result.up) {
      const shouldSend = await shouldAlert(result.url);
      if (shouldSend) {
        await sendSlackAlert(result.url, result.status, result.response_time_ms);
      }
    }
  }
}

module.exports = { processAlerts };

The cooldown logic is critical. Without it, you'll get a Slack message every single minute something is down. With it, you only get alerted at the transition — when a site goes from UP to DOWN. Add a recovery notification the same way.

Step 5: Dashboard (30 Lines)

// server.js
const express = require('express');
const { db, getRecentChecks } = require('./db');
const { ENDPOINTS } = require('./checker');

const app = express();

app.get('/', (req, res) => {
  const statuses = ENDPOINTS.map(ep => {
    const checks = getRecentChecks.all(ep.url, 60); // Last 60 checks
    const current = checks[0];
    const uptimePercent = checks.filter(c => c.up).length / checks.length * 100;
    const avgResponseTime = checks.reduce((sum, c) => sum + c.response_time_ms, 0) / checks.length;

    return {
      url: ep.url,
      up: current?.up ?? false,
      status: current?.status,
      responseTime: current?.response_time_ms,
      uptime: uptimePercent.toFixed(1),
      avgResponseTime: Math.round(avgResponseTime),
    };
  });

  res.send(`
    <html><head><title>Uptime Monitor</title>
    <style>
      body { font-family: system-ui; max-width: 800px; margin: 2rem auto; padding: 0 1rem; }
      .endpoint { border: 1px solid #ddd; border-radius: 8px; padding: 1rem; margin: 0.5rem 0; }
      .up { border-left: 4px solid #22c55e; } .down { border-left: 4px solid #ef4444; }
      .dot { display: inline-block; width: 12px; height: 12px; border-radius: 50%; margin-right: 8px; }
      .dot.up { background: #22c55e; } .dot.down { background: #ef4444; }
    </style></head><body>
    <h1>📊 Uptime Monitor</h1>
    ${statuses.map(s => `
      <div class="endpoint ${s.up ? 'up' : 'down'}">
        <span class="dot ${s.up ? 'up' : 'down'}"></span>
        <strong>${s.url}</strong><br>
        Status: ${s.status || 'N/A'} | Response: ${s.responseTime || 'N/A'}ms |
        Uptime (1hr): ${s.uptime}% | Avg: ${s.avgResponseTime}ms
      </div>
    `).join('')}
    <p style="color:#888;margin-top:2rem">Checks every ${process.env.MONITOR_INTERVAL || 1} min</p>
    </body></html>
  `);
});

app.listen(process.env.PORT || 3000);

No React, no bundler, no CSS framework. Just server-rendered HTML. It loads in under 100ms and shows everything you need at a glance.

Step 6: Tie It All Together

// index.js
require('dotenv').config();
const cron = require('node-cron');
const { runAllChecks } = require('./checker');
const { processAlerts } = require('./alerter');
require('./server'); // Start dashboard

// Run every minute (or adjust the cron expression)
const interval = process.env.MONITOR_INTERVAL || 1;
cron.schedule(`*/${interval} * * * *`, async () => {
  console.log(`[${new Date().toISOString()}] Running checks...`);
  const results = await runAllChecks();
  await processAlerts(results);

  const downCount = results.filter(r => !r.up).length;
  if (downCount > 0) {
    console.log(`⚠️  ${downCount} endpoint(s) DOWN`);
  } else {
    console.log(`✅ All ${results.length} endpoints UP`);
  }
});

console.log('🚀 Uptime monitor running. Dashboard at http://localhost:3000');

node index.js

That's it. You now have a fully functional uptime monitor with alerting and a dashboard.

Real-World Scenarios

Personal Projects (What I Built This For)

I monitor 7 endpoints across 3 personal projects. The total cost is $0/month, running on a $4 DigitalOcean droplet that's also hosting the projects themselves. Response time data lives in a 12MB SQLite file.

Small Team (2-5 People)

Add Slack alerting per channel (one per service), and you've got team-wide visibility. Rotate the SQLite database monthly with a cron job that archives old data. You're still at $0.

The Cost Comparison

Approach	Monthly Cost	Annual Cost
Uptime Robot (Pro)	$7	$84
Pingdom	$15	$180
Better Stack	$24	$288
This solution	$0*	$0*

*Assuming you already have a server. If not, a $4/month droplet handles this + your apps.

FAQ

What if my monitor server goes down?

This is the classic "who monitors the monitor?" problem. Two options: (1) Use a free-tier external service as a backup check on your monitor, or (2) Set up a simple healthcheck on a separate $0 cloud function. For personal projects, this edge case rarely matters.

Can I scale this to 1,000+ endpoints?

Yes, but switch SQLite to PostgreSQL. Add connection pooling and batch your inserts. The checker logic stays the same.

What about SSL certificate monitoring?

Add this check to checker.js:

const https = require('https');
function checkSSLCert(url) {
  return new Promise((resolve) => {
    const req = https.request(url, { method: 'HEAD' }, (res) => {
      const cert = res.socket.getPeerCertificate();
      const daysLeft = Math.floor(
        (new Date(cert.valid_to) - Date.now()) / (1000 * 60 * 60 * 24)
      );
      resolve({ valid: daysLeft > 0, daysLeft, issuer: cert.issuer?.O });
    });
    req.on('error', () => resolve({ valid: false }));
    req.end();
  });
}

How do I handle cron in production?

Use PM2 for process management: pm2 start index.js --name uptime-monitor. It handles restarts, logs, and monitoring. No Docker needed for something this simple.

Won't SQLite corrupt under concurrent writes?

SQLite uses WAL mode by default with better-sqlite3. For this workload (one write per minute per endpoint), corruption is essentially impossible. If you're worried, add db.pragma('journal_mode = WAL'); explicitly.

What I Learned

Most monitoring features are noise. I was paying for 40+ integrations, custom status pages, and team management I never used. Identify what you actually need.
SQLite is underrated. For read-heavy monitoring data with infrequent writes, it's perfect. Don't default to PostgreSQL for everything.
Alerting is harder than monitoring. The checking part is trivial. The hard part is not alerting — cooldowns, deduplication, and escalation policies. Keep it simple at first.
Server-rendered HTML is fine. My dashboard is 30 lines of template literals and loads instantly. Not everything needs React.
Own your data. When Uptime Robot had an outage last year, I couldn't access my monitoring data. With SQLite on my own server, that data is always mine.

Conclusion

Building a custom uptime monitor took me a weekend, costs nothing to run, and does exactly what I need. No feature bloat, no subscription, no vendor lock-in.

That said — if you're running production systems at a company, use a real service. Monitoring is critical infrastructure. The $15/month is worth the reliability, compliance, and support. But for personal projects and small teams? Roll your own. It's more fun, you'll learn something, and you'll save money.

The full code is available in the examples above — copy, paste, deploy. If you build something with this, I'd love to hear about it.

Happy monitoring! 🚀

The Vercel April 2026 Security Incident: What Every Developer Actually Needs to Know

AttractivePenguin — Mon, 20 Apr 2026 08:31:20 +0000

When 694 Hacker News points and a parallel Lobsters thread blow up in the same day, something real happened. Here's the breakdown — and more importantly, what you should do about it.

Last week, Vercel dropped a security bulletin that landed at the top of Hacker News and quickly spread to Lobsters, Reddit, and every tech Slack you belong to. The reaction wasn't just "oh no, another breach" — it was something sharper: we trusted this platform with our entire deployment pipeline, and now what?

If you haven't read the full incident report yet, don't worry. I have. Let me save you the corporate-speak and tell you what actually happened, why it matters even if you're not on Vercel, and what concrete steps you should take this week.

What Happened

Vercel's April 2026 incident involved a compromise in their edge function infrastructure. Without overstating what's been publicly confirmed: environment variables — including secrets stored in Vercel's project settings — were potentially exposed in a subset of deployments during a window in early April.

The attack vector exploited a misconfiguration in how Vercel's build pipeline isolated tenant environments during a rollout of their updated edge runtime. In plain terms: the wall between your project's environment and another tenant's wasn't airtight for a brief period, and a sophisticated actor noticed.

Vercel's detection was reasonably fast — hours, not days — and they rotated affected credentials server-side before the bulletin went out. But here's the thing: by the time they detected it, the exposure window had already closed. They were doing forensics on what had already happened, not stopping an active attack.

That's the uncomfortable truth about most cloud platform incidents.

Why This Hit Different

Every week there's some breach somewhere. Most devs scroll past. This one stuck because it violated a core psychological contract.

The pitch of platforms like Vercel is: stop worrying about infrastructure, just ship. Environment variables are the canonical example. You don't manage .env files in production — you just drop your DATABASE_URL and STRIPE_SECRET_KEY into the dashboard and trust the platform to keep them locked down.

That trust is the product. And when it cracks, even briefly, even with a fast response, the questions start:

How many of my secrets were actually exposed?
How would I even know if something was accessed?
What's my blast radius if a bad actor grabbed my production database credentials?

Most developers can't answer the third question. That's the real problem this incident surfaced.

The Supply Chain Angle Nobody's Talking About Enough

The HN thread (predictably) devolved into a Vercel vs. self-hosted debate within 20 comments. That's the wrong framing.

The more interesting angle: this is a supply chain problem wearing a platform problem's clothes.

Your app doesn't just run on Vercel. It runs on Vercel, which runs on AWS, which uses third-party edge PoPs, which might involve CDN providers — each layer has its own security posture you're implicitly trusting. When you push to main and it "just works," you're actually delegating trust across a dependency chain you've never audited.

This isn't unique to Vercel. It's true of Netlify, Railway, Render, Fly.io — any managed deployment platform. The convenience is real; so is the opaqueness.

The lesson isn't "self-host everything" (please don't, unless you genuinely enjoy 3am PagerDuty alerts). The lesson is know your trust surface.

Concrete Steps You Should Take This Week

Enough analysis. Here's what to actually do.

1. Audit Your Stored Secrets Right Now

# If you use the Vercel CLI, list your project's env vars
vercel env ls --environment=production

# Cross-reference against what should be there
# Look for anything that shouldn't exist or has an unexpected value

If anything looks off, rotate it immediately — not "when you get a chance." Now.

2. Scope Your Database Credentials

This is the one that will save you when (not if) credentials leak somewhere.

-- Don't give your app a superuser credential
-- Create a role scoped to exactly what it needs

CREATE ROLE app_user LOGIN PASSWORD 'your-password';
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO app_user;

-- Explicitly deny what it shouldn't touch
REVOKE ALL ON TABLE audit_log FROM app_user;
REVOKE ALL ON TABLE admin_settings FROM app_user;

If your production DATABASE_URL is a superuser — change that this week. A compromised credential to a read/write-limited role is a bad day. A compromised superuser credential is an existential event.

3. Implement Secret Rotation (Actually, Not In Theory)

Most teams have "we should rotate secrets periodically" somewhere in a Notion doc that nobody reads. Here's a minimal setup using GitHub Actions that forces you to think about it:

# .github/workflows/secret-rotation-reminder.yml
name: Secret Rotation Reminder

on:
  schedule:
    - cron: '0 9 1 * *'  # First of every month, 9am

jobs:
  remind:
    runs-on: ubuntu-latest
    steps:
      - name: Create rotation issue
        uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: `[Security] Monthly secret rotation — ${new Date().toISOString().slice(0,7)}`,
              body: `## Secrets to review this month\n\n- [ ] Database credentials\n- [ ] API keys (Stripe, SendGrid, etc.)\n- [ ] Webhook signing secrets\n- [ ] OAuth client secrets\n\nRotate anything that hasn't been rotated in 90+ days.`,
              labels: ['security']
            })

Annoying? A little. Better than discovering you haven't rotated your Stripe key in two years when someone starts making fraudulent charges? Yes.

4. Enable Audit Logging on Your Critical Services

If you're using Supabase, PlanetScale, or any modern database platform, they have audit log features. Turn them on. Most people don't because it's buried in the settings and nobody thinks they'll need it.

You will need it exactly when you can't afford to not have it.

# Supabase: audit logging is under Settings > Logs
# PlanetScale: Audit log is under your organization settings
# AWS RDS: Enable CloudTrail + RDS logs in parameter group

# Minimal CloudWatch log subscription for RDS slow/error logs:
aws rds modify-db-instance \
  --db-instance-identifier your-instance \
  --cloudwatch-logs-export-configuration '{"EnableLogTypes":["error","audit"]}' \
  --apply-immediately

5. Know Your Blast Radius Before You Need To

Right now, before anything bad happens: write a one-pager (or even a Slack message to yourself) answering these questions:

If my production DATABASE_URL leaked, what could an attacker do?
If my STRIPE_SECRET_KEY leaked, how quickly could I revoke it and what's the damage window?
If my OAuth client secret leaked, who can I call to rotate it immediately?
What's the escalation path if I discover a breach at 2am on a Saturday?

This isn't paranoia. It's the same reason you write a runbook before your service goes down, not after.

The Broader Takeaway on Platform Trust

Vercel handled this incident reasonably well by industry standards — fast detection, proactive rotation, transparent communication. But "reasonably well" in the security industry is a low bar, and this response doesn't change the underlying dynamic.

Centralized edge infrastructure is a single point of failure. When you consolidate deployments onto one platform, you get convenience, great DX, and impressive performance numbers. You also get correlated risk — an incident at Vercel affects thousands of companies simultaneously in a way that a self-hosted setup never could.

That's not an argument against Vercel. It's an argument for not treating "deployed to Vercel" as "secured."

Your platform handles runtime isolation. You still own:

Credential hygiene
Blast radius scoping
Monitoring and detection
Incident response playbooks

The platform is a vendor. Treat it like one — with trust, but with verification.

Hot Take Corner

Here it is, since we're being honest: the "just push to prod" culture has made the average developer dangerously under-practiced at security fundamentals.

When you self-host, you're forced to think about firewalls, network segmentation, credential management — because nothing works until you do. When the platform handles it, those muscles atrophy. The Vercel incident is a stress test that revealed how many devs don't know their blast radius, don't have rotation procedures, and don't have audit logs turned on.

The platform didn't fail you. The ecosystem of "don't worry about it, just ship" set unrealistic expectations about what platforms are responsible for and what you still need to own.

What's Next

Watch Vercel's follow-up communications closely — specifically whether they announce architectural changes to tenant isolation, not just "we've improved our monitoring." The latter is table stakes. The former is what actually prevents recurrence.

If you're evaluating deployment platforms right now, add tenant isolation architecture to your vendor questions. It's not fun to ask, but it's the right question.

And if this incident made you realize you can't answer "what's my blast radius" — block two hours this week. Write it down. You'll sleep better.

What did I miss? Drop your take in the comments — especially if you were directly affected and are willing to share what your response looked like.

How a Shell Script Can Compile C Code: Inside C89cc

AttractivePenguin — Sun, 19 Apr 2026 09:06:57 +0000

You read that right. There exists a complete C89/ELF64 compiler written entirely in portable shell script. No C bootstrap. No Python helper. No cheating. Just pure, unadulterated /bin/sh.

It's called c89cc, and when it hit 117 points on Hacker News, the developer community's collective jaw hit the floor. Because if you can write a C compiler in shell script, what can't you do with shell?

In this article, we'll explore how c89cc works, why it matters, and what you can learn from it — whether you're a shell scripting enthusiast, a compiler nerd, or just someone who appreciates a beautiful engineering feat.

Why This Matters

Shell scripting is the duct tape of the Unix world. We use it for CI pipelines, deployment scripts, system administration, and quick automations. But a compiler? That's supposed to be the domain of lex, yacc, LLVM, and carefully crafted C/C++ codebases spanning hundreds of thousands of lines.

c89cc challenges that assumption. It proves that shell scripting is far more capable than most developers give it credit for. More importantly, it's an incredible educational tool: by reading a compiler written in shell, you can understand every layer of the compilation pipeline without getting lost in macro magic or template metaprogramming.

Three reasons you should care:

It demystifies compilers. Shell is readable. When the lexer, parser, and code generator are all written in shell, you can trace exactly how source code becomes a binary.
It pushes shell's limits. Understanding how the author handled memory, string processing, and binary output in pure shell teaches you techniques you'll use in everyday scripting.
It's just cool. Sometimes engineering should be about the joy of "because I can."

How c89cc Works: The Compilation Pipeline

Like any compiler, c89cc follows the standard pipeline: preprocessing, lexical analysis, parsing, semantic analysis, and code generation. The difference is that every stage is shell functions operating on text files and shell variables.

Step 1: Preprocessing

The preprocessor handles #include, #define, and conditional compilation. In c89cc, this is done with sed, awk, and shell functions that expand macros line by line.

# Simplified view of how c89cc might handle #define expansion
preprocess() {
    while IFS= read -r line; do
        case "$line" in
            '#define'*)
                name=$(echo "$line" | awk '{print $2}')
                value=$(echo "$line" | awk '{print $3}')
                eval "define_${name}=\"${value}\""
                ;;
            *)
                # Expand known macros
                for macro in $(set | grep '^define_' | cut -d= -f1); do
                    mname="${macro#define_}"
                    line="${line//${mname}/${!macro}}"
                done
                echo "$line"
                ;;
        esac
    done < "$1"
}

Step 2: Lexical Analysis (Tokenization)

The lexer breaks C source into tokens — keywords, identifiers, operators, literals, and punctuation. In c89cc, this is implemented as a state machine using shell case statements and pattern matching.

# Simplified token categorization
tokenize() {
    local char="$1"
    case "$char" in
        [a-zA-Z_])  echo "IDENTIFIER" ;;
        [0-9])       echo "NUMBER" ;;
        '+')         echo "PLUS" ;;
        '-')         echo "MINUS" ;;
        '*')         echo "STAR" ;;
        '/')         echo "SLASH" ;;
        ';')         echo "SEMICOLON" ;;
        '{')         echo "LBRACE" ;;
        '}')         echo "RBRACE" ;;
        *)           echo "UNKNOWN:$char" ;;
    esac
}

The real lexer is far more sophisticated — handling multi-character operators (==, !=, &&), string literals with escape sequences, and comments. But the core idea remains: a shell case statement driving a state machine.

Step 3: Parsing (Building the AST)

The parser consumes tokens and builds an Abstract Syntax Tree. c89cc uses a recursive descent parser — each grammar rule is a shell function that calls other shell functions.

# Recursive descent: parse an expression
parse_expression() {
    local left=$(parse_term)
    while [ "$current_token" = "PLUS" ] || [ "$current_token" = "MINUS" ]; do
        local op="$current_token"
        advance_token
        local right=$(parse_term)
        left="{$op, $left, $right}"
    done
    echo "$left"
}

parse_term() {
    local left=$(parse_factor)
    while [ "$current_token" = "STAR" ] || [ "$current_token" = "SLASH" ]; do
        local op="$current_token"
        advance_token
        local right=$(parse_factor)
        left="{$op, $left, $right}"
    done
    echo "$left"
}

parse_factor() {
    case "$current_token" in
        "NUMBER")  local val="$current_value"; advance_token; echo "$val" ;;
        "IDENTIFIER") local name="$current_value"; advance_token; echo "$name" ;;
        "LPAREN")  advance_token; local expr=$(parse_expression); advance_token; echo "$expr" ;;
    esac
}

Each function returns its sub-tree as a string (shell's only real data structure), and the parent function combines them. It's functional programming in shell — and it works.

Step 4: Code Generation (ELF64 Output)

This is where it gets wild. c89cc generates a valid ELF64 binary — headers, section tables, program headers, and x86-64 machine code — all by writing raw bytes using shell's printf with octal escapes.

# Writing the ELF header (simplified)
write_elf_header() {
    # Magic number: 0x7f 'E' 'L' 'F'
    printf '\177ELF' > "$output"
    # ELFCLASS64
    printf '\002' >> "$output"
    # Little endian
    printf '\001' >> "$output"
    # ELF version
    printf '\001' >> "$output"
    # OS/ABI, padding
    printf '\000' >> "$output"
    printf '%0.s\000' {1..8} >> "$output"
    # ET_EXEC (executable file)
    printf '\002\000' >> "$output"
    # EM_X86_64
    printf '\062\000' >> "$output"
    # ...
}

The code generator traverses the AST and emits x86-64 instructions using the same printf technique. Each CPU opcode is encoded as its byte sequence.

Real-World Lessons for Shell Scripters

You're probably not going to write your next compiler in shell. But the techniques c89cc uses are directly applicable to everyday scripting:

1. Shell Can Handle Binary Data

Most people think shell can only process text. printf with octal escapes (\177) and the $'\xNN' syntax let you write arbitrary bytes. This is useful for generating binary files, protocol messages, or even small images.

# Write a minimal PNG header
printf '\x89PNG\r\n\x1a\n' > image.png

2. Functional Patterns in Shell

c89cc uses functions that echo their results, and callers capture them with $(). This is essentially functional programming — pure functions composing via pipe-like semantics.

# Compose functions instead of mutating globals
result=$(validate $(sanitize $(read_input)))

3. State Machines with `case`

The lexer's state machine pattern is useful anywhere you need to process input character by character — log parsing, protocol handling, simple parsers.

4. Don't Underestimate Shell Performance

Yes, c89cc is slow. Compiling a simple program can take minutes instead of milliseconds. But for one-off build scripts, prototyping, and CI checks, shell is fast enough. The bottleneck is usually I/O, not the shell itself.

FAQ

Q: Can c89cc compile itself (bootstrap)?
A: Not yet. The C89 feature set it supports is impressive but incomplete. Self-compilation is the holy grail of compiler projects, and shell adds its own constraints. But the subset it handles — functions, pointers, structs, control flow — is substantial.

Q: Is this production-ready?
A: Absolutely not, and the author doesn't claim it is. This is an educational and artistic project. Use GCC, Clang, or TCC for real work.

Q: Why shell specifically? Why not Python or Go?
A: The constraint is the point. Shell is available everywhere — every Unix system, every Docker container, every CI runner. Writing a compiler in the most portable language possible is a statement about accessibility and minimalism.

Q: How does it handle memory?
A: Shell doesn't have pointers or manual memory management. c89cc uses temporary files and shell variables to simulate memory. Symbol tables are associative arrays (in bash/ksh) or files keyed by name.

Q: What C89 features are supported?
A: The core set: integer types, functions, pointers, arrays, structs, control flow (if/else, while, for, switch), and basic preprocessor directives. Notably missing: floating point, complex macro expansion, and full standard library support.

Running c89cc Yourself

Want to see it in action? Here's how:

# Clone the repo
git clone https://github.com/yiyus/c89cc.git
cd c89cc

# Try compiling a simple C program
cat > hello.c << 'CEOF'
int main() {
    write(1, "Hello from shell-compiled C!\n", 29);
    return 0;
}
CEOF

# Compile with c89cc (this will take a while!)
./c89cc.sh hello.c -o hello

# Run the result
./hello
# Output: Hello from shell-compiled C!

A word of warning: Compilation is slow. A program that GCC compiles in milliseconds might take c89cc several minutes. This is expected — shell is interpreting each line, doing string processing for every token, every parse step, every byte of output. Grab a coffee.

Conclusion

c89cc isn't about replacing your toolchain. It's about expanding what you believe is possible.

When someone says "shell scripting can't do X," c89cc is the counterargument. A C compiler — with a lexer, recursive descent parser, and ELF64 code generator — all in portable shell. It works. It produces real executables. And you can read every line of its source code in an afternoon.

The next time you're writing a shell script and thinking "this is getting too complex, I should switch to Python," remember c89cc. Maybe shell can handle it. And maybe the constraint of staying in shell will lead you to a simpler, more elegant solution.

The repo is at github.com/yiyus/c89cc. Go read the source. It's a masterclass in what shell can do.

Have you built something surprising in shell? I'd love to hear about it in the comments.

You're Probably Wasting Money on AI Coding Tools (And You Have No Idea How Much)

AttractivePenguin — Sun, 19 Apr 2026 08:34:42 +0000

You're Probably Wasting Money on AI Coding Tools (And You Have No Idea How Much)

AI coding assistants have officially crossed the threshold from "cool experiment" to "monthly line item on the credit card." Claude Code, Cursor, GitHub Copilot, Codex — most devs I talk to are running at least two of these simultaneously, and the costs are adding up in ways that aren't obvious until something like codeburn hits 2,700 GitHub stars in under a week and you realize: oh, this is a whole category of problem now.

The problem isn't that AI tools are expensive. It's that they're opaque. You get a monthly bill that says "$40" or "$120" or, if you're a team lead with budget authority and a particularly aggressive engineering culture, "please explain this $3,400 charge." But you have almost no visibility into where those tokens went.

That's changing. And understanding the new generation of token cost observability tooling will make you a smarter buyer — and a more efficient AI user.

Why Token Costs Are Harder to Track Than You Think

Unlike traditional SaaS pricing (flat seat fees, predictable per-request charges), LLM APIs price on a dimension most devs aren't trained to think about: context window consumption.

Every message you send to Claude or GPT includes not just your current prompt, but potentially:

Your entire conversation history
System prompts (sometimes thousands of tokens of boilerplate)
File contents you've attached or that the tool auto-includes
Tool call outputs and responses
Re-injected context from previous turns

When you type a two-word response in a long Cursor session, you might be sending 20,000 tokens of context along with it. Multiply that by however many times you hit "Apply" on a suggestion, and the math gets ugly fast.

Here's a rough back-of-napkin example:

Session: 3-hour debugging session in Cursor
Average context per request: ~18,000 tokens (input)
Average output per request: ~800 tokens
Number of requests: ~40

Total input tokens:  720,000
Total output tokens:  32,000

Claude Sonnet pricing (hypothetical):
  Input:  $3.00 / 1M tokens → $2.16
  Output: $15.00 / 1M tokens → $0.48

Session cost: ~$2.64

That's not outrageous. But scale it: 5 devs, 8 working hours, multiple AI tools, and you're looking at a meaningful monthly burn that nobody's tracking in any structured way.

The Token Sinks Nobody Talks About

After digging into observability data from tools like codeburn and similar projects, a few patterns show up repeatedly as the biggest cost drivers:

1. The "Re-explain the whole project" problem

AI coding tools often re-inject your entire codebase context on every request. Some tools are smarter about chunking and retrieval, but many — especially agentic workflows — err on the side of "more context = better results." Which is often true! But it's expensive.

The fix: be explicit about what files you want the AI to have access to. Don't let tools auto-index your entire monorepo when you're just trying to fix a CSS bug.

2. Failed attempts that cost as much as successful ones

This is the brutal one. If you ask the AI to refactor a complex function and it gets it wrong three times before nailing it on the fourth try — you paid for all four attempts. The failed ones aren't discounted.

# Rough cost comparison: one complex refactoring task
# Attempt 1 (failed): 12,000 tokens in + 3,000 out
# Attempt 2 (failed): 14,000 tokens in + 2,800 out  
# Attempt 3 (failed): 16,000 tokens in + 4,200 out
# Attempt 4 (success): 18,000 tokens in + 5,100 out
# Total: 60,000 tokens in + 15,100 out
# vs. getting it right first try: 12,000 + 5,100

Clear, specific prompts are not just a workflow optimization — they're a cost optimization. Ambiguity is literally billable.

3. System prompt bloat

Many AI tools (and especially the frameworks you build on top of them) use massive system prompts. I've seen system prompts over 8,000 tokens that get prepended to every single request. If you're making 100 requests a day, that's 800,000 tokens of system prompt input that you're paying for regardless of whether that context is relevant.

Audit your system prompts. Trim the fat. The difference between a 2,000-token and 8,000-token system prompt adds up to real money across a team.

What Token Cost Dashboards Actually Show You

Tools like codeburn operate by intercepting or parsing usage data from AI tool APIs and presenting it in a usable format. The better ones break down costs by:

Tool (which AI assistant generated the cost)
Session/project (which codebase or conversation)
Time (hourly/daily/weekly burn rates)
Model (cost per model if you're mixing Sonnet vs. Opus vs. Haiku)
Token type (input vs. output, because output tokens cost dramatically more)

A typical TUI interface looks something like this conceptually:

┌─────────────────────────────────────────────────────┐
│  AI Token Cost Dashboard              [April 2026]  │
├─────────────────────────────────────────────────────┤
│  Today's spend:    $4.23                            │
│  This week:        $31.87                           │
│  Monthly projection: $127.48                        │
├──────────────────┬──────────────────────────────────┤
│  By Tool         │  By Project                     │
│  Cursor    $2.11 │  api-refactor      $1.84        │
│  Claude    $1.47 │  frontend-cleanup  $1.22        │
│  Copilot   $0.65 │  misc/scratch      $1.17        │
└──────────────────┴──────────────────────────────────┘

The insight isn't just "here's what you spent." It's "here's the pattern." When you can see that your api-refactor project costs 3x more per hour than other projects, you start asking why. Usually the answer involves large context windows, lots of failed attempts, or a very chatty system prompt.

Setting Budgets Without Crippling Your Workflow

Here's my hot take: hard token limits are usually a mistake. If you hit your budget at 3pm and the AI shuts off, you lose the last hour of productivity before a deadline. That's a bad trade.

Better approach: soft budgets with visibility and friction.

What actually works:

Daily awareness notifications — get a Slack/Telegram ping when you hit 50% and 80% of your daily budget. Not a hard stop, just awareness. You'll naturally become more deliberate.

Project-level budgets — track cost per project, not just total spend. It reframes the question from "am I spending too much on AI?" to "is this project's AI spend proportionate to its business value?"

Model routing by task complexity — use cheap models (Haiku, GPT-4o-mini) for simple completions, expensive models (Opus, o3) only when complexity justifies it. This is the single highest-leverage optimization most devs aren't doing.

# Rough model routing logic
def pick_model(task_complexity: str) -> str:
    match task_complexity:
        case "simple":    return "claude-haiku-3"
        case "moderate":  return "claude-sonnet-4"
        case "complex":   return "claude-opus-4"
        case _:           return "claude-sonnet-4"  # sensible default

Weekly cost reviews — 10 minutes on Friday looking at what drove your AI spend that week. You'll find patterns: certain task types, certain times of day, certain projects that are systematically expensive.

For Team Leads: This Is Now a Budget Category

If you're managing a team of devs with access to AI tools, you need to treat this like cloud compute — not like a subscription service you pay for and forget.

Key things to track:

Per-dev monthly spend — outliers indicate either very heavy users (good or bad, depends on output) or tooling misconfiguration
Cost per shipped PR/feature — the actual ROI signal. High AI spend per output is a yellow flag
Model mix — are your devs defaulting to the most expensive model even for trivial tasks?

Most enterprise AI tool vendors offer usage exports or API access to cost data. If they don't, that's a vendor evaluation criterion worth taking seriously.

The Uncomfortable Truth About AI Coding ROI

Here's where I'll be blunt: most devs don't actually know if their AI tools are paying off. They feel more productive. They ship faster some days. But without cost visibility, you can't calculate ROI — you're just guessing.

Token cost observability is the first step toward treating AI spend like any other engineering cost: something you measure, optimize, and make deliberate decisions about.

The devs who get good at this in the next 12 months will have a real competitive advantage — not just in their own productivity, but in being able to make credible arguments about AI tool budgets, justify new tool adoption, and spot when an AI-heavy approach is actually the wrong call for a given task.

That's not a small thing. The ability to say "here's what we spent on AI for this feature, here's the output, here's whether it was worth it" is a superpower in a world where everyone's AI spend is going up and almost nobody can answer that question.

Getting Started

If you want to start tracking your AI costs today:

Check if your tools expose usage data — most do, either in the UI or via API
Try codeburn or a similar TUI — even 3 days of data will surprise you
Start a simple tracking habit — even a weekly note of your total spend builds awareness
Audit one expensive session — pick a long session and try to reconstruct why it cost what it did

The goal isn't to spend less on AI. The goal is to spend right on AI — knowing where every dollar goes and making sure it's earning its keep.

Token cost observability is early-stage but growing fast. If you're building in this space or have tooling recommendations, drop them in the comments.

Kubernetes Observability in 30 Minutes: Prometheus, Grafana, and Custom Alerts That Actually Tell You Something

AttractivePenguin — Sat, 18 Apr 2026 09:05:57 +0000

Kubernetes Observability in 30 Minutes: Prometheus, Grafana, and Custom Alerts That Actually Tell You Something

You deployed your app to Kubernetes. Pods are running. Services are up. Everything looks green. But then someone asks: "How do you know it's actually healthy?"

That question hits different when your only answer is kubectl get pods. Running containers ≠ working application. Real observability means you can answer: Are requests succeeding? Is the database slow? Is the cache doing anything? When something breaks at 3 AM, what tells you before your users do?

This tutorial walks through setting up production-grade Kubernetes observability from scratch — Prometheus for metrics collection, Grafana for dashboards, and Alertmanager for notifications — all deployed via GitOps. By the end, you'll have a system that doesn't just show green bars but tells you meaningful stories about your application's health.

Why Observability Matters (And Why dashboards Aren't Enough)

Most Kubernetes tutorials stop at deployment. They show you how to get containers running but skip the part where you actually understand what's happening inside them. CloudWatch shows node metrics. kubectl top shows resource usage. Neither tells you that your cache hit rate dropped to 20% or that database inserts are taking 3x longer than usual.

The difference between monitoring and observability is the difference between a smoke alarm and a dashboard that tells you which room is on fire, how fast it's spreading, and whether the sprinklers are working. You need:

Application metrics — request rates, error rates, latencies, cache performance
Infrastructure metrics — CPU, memory, disk, network per pod and node
Correlation — the ability to see that latency spikes when the cache miss rate climbs
Alerting — proactive notification when things go wrong, not just dashboards you remember to check

Prometheus + Grafana + Alertmanager gives you all four. Let's build it.

Prerequisites

A running Kubernetes cluster (EKS, GKE, AKS, or minikube — any works)
kubectl configured and pointing at your cluster
ArgoCD installed (for GitOps deployment) or willingness to apply manifests directly
A Node.js application to instrument (I'll show the pattern; adapt it to your stack)
Basic familiarity with Kubernetes resources (Deployments, Services, ConfigMaps)

Step 1: Instrument Your Application

Before Prometheus can scrape anything, your app needs to emit metrics. For Node.js, the prom-client library makes this straightforward.

Install the dependency:

npm install prom-client

Create a metrics module (src/metrics.js):

const client = require('prom-client');
const register = new client.Registry();

// Default metrics (GC, event loop, memory, etc.)
client.collectDefaultInfo({ register });

// HTTP request duration histogram
const httpRequestDuration = new client.Histogram({
  name: 'http_request_duration_seconds',
  help: 'Duration of HTTP requests in seconds',
  labelNames: ['method', 'route', 'status_code'],
  buckets: [0.01, 0.05, 0.1, 0.5, 1, 2, 5, 10],
  registers: [register],
});

// Database query duration histogram
const dbQueryDuration = new client.Histogram({
  name: 'db_query_duration_seconds',
  help: 'Duration of database queries in seconds',
  labelNames: ['operation', 'table'],
  buckets: [0.001, 0.005, 0.01, 0.05, 0.1, 0.5],
  registers: [register],
});

// Cache operations counter
const cacheOperations = new client.Counter({
  name: 'cache_operations_total',
  help: 'Total cache operations',
  labelNames: ['operation', 'result'],
  registers: [register],
});

// Active database connections gauge
const dbConnectionsActive = new client.Gauge({
  name: 'db_connections_active',
  help: 'Number of active database connections',
  registers: [register],
});

module.exports = {
  register,
  httpRequestDuration,
  dbQueryDuration,
  cacheOperations,
  dbConnectionsActive,
};

Add HTTP middleware to track every request (src/middleware.js):

const { httpRequestDuration } = require('./metrics');

function metricsMiddleware(req, res, next) {
  // Exclude the /metrics endpoint itself from tracking
  if (req.path === '/metrics') return next();

  const end = httpRequestDuration.startTimer();
  res.on('finish', () => {
    end({
      method: req.method,
      route: req.route?.path || req.path,
      status_code: res.statusCode,
    });
  });
  next();
}

module.exports = { metricsMiddleware };

Track database and cache operations wherever they occur:

const { dbQueryDuration, cacheOperations, dbConnectionsActive } = require('./metrics');

// Database wrapper
async function query(operation, table, fn) {
  const end = dbQueryDuration.startTimer({ operation, table });
  try {
    const result = await fn();
    return result;
  } finally {
    end();
  }
}

// Cache tracking
async function cacheGet(key) {
  const result = await redis.get(key);
  cacheOperations.inc({
    operation: 'get',
    result: result !== null ? 'hit' : 'miss',
  });
  return result;
}

async function cacheSet(key, value) {
  await redis.set(key, value);
  cacheOperations.inc({ operation: 'set', result: 'success' });
}

Expose the metrics endpoint (src/server.js):

const { register } = require('./metrics');

app.get('/metrics', async (req, res) => {
  res.set('Content-Type', register.contentType);
  res.end(await register.metrics());
});

Hit http://localhost:3000/metrics and you should see ~100 lines of Prometheus-format metrics: counters, histograms, gauges, all labeled and ready to scrape.

Step 2: Deploy the Observability Stack

The kube-prometheus-stack Helm chart bundles Prometheus, Grafana, Alertmanager, Node Exporter, and kube-state-metrics — everything you need in one deploy.

If you're using ArgoCD, create an Application manifest (monitoring/argocd-app.yaml):

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: monitoring
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://prometheus-community.github.io/helm-charts
    chart: kube-prometheus-stack
    targetRevision: "58.0.0"
    helm:
      values: |
        grafana:
          service:
            type: LoadBalancer
          adminPassword: admin
        prometheus:
          prometheusSpec:
            retention: 7d
            resources:
              requests:
                memory: 512Mi
  destination:
    server: https://kubernetes.default.svc
    namespace: monitoring
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Or install directly with Helm:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
kubectl create namespace monitoring
helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --set grafana.service.type=LoadBalancer \
  --set grafana.adminPassword=admin \
  --set prometheus.prometheusSpec.retention=7d

Wait 2-3 minutes for all pods to come up:

kubectl get pods -n monitoring -w

You should see: prometheus-monitoring-0, monitoring-grafana-..., alertmanager-monitoring-0, and several exporters.

Step 3: Connect Prometheus to Your Application

This is where most people get stuck. The instinct is to use additionalScrapeConfigs in the Helm values. Don't. The correct approach is a ServiceMonitor — a CRD that the Prometheus Operator watches to discover scrape targets automatically.

First, make sure your application's Service has a named port:

apiVersion: v1
kind: Service
metadata:
  name: gitops-api
  namespace: three-tier
  labels:
    app: gitops-api
spec:
  selector:
    app: gitops-api
  ports:
    - name: http          # This name matters!
      port: 80
      targetPort: 3000

Then create the ServiceMonitor (monitoring/servicemonitor.yaml):

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: gitops-api
  namespace: monitoring
  labels:
    release: monitoring   # Must match your Helm release label
spec:
  selector:
    matchLabels:
      app: gitops-api
  namespaceSelector:
    matchNames:
      - three-tier
  endpoints:
    - port: http
      path: /metrics
      interval: 15s

The release: monitoring label is critical — the Prometheus Operator uses it to discover ServiceMonitors. If this label doesn't match your Helm release name, Prometheus silently ignores your ServiceMonitor and you'll spend hours debugging why targets aren't showing up.

Apply it:

kubectl apply -f monitoring/servicemonitor.yaml

Verify the target is discovered in Prometheus: open the Prometheus UI (kubectl port-forward svc/prometheus-operated 9090 -n monitoring) and navigate to Status → Targets. You should see your application endpoint with state "UP".

Step 4: Build Meaningful Dashboards

Skip the pretty-but-useless dashboards. Build ones that tell a story. Here's a layout that covers the three layers that matter:

Row 1 — HTTP Layer (Is the API serving traffic?)

{
  "title": "Request Rate",
  "type": "timeseries",
  "targets": [
    { "expr": "sum(rate(http_request_duration_seconds_count[5m])) by (route)", "legendFormat": "{{route}}" }
  ]
}

Add panels for Error Rate (rate(...{status_code=~"5.."}) / rate(...) as percentage) and P95 Latency (histogram_quantile(0.95, rate(..._bucket[5m])) by route).

Row 2 — Data Layer (Is the backend keeping up?)

Requests by Status (pie chart: 200/201/404/503 distribution)
Cache Hit/Miss Ratio (pie chart: cache_operations_total{operation="get",result="hit"} vs miss)
DB Query Duration P95 (by operation type — inserts vs selects)
Active DB Connections (gauge, 0–10 scale, yellow at 6, red at 8)

Row 3 — Infrastructure (Do we need more resources?)

DB Queries per Second (insert and select rates)
Pod Memory Usage
Pod CPU Usage

Save the dashboard JSON as a ConfigMap and deploy it via ArgoCD so it's version-controlled and reproducible:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-dashboard
  namespace: monitoring
  labels:
    grafana_dashboard: "1"
data:
  app-dashboard.json: |
    {YOUR_DASHBOARD_JSON_HERE}

Grafana's sidecar automatically picks up ConfigMaps with the grafana_dashboard label and imports them.

Step 5: Set Up Alerts That Don't Cry Wolf

Nine custom alert rules across four categories — enough to catch real problems without paging you at 3 AM for a blip:

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: app-alerts
  namespace: monitoring
  labels:
    release: monitoring
spec:
  groups:
    - name: api-health
      rules:
        - alert: HighErrorRate
          expr: |
            sum(rate(http_request_duration_seconds_count{status_code=~"5.."}[5m]))
            / sum(rate(http_request_duration_seconds_count[5m])) > 0.05
          for: 5m
          labels:
            severity: critical
          annotations:
            summary: "Error rate above 5%"
            description: "API error rate is {{ $value | humanizePercentage }}"

        - alert: HighLatency
          expr: |
            histogram_quantile(0.95,
              sum(rate(http_request_duration_seconds_bucket[5m])) by (le, route)
            ) > 2
          for: 10m
          labels:
            severity: warning
          annotations:
            summary: "P95 latency above 2s"
            description: "Route {{ $labels.route }} P95 latency is {{ $value }}s"

    - name: database
      rules:
        - alert: SlowDatabaseQueries
          expr: |
            histogram_quantile(0.95,
              sum(rate(db_query_duration_seconds_bucket[5m])) by (le, operation)
            ) > 0.5
          for: 10m
          labels:
            severity: warning

        - alert: ConnectionPoolExhaustion
          expr: db_connections_active > 8
          for: 5m
          labels:
            severity: critical
          annotations:
            summary: "DB connection pool nearing exhaustion"

    - name: cache
      rules:
        - alert: LowCacheHitRate
          expr: |
            sum(rate(cache_operations_total{operation="get",result="hit"}[5m]))
            / sum(rate(cache_operations_total{operation="get"}[5m])) < 0.5
          for: 15m
          labels:
            severity: warning
          annotations:
            summary: "Cache hit rate below 50%"

    - name: pods
      rules:
        - alert: PodCrashLooping
          expr: rate(kube_pod_container_status_restarts_total[15m]) > 0
          for: 5m
          labels:
            severity: critical

        - alert: PodMemoryPressure
          expr: |
            container_memory_working_set_bytes
            / container_spec_memory_limit_bytes > 0.85
          for: 10m
          labels:
            severity: warning

Key design principles for alerts:

Always use for: clauses. A 30-second spike shouldn't wake you up. for: 5m means the condition must persist for 5 minutes before alerting.
Thresholds should be meaningful. 5% error rate is objectively bad. 2-second P95 latency means real users are suffering. Don't alert on 1% error rates — you'll burn out on noise.
Group by severity. critical = page someone now. warning = investigate in the morning.

Step 6: Configure Alertmanager Routing

Alertmanager decides where alerts go. A basic config that routes critical alerts to Slack and warnings to email:

apiVersion: v1
kind: Secret
metadata:
  name: alertmanager-monitoring-config
  namespace: monitoring
type: Opaque
stringData:
  alertmanager.yaml: |
    route:
      receiver: slack
      group_by: [alertname, severity]
      group_wait: 30s
      group_interval: 5m
      repeat_interval: 4h
      routes:
        - match:
            severity: critical
          receiver: slack-urgent
          repeat_interval: 1h
        - match:
            severity: warning
          receiver: email
    receivers:
      - name: slack
        slack_configs:
          - api_url: https://hooks.slack.com/services/YOUR/WEBHOOK/URL
            channel: '#monitoring'
      - name: slack-urgent
        slack_configs:
          - api_url: https://hooks.slack.com/services/YOUR/WEBHOOK/URL
            channel: '#incidents'
      - name: email
        email_configs:
          - to: oncall@yourcompany.com
            from: alertmanager@yourcompany.com
            smarthost: smtp.yourcompany.com:587

Real-World Scenarios

Scenario 1: The Silent Degradation

Your cache hit rate slowly drops from 85% to 40% over two hours. No pods crash. No 500 errors. But database query latency triples because every request now hits Postgres instead of Redis. The LowCacheHitRate alert catches this 15 minutes in, long before users notice.

Scenario 2: The Connection Leak

A new deploy introduces a database connection leak. Active connections climb from 3 to 8 over 10 minutes. The ConnectionPoolExhaustion alert fires at 8 connections, you roll back before the pool hits 10 and the app becomes unresponsive.

Scenario 3: The Noisy Neighbor

Another team deploys a memory-hungry job on the same node. Your pod's memory pressure crosses 85%. The PodMemoryPressure warning gives you time to request a node with more capacity or move the workload before OOMKill hits.

FAQ / Troubleshooting

Q: Prometheus shows my target as "DOWN" or missing entirely.
A: Check three things: (1) Does your Service have a named port (name: http, not just port: 80)? ServiceMonitors reference ports by name. (2) Does your ServiceMonitor have the release label matching your Helm release? (3) Is the namespace in namespaceSelector.matchNames correct?

Q: My ServiceMonitor exists but metrics aren't appearing in Grafana.
A: Go to Prometheus UI → Status → Targets. If the target isn't listed, the ServiceMonitor isn't being picked up. Check label selectors. If it's listed but showing errors, the /metrics endpoint might not be reachable from the cluster network.

Q: Alerts aren't firing.
A: Check Prometheus UI → Alerts. Are the rules loaded? Is the expression evaluating? Test your PromQL directly in the query bar. Common mistake: metric names with typos or label mismatches.

Q: Dashboard shows "No data."
A: Verify the data source is configured (Grafana → Settings → Data Sources → Prometheus). Check that the namespace in your query matches where the metrics are. Use {namespace="three-tier"} to scope queries.

Q: How much resources does this stack need?
A: For a small cluster (< 20 nodes), budget: Prometheus 512Mi-1Gi RAM, Grafana 256Mi, Alertmanager 128Mi. The Helm chart defaults are reasonable for dev/test. Increase retention and memory for production.

Conclusion

Observability isn't a nice-to-have — it's the difference between guessing and knowing. The combination of application-level metrics (prom-client), infrastructure metrics (node exporter, kube-state-metrics), powerful querying (PromQL), visualization (Grafana), and proactive alerting (Alertmanager) gives you a complete picture of your system's health.

The whole stack deploys via GitOps — push to main, ArgoCD syncs. No manual dashboard creation. No ad-hoc alert rules. Everything version-controlled, reproducible, and auditable.

Start with the metrics that answer "is it working?" — request rate, error rate, latency. Add depth from there: cache performance, database health, resource pressure. Let the alerts do the watching so you don't have to.

Your future self — the one getting paged at 3 AM — will thank you.

Forem: AttractivePenguin

Idempotency Keys in Production: The Race Conditions, Expiration Traps, and Edge Cases Most Tutorials Skip

Idempotency Keys in Production: The Race Conditions, Expiration Traps, and Edge Cases Most Tutorials Skip

Why Idempotency Keys Exist

The Tutorial Version (And Why It Breaks)

Trap #1: The Concurrency Race Condition

The Fix: Atomic Key Claims

Trap #2: Idempotency ≠ Caching

Trap #3: Key Reuse With Different Payloads

Trap #4: Key Expiration Creates Duplicates

The Fix: Expiration Must Outlive the Transaction

The Client Side: One Key Per Intent

The Full Production Flow

How to Test It

Troubleshooting FAQ

The Checklist

Core Web Vitals in 2026: The Practical Fixes for INP, LCP, and CLS That Actually Work

Core Web Vitals in 2026: The Practical Fixes for INP, LCP, and CLS That Actually Work

Why Your Lighthouse Score Is Lying to You

Fix 1: INP — The Architecture Problem Nobody Diagnoses

The Fix: Break the Task, Then Yield

Layer 2: Defer Heavy Renders with useDeferredValue

Layer 3: Offload Hidden Long Tasks

Fix 2: LCP — It's Almost Always the Hero Image

Step 1: Identify the LCP Element

Step 2: Preload the LCP Resource

Step 3: Set Fetch Priority

Step 4: Modern Formats + Responsive Sizes

Step 5: Never Lazy-Load the LCP Image

When LCP Is a Font, Not an Image

Fix 3: CLS — Three Rules That Cover Everything

Rule 1: Explicit Dimensions on Every Image, Video, and Iframe

Rule 2: Reserve Space for Late-Injected Content

Rule 3: Match Font Metrics with size-adjust

The Monitoring Setup That Makes This Stick

1. Ship Real-User Metrics

2. Slice the Data

3. Set Alerts at 80% of Thresholds

FAQ: Common Gotchas

The Three Things I'd Tell You on Day One

Stop Defaulting to WebSockets: A Practical Guide to SSE, Polling, and Knowing When You Actually Need Them

Stop Defaulting to WebSockets: A Practical Guide to SSE, Polling, and Knowing When You Actually Need Them

The One Question That Decides Everything

Approach 1: Polling — The Underrated Workhorse

When to Use Polling

The Code

Why It Works

The Drawback

Approach 2: Server-Sent Events — The Sweet Spot

When to Use SSE

The Code

Named Events for Structured Data

SSE with Node.js Streams (Production Grade)

Why SSE Beats WebSockets for Server-Push

The Limitation

Approach 3: WebSockets — When You Actually Need Them

Real Use Cases Where WebSockets Win

The Code (with ws library)

What You're Signing Up For

Decision Framework: Pick in 60 Seconds

Real-World Scenario: Building a Notification System

The Wrong Approach: WebSocket

The Right Approach: SSE + fetch

FAQ

Isn't polling wasteful?

Can SSE send binary data?

What about HTTP/2 Server Push?

Do SSE connections survive behind nginx/Cloudflare?

What if I need both server-push and occasional client sends?

Can I use SSE with React?

Conclusion

AI Burnout: Do I Still Belong in Tech?

AI Burnout: "Do I Still Belong in Tech?"

The Old Bargain

What the Burnout Actually Feels Like

The Psychology of Craft and Identity

What Actually Helps (and What Doesn't)

What doesn't help:

What actually helps:

The Harder Question

Layer 2: Defer Heavy Renders with `useDeferredValue`

Rule 3: Match Font Metrics with `size-adjust`