Forem: Pablo Albaladejo

Deploy Observable AI Streaming on AWS: CDK Stack with Tests

Pablo Albaladejo — Sun, 15 Feb 2026 11:50:13 +0000

This is the final post in a four-part series. In Part 1, we set up the architecture: API Gateway REST streaming through Lambda with Middy and the Vercel AI SDK. In Part 2, we discovered that Middy's after hook fires before the stream body is consumed, making it useless for observability.

In Part 3, we built the solution: a TransformStream pipeline that defers logging and data publishing to the flush() callback, which the WHATWG Streams spec guarantees runs only after all chunks have been consumed.

Now we deploy it. This post walks through every piece: the CDK infrastructure, the handler composition, the test suite, and the deployment. The companion repo contains the complete, runnable code as step-4.

By the end, you will have a working stack that streams structured JSON from Bedrock Claude through API Gateway to the client, with observability that fires at the right time: after the stream completes, not before.

TL;DR: A single CDK stack deploys API Gateway REST with streaming, a Lambda function running Middy + AI SDK v6, and Bedrock IAM permissions. The handler composes AI SDK middlewares (logging + data store) with a Middy middleware (publishLog) that uses the Dual Middleware Bridge and TransformStream Pipeline patterns. Deploy with npx cdk deploy, test with curl --no-buffer. Full test suite included: 13 tests covering flush timing, error classification, and the streaming pipeline.

How do you deploy API Gateway REST streaming with CDK?

The infrastructure is a single CDK stack with three resources: a Lambda function, an API Gateway REST API, and the IAM permissions to call Bedrock.

// lib/streaming-stack.ts

import * as cdk from 'aws-cdk-lib';
import * as apigateway from 'aws-cdk-lib/aws-apigateway';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as logs from 'aws-cdk-lib/aws-logs';
import type { Construct } from 'constructs';
import * as path from 'node:path';

export class StreamingStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // --- Lambda Function ---

    const fn = new lambda.Function(this, 'StreamingHandler', {
      runtime: lambda.Runtime.NODEJS_22_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset(path.join(__dirname, '..', 'lambda'), {
        bundling: {
          image: lambda.Runtime.NODEJS_22_X.bundlingImage,
          command: [
            'bash',
            '-c',
            [
              'npx esbuild index.ts --bundle --platform=node --target=node22 --outfile=/asset-output/index.mjs --format=esm --external:@aws-sdk/*',
              'cp /asset-output/index.mjs /asset-output/index.js',
            ].join(' && '),
          ],
          local: {
            tryBundle(outputDir: string): boolean {
              try {
                const { execSync } = require('node:child_process');
                execSync(
                  `npx esbuild lambda/index.ts --bundle --platform=node --target=node22 --outfile=${outputDir}/index.js --format=esm --external:@aws-sdk/*`,
                  { cwd: path.join(__dirname, '..') },
                );
                return true;
              } catch {
                return false;
              }
            },
          },
        },
      }),
      memorySize: 512,
      timeout: cdk.Duration.seconds(60),
      logRetention: logs.RetentionDays.ONE_WEEK,
      environment: {
        NODE_OPTIONS: '--enable-source-maps',
      },
    });

    // Grant Bedrock model invocation permissions
    fn.addToRolePolicy(
      new iam.PolicyStatement({
        actions: [
          'bedrock:InvokeModel',
          'bedrock:InvokeModelWithResponseStream',
        ],
        resources: ['arn:aws:bedrock:*::foundation-model/*'],
      }),
    );

    // --- API Gateway REST API ---

    const api = new apigateway.RestApi(this, 'StreamingApi', {
      restApiName: 'streaming-llm-api',
      description: 'API Gateway REST with Lambda response streaming',
      deployOptions: {
        stageName: 'v1',
      },
    });

    const streamResource = api.root.addResource('stream');

    // OPTIONS for CORS preflight
    streamResource.addCorsPreflight({
      allowOrigins: apigateway.Cors.ALL_ORIGINS,
      allowMethods: ['POST', 'OPTIONS'],
      allowHeaders: ['Content-Type'],
    });

    // POST /stream -- Lambda proxy integration
    const postMethod = streamResource.addMethod(
      'POST',
      new apigateway.LambdaIntegration(fn, { proxy: true }),
    );

    // --- CfnMethod escape hatch for streaming ---
    const cfnMethod = postMethod.node.defaultChild as apigateway.CfnMethod;

    cfnMethod.addPropertyOverride(
      'Integration.ResponseTransferMode', 'STREAM',
    );
    cfnMethod.addPropertyOverride('Integration.TimeoutInMillis', 60_000);
    cfnMethod.addPropertyOverride(
      'Integration.Uri',
      cdk.Fn.sub(
        'arn:aws:apigateway:${AWS::Region}:lambda:path/2021-11-15/functions/${FnArn}/response-streaming-invocations',
        { FnArn: fn.functionArn },
      ),
    );

    // --- Outputs ---

    new cdk.CfnOutput(this, 'ApiUrl', {
      value: `${api.url}stream`,
      description: 'POST endpoint for streaming LLM responses',
    });
  }
}

A few things to note.

Node.js 22 with esbuild bundling. The local.tryBundle hook runs esbuild on your machine if it is available, falling back to Docker if not.

We bundle as ESM, exclude @aws-sdk/* (already in the Lambda runtime), and enable source maps for readable stack traces.

Bedrock permissions. We grant both bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream. The AI SDK uses the streaming variant, but having both keeps the door open for non-streaming calls in the same handler.

The CfnMethod escape hatch. As we covered in Part 1, CDK's L2 addMethod construct does not expose ResponseTransferMode or the response-streaming-invocations Lambda invocation URI.

We drop down to the L1 CfnMethod and override three properties: the transfer mode, the timeout, and the integration URI. This is the only CDK workaround required to enable response streaming through API Gateway REST.

CORS via addCorsPreflight. This generates the OPTIONS mock integration automatically. The httpCors Middy middleware adds the headers to the actual POST response.

How do you compose Middy and AI SDK middlewares for streaming?

The handler wires together two middleware systems: AI SDK middlewares (which operate on the LLM stream) and Middy middlewares (which operate on the HTTP response).

// lambda/index.ts

import middy from '@middy/core';
import httpCors from '@middy/http-cors';
import type { APIGatewayProxyEvent } from 'aws-lambda';
import { DataStoreMiddleware } from './ai-middleware/data-store-middleware';
import { LoggingMiddleware } from './ai-middleware/logging-middleware';
import { publishLog } from './middleware/publish-log';
import { streamErrorHandler } from './middleware/stream-error-handler';
import { streamingService } from './streaming-service';
import { llmLogDataStore } from './utils/log-data-store';

type HttpStreamResponse = {
  body: ReadableStream<Uint8Array> | string;
  headers: Record<string, string>;
  statusCode: number;
};

const logger = {
  info: (...args: Array<unknown>) => console.log('[INFO]', ...args),
  warn: (...args: Array<unknown>) => console.warn('[WARN]', ...args),
  error: (...args: Array<unknown>) => console.error('[ERROR]', ...args),
};

const onError = streamErrorHandler(logger);

// AI SDK middlewares -- operate on the LanguageModel stream
const aiMiddlewares = [
  LoggingMiddleware(logger),
  DataStoreMiddleware(llmLogDataStore.set),
];

const streamHandler = async (
  event: APIGatewayProxyEvent,
): Promise<HttpStreamResponse> => {
  const body = JSON.parse(event.body ?? '{}');
  const prompt: string =
    body.prompt ?? 'Summarize the benefits of serverless architecture';

  const result = streamingService({ prompt }, onError, aiMiddlewares);
  const response = result.toTextStreamResponse();

  return {
    body: response.body ?? '',
    headers: Object.fromEntries(response.headers.entries()),
    statusCode: response.status,
  };
};

// Middy middleware chain:
//   1. httpCors         -- adds CORS headers
//   2. publishLog       -- before: clears store
//                          after (streaming): wraps body with TransformStream
//                          after (non-streaming): publishes immediately
//
// Data flow for a streaming response:
//
//   streamText()
//     -> AI SDK stream (LanguageModelV3StreamPart)
//       -> LoggingMiddleware.wrapStream    (flush: logs completion)
//       -> DataStoreMiddleware.wrapStream  (flush: stores data)
//     -> toTextStreamResponse()
//       -> ReadableStream<Uint8Array>
//         -> publishLog TransformStream    (flush: publishes stored data)
//           -> Middy pipes to Lambda responseStream
//             -> API Gateway streams to client

export const handler = middy<APIGatewayProxyEvent, HttpStreamResponse>({
  streamifyResponse: true,
})
  .use(httpCors({ origins: ['*'] }))
  .use(publishLog(llmLogDataStore, logger))
  .handler(streamHandler);

The data flow comment is the map of the entire pipeline. Read it bottom to top to see how data reaches the client. Read it top to bottom to see where each flush() fires.

flowchart LR
    A["streamText()"] --> B["LoggingMiddleware\n(TransformStream)"]
    B --> C["DataStoreMiddleware\n(TransformStream)"]
    C --> D["toTextStreamResponse()"]
    D --> E["publishLog\n(TransformStream)"]
    E --> F["Lambda\nresponseStream"]
    F --> G["API Gateway\n→ Client"]

    B -. "flush: log completion" .-> B
    C -. "flush: store data" .-> C
    E -. "flush: publish log" .-> E

    style B fill:#e1f5fe
    style C fill:#e1f5fe
    style E fill:#fff3e0

Three TransformStream instances are chained together:

LoggingMiddleware: wraps the AI SDK's raw LanguageModelV3StreamPart stream. On flush, it logs the chunk count and model ID.
DataStoreMiddleware: wraps the same stream (after logging). On flush, it writes the full request params and response data into the in-memory store.
publishLog: wraps the final ReadableStream<Uint8Array> (after toTextStreamResponse() converts the AI SDK stream to bytes). On flush, it reads the store and publishes the log.

The in-memory llmLogDataStore is the bridge between the two middleware systems. AI SDK middlewares write to it. The Middy middleware reads from it.

Both are connected through flush() ordering: the AI SDK flushes fire first (they are upstream in the pipeline), then the Middy flush fires last (it is downstream, wrapping the byte stream).

Testing the Pipeline

We test three units independently: the stream transform utility, the publish log middleware, and the stream error handler.

createStreamTransform

This is the generic utility that wraps any ReadableStream with a TransformStream and calls a callback on flush.

// test/create-stream-transform.test.ts

describe('createStreamTransform', () => {
  it('passes all chunks through unchanged', async () => {
    const source = new ReadableStream<string>({
      start(controller): void {
        controller.enqueue('hello');
        controller.enqueue(' ');
        controller.enqueue('world');
        controller.close();
      },
    });

    const onFlush = vi.fn();
    const transformed = createStreamTransform(source, onFlush);
    const result = await readStream(transformed);

    expect(result).toEqual(['hello', ' ', 'world']);
  });

  it('calls onFlush with all accumulated chunks after stream ends', async () => {
    const source = new ReadableStream<string>({
      start(controller): void {
        controller.enqueue('a');
        controller.enqueue('b');
        controller.enqueue('c');
        controller.close();
      },
    });

    const onFlush = vi.fn();
    const transformed = createStreamTransform(source, onFlush);
    await readStream(transformed);

    expect(onFlush).toHaveBeenCalledOnce();
    expect(onFlush).toHaveBeenCalledWith(['a', 'b', 'c']);
  });

  it('swallows errors thrown in onFlush', async () => {
    const source = new ReadableStream<string>({
      start(controller): void {
        controller.enqueue('data');
        controller.close();
      },
    });

    const onFlush = vi.fn().mockRejectedValue(new Error('publish failed'));
    const transformed = createStreamTransform(source, onFlush);

    const result = await readStream(transformed);
    expect(result).toEqual(['data']);
    expect(onFlush).toHaveBeenCalledOnce();
  });

  it('does not call onFlush when stream is cancelled', async () => {
    const source = new ReadableStream<string>({
      start(controller): void {
        controller.enqueue('first');
        // Stream stays open -- simulates an ongoing LLM stream
      },
    });

    const onFlush = vi.fn();
    const transformed = createStreamTransform(source, onFlush);
    const reader = transformed.getReader();

    await reader.read(); // Read 'first'
    await reader.cancel('client disconnected');

    expect(onFlush).not.toHaveBeenCalled();
  });
});

Four behaviors, four tests. The critical ones are the last two: flush swallows errors (observability never breaks the client stream), and flush does not fire on cancellation (a client disconnect should not trigger publication of incomplete data).

publishLog

This tests the dual-path behavior of the Middy middleware: immediate publication for non-streaming responses, deferred publication for streaming responses.

// test/publish-log.test.ts

describe('publishLog middleware', () => {
  it('publishes immediately for non-streaming responses', async () => {
    const store = createLlmLogDataStore();
    const logger = createMockLogger();
    const middleware = publishLog(store, logger);

    store.set({
      llmParam: { model: 'test' },
      llmResult: { text: 'hello' },
    });

    const request = {
      response: {
        body: JSON.stringify({ result: 'ok' }),
        headers: {},
        statusCode: 200,
      },
    };

    await middleware.after!(request as never);

    expect(logger.info).toHaveBeenCalledWith(
      'LLM log data (non-streaming)',
      expect.objectContaining({ data: expect.any(Object) }),
    );
  });

  it('defers publication to flush() for streaming responses', async () => {
    const store = createLlmLogDataStore();
    const logger = createMockLogger();
    const middleware = publishLog(store, logger);

    const sourceStream = new ReadableStream<Uint8Array>({
      start(controller): void {
        controller.enqueue(new TextEncoder().encode('chunk1'));
        controller.enqueue(new TextEncoder().encode('chunk2'));
        controller.close();
      },
    });

    store.set({
      llmParam: { model: 'test' },
      llmResult: { tokens: 42 },
    });

    const request = {
      response: {
        body: sourceStream,
        headers: { 'content-type': 'text/plain' },
        statusCode: 200,
      },
    };

    // After hook wraps the stream -- does NOT publish yet
    await middleware.after!(request as never);
    expect(logger.info).not.toHaveBeenCalled();

    // Consume the wrapped stream -- flush() fires and publishes
    const body = await readStream(
      request.response.body as ReadableStream<Uint8Array>,
    );

    expect(body).toBe('chunk1chunk2');
    expect(logger.info).toHaveBeenCalledWith(
      'LLM log data (streaming)',
      expect.objectContaining({ data: expect.any(Object) }),
    );
  });
});

The key assertion is in the streaming test: after the after hook runs, logger.info has not been called. It is only called after we consume the stream with readStream(). This proves that the deferred execution model works.

streamErrorHandler

Streaming introduces error types you do not see in request/response handlers. The streamErrorHandler classifies them.

// test/stream-error-handler.test.ts

describe('streamErrorHandler', () => {
  it('logs AbortError at warn level', () => {
    const logger = createMockLogger();
    const handler = streamErrorHandler(logger);

    const error = new Error('The operation was aborted');
    error.name = 'AbortError';

    handler({ error });

    expect(logger.warn).toHaveBeenCalledOnce();
    expect(logger.error).not.toHaveBeenCalled();
  });

  it('logs ERR_INVALID_STATE as abort at warn level', () => {
    const logger = createMockLogger();
    const handler = streamErrorHandler(logger);

    const error = new TypeError('Invalid state: Reader released');
    (error as unknown as { code: string }).code = 'ERR_INVALID_STATE';

    handler({ error });

    expect(logger.warn).toHaveBeenCalledOnce();
    expect(logger.error).not.toHaveBeenCalled();
  });

  it('detects nested AbortError in cause chain', () => {
    const logger = createMockLogger();
    const handler = streamErrorHandler(logger);

    const cause = new Error('aborted');
    cause.name = 'AbortError';
    const error = new Error('AI SDK error', { cause });

    handler({ error });

    expect(logger.warn).toHaveBeenCalledOnce();
    expect(logger.error).not.toHaveBeenCalled();
  });

  it('logs non-abort errors at error level', () => {
    const logger = createMockLogger();
    const handler = streamErrorHandler(logger);

    handler({ error: new Error('connection timeout') });

    expect(logger.error).toHaveBeenCalledOnce();
    expect(logger.warn).not.toHaveBeenCalled();
  });
});

AbortError and ERR_INVALID_STATE are normal in streaming. They mean the client disconnected. The AI SDK wraps these in its own errors, so we walk the cause chain.

These go to warn, not error, because they are not actionable failures. Everything else goes to error.

The awslambda mock fixture

Testing streamifyResponse handlers outside the Lambda runtime requires mocking the awslambda global. The test fixture creates a minimal implementation:

// test/fixtures/streaming.fixtures.ts

export const setupAwsLambdaStreamingMock = (): void => {
  (globalThis as unknown as { awslambda: unknown }).awslambda = {
    HttpResponseStream: {
      from: (
        stream: LambdaResponseStream,
        metadata: HttpResponseMetadata,
      ): LambdaResponseStream => {
        stream.__httpResponseMetadata = metadata;
        return stream;
      },
    },
    streamifyResponse: <T>(handler: T): T => handler,
  };
};

streamifyResponse is a pass-through (it returns the handler as-is). HttpResponseStream.from attaches metadata to the writable stream. This is enough to test the full Middy chain locally without deploying.

Deployment and Verification

Prerequisites

Node.js 22+
AWS CLI configured with credentials
AWS CDK CLI (npm install -g aws-cdk)
An AWS account with Bedrock model access enabled for Claude 3.5 Sonnet v2

Deploy

cd step-4
npm install
npx cdk deploy

CDK will output the API URL:

Outputs:
StreamingLlmStack.ApiUrl = https://abc123.execute-api.us-east-1.amazonaws.com/v1/stream

Test with curl

curl -i --no-buffer -X POST \
  https://<api-id>.execute-api.<region>.amazonaws.com/v1/stream \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "Explain the benefits of serverless architecture"}'

The --no-buffer flag tells curl to display chunks as they arrive instead of buffering the entire response. You should see:

Headers arrive immediately. Transfer-Encoding: chunked, Content-Type: text/plain; charset=utf-8.
A 1-2 second pause. This is Lambda's initial buffering, not a bug. Lambda buffers roughly the first 6 KB before beginning to stream.
Chunks arriving progressively. JSON fragments flowing in as Bedrock generates them.
The connection closes when the stream is complete.

Verify observability

Open CloudWatch Logs for the Lambda function. After the response finishes streaming, you should see two log entries:

[INFO] LLM streaming call completed: from the LoggingMiddleware flush, with chunk count and model ID.
[INFO] LLM log data (streaming): from the publishLog flush, with the full request parameters and response data.

Both entries appear after the last byte has been sent to the client. This is the whole point.

If we had relied on Middy's after hook, these logs would appear before any streaming started, with no data to report.

Run the tests locally

npm test

Tear down

npx cdk destroy

What are the production considerations for streaming Lambda responses?

The companion code is intentionally minimal. Here is what you would add before shipping to production.

Structured logging. Replace the console.log wrapper with Powertools for AWS Lambda (TypeScript). It gives you structured JSON logs, correlation IDs, and log sampling out of the box.

Event publishing. The publishLog middleware currently writes to CloudWatch. In production, replace logger.info('LLM log data (streaming)', { data }) with a publish to SNS, Kinesis Data Firehose, or whatever event bus your platform uses. The flush() pattern works the same regardless of the destination.

Authentication. Add a Middy before hook that validates a JWT or calls an authorizer. It runs before the handler, so it has no interaction with the streaming pipeline.

Request validation. Add a Zod schema and a parser middleware to validate the incoming body. Reject bad input before you invoke Bedrock.

VPC considerations. Lambda Function URLs do not support response streaming when the function is in a VPC. API Gateway REST does, because it uses the InvokeWithResponseStream API from its own managed infrastructure.

If your Lambda needs VPC access (to reach an RDS database, for example), API Gateway REST is your path.

Timeout alignment. Set the API Gateway integration timeout equal to or greater than the Lambda function timeout. In our stack, both are 60 seconds. If the API Gateway timeout is shorter, it will close the connection while Lambda is still streaming, and the client will get a truncated response with no error signal.

Pricing and quotas. Lambda response streaming uses InvokeWithResponseStream, which is billed identically to standard Invoke (duration plus memory). However, streaming responses are not interrupted when the client disconnects. You are billed for the full function duration even if the client closes the connection mid-stream.

API Gateway REST has a hard 10 MB payload limit for streamed responses. Lambda supports up to 200 MB via InvokeWithResponseStream. For LLM responses (typically tens of KB), neither limit is a concern.

The default integration timeout of 29 seconds can be increased for regional and private APIs through Service Quotas.

Series Recap

Four posts, one problem, one solution.

Part 1: The Architecture. API Gateway REST can stream Lambda responses using ResponseTransferMode: STREAM and the response-streaming-invocations URI. Middy's streamifyResponse option preserves the full middleware chain. The AI SDK provides a clean interface to Bedrock with streamText, Output.object(), and wrapLanguageModel.

Part 2: The Problem. Middy's after hook fires when the handler returns, not when the stream body has been consumed. For streaming responses, the handler returns a ReadableStream that has not been read yet. Any observability work in after runs against empty data.

Part 3: The Solution. The WHATWG Streams TransformStream has a flush() callback that fires after all chunks have passed through. We use this as a deferred execution point: accumulate data during transform(), publish it during flush(). The AI SDK's wrapStream middleware pattern lets us do the same at the LLM stream level.

Part 4: The Implementation. A complete CDK stack, a composed Lambda handler, and a test suite that verifies every behavior.

The key takeaway: when working with streaming responses in serverless, attach your observability to the data pipeline, not to lifecycle hooks. Middleware frameworks were designed for request/response. Streams are a different paradigm.

The WHATWG Streams spec gives you flush() as a guaranteed execution point after stream completion. Use it.

The complete source code is available in the step-4 branch of the companion repository.

References

Building responsive APIs with Amazon API Gateway response streaming: AWS announcement (Nov 2025)
Response streaming for Lambda functions: AWS Lambda Developer Guide
Bedrock model access configuration: enabling Claude in your account
WHATWG Streams Standard: TransformStream specification
AI SDK streamText reference: Vercel AI SDK
AI SDK Middleware: LanguageModelMiddleware
AI SDK Amazon Bedrock Provider: Bedrock integration
Middy streaming integration: Middy documentation
Powertools for AWS Lambda (TypeScript): structured logging
Running code after returning a response from Lambda: post-response execution caveats

Deferred Observability for Streaming Lambda with TransformStream flush()

Pablo Albaladejo — Sun, 15 Feb 2026 11:50:12 +0000

In Part 2, we diagnosed a fundamental timing problem: Middy's after hook fires before the stream body is consumed by the Lambda runtime. The LLM data we need to publish simply does not exist yet when the hook runs. No amount of clever async orchestration can fix this. The data is not late; it has not been produced.

That diagnosis led us to a core insight: observability must follow data, not lifecycle. HTTP middleware frameworks model request handling as a sequence of phases: before, handler, after. That model works when the response is a finished value. It breaks when the response is a stream that has not been read yet.

So we stop relying on lifecycle hooks for publication. Instead, we attach observability to the data pipeline itself.

The mechanism is a TransformStream whose flush() callback fires at exactly the right moment: after every byte has flowed through, right before the stream signals completion to the consumer. This is not an implementation detail we stumbled into. It is a guarantee defined in the WHATWG Streams Standard.

TL;DR: Use TransformStream.flush() to defer observability work until after a stream completes. The WHATWG Streams spec guarantees flush() fires only after all chunks have passed through. We build two layers: AI SDK middlewares (wrapStream) collect data during streaming, and a Middy middleware wraps the HTTP response with a final TransformStream that publishes everything on flush(). An in-memory store bridges the two middleware systems (we call this the Dual Middleware Bridge pattern).

How does TransformStream flush() guarantee deferred execution?

The Web Streams API ships a TransformStream class that sits between a readable and a writable side. You give it a transformer object with up to three callbacks:

transform(chunk, controller): called for every chunk that enters the writable side. You process the chunk (or pass it through unchanged) and enqueue it to the readable side via controller.enqueue().
flush(controller): called once, after the writable side closes successfully and every transform() promise has resolved.
cancel(reason): called if the readable side is cancelled by the consumer.

The flush() contract comes directly from the WHATWG Streams Standard:

flush() is called only after all calls to transform() have completed successfully and the writable side has been closed.

When does flush() not fire?

Scenario	`flush()` called?
All chunks processed, stream closed normally	Yes
`transform()` rejects (throws)	No
Writable side is aborted	No
Readable side is cancelled (e.g., client disconnect)	No. `cancel()` fires instead

This is exactly the semantic we need. flush() gives us a guaranteed callback that fires once, after all data has passed through, and only if the stream completed successfully.

If the client disconnects and the stream aborts, flush() never fires, which is correct because partial LLM output is not worth publishing.

The createStreamTransform Utility

We start with a small, generic utility. All the code in this post comes from step-3 of the companion repo.

// lambda/utils/create-stream-transform.ts

export const createStreamTransform = <T>(
  stream: ReadableStream<T>,
  onFlush: (chunks: Array<T>) => void | Promise<void>,
): ReadableStream<T> => {
  const chunks: Array<T> = [];

  return stream.pipeThrough(
    new TransformStream<T, T>({
      transform(chunk: T, controller): void {
        chunks.push(chunk);
        controller.enqueue(chunk);
      },

      async flush(): Promise<void> {
        try {
          await onFlush(chunks);
        } catch {
          // Swallow errors: observability failures must never
          // break the stream pipeline reaching the client.
        }
      },
    }),
  );
};

This is a tap: it observes the stream without modifying it. Every chunk passes through to the next stage unchanged, while being accumulated in an internal array. When the stream completes, onFlush receives the full array of chunks.

The try/catch in flush() is critical. If onFlush throws (say, an SNS publish fails or a CloudWatch API call times out), the error is swallowed.

If we let it propagate, the WHATWG spec says the readable side enters an error state, which would break the stream for the client. The rule is simple: observability failures must never become user-facing failures.

Note the generic type parameter <T>. This utility works with LanguageModelV3StreamPart (AI SDK's internal stream type) and Uint8Array (the serialized HTTP response bytes) alike. We will use it at both levels.

Two Middleware Systems

Our architecture involves two independent middleware systems that need to cooperate:

AI SDK middleware (LanguageModelMiddleware with wrapStream): operates on the raw LLM stream. Each chunk is a LanguageModelV3StreamPart: a typed union of token deltas, usage metadata, finish reasons, and so on.
Middy middleware: operates on the HTTP response after toTextStreamResponse() has serialized the AI SDK stream into a ReadableStream<Uint8Array>. At this level, chunks are raw bytes.

These two systems run at different layers and at different times. The AI SDK middlewares wrap the model before streamText() is called. The Middy middleware wraps the response body after the handler returns.

They need a bridge.

The bridge is an in-memory store: a simple Map that the AI SDK middleware writes to (during its flush()) and the Middy middleware reads from (during its own flush()):

// lambda/utils/log-data-store.ts

export type LlmLogData =
  | {
      llmParam: Record<PropertyKey, unknown>;
      llmResult: Record<PropertyKey, unknown>;
    }
  | undefined;

export type LlmLogDataStore = {
  clear: () => void;
  get: () => LlmLogData;
  set: (data: LlmLogData) => void;
};

export const createLlmLogDataStore = (): LlmLogDataStore => {
  const storage = new Map<string, LlmLogData>();
  const KEY = 'llmLogData';

  return {
    clear: (): void => { storage.clear(); },
    get: (): LlmLogData => storage.get(KEY),
    set: (data: LlmLogData): void => { storage.set(KEY, data); },
  };
};

export const llmLogDataStore = createLlmLogDataStore();

A Map is safe here because Lambda is single-threaded. One request at a time, one reader and one writer, no race conditions. The before hook clears the store at the start of each invocation.

AI SDK Middlewares

We build two AI SDK middlewares, both using createStreamTransform.

LoggingMiddleware

Logs model response metadata (chunk count, model ID) when the stream completes:

// lambda/ai-middleware/logging-middleware.ts

import type { LanguageModelMiddleware } from 'ai';
import { createStreamTransform } from '../utils/create-stream-transform';

type Logger = {
  info: (...args: Array<unknown>) => void;
};

export const LoggingMiddleware = (logger: Logger): LanguageModelMiddleware => ({
  specificationVersion: 'v3',

  wrapGenerate: async ({ doGenerate }) => {
    const result = await doGenerate();
    logger.info('LLM call completed', {
      usage: result.usage,
      modelId: result.response?.modelId,
    });
    return result;
  },

  wrapStream: async ({ doStream }) => {
    const result = await doStream();

    return {
      ...result,
      stream: createStreamTransform(result.stream, (streamParts) => {
        logger.info('LLM streaming call completed', {
          chunkCount: streamParts.length,
        });
      }),
    };
  },
});

DataStoreMiddleware

Captures both the input parameters and the streaming result, then writes them to the shared store:

// lambda/ai-middleware/data-store-middleware.ts

import type { LanguageModelMiddleware } from 'ai';
import { createStreamTransform } from '../utils/create-stream-transform';
import type { LlmLogData } from '../utils/log-data-store';

export const DataStoreMiddleware = (
  setData: (data: LlmLogData) => void,
): LanguageModelMiddleware => ({
  specificationVersion: 'v3',

  wrapGenerate: async ({ doGenerate, params }) => {
    const result = await doGenerate();
    setData({
      llmParam: params as Record<PropertyKey, unknown>,
      llmResult: result as Record<PropertyKey, unknown>,
    });
    return result;
  },

  wrapStream: async ({ doStream, params }) => {
    const result = await doStream();

    return {
      ...result,
      stream: createStreamTransform(result.stream, (streamParts) => {
        setData({
          llmParam: params as Record<PropertyKey, unknown>,
          llmResult: {
            request: result.request,
            response: result.response,
            streamParts,
          },
        });
      }),
    };
  },
});

Both middlewares follow the same shape. In wrapGenerate, the data is available synchronously after doGenerate() resolves, so no TransformStream is needed.

In wrapStream, data only becomes available after the entire stream is consumed, so we use createStreamTransform to defer our work to flush().

The middlewares are composed onto the model via wrapLanguageModel():

const aiMiddlewares = [
  LoggingMiddleware(logger),
  DataStoreMiddleware(llmLogDataStore.set),
];

const model = wrapLanguageModel({
  model: baseModel,
  middleware: aiMiddlewares,
});

How do you publish observability data after a stream completes?

This is the Middy middleware that actually solves the timing problem from Part 2. It is a dual-path middleware: it handles both streaming and non-streaming responses.

// lambda/middleware/publish-log.ts

import type middy from '@middy/core';
import type { LlmLogDataStore } from '../utils/log-data-store';

type StreamResponse = {
  body: ReadableStream<Uint8Array>;
  headers: Record<string, string>;
  statusCode: number;
};

type Logger = {
  info: (...args: Array<unknown>) => void;
  error: (...args: Array<unknown>) => void;
};

const isStreamResponse = (response: unknown): response is StreamResponse =>
  typeof response === 'object' &&
  response !== null &&
  'body' in response &&
  (response as StreamResponse).body instanceof ReadableStream;

export const publishLog = (
  llmLogDataStore: LlmLogDataStore,
  logger: Logger,
): middy.MiddlewareObj => {
  const publishImmediately = async (): Promise<void> => {
    const data = llmLogDataStore.get();
    if (!data) return;

    try {
      logger.info('LLM log data (non-streaming)', { data });
    } catch (error) {
      logger.error('Failed to publish LLM log data', { error });
    } finally {
      llmLogDataStore.clear();
    }
  };

  const publishAfterStreamConsumption = (request: middy.Request): void => {
    const response = request.response as StreamResponse;

    const publishTransform = new TransformStream<Uint8Array, Uint8Array>({
      transform(chunk: Uint8Array, controller): void {
        controller.enqueue(chunk);
      },

      async flush(): Promise<void> {
        try {
          const data = llmLogDataStore.get();
          if (!data) return;

          logger.info('LLM log data (streaming)', { data });
        } catch (error) {
          logger.error('Failed to publish LLM log data', { error });
        } finally {
          llmLogDataStore.clear();
        }
      },
    });

    (request.response as StreamResponse).body =
      response.body.pipeThrough(publishTransform);
  };

  return {
    before: (): void => {
      llmLogDataStore.clear();
    },

    after: async (request: middy.Request): Promise<void> => {
      if (isStreamResponse(request.response)) {
        publishAfterStreamConsumption(request);
        return;
      }

      await publishImmediately();
    },

    onError: async (): Promise<void> => {
      await publishImmediately();
    },
  };
};

Read the after hook carefully. For streaming responses, it does not publish. It wraps the response body with a new TransformStream.

The actual publication happens inside that TransformStream's flush(), which fires later, after the Lambda runtime has consumed every byte.

For non-streaming responses (e.g., a handler that uses generateObject instead of streamText), the data is already available, so we publish immediately. The isStreamResponse type guard decides which path to take.

The before hook clears the store at the start of each invocation.

The onError hook publishes whatever data exists if the handler throws before streaming starts. Partial data can still be valuable for debugging failures.

The Complete Pipeline

When the Lambda runtime consumes the response body, data flows through a chain of TransformStreams:

streamText()
  -> AI SDK stream (LanguageModelV3StreamPart)
    -> LoggingMiddleware.wrapStream    (flush: logs completion)
    -> DataStoreMiddleware.wrapStream  (flush: stores data in Map)
  -> toTextStreamResponse()
    -> ReadableStream<Uint8Array>
      -> publishLog TransformStream    (flush: reads Map, publishes)
        -> Middy pipes to Lambda responseStream
          -> API Gateway streams to client

Here is the same pipeline as a visual diagram:

flowchart LR
    A["streamText()"] --> B["LoggingMiddleware\n(TransformStream)"]
    B --> C["DataStoreMiddleware\n(TransformStream)"]
    C --> D["toTextStreamResponse()"]
    D --> E["publishLog\n(TransformStream)"]
    E --> F["Lambda\nresponseStream"]
    F --> G["API Gateway\n→ Client"]

    B -. "flush: log completion" .-> B
    C -. "flush: store data" .-> C
    E -. "flush: publish log" .-> E

    style B fill:#e1f5fe
    style C fill:#e1f5fe
    style E fill:#fff3e0

The three colored boxes are TransformStream instances. The blue ones are AI SDK middlewares operating on LanguageModelV3StreamPart chunks. The orange one is the Middy middleware operating on Uint8Array bytes after toTextStreamResponse() serialization.

The flush() callbacks execute in pipeline order. This is guaranteed by how pipeThrough chains work: the upstream TransformStream must close its readable side before the downstream TransformStream's writable side closes, which means upstream flush() completes before downstream flush() is invoked.

The sequence:

LoggingMiddleware.flush(): logs the LLM streaming result to CloudWatch.
DataStoreMiddleware.flush(): writes llmParam and llmResult into the in-memory store.
publishLog.flush(): reads from the store and publishes to the observability backend (SNS, Kinesis, CloudWatch, or whatever you need).

Step 2 sets the data. Step 3 reads it. This ordering is not a coincidence we hope holds. It is a structural guarantee of pipeThrough chains in the WHATWG spec.

Upstream flushes complete before downstream flushes begin.

Here is how the handler wires everything together:

// lambda/index.ts

const aiMiddlewares = [
  LoggingMiddleware(logger),
  DataStoreMiddleware(llmLogDataStore.set),
];

const streamHandler = async (
  event: APIGatewayProxyEvent,
): Promise<HttpStreamResponse> => {
  const body = JSON.parse(event.body ?? '{}');
  const prompt: string = body.prompt ?? 'Summarize the benefits of serverless architecture';

  const result = streamingService({ prompt }, onError, aiMiddlewares);
  const response = result.toTextStreamResponse();

  return {
    body: response.body ?? '',
    headers: Object.fromEntries(response.headers.entries()),
    statusCode: response.status,
  };
};

export const handler = middy<APIGatewayProxyEvent, HttpStreamResponse>({
  streamifyResponse: true,
})
  .use(httpCors({ origins: ['*'] }))
  .use(publishLog(llmLogDataStore, logger))
  .handler(streamHandler);

Nothing in this handler blocks on observability. The stream starts flowing to the client immediately. The TransformStreams execute their flush() callbacks transparently as part of the pipe chain, after every byte has been delivered.

Why is TransformStream flush() better than AI SDK onFinish?

AI SDK provides an onFinish callback in streamText(). You might wonder why we did not just use that.

The problem is that onFinish is fire-and-forget. The AI SDK calls your callback but does not await the returned promise. From the AI SDK source:

// Simplified from AI SDK internals
onFinish?.({ usage, object, ... }); // No await

This means:

If your onFinish handler does async work (publishing to SNS, writing to a database), the Lambda runtime may freeze the execution context before that work completes.
There is no backpressure. The stream closes and the response ends regardless of whether your callback finished.
There is no error propagation. If onFinish throws, nobody catches it.

Our flush() approach is fundamentally different:

flush() is part of the pipeline. It executes as a step in the pipe chain. The downstream consumer does not see the stream as "closed" until flush() resolves. The Lambda runtime will not freeze the context while flush() is awaiting an SNS publish.
flush() only fires on success. If the stream aborts (client disconnect, timeout), flush() never runs. We do not publish partial, potentially misleading data.
flush() ordering is deterministic. In a chain of pipeThrough calls, upstream flushes complete before downstream flushes begin. We rely on this to ensure the data store is populated before we read from it.

The difference is not subtle. onFinish is a notification. flush() is a pipeline stage. One is best-effort. The other is structurally guaranteed.

Recap

The pattern we built in this post can be summarized in one sentence: attach observability to the data pipeline using TransformStream flush(), instead of relying on middleware lifecycle hooks that fire at the wrong time.

Three properties make this work:

WHATWG flush() guarantee: fires once, after all transform() promises resolve and the writable side closes successfully. This is not a Node.js quirk. It is a web standard implemented consistently across runtimes.
Pipeline ordering: in a pipeThrough chain, upstream flush() completes before downstream flush() begins. This lets us write data in one TransformStream and read it in another, with guaranteed ordering.
Error isolation: try/catch inside flush() ensures observability failures never break the user-facing stream. The client gets their complete response regardless of whether our publish succeeded.

The createStreamTransform utility is 24 lines of code. The publishLog middleware is under 90. Together, they solve a problem that has no clean solution in traditional middleware architectures.

What's Next

We have the pattern. We have the code. But how do we know it actually works? How do we verify that flush() fires in the right order, that errors are properly swallowed, that aborted streams skip publication?

In Part 4, we put it all together: a complete CDK stack you can deploy in 5 minutes, with tests that verify the TransformStream timing guarantees.

We will write unit tests that prove the flush ordering, integration tests that confirm end-to-end observability, and cover the edge cases: client disconnects, LLM timeouts, and publish failures.

References

WHATWG Streams Standard: TransformStream: the specification that guarantees flush() behavior
MDN: TransformStream: API reference
MDN: ReadableStream.pipeThrough(): chaining TransformStreams
Node.js Web Streams API: Node.js 22 implementation
AI SDK Middleware: LanguageModelMiddleware with wrapStream
AI SDK wrapLanguageModel: composing model with middlewares
Cloudflare Workers TransformStream: validation of the "tap" pattern in production
New Relic: Serverless response streaming with AI observability: similar generator proxy pattern
Is onFinish fire-and-forget?: Vercel community confirmation

The Middy After Hook Problem: Why Streaming Lambda Observability Is Broken

Pablo Albaladejo — Sun, 15 Feb 2026 11:50:08 +0000

In Part 1 we built a streaming Lambda that pipes an LLM response through API Gateway to the client using Middy and the AI SDK. It works. The client sees tokens arrive in real time, structured JSON builds incrementally in the browser, and the architecture is clean.

Now we want observability. We want to log the full LLM response, publish token usage metrics, and record the structured output for auditing. The natural place for this in a Middy-based Lambda is an after hook. That is exactly what we tried. And it is exactly what does not work.

The problem is a timing issue between Middy's middleware lifecycle and the streaming response lifecycle. It is not documented in the Middy docs, the AWS Lambda streaming docs, or any community resource we could find.

We spent significant time diagnosing it, and this post is the documentation that should have existed.

TL;DR: Middy's after hook fires at T1 (when the handler returns the ReadableStream), but the stream body is consumed at T2-T4. Any observability code in after (logging, metrics, publishing) sees empty data because the stream has not been read yet. onFinish is fire-and-forget and Lambda post-response execution is unreliable. The solution is in Part 3: TransformStream.flush().

Why does Middy's after hook see empty data when streaming?

Here is what you would naturally write. A Middy after hook that reads from a data store populated by the LLM call and publishes it:

const publishLlmLog: middy.MiddlewareObj = {
  after: async (request) => {
    const payload = llmLogDataStore.get();
    if (!payload) return;

    await publishToSns({
      payload: {
        llmParam: payload.llmParam,
        llmResult: payload.llmResult,
      },
      projectName: 'my-service',
    });

    llmLogDataStore.clear();
  },
};

This pattern works perfectly for non-streaming responses. The handler calls the LLM, waits for the full result, stores it, and returns a JSON body.

Middy's after hook fires after the handler returns, reads the store, publishes the data. Everything is sequential and deterministic.

For a streaming Lambda, the handler looks like this:

const streamHandler = async (
  event: APIGatewayProxyEvent,
): Promise<HttpStreamResponse> => {
  const body = JSON.parse(event.body ?? '{}');
  const prompt: string = body.prompt ?? 'Summarize the benefits of serverless architecture';

  const result = streamingService({ prompt }, onError);
  const response = result.toTextStreamResponse();

  return {
    body: response.body ?? '',
    headers: Object.fromEntries(response.headers.entries()),
    statusCode: response.status,
  };
};

The handler returns a ReadableStream as the body. It does not wait for the stream to finish. The LLM has barely started generating tokens. The data store is empty.

When does Middy's after hook fire relative to stream consumption?

This is the exact sequence of events when Middy processes a streaming response:

T0  Handler returns { body: ReadableStream, statusCode: 200 }
    |
T1  Middy after hook fires             <-- stream is UNREAD
    |                                       llmLogDataStore.get() === undefined
    |
T2  Middy pipes response.body to Lambda responseStream
    |
T3  Client receives chunks             <-- tokens flowing
    |   ...
    |   ...
T4  Stream ends                         <-- data is HERE
        DataStoreMiddleware.flush() populates the store
        But the after hook already ran at T1.

The after hook runs at T1. The data you need only exists at T4. The gap between T1 and T4 can be seconds or minutes, depending on how long the LLM takes to generate the full response.

Here is another way to see it. In a non-streaming Lambda, the timeline is linear:

Handler executes -> LLM completes -> Store populated -> Handler returns -> After hook fires
                                                                           (store has data)

In a streaming Lambda, the handler returns before the LLM completes:

Handler executes -> LLM starts -> Handler returns -> After hook fires -> LLM streams -> LLM completes
                                                      (store is empty)                   (store has data)

The after hook and the stream consumption are on different timelines. The handler returning does not mean the work is done. It means the work has barely started.

Why Tests Don't Catch It

This is the part that delayed our diagnosis. The unit tests passed. Every time.

In a typical unit test for this handler, you mock the AI SDK service:

vi.mock('./streaming-service', () => ({
  streamingService: () => ({
    toTextStreamResponse: () => new Response('{"summary":"test"}'),
  }),
}));

The mock returns a resolved Response with a complete body. When the test calls the handler, the mock populates the data store synchronously. By the time the after hook runs in the test, the store has data. The test passes.

In production, streamText() returns immediately with an unresolved stream. The AI SDK creates a ReadableStream backed by an HTTP connection to the LLM. The data flows asynchronously through internal TransformStream chains as the model generates tokens.

The DataStoreMiddleware uses a TransformStream with a flush() callback that only fires after the last chunk passes through:

// This TransformStream wraps the LLM's internal stream.
// flush() fires only when the stream closes -- at T4.
return {
  ...result,
  stream: createStreamTransform(result.stream, (streamParts) => {
    setData({
      llmParam: params,
      llmResult: {
        request: result.request,
        response: result.response,
        streamParts,
      },
    });
  }),
};

The flush() callback runs at T4. The after hook runs at T1. In the mock, both happen at T0. The test collapses the entire timeline into a single tick and hides the race condition.

This is not a testing mistake. It is a fundamental limitation of mocking streams. The mock correctly simulates the interface but eliminates the temporal behavior that causes the bug.

Three Approaches That Don't Work

Before we found the solution (covered in Part 3), we explored three approaches that each fail for a different reason.

1. Move publishing to the AI SDK's `onFinish` callback

The AI SDK provides an onFinish callback on streamText(). It fires when the stream completes. That sounds like exactly what we need:

streamText({
  model,
  prompt,
  output: Output.object({ schema }),
  onFinish: async ({ object, usage }) => {
    await publishToSns({ llmResult: { object, usage } });
  },
});

The problem: onFinish is fire-and-forget. The AI SDK calls it but does not await the returned promise. The promise resolves (or rejects) silently in the background.

In Lambda, the execution environment can freeze or terminate after responseStream.end(). If onFinish triggers an async operation like an SNS publish, there is no guarantee it completes before the environment is reclaimed.

This is a known limitation. The Vercel community has discussed it in the context of Next.js serverless functions.

On Vercel's platform, waitUntil() can extend the function lifetime. On Lambda, no such mechanism exists for streaming functions.

2. Await the stream in the after hook, then reconstruct it

You could consume the ReadableStream in the after hook to extract the data:

const afterHook: middy.MiddlewareObj = {
  after: async (request) => {
    const response = request.response as HttpStreamResponse;
    if (!(response.body instanceof ReadableStream)) return;

    const reader = response.body.getReader();
    const chunks: Array<Uint8Array> = [];
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      if (value) chunks.push(value);
    }

    // Now we have the data... but the stream is consumed.
    // We need to give Middy a new stream to pipe to the client.
    response.body = new ReadableStream({
      start(controller) {
        for (const chunk of chunks) {
          controller.enqueue(chunk);
        }
        controller.close();
      },
    });

    await publishData(chunks);
  },
};

This technically works, but it defeats the purpose of streaming. You buffer the entire response in memory before the client receives a single byte. The TTFB becomes the full LLM generation time.

You have rebuilt a buffered response with extra steps.

3. Use Lambda post-response execution

Lambda continues executing code after responseStream.end() is called. You could set a flag when the stream completes and run your publish logic after:

// After the stream ends, Lambda still has execution time
void (async () => {
  for await (const chunk of result.textStream) {
    passThrough.write(chunk);
  }
  passThrough.end();

  // Post-stream work
  const data = llmLogDataStore.get();
  if (data) {
    await publishToSns(data);
  }
})();

The AWS documentation explicitly warns against relying on this:

"The runtime does not wait for asynchronous work to complete after the response stream has errored or been destroyed."

If the client disconnects mid-stream (a common event in production), the response stream is destroyed. Any async work scheduled after that point may not complete. Your publish call might execute, or it might not. There is no guarantee.

This approach also moves your observability logic into the handler's streaming IIFE, coupling it to the handler code and bypassing Middy's middleware chain entirely.

How do you handle errors in streaming Lambda responses?

Before we move to the solution, we need to address a related problem: error handling in a streaming Lambda has two distinct phases, and your error handler must account for both.

Pre-stream errors

Before the handler returns a ReadableStream, errors behave normally. A validation failure, an auth error, a missing parameter: these throw before any bytes are written.

Middy catches them, runs the onError chain, and returns a standard HTTP error response with an appropriate status code. This is the familiar path.

Mid-stream errors

Once the handler returns a ReadableStream and Middy starts piping it to the client, the HTTP status code (200) and headers are already sent. If the LLM fails mid-generation, or the client disconnects, or a timeout fires, the error manifests as a stream destruction.

There is no way to change the status code retroactively.

The AI SDK wraps these errors in nested cause chains. A client disconnect might surface as:

Error: Stream processing failed
  cause: TypeError [ERR_INVALID_STATE]: The reader is not attached to a stream

Or as a direct AbortError from the AbortController. Or as an AI SDK wrapper error with a nested AbortError in the cause property.

The streamErrorHandler in the step-2 companion code handles this by recursively checking the cause chain:

const isAbortError = (error: Error): boolean => {
  if (error.name === 'AbortError') return true;

  if (
    error.name === 'TypeError' &&
    'code' in error &&
    error.code === 'ERR_INVALID_STATE'
  )
    return true;

  if (error.cause instanceof Error) {
    return isAbortError(error.cause);
  }

  return false;
};

This function handles three cases:

Direct AbortError: the AbortController signal fired (timeout or explicit abort).
ERR_INVALID_STATE: a TypeError thrown when reading from a detached stream reader. This happens when the client disconnects and the underlying stream is destroyed while a for await loop is still reading from it.
Nested cause chains: the AI SDK wraps errors. A stream error from the HTTP connection gets wrapped in an AI SDK error, which may itself be wrapped in another error by the streaming pipeline. The AbortError can be several levels deep in the cause chain.

The log level strategy follows from the semantics: AbortErrors are expected in production (users close tabs, navigate away, connections drop) and get logged at warn level. All other errors are genuine failures and get logged at error level.

export const streamErrorHandler = (logger) => {
  return ({ error }: { error: unknown }): void => {
    if (!(error instanceof Error)) {
      logger.error('Unknown streaming error', { error });
      return;
    }

    const cause = extractCause(error);

    if (isAbortError(error)) {
      logger.warn('Stream aborted (client disconnect or timeout)', {
        cause,
        errorType: 'AbortError',
        message: error.message,
      });
      return;
    }

    logger.error('Streaming error', {
      cause,
      errorType: 'StreamError',
      message: error.message,
      stack: error.stack,
    });
  };
};

We also extract AI SDK-specific properties (text, usage) from the cause when present. The NoObjectGeneratedError, for example, includes the partial text the model generated before failing and the token usage at the time of failure.

These are valuable for debugging and cost tracking.

The Documentation Gap

We searched for this problem in every place you would expect to find it:

Middy documentation: The streaming section describes streamifyResponse: true and the PassThrough/ReadableStream pattern. It does not mention the after hook timing issue.
AWS Lambda streaming docs: Cover awslambda.streamifyResponse, response format, and limits. No mention of middleware timing.
AI SDK documentation: Covers onFinish, streamText, and stream protocols. Does not address the fire-and-forget nature of onFinish in serverless contexts.
Community posts and Stack Overflow: Multiple posts about Lambda streaming setup. None about middleware lifecycle interaction with streams.

The combination of Middy's streaming mode, the AI SDK's stream lifecycle, and Lambda's execution model creates a timing problem that each project documents only their part of. No single source connects the three.

What's Next

The problem is clear: Middy's after hook fires before the stream is consumed. The data we need for observability only exists after the stream ends. The three obvious workarounds each fail for a different structural reason.

The solution exists, and it is elegant. In Part 3, we will show how TransformStream's flush() callback gives us exactly what we need: a guaranteed execution point after the entire stream is consumed, running inline with the stream pipeline, before the Lambda execution environment can freeze.

It works within Middy's after hook, preserves the streaming behavior for the client, and requires no changes to the handler code.

The stream itself becomes the mechanism for its own observability.

References

Middy streaming integration: Middy documentation
Response streaming for Lambda functions: AWS Lambda Developer Guide
Running code after returning a response from Lambda: AWS blog on post-response execution
AI SDK streamText reference: Vercel AI SDK
Is onFinish fire-and-forget?: Vercel community discussion
Node.js AbortError is not DOMException: Node.js issue on AbortError behavior
AI SDK Issue #4101: Invalid state after controller closed: AI SDK error wrapping behavior
Understanding AbortController in Node.js: AbortSignal patterns

Streaming LLM Responses Through API Gateway REST with Lambda

Pablo Albaladejo — Sun, 15 Feb 2026 11:33:39 +0000

The problem with buffered LLM responses

When you call an LLM from a Lambda function and wait for the full response before returning it, the user stares at a spinner for 5 to 15 seconds. The model is generating tokens the entire time, but the client receives nothing until the very last one. For short completions this is tolerable. For structured output (summaries, extracted entities, multi-field JSON), the delay kills the experience.

The fix is response streaming: send tokens to the client as the model produces them. The user sees content appearing incrementally. Time-to-first-byte drops from seconds to hundreds of milliseconds. The perceived latency improves dramatically even though the total generation time stays the same.

In November 2025, AWS added native response streaming support to API Gateway REST APIs. Before that, streaming from Lambda required Function URLs (limited auth, no VPC support) or WebSocket APIs (more infrastructure, more complexity). Now we get streaming through the same REST API most teams already have in production.

This is the first post in a four-part series:

The infrastructure and handler setup (this post)
A subtle middleware timing bug that breaks observability when streaming (Part 2)
A deferred observability pattern that solves it (Part 3)
Production-grade testing for the whole thing (Part 4)

Every post has a companion step in the repository with working code you can deploy.

TL;DR: Stream structured LLM responses from Bedrock Claude through API Gateway REST and Lambda using Middy's streamifyResponse and AI SDK v6 streamText() with Output.object(). The CDK stack uses a CfnMethod escape hatch to enable ResponseTransferMode: STREAM. Time-to-first-byte drops from seconds to hundreds of milliseconds. Full working code included.

What this post covers

We will wire up a complete streaming pipeline: API Gateway REST receives a request, invokes a Lambda function in streaming mode, the Lambda uses Middy and the Vercel AI SDK to call Bedrock Claude via streamText, and the structured JSON streams back to the client chunk by chunk.

By the end, you will have a deployed endpoint you can curl and watch tokens arrive in real time.

How do you stream LLM responses through API Gateway REST?

Here is the data flow:

Client (curl / browser)
  |
  | POST /stream
  v
API Gateway REST (responseTransferMode: STREAM)
  |
  | response-streaming-invocations
  v
Lambda (Middy, streamifyResponse: true)
  |
  | streamText()
  v
Bedrock Claude 3.5 Sonnet

The client sends a POST request. API Gateway forwards it to Lambda using the streaming invocation endpoint. Lambda calls Bedrock Claude through the AI SDK's streamText, which returns a ReadableStream.

Middy detects the stream in the response body and pipes it to Lambda's response stream. API Gateway forwards each chunk to the client as it arrives.

Why API Gateway REST instead of Function URLs? Three reasons:

VPC support. Function URLs do not support response streaming from within a VPC. If your Lambda runs in a VPC to reach a database or private service, Function URLs are not an option.
Built-in auth and policies. REST APIs support IAM and Cognito authorizers, request validation, and usage plans that most teams already depend on.
Battle-tested CORS. CORS handling through mock integrations and gateway responses is battle-tested. Function URLs require you to handle CORS entirely in your Lambda code.

How do you configure API Gateway REST for response streaming?

Two properties on the API Gateway method integration enable streaming:

ResponseTransferMode: STREAM tells API Gateway to forward response bytes to the client as they arrive, rather than buffering the entire response.
The streaming invocation URI replaces the standard Lambda invoke path. Instead of .../invocations, you use .../response-streaming-invocations. This tells API Gateway to use the InvokeWithResponseStream API instead of the standard Invoke.

Wire format

When Lambda streams through an API Gateway proxy integration, the stream must follow a specific wire format:

JSON metadata: a valid JSON object containing statusCode, headers, and optionally cookies. This can be as minimal as {}.
An 8-null-byte delimiter: eight 0x00 bytes that separate metadata from body.
The response payload: the actual content chunks.

The delimiter must appear within the first 16 KB of the stream. If you use Middy with streamifyResponse: true, it handles this format automatically. You never write the delimiter yourself.

Timeouts and bandwidth

API Gateway REST streaming supports timeouts up to 15 minutes. There are two idle timeouts to be aware of:

Regional and private endpoints: 5 minutes of idle time before the connection terminates.
Edge-optimized endpoints: 30 seconds. For LLM streaming, avoid edge-optimized endpoints.

Bandwidth is throttled at two layers:

Layer	Uncapped window	Throttled rate
Lambda	First 6 MB	2 MB/s
API Gateway	First 10 MB	2 MB/s

For LLM responses (typically tens of KB), you will never hit these limits. They matter for large file downloads, not token streaming.

There are additional quotas to be aware of when using API Gateway REST with streaming:

Quota	Value	Notes
Max streamed payload (API Gateway)	10 MB	Hard limit, not increasable
Max streamed payload (Lambda)	200 MB	Via `InvokeWithResponseStream`
Integration timeout (default)	29 seconds	Increasable for regional and private APIs
Idle connection timeout	310 seconds	General API Gateway limit (separate from the streaming idle timeouts above)

For LLM streaming, the integration timeout is the one that matters most. The default 29-second limit may be too short for complex prompts. You can request an increase through Service Quotas, or use a longer timeout in your CDK stack (we use 60 seconds).

Pricing. Lambda response streaming uses InvokeWithResponseStream, which is billed identically to standard Invoke: duration plus memory.

However, streaming responses are not interrupted when the client disconnects. You are billed for the full function duration even if the client closes the connection mid-stream. Keep this in mind when setting timeouts.

CDK infrastructure

CDK's L2 LambdaIntegration construct does not expose ResponseTransferMode or the streaming invocation URI. We use the CfnMethod escape hatch to override the underlying CloudFormation resource directly.

Here is the full stack. Read it top to bottom. We will break down the important parts after.

// lib/streaming-stack.ts
import * as cdk from 'aws-cdk-lib';
import * as apigateway from 'aws-cdk-lib/aws-apigateway';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as logs from 'aws-cdk-lib/aws-logs';
import type { Construct } from 'constructs';
import * as path from 'node:path';

export class StreamingStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const fn = new lambda.Function(this, 'StreamingHandler', {
      runtime: lambda.Runtime.NODEJS_22_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset(path.join(__dirname, '..', 'lambda'), {
        bundling: {
          image: lambda.Runtime.NODEJS_22_X.bundlingImage,
          command: [
            'bash', '-c',
            'npx esbuild index.ts --bundle --platform=node --target=node22 --outfile=/asset-output/index.mjs --format=esm --external:@aws-sdk/* && cp /asset-output/index.mjs /asset-output/index.js',
          ],
        },
      }),
      memorySize: 512,
      timeout: cdk.Duration.seconds(60),
      logRetention: logs.RetentionDays.ONE_WEEK,
    });

    fn.addToRolePolicy(
      new iam.PolicyStatement({
        actions: [
          'bedrock:InvokeModel',
          'bedrock:InvokeModelWithResponseStream',
        ],
        resources: ['arn:aws:bedrock:*::foundation-model/*'],
      }),
    );

    const api = new apigateway.RestApi(this, 'StreamingApi', {
      restApiName: 'streaming-llm-api',
      deployOptions: { stageName: 'v1' },
    });

    const streamResource = api.root.addResource('stream');

    streamResource.addCorsPreflight({
      allowOrigins: apigateway.Cors.ALL_ORIGINS,
      allowMethods: ['POST', 'OPTIONS'],
      allowHeaders: ['Content-Type'],
    });

    const postMethod = streamResource.addMethod(
      'POST',
      new apigateway.LambdaIntegration(fn, { proxy: true }),
    );

    // Escape hatch: override CloudFormation for streaming
    const cfnMethod = postMethod.node.defaultChild as apigateway.CfnMethod;

    cfnMethod.addPropertyOverride(
      'Integration.ResponseTransferMode', 'STREAM',
    );
    cfnMethod.addPropertyOverride(
      'Integration.TimeoutInMillis', 60_000,
    );
    cfnMethod.addPropertyOverride(
      'Integration.Uri',
      cdk.Fn.sub(
        'arn:aws:apigateway:${AWS::Region}:lambda:path/2021-11-15/functions/${FnArn}/response-streaming-invocations',
        { FnArn: fn.functionArn },
      ),
    );

    new cdk.CfnOutput(this, 'ApiUrl', {
      value: `${api.url}stream`,
    });
  }
}

The three addPropertyOverride calls are the entire difference between a buffered API and a streaming one. Everything else is standard CDK. Note that we also grant bedrock:InvokeModelWithResponseStream because Bedrock needs this permission for streaming model calls, not just InvokeModel.

Lambda handler with Middy

The Lambda handler is where the streaming behavior is configured on the compute side. Middy's streamifyResponse: true option wraps the handler with awslambda.streamifyResponse() internally, so we do not call the native AWS streaming API ourselves.

// lambda/index.ts
import middy from '@middy/core';
import httpCors from '@middy/http-cors';
import type { APIGatewayProxyEvent } from 'aws-lambda';
import { streamingService } from './streaming-service';

type HttpStreamResponse = {
  body: ReadableStream<Uint8Array> | string;
  headers: Record<string, string>;
  statusCode: number;
};

const streamHandler = async (
  event: APIGatewayProxyEvent,
): Promise<HttpStreamResponse> => {
  const body = JSON.parse(event.body ?? '{}');
  const prompt: string =
    body.prompt ?? 'Summarize the benefits of serverless architecture';

  const result = streamingService({ prompt });
  const response = result.toTextStreamResponse();

  return {
    body: response.body ?? '',
    headers: Object.fromEntries(response.headers.entries()),
    statusCode: response.status,
  };
};

export const handler = middy<APIGatewayProxyEvent, HttpStreamResponse>({
  streamifyResponse: true,
})
  .use(httpCors({ origins: ['*'] }))
  .handler(streamHandler);

The handler returns an object with { body, headers, statusCode }. When body is a ReadableStream<Uint8Array>, Middy detects this and pipes it to Lambda's response stream. When body is a plain string, Middy writes it as a buffered response.

This is the key difference from the native awslambda.streamifyResponse() pattern, where you receive a Writable stream as a handler argument and write to it directly.

The Middy pattern is valuable because it preserves the middleware chain. Middlewares like httpCors run in the before phase (setting up CORS headers) and the after phase (attaching them to the response).

The handler remains a pure function that takes an event and returns a response object; the streaming mechanics are abstracted away.

The HttpStreamResponse type accepts either ReadableStream<Uint8Array> or string for the body. This matters: if your handler needs to return an error before starting the stream (validation failure, auth error), it returns a plain string body with the appropriate status code. The stream is only used for the happy path.

How do you use AI SDK streamText with Bedrock Claude on Lambda?

The service layer calls Bedrock Claude through the AI SDK's streamText function with Output.object() for structured generation. This sends the prompt to the model and returns a streaming result that produces structured JSON validated against a Zod schema.

// lambda/streaming-service.ts
import { bedrock } from '@ai-sdk/amazon-bedrock';
import { Output, streamText } from 'ai';
import { z } from 'zod';

const responseSchema = z.object({
  summary: z.string().describe('A concise summary of the topic'),
  keywords: z.array(z.string()).describe('Relevant keywords'),
});

type StreamParams = {
  prompt: string;
};

export const streamingService = (params: StreamParams) => {
  const model = bedrock('anthropic.claude-3-5-sonnet-20241022-v2:0');

  return streamText({
    model,
    prompt: params.prompt,
    output: Output.object({ schema: responseSchema }),
    system:
      'You are a helpful assistant. Respond with a structured JSON object containing a summary and keywords about the topic the user asks about.',
    maxOutputTokens: 2000,
    temperature: 0.7,
  });
};

streamText with Output.object() does three things for us. It instructs the model to produce JSON matching our Zod schema. It streams the response token by token. And it exposes toTextStreamResponse(), which wraps the token stream in a standard Web Response object with a ReadableStream body.

That Response is what we destructure in the handler to extract body, headers, and status.

Why Output.object() instead of plain streamText? We want structured JSON output. The output: Output.object({ schema }) option instructs the model to generate a JSON object conforming to our Zod schema, and the AI SDK validates the final result against it.

Without the output option, streamText returns free-form text. For structured output use cases (data extraction, form generation, classification results), Output.object() is the right tool.

Note that partial objects streamed during generation are not validated against the schema. Only the final complete object is. If you need validation during streaming, you would validate partial results yourself. For most UI rendering use cases, the partial objects are good enough to display incrementally.

Testing the endpoint

Deploy the stack and grab the API URL from the CloudFormation output:

cd step-1
npm install
npx cdk deploy

Then test with curl. The --no-buffer flag is critical. Without it, curl buffers output and you will not see the streaming effect:

curl -i --no-buffer \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Explain the CAP theorem"}' \
  https://your-api-id.execute-api.us-east-1.amazonaws.com/v1/stream

You will see the JSON object build up incrementally:

HTTP/2 200
content-type: text/plain; charset=utf-8
access-control-allow-origin: *

{"summary":"The CAP theorem, also known as Brewer's theorem, states
that a distributed data store can only guarantee two of the following
three properties simultaneously...","keywords":["CAP theorem",
"consistency","availability","partition tolerance",...]}

A note on first-byte latency. You may notice a 1 to 2 second delay before the first chunk arrives, even though streaming is working correctly. This is expected Lambda behavior: Lambda buffers chunks smaller than approximately 10 KB before flushing them to the client.

The first few tokens from the model are small, so they accumulate in the buffer. Once enough data builds up or the buffer timeout triggers, the first chunk flushes and subsequent chunks flow smoothly. This is not a bug in your code or configuration.

What we built

We now have a working streaming pipeline. A POST request hits API Gateway, which invokes Lambda in streaming mode. Middy wraps the handler with streamifyResponse, the AI SDK calls Bedrock Claude with streamText and Output.object(), and the structured JSON streams back to the client as it generates.

The key pieces are:

API Gateway: ResponseTransferMode: STREAM and the response-streaming-invocations URI, set through CDK escape hatches.
Lambda/Middy: streamifyResponse: true with a handler that returns { body: ReadableStream }.
AI SDK: streamText with Output.object() and a Zod schema, converted to a Web Response via toTextStreamResponse().

What is next

This clean architecture hides a subtle timing bug. When Middy streams a response, the after middleware hook fires at the wrong moment: before the stream has finished writing. Any observability logic you put in after (logging LLM metrics, recording token counts, publishing to CloudWatch) will execute with incomplete data or not at all.

Nobody has documented this behavior, and it breaks every middleware-based observability pattern you would naturally reach for.

In the next post, we will reproduce this bug, trace it through Middy's source code, and understand exactly why it happens.

All code for this series is available in the companion repository. Each post corresponds to a step-N/* branch with a self-contained, deployable CDK project.

References

Building responsive APIs with Amazon API Gateway response streaming: AWS announcement (Nov 2025)
Response streaming for Lambda functions: AWS Lambda Developer Guide
Introducing AWS Lambda response streaming: original launch (Apr 2023)
Serverless strategies for streaming LLM responses: AWS comparison of streaming approaches (Nov 2025)
Middy streaming integration: Middy documentation
AI SDK streamText reference: Vercel AI SDK
AI SDK Structured Output: Vercel AI SDK
AI SDK Amazon Bedrock Provider: Vercel AI SDK
Lambda response streaming flush behavior: chunk buffering analysis

Stop Chatting, Start Specifying: Spec-Driven Design with Kiro IDE

Pablo Albaladejo — Mon, 01 Dec 2025 21:36:29 +0000

Building Kaiord with Kiro: From Specs to Production

How Spec-Driven Development and Test-Driven Development Changed the Way I Code

The Problem with Chat-First Development

We've all been there. You open your AI assistant and start chatting:

"Can you help me convert FIT files to JSON?"

The AI immediately writes code. But wait, that's not exactly what you wanted. So you clarify:

"Actually, I need it to preserve all the workout data..."

More code appears. Still not quite right. Another message:

"No, I meant it should also handle repetition blocks..."

This back-and-forth conversation fills up the context window. By the time you get working code, there's no space left for the AI to help with the details. The problem: we're asking AI to write code before understanding what we need.

Enter Spec-Driven Development

When I discovered Kiro and its spec-driven approach, everything changed. Instead of jumping straight to code, I learned to think first, code later.

Here's how it works:

Step 1: Create a Specification with Kiro

Before writing any code, Kiro helps me create three documents by asking me questions:

requirements.md - What the feature needs to do
design.md - How it will work
tasks.md - Step-by-step implementation plan

I tell Kiro "I need to convert FIT files to KRD format," and Kiro starts asking questions:

"What data needs to be preserved?"
"What are the acceptance criteria?"
"What tolerances are acceptable for round-trip conversion?"

Based on my answers, Kiro generates the specification documents. Let me show you a real example from Kaiord. When I needed to convert FIT files (Garmin's binary format) to KRD (my JSON format), Kiro created this spec:

# fit-to-krd-conversion/requirements.md

## User Story
As a cyclist or runner, I want to convert FIT workout files to KRD format, 
so that I can edit workout structures in a human-readable format and validate 
them against a standard schema.

## Acceptance Criteria
- WHEN the converter receives a valid FIT file
- THEN it SHALL produce a valid KRD document
- AND it SHALL preserve all workout steps
- AND it SHALL validate against the KRD schema

You can see the full spec in .kiro/specs/core/fit-to-krd-conversion/requirements.md.

Why this matters: The spec becomes Kiro's guide. Instead of guessing what I want, Kiro has a clear document that captures my requirements through our conversation.

Step 2: Kiro Defines the Architecture

After understanding my requirements, Kiro proposes the architecture in the design document. It asks me questions like:

"Should we use hexagonal architecture with ports and adapters?"
"What external libraries will we use?"
"What are the key design goals?"

Based on my answers, Kiro generates the design document describing how the conversion will work:

# design.md

## Architecture

We'll create:
- Port interface: FitReader (domain/ports)
- Adapter implementation: garmin-fitsdk (adapters/fit)
- Use case: convertFitToKrd (application/use-cases)

### Key Design Goals

1. **Round-trip safety**: FIT → KRD → FIT preserves data within tolerances
2. **Hexagonal architecture**: Domain logic isolated from external dependencies
3. **Dependency injection**: Swappable FIT SDK implementations via ports
4. **Schema validation**: All KRD output validated against JSON schema
5. **Type safety**: Strict TypeScript with no implicit `any`

This planning phase saved me hours of refactoring later. The code structure was clear before Kiro wrote a single line.

See .kiro/steering/architecture.md for the full architecture rules.

Step 3: Kiro Breaks Down Tasks

Finally, Kiro breaks the work into small, testable pieces in the tasks document:

# tasks.md

- [ ] Define FitReader port interface
- [ ] Implement Garmin FIT SDK adapter
- [ ] Create convertFitToKrd use case
- [ ] Write unit tests (AAA pattern)
- [ ] Add round-trip validation
- [ ] Document API examples

Each task is small enough for Kiro to implement in one sitting, making progress visible and momentum sustainable. I review each task, and Kiro implements it following TDD.

Steering Documents: Teaching Kiro My Rules

Before Kiro writes any code, I need to teach it how I want things done. This is where steering documents come in.

Steering documents are markdown files in .kiro/steering/ that define the rules Kiro must follow. Think of them as a style guide, architecture manual, and best practices document all in one—but for AI.

Here's what I defined for Kaiord:

My Steering Documents

.kiro/steering/
├── architecture.md       # Hexagonal architecture rules
├── tdd.md               # Test-driven development workflow
├── code-style.md        # Code quality standards
├── zod-patterns.md      # Schema-first patterns
├── testing.md           # Testing strategies
├── error-patterns.md    # Error handling patterns
└── ... 18 documents total

Each document tells Kiro:

What patterns to follow (e.g., "Use hexagonal architecture")
What to avoid (e.g., "Never use any types")
How to structure code (e.g., "Files ≤ 100 lines, functions < 40 LOC")
When to apply rules (e.g., "Write tests first, then implementation")

The key insight: Kiro reads these documents automatically while working. I don't need to repeat myself in every conversation. The rules are always there, guiding every decision Kiro makes.

Example: Architecture Steering Document

Here's a snippet from my .kiro/steering/architecture.md:

# Architecture — Hexagonal & DI

Layers:

- **domain/** — pure KRD types & rules
- **application/** — use-cases; depends on `ports/` only
- **ports/** — I/O contracts (Fit/Tcx/Zwift Reader/Writer)
- **adapters/** — concrete implementations (e.g., @garmin/fitsdk)
- **cli/** — end-user commands; depends on `application`

Rules:

- `domain` depends on no one
- `application` MUST NOT import external libs nor `adapters/`
- `adapters` implement `ports` and may use external libs

When Kiro generates code, it automatically follows these architecture rules. If I accidentally try to import an adapter in a use case, Kiro warns me about the violation. I never have to remind Kiro about architectural boundaries—it's all in the steering document.

Growing Together: Managing Codebase Complexity

As a codebase grows, it's easy for things to get messy—especially when an AI can generate hundreds of lines in seconds. To keep our collaboration smooth and productive, I found that Kiro works best when we have clear processes and healthy boundaries.

Think of it as setting up guardrails so we can run fast without falling off a cliff.

1. The Safety Net: Test-Driven Development (TDD)

For me, TDD isn't just a methodology; it's how I communicate my intent to Kiro. It acts as our safety net, allowing us to build features and refactor code with confidence.

With TDD rules defined in my steering documents, Kiro follows this workflow automatically:

Kiro writes the test first (it fails - red)
Kiro writes code to pass the test (it passes - green)
We refactor together (keep tests green - refactor)

Here's a real example from packages/core/src/application/use-cases/convert-fit-to-krd.ts:

Step 1: Kiro Writes the Test (Red)

// packages/core/src/application/use-cases/convert-fit-to-krd.test.ts
describe("convertFitToKrd", () => {
  it("should convert FIT workout to KRD format", async () => {
    // Arrange
    const fitBuffer = loadFitFixture("WorkoutIndividualSteps.fit");
    const fitReader = createMockFitReader();
    const validator = createSchemaValidator();

    // Act
    const krd = await convertFitToKrd(fitReader, validator)({
      fitBuffer,
    });

    // Assert
    expect(krd.version).toBe("1.0");
    expect(krd.type).toBe("workout");
    expect(krd.extensions.workout.steps).toHaveLength(4);
  });
});

This test fails at first because convertFitToKrd doesn't exist yet.

Step 2: Kiro Writes Minimal Code (Green)

Only after writing the test does Kiro implement the actual conversion function:

// packages/core/src/application/use-cases/convert-fit-to-krd.ts
export const convertFitToKrd =
  (fitReader: FitReader, validator: SchemaValidator) =>
  async (params: ConvertFitToKrdParams): Promise<KRD> => {
    // Read FIT file
    const krd = await fitReader(params.fitBuffer);

    // Validate result
    const errors = validator.validate(krd);
    if (errors.length > 0) {
      throw new KrdValidationError("Validation failed", errors);
    }

    // Return validated KRD
    return krd;
  };

The test passes! But we're not done yet.

Step 3: We Refactor Together

Now Kiro and I improve the code while keeping tests green. Kiro might suggest:

Extracting helper functions
Improving error messages
Adding edge case handling

I review the suggestions, and Kiro applies the changes—all while the tests continue to pass.

The AAA Pattern

All our tests follow the AAA pattern (Arrange-Act-Assert):

it("should validate KRD against schema", () => {
  // Arrange
  const validator = createSchemaValidator(mockLogger);
  const invalidKrd = { version: "1.0" }; // missing required fields

  // Act
  const errors = validator.validate(invalidKrd);

  // Assert
  expect(errors).toHaveLength(2);
  expect(errors[0].field).toBe("type");
  expect(errors[1].field).toBe("metadata");
});

This pattern makes tests easy to read and understand. The Arrange section sets up test data, the Act section calls the function, and the Assert section checks the result.

The result: The code works the first time. No debugging sessions trying to figure out why something breaks.

You can see the TDD guidelines I defined in .kiro/steering/tdd.md. Kiro follows these rules for every function in Kaiord.

2. Healthy Boundaries: Small Files, Happy Code

Just like us, Kiro does better work when tasks are bite-sized. I've set up some helpful constraints that encourage us to keep things simple:

Keep files small (≤ 100 lines)
Keep functions focused (< 40 LOC)
Be specific (NO any types)

These aren't just rigid rules; they actually help Kiro write better, more maintainable code. When a file starts getting too long, Kiro notices and suggests, "Hey, maybe we should split this up?"

Example: When we were building the FIT converter, Kiro naturally organized the code into small, focused modules instead of one giant file:

adapters/fit/
├── garmin-fitsdk.ts          # Main entry (< 100 lines)
├── duration.mapper.ts         # Duration mapping (< 100 lines)
├── target.converter.ts        # Target conversion (< 100 lines)
└── workout.mapper.ts          # Workout mapping (< 100 lines)

Each file stays under 100 lines, making the code easy to understand for both of us.

These automated checks run in the background, gently nudging us back to the right path if we stray. The result: A codebase that feels clean and manageable, no matter how much feature work we do.

Architecture Enforcement

I defined hexagonal architecture rules in steering docs that Kiro enforces. The key rule:

Domain depends on nothing. Application depends only on domain and ports. Adapters implement ports.

When Kiro generates code, it automatically:

Creates port interfaces (contracts)
Implements adapters (external libraries)
Writes use cases (business logic)
Wires everything in providers (dependency injection)

If I accidentally try to import an adapter in a use case, Kiro warns me:

⚠️ Architecture violation detected
   Application layer should not import adapters
   Import the port interface instead

This keeps the codebase maintainable and testable. See .kiro/steering/architecture.md for the full rules I defined.

Real Examples from Kaiord

Let me show you three real examples where spec-driven development with Kiro made a huge difference:

Example 1: FIT to KRD Conversion

The challenge: Convert Garmin's binary FIT format to my JSON format without losing data.

The spec: .kiro/specs/core/fit-to-krd-conversion/requirements.md

What Kiro generated:

Port interface in packages/core/src/ports/fit-reader.ts
Adapter implementation in packages/core/src/adapters/fit/garmin-fitsdk.ts
Use case in packages/core/src/application/use-cases/convert-fit-to-krd.ts
Complete test suite with 90%+ coverage
Round-trip validation to ensure no data loss

Time saved: What would have taken me 2 weeks of trial-and-error took 3 days with clear specifications.

Example 2: Duration Types with Discriminated Unions

The challenge: Support 14 different duration types (time, distance, calories, heart rate conditions, power conditions, etc.)

The spec: .kiro/specs/core/zod-schema-migration/design.md

What Kiro generated:

// From https://github.com/pablo-albaladejo/kaiord/blob/main/packages/core/src/domain/schemas/duration.ts
export const durationSchema = z.discriminatedUnion("type", [
  // Simple durations
  z.object({ type: z.literal("time"), seconds: z.number() }),
  z.object({ type: z.literal("distance"), meters: z.number() }),
  z.object({ type: z.literal("calories"), calories: z.number() }),

  // Conditional durations
  z.object({
    type: z.literal("heart_rate_less_than"),
    heartRate: z.number(),
    seconds: z.number().optional()
  }),
  z.object({
    type: z.literal("power_greater_than"),
    power: z.number(),
    seconds: z.number().optional()
  }),

  // ... 14 types total
]);

The benefit: TypeScript knows exactly which fields exist for each duration type. No more runtime errors from accessing wrong fields!

Example 3: Round-Trip Validation

The challenge: Ensure conversions preserve data (FIT → KRD → FIT should match original).

The implementation:

// packages/core/src/application/use-cases/validate-round-trip.ts
export const validateRoundTrip = (
  fitReader: FitReader,
  fitWriter: FitWriter,
  checker: ToleranceChecker,
  logger: Logger
) => async (params: ValidateRoundTripParams): Promise<void> => {
  // Convert FIT → KRD → FIT
  const krd = await fitReader(params.fitBuffer);
  const convertedFit = await fitWriter(krd);

  // Check tolerances
  const deviations = checker.checkTolerances(
    params.fitBuffer,
    convertedFit
  );

  if (deviations.length > 0) {
    throw new ToleranceExceededError("Tolerances exceeded", deviations);
  }
};

Tolerances:

Time: ±1 second
Power: ±1 watt or ±1% FTP
Heart Rate: ±1 bpm
Cadence: ±1 rpm

This ensures data integrity across all conversions.

Automation with Agent Hooks

One of Kiro's most powerful features is agent hooks - automated workflows that trigger based on events.

I created a hook that automatically generates GitHub pull requests from my specs. Here's how it works:

Hook configuration: .kiro/hooks/create-github-pr.kiro.hook

When I finish a feature, the hook:

Reads my spec from .kiro/specs/[feature-name]/
Gets the requirements, design, and tasks
Generates a PR title from the main goal
Fills the PR template with spec content
Creates the PR using GitHub MCP (Model Context Protocol)
Adds appropriate labels

Time saved: Creating a PR went from 10 minutes to 30 seconds.

Bonus: Every PR has complete documentation because it's generated from the spec!

The Numbers Tell the Story

After building Kaiord with Kiro's spec-driven approach:

Code Quality

86.5% test coverage - More than most production apps
300+ unit tests - TDD from day one
55+ E2E tests - Playwright for integration testing
0 any types - Complete type safety
550+ TypeScript files - All following architecture rules

Test Coverage by Package

Core Package: 88% (target: ≥80%)
Converters/Mappers: 94% (target: ≥90%)
CLI Package: 82% (target: ≥70%)
Web SPA: 86.5% (target: ≥70%)

Kiro Integration

30+ specifications in .kiro/specs/
18 steering documents in .kiro/steering/
8 agent hooks in .kiro/hooks/
100% of code generated with Kiro's help

Production Ready

3 npm packages published:
- @kaiord/core - Core library
- @kaiord/cli - Command-line tool
1 web app deployed: Live demo
5 CI/CD workflows - Automated testing, linting, deployment

What I Learned

1. Planning Saves Time

Writing specs feels slow at first. But it's much faster than:

Rewriting code because requirements changed
Debugging code you don't understand
Explaining to the AI what you already explained 5 times

Example: The FIT conversion spec took 2 hours to write. It saved me 2 weeks of confusion.

2. Tests Give Confidence

Following TDD means I know my code works. When I add features, the tests tell me immediately if I broke something.

Example: When I added support for swimming workouts, my existing tests caught 3 bugs before I even ran the app.

3. Architecture Matters

Using hexagonal architecture (ports and adapters) made my code:

Easy to test (mock the ports)
Easy to extend (add new adapters)
Easy to understand (clear boundaries)

Example: Adding TCX format support required zero changes to existing code - just a new adapter.

4. AI Works Better with Structure

Kiro's spec-driven approach gives AI the context it needs to help effectively. Instead of guessing, it implements what's documented.

5. Specs Save Time

Clear specs mean:

Less confusion during implementation
Fewer bugs from misunderstood requirements
Better code reviews
Easier onboarding for new developers

6. TDD Prevents Bugs

Writing tests first catches bugs early. When we write a test, we think about:

What should happen in normal cases?
What should happen in edge cases?
What should happen when errors occur?

This thinking prevents many bugs before they're written.

Try It Yourself

Want to see spec-driven development in action? Check out Kaiord:

Source code: github.com/pablo-albaladejo/kaiord
Live demo: pablo-albaladejo.github.io/kaiord
Specifications: Browse .kiro/specs/ to see real examples
Steering docs: Read .kiro/steering/ for architecture rules

Key Files to Explore

Specifications (see how Kiro plans):

.kiro/specs/core/fit-to-krd-conversion/requirements.md - How Kiro planned FIT conversion
.kiro/specs/core/zod-schema-migration/design.md - Schema-first approach

Steering Docs (see the rules Kiro follows):

.kiro/steering/architecture.md - Hexagonal architecture rules
.kiro/steering/tdd.md - Test-driven development workflow
.kiro/steering/zod-patterns.md - Schema-first patterns

Implementation (see the code Kiro generated):

packages/core/src/domain/schemas/duration.ts - Discriminated unions with Zod
packages/core/src/application/use-cases/convert-fit-to-krd.ts - Clean use case
packages/core/src/ports/fit-reader.ts - Port interface
packages/core/src/adapters/fit/garmin-fitsdk.ts - Adapter implementation

Final Thoughts

Building Kaiord taught me that AI doesn't replace thinking - it amplifies it.

With Kiro's spec-driven approach:

I think more clearly about what I need
I document decisions before coding
I test continuously (TDD)
I maintain clean architecture
AI implements what I specify

The result? Production-ready code that:

Works reliably (86.5% test coverage)
Makes sense (clean architecture)
Can grow (extensible design)
Helps others (open source)

Spec-driven development isn't slower - it's smarter. The time you spend planning pays back 10x in code that works the first time.

Resources

Kaiord Repository: github.com/pablo-albaladejo/kaiord
Live Web App: pablo-albaladejo.github.io/kaiord
npm Packages:
- @kaiord/core
- @kaiord/cli
Kiro IDE: kiro.dev
Kiroween Hackathon: kiroween.devpost.com

Forem: Pablo Albaladejo

Deploy Observable AI Streaming on AWS: CDK Stack with Tests

How do you deploy API Gateway REST streaming with CDK?

How do you compose Middy and AI SDK middlewares for streaming?

Testing the Pipeline

createStreamTransform

publishLog

streamErrorHandler

The awslambda mock fixture

Deployment and Verification

Prerequisites

Deploy

Test with curl

Verify observability

Run the tests locally

Tear down

What are the production considerations for streaming Lambda responses?

Series Recap

References

Deferred Observability for Streaming Lambda with TransformStream flush()

How does TransformStream flush() guarantee deferred execution?

The createStreamTransform Utility

Two Middleware Systems

AI SDK Middlewares

LoggingMiddleware

DataStoreMiddleware

How do you publish observability data after a stream completes?

The Complete Pipeline

Why is TransformStream flush() better than AI SDK onFinish?

Recap

What's Next

References

The Middy After Hook Problem: Why Streaming Lambda Observability Is Broken

Why does Middy's after hook see empty data when streaming?

When does Middy's after hook fire relative to stream consumption?

Why Tests Don't Catch It

Three Approaches That Don't Work

1. Move publishing to the AI SDK's onFinish callback

2. Await the stream in the after hook, then reconstruct it

3. Use Lambda post-response execution

How do you handle errors in streaming Lambda responses?

Pre-stream errors

Mid-stream errors

The Documentation Gap

What's Next

References

Streaming LLM Responses Through API Gateway REST with Lambda

The problem with buffered LLM responses

What this post covers

How do you stream LLM responses through API Gateway REST?

How do you configure API Gateway REST for response streaming?

Wire format

Timeouts and bandwidth

CDK infrastructure

Lambda handler with Middy

How do you use AI SDK streamText with Bedrock Claude on Lambda?

Testing the endpoint

What we built

What is next

References

Stop Chatting, Start Specifying: Spec-Driven Design with Kiro IDE

Building Kaiord with Kiro: From Specs to Production

The Problem with Chat-First Development

Enter Spec-Driven Development

Step 1: Create a Specification with Kiro

Step 2: Kiro Defines the Architecture

Step 3: Kiro Breaks Down Tasks

Steering Documents: Teaching Kiro My Rules

My Steering Documents

Example: Architecture Steering Document

Growing Together: Managing Codebase Complexity

1. The Safety Net: Test-Driven Development (TDD)

Step 1: Kiro Writes the Test (Red)

Step 2: Kiro Writes Minimal Code (Green)

Step 3: We Refactor Together

The AAA Pattern

2. Healthy Boundaries: Small Files, Happy Code

Architecture Enforcement

Real Examples from Kaiord

Example 1: FIT to KRD Conversion

1. Move publishing to the AI SDK's `onFinish` callback