Forem: Megan Lee

How to build better AI apps in React with MediaPipe’s latest APIs

Megan Lee — Fri, 25 Jul 2025 15:00:00 +0000

Written by Emmanuel John✏️

MediaPipe has been around for a while. Google originally released it as a framework for running smaller on-device models in the browser and on mobile, mostly focused on vision tasks like face detection, hand tracking, pose estimation, etc.

It now operates as Google’s open-source machine learning framework, built with TensorFlow andspecializing in helping devs create robust, on-device AI applications for real-time, static, and streaming media across different platforms. The most recent release enables Vision Models to run fully on-device across platforms, without needing to run models server-side.

In this article, we will explore MediaPipe's latest APIs while building an object detection and categorization application in React. We will specifically focus on the MediaPipe Tasks API for object detection. Here’s what our final app will look like: Before moving forward, you should have the following:

React knowledge (Hooks, components)
Node.js and npm installed
A webcam (optional, but required for real-time testing)

Why object detection? And why MediaPipe?

Object detection has real-world applications across a wide range of industries. Developers building apps for retail, logistics, accessibility, smart homes, fitness, or security often need to detect and identify real-world items in images or video streams. For example, an e-commerce app could let users scan products with their camera to auto-fill search queries. In health and fitness, apps can use object detection to recognize body positions, equipment, or form during workouts.

For scientific research, an AI-driven science app could identify lab tools or specimens through a webcam. By learning how to integrate object detection into a React app, developers can create smarter, more intuitive interfaces powered by computer vision, all running efficiently in the browser.

MediaPipe is one of Google’s most impressive contributions to real-time, on-device AI. The release of the MediaPipe Tasks API makes it possible to add real-time vision models into your web app with a few lines of JavaScript.

Introduction to MediaPipe’s new APIs

MediaPipe’s latest APIs, or MediaPipe Solutions suite, consist of the following:

MediaPipe Tasks

This offers a simple, cross-platform API for installing typical ML solutions with as little code as possible. It supports gesture recognition and image classification.

MediaPipe Model Maker

This simplifies the customization of pre-trained MediaPipe models using your data. It makes it possible to fine-tune pre-existing models using comparatively small datasets.

MediaPipe Studio

This provides a no-code approach to exploring MediaPipe solutions with your data. We will be using MediaPipe’s Tasks for object detection in our tutorial.

Project setup

I used Vite to initialize our app. For this project, I will be using Tailwind CSS with React. To start, open your terminal and navigate to the desired folder where you want your code to be. Then run the following command:

npm create vite@latest project_name

Choose JavaScript as your preferred language. Next, CD into your project directory or folder and then enter this:

npm install tailwindcss @tailwindcss/vite

Open the folder in VS Code or any other text editor you choose. Next, modify the vite.config.js file and paste in the following code:

import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";
import tailwindcss from "@tailwindcss/vite";

export default defineConfig({
  plugins: [react(), tailwindcss()],
});

Remove all code from the index.css file and paste this:

@import "tailwindcss";

Our React project is set up and ready. Let's get into installing the required dependencies.

Install the required dependency

We need to install the MediaPipe Tasks Vision package for this project. Navigate to your project directory and run:

npm i @mediapipe/tasks-vision

If the installation was successful, you will see a screen similar to the one below:

Getting started

To get started, we are going to create 3 folders for our project:

components
hooks
utils

This is what our UI will look like:

Setting up our utility functions

We will create four utility functions inside the “utils” folder, each building upon another.

drawDetections.js
initializeObjectDetector.js
detectObjectsAndDraw.js
cleanupMediaPipeResources.js

In the “utils” folder, create a drawDetections.js file and paste the following:

// drawDetections.js

export const drawDetections = (context, detections) => {
  detections.forEach(detection => {
    const bbox = detection.boundingBox;
    const category = detection.categories[0]; 
    context.strokeStyle = 'red'; 
    context.lineWidth = 2;      
    context.strokeRect(bbox.originX, bbox.originY, bbox.width, bbox.height);        
    context.fillStyle = 'red';   
    context.font = '16px Arial'; 
    const text = `${category.categoryName} (${(category.score * 100).toFixed(2)}%)`;
    const textX = bbox.originX;
    const textY = bbox.originY > 10 ? bbox.originY - 5 : 10;
    context.fillText(text, textX, textY);
  });
};

The drawDetections.js function is responsible for visualizing the detected objects on the HTML canvas. It takes the 2D rendering context of the canvas and the MediaPipe detection results as input.

For each detected object, it draws a colored box and a text label on the canvas. Next, create an initializeObjectDetector.js file and populate it with the code below:

// initializeObjectDetector.js
import { ObjectDetector, FilesetResolver } from '@mediapipe/tasks-vision';

export const initializeObjectDetector = async (modelPath, scoreThreshold, maxResults) => {
  const visionFilesetResolver = await FilesetResolver.forVisionTasks(
    '<https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm>'
  );
  const objectDetector = await ObjectDetector.createFromOptions(
    visionFilesetResolver, 
    {
      baseOptions: {
        modelAssetPath: modelPath,
      },
      runningMode: 'LIVE_STREAM', 
      scoreThreshold: scoreThreshold, 
      maxResults: maxResults,        
    }
  );

  return objectDetector; 
};

The initializeObjectDetector function calls on FilesetResolver.forVisionTasks() to load the WebAssembly(WASM) files.

Then, it utilizes ObjectDetector.createFromOptions() to combine the loaded files with our AI model and the specified options, returning the ObjectDetector instance. Next, we will create a detectObjectsAndDraw function. This function accepts the following props:

videoElement
canvasElement
objectDetector
setDetectionsCallback

We’ll create the videoElement and the setDetectionsCallback state in the next section:

//detectObjectsAndDraw.js
import { drawDetections } from "./drawDetections";

export const detectObjectsAndDraw = (
  videoElement,
  canvasElement,
  objectDetector,
  setDetectionsCallback
) => {
  const context = canvasElement.getContext("2d");

  canvasElement.width = videoElement.videoWidth;
  canvasElement.height = videoElement.videoHeight;
  context.clearRect(0, 0, canvasElement.width, canvasElement.height);
  context.drawImage(
    videoElement,
    0,
    0,
    canvasElement.width,
    canvasElement.height
  );

  try {
    const detectionsResult = objectDetector.detectForVideo(
      videoElement,
      performance.now()
    );
    setDetectionsCallback(detectionsResult.detections);
    drawDetections(
      context,
      detectionsResult.detections,
      canvasElement.width,
      canvasElement.height
    );

  } catch (err) {
    console.error("Error during object detection or drawing:", err);
    setDetectionsCallback([]);
    return [];
  }
};

The detectObjectsAndDraw function draws the current video frame onto the canvas. It then runs the objectDetector on this frame to identify objects.

After that, it updates the setDetectionsCallback state with the detected objects. It finally calls the drawDetections utility to overlay bounding boxes and labels on the canvas. For this section, we will finally create the cleanupMediaPipeResources.js:

//cleanupMediaPipeResources.js
export const cleanupMediaPipeResources = ({ animationFrameId, videoRef, objectDetectorRef }) => {
  if (animationFrameId.current) {
    cancelAnimationFrame(animationFrameId.current);
    animationFrameId.current = null; 
  }

  if (videoRef.current && videoRef.current.srcObject) {
    videoRef.current.srcObject.getTracks().forEach(track => {
      track.stop(); 
    });
    videoRef.current.srcObject = null;
  }
  if (objectDetectorRef.current) {
    objectDetectorRef.current.close();
    objectDetectorRef.current = null; 
  }
};

The cleanupMediaPipeResources.js manages system resources after running our application. It is responsible for stopping the animation frame loop, releasing the webcam stream, and closing the MediaPipe detector instance to prevent memory leaks and release hardware resources. In the next section, we will set up our MediaPipe logic.

Getting the logic together

First, create a useMediaPipeObjectDetection.js file in the “hooks” folder. Then import [useEffect](https://blog.logrocket.com/useeffect-react-hook-complete-guide/), useRef, useState, and useCallback like so:

import { useEffect, useRef, useState, useCallback } from 'react';
import { detectObjectsAndDraw } from '../utils/detectObjectsAndDraw';

I have also imported detectObjectsAndDraw which we created earlier. These are the stages our code will follow:

Perform detection loop
Start Webcam
Initialize Detector
Initialization on mount and unmount

Perform detection loop

Below is our performDetectionLoop function, together with the supporting useRefs and useState we need for it to work:

 const videoRef = useRef(null);
  const canvasRef = useRef(null);
  const objectDetectorRef = useRef(null);
  const animationFrameId = useRef(null);

 const [detections, setDetections] = useState([]);

  const performDetectionLoop = useCallback(() => {
    if (!videoRef.current || !canvasRef.current || !objectDetectorRef.current) {
      animationFrameId.current = requestAnimationFrame(performDetectionLoop);
      return;
    }
    const video = videoRef.current;
    const canvas = canvasRef.current;
    const detector = objectDetectorRef.current;
    if (video.videoWidth === 0 || video.videoHeight === 0) {
      animationFrameId.current = requestAnimationFrame(performDetectionLoop);
      return;
    }
    detectObjectsAndDraw(video, canvas, detector, setDetections);
    animationFrameId.current = requestAnimationFrame(performDetectionLoop);
  }, [setDetections]);

The performDetectionLoop function checks if video, canvas, and detector are ready. When they‘re ready, it calls the detectObjectsAndDraw utility function to process the current video frame. It finally updates the detection state through a callback from the detectObjectsAndDraw function. Your useMediaPipeObjectDetection.js should now look like this:

import { useEffect, useRef, useState, useCallback } from 'react';
import { detectObjectsAndDraw } from '../utils/detectObjectsAndDraw';

const useMediaPipeObjectDetection = () => {

 const videoRef = useRef(null);
  const canvasRef = useRef(null);
  const objectDetectorRef = useRef(null);
  const animationFrameId = useRef(null);

 const [detections, setDetections] = useState([]);

  const performDetectionLoop = useCallback(() => {
    if (!videoRef.current || !canvasRef.current || !objectDetectorRef.current) {
      animationFrameId.current = requestAnimationFrame(performDetectionLoop);
      return;
    }
    const video = videoRef.current;
    const canvas = canvasRef.current;
    const detector = objectDetectorRef.current;
    if (video.videoWidth === 0 || video.videoHeight === 0) {
      animationFrameId.current = requestAnimationFrame(performDetectionLoop);
      return;
    }
    detectObjectsAndDraw(video, canvas, detector, setDetections);
    animationFrameId.current = requestAnimationFrame(performDetectionLoop);
  }, [setDetections]);    
};

export default useMediaPipeObjectDetection;

Start Webcam

The startWebcam code is below:

 //Start Webcam Function
  const startWebcam = useCallback(async () => {
    if (loading || error) return;

    try {
      const stream = await navigator.mediaDevices.getUserMedia({ video: true });

      if (videoRef.current) {
        videoRef.current.srcObject = stream;

        videoRef.current.onloadedmetadata = () => {
          videoRef.current.play();
          animationFrameId.current =
            requestAnimationFrame(performDetectionLoop);
        };
      }
    } catch (err) {
      console.error("Error accessing webcam:", err);
      setError(err);
    }
  }, [loading, error, performDetectionLoop]);

We also created two states for the startWebcam function:

const [loading, setLoading] = useState(true); 
const [error, setError] = useState(null);

In startWebcam function, we made an asynchronous call asking the user's browser for permission to access their webcam. If permission is granted, we get the live feed from the camera.

We then use the videoRef Hook to get a direct reference to the <video> HTML element. Once we have the MediaStream, we assign it to the srcObject property of the video element, videoRef.current.srcObject = stream. This tells the video element to display the live camera feed.

We use the onloadedmetadata event to listen for when metadata is loaded so that we can play the video We then use the videoRef.current.play() to start the video playback.

Once the video starts playing and its metadata is available, we start our continuous object detection process. We do this by calling requestAnimationFrame, passing in the performDetectionLoop function.

Initialize Detector

initializeDetector uses the useCallback Hook to call the initializeObjectDetector function we created in our“utils” folder. We first have to import the function from the “utils” folder:

import { initializeObjectDetector } from "../utils/initializeObjectDetector";

Next, copy the code below into your useMediaPipeObjectDetection.js file:

//InitializeDetector Function
  const initializeDetector = useCallback(async () => {
    try {
      setLoading(true);
      setError(null);
      const detector = await initializeObjectDetector(
        modelPath,
        scoreThreshold,
        maxResults
      );
      objectDetectorRef.current = detector;
      setLoading(false);
      console.log("Object Detector initialized!");
    } catch (err) {
      console.error("Failed to initialize Object Detector:", err);
      setError(err);
      setLoading(false);
    }
  }, [modelPath, scoreThreshold, maxResults]);

Next, we have two useEffects that get triggered when components mount and unmount. The second useEffect Hook calls on the cleanupMediaPipeResources function we created earlier. We will need to import the cleanupMediaPipeResources function. First, import it like so:

import { cleanupMediaPipeResources } from "../utils/cleanupMediaPipeResources";

Then copy and paste the code below:

 //Effects
  useEffect(() => {
    initializeDetector();
  }, [initializeDetector]);

  useEffect(() => {
    return () => {
      cleanupMediaPipeResources({
        animationFrameId,
        videoRef,
        objectDetectorRef,
      });
    };
  }, []);

Our completed useMediaPipeObjectDetection.js file now looks like this:

// useMediaPipeObjectDetection.js

import { useEffect, useRef, useState, useCallback } from "react";
import { detectObjectsAndDraw } from "../utils/detectObjectsAndDraw";
import { initializeObjectDetector } from "../utils/initializeObjectDetector";
import { cleanupMediaPipeResources } from "../utils/cleanupMediaPipeResources";

const useMediaPipeObjectDetection = (
  modelPath,
  scoreThreshold = 0.5,
  maxResults = 5
) => {
  const videoRef = useRef(null);
  const canvasRef = useRef(null);
  const objectDetectorRef = useRef(null);
  const animationFrameId = useRef(null);

  const [detections, setDetections] = useState([]);
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState(null);

  // Core Detection Loop Logic
  const performDetectionLoop = useCallback(() => {
    if (!videoRef.current || !canvasRef.current || !objectDetectorRef.current) {
      animationFrameId.current = requestAnimationFrame(performDetectionLoop);
      return;
    }
    const video = videoRef.current;
    const canvas = canvasRef.current;
    const detector = objectDetectorRef.current;
    if (video.videoWidth === 0 || video.videoHeight === 0) {
      animationFrameId.current = requestAnimationFrame(performDetectionLoop);
      return;
    }
    detectObjectsAndDraw(video, canvas, detector, setDetections);
    animationFrameId.current = requestAnimationFrame(performDetectionLoop);
  }, [setDetections]);

  //Start Webcam Function
  const startWebcam = useCallback(async () => {
    if (loading || error) return;

    try {
      const stream = await navigator.mediaDevices.getUserMedia({ video: true });

      if (videoRef.current) {
        videoRef.current.srcObject = stream;

        videoRef.current.onloadedmetadata = () => {
          videoRef.current.play();
          animationFrameId.current =
            requestAnimationFrame(performDetectionLoop);
        };
      }
    } catch (err) {
      console.error("Error accessing webcam:", err);
      setError(err);
    }
  }, [loading, error, performDetectionLoop]);

  //InitializeDetector Function

  const initializeDetector = useCallback(async () => {
    try {
      setLoading(true);
      setError(null);
      const detector = await initializeObjectDetector(
        modelPath,
        scoreThreshold,
        maxResults
      );
      objectDetectorRef.current = detector;
      setLoading(false);
      console.log("Object Detector initialized!");
    } catch (err) {
      console.error("Failed to initialize Object Detector:", err);
      setError(err);
      setLoading(false);
    }
  }, [modelPath, scoreThreshold, maxResults]);

  //Effects for Lifecycle Management
  useEffect(() => {
    initializeDetector();
  }, [initializeDetector]);

  useEffect(() => {
    return () => {
      cleanupMediaPipeResources({
        animationFrameId,
        videoRef,
        objectDetectorRef,
      });
    };
  }, []);
  return { videoRef, canvasRef, detections, loading, error, startWebcam };
};

export default useMediaPipeObjectDetection;

Next, we will work on our app's user interface.

Building the UI

In this section, we will build out our UI components. We have 4 UI components in total:

VideoCanvasDisplay
DetectionResults
ObjectDetectionApp
App

In our UI setup, the ObjectDetectionApp renders VideoCanvasDisplay and DetectionResults. The ObjectDetectionApp component then gets rendered in the App.jsx file like so:

import React from "react";
import ObjectDetectionApp from "./components/ObjectDetectionApp";

function App() {

  const MODEL_PATH =
    "<https://storage.googleapis.com/mediapipe-models/object_detector/efficientdet_lite0/int8/1/efficientdet_lite0.tflite>";

  return (
    <div className="">
      <div>
        <ObjectDetectionApp modelPath={MODEL_PATH} />
      </div>
    </div>
  );
}

export default App;

To make for easy change, we passed in the MODEL_PATH, or the model URL as props to the ObjectDetectionApp. Navigate inside the “components” folder and create the VideoCanvasDisplay.jsx file. Then copy and paste the code below:

// VideoCanvasDisplay.jsx
import React from "react";

const VideoCanvasDisplay = ({ videoRef, canvasRef }) => {
  return (
    <div className="relative w-full max-w-xl mx-auto h-auto border border-gray-200 shadow-sm">
      <video
        ref={videoRef}
        className="block w-full h-auto"
        autoPlay
        muted
        playsInline
      ></video>
      <canvas
        ref={canvasRef}
        className="absolute top-0 left-0 w-full h-full"
      ></canvas>
    </div>
  );
};

export default VideoCanvasDisplay;

The component displays video output from the webcam and the HTML canvas on top of the video with some Tailwind CSS styling. Next is the DetectionResults component. As the name suggests, it displays the object detection results:

// DetectionResults.jsx
import React from 'react';
const DetectionResults = ({ detections }) => {
  return (
    <div>
      <h2 className=" text-center text-xl font-semibold text-gray-800 mb-3">Detected Objects</h2>
      {detections.length > 0 ? (
        <ul className="list-none p-0">
          {detections.map((detection, index) => (
            <li 
              key={index} 
              className="mb-2 p-3 bg-gray-50 rounded-md border-l-4 border-blue-600 shadow-sm text-gray-700"
            >
              <strong className="font-medium">{detection.categories[0].categoryName}</strong> (Score:{' '}
              <span className="font-mono text-sm">{(detection.categories[0].score * 100).toFixed(2)}%</span>)
            </li>
          ))}
        </ul>
      ) : (
        <p className="text-gray-500 italic">No objects detected yet. Start the webcam.</p>
      )}
    </div>
  );
};

export default DetectionResults;

Finally, there is the ObjectDetectionApp component, which renders the VideoCanvasDisplay and DetectionResultscomponents:

import React from "react";
import useMediaPipeObjectDetection from "../hooks/useMediaPipeObjectDetection";
import VideoCanvasDisplay from "../components/VideoCanvasDisplay";
import DetectionResults from "../components/DetectionResults";

const ObjectDetectionApp = ({ modelPath }) => {
  const { videoRef, canvasRef, detections, loading, error, startWebcam } =
    useMediaPipeObjectDetection(modelPath);

  return (
    <div
      className="relative flex size-full min-h-screen flex-col bg-[#faf8fc]"
      style={{ fontFamily: '"Noto Serif", "Noto Sans", sans-serif' }}
    >
      <div className="px-4 md:px-10 lg:px-40 flex flex-1 justify-center py-5">
        <div className="layout-content-container flex flex-col max-w-[960px] flex-1">
          <div className="flex w-full min-w-72 grow flex-col items-stretch justify-center gap-1 py-4 @xl:px-4">
            <div className=" flex flex-col items-center gap-y-10">
              {loading && !error && (
                <p className="text-center text-gray-600 text-lg">
                  Loading MediaPipe model...
                </p>
              )}
              {error && (
                <div className="text-red-700 bg-red-100 border border-red-500 p-3 rounded-md mb-4">
                  <p className="font-medium">
                    Error:{" "}
                    {error.message || "Failed to initialize object detector."}
                  </p>
                  <p className="text-sm">
                    Please ensure your webcam is accessible
                  </p>
                </div>
              )}

              {!loading && !error && (
                <>
                  <div
                    onClick={startWebcam}
                    disabled={loading}
                    className="flex flex-1 gap-3 flex-wrap px-4 py-3 max-w-[480px] justify-center"
                  >
                    <button className="py-4 px-20  text-lg cursor-pointer items-center justify-center overflow-hidden rounded-full  bg-[#ede7f3] text-[#140e1b] font-bold leading-normal tracking-[0.015em] grow focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-opacity-75 transition-colors duration-300 ease-in-out disabled:opacity-50 disabled:cursor-not-allowed">
                      <span className="truncate">Search with camera</span>
                    </button>
                  </div>

                  <VideoCanvasDisplay
                    videoRef={videoRef}
                    canvasRef={canvasRef}
                  />
                  <DetectionResults detections={detections} />
                </>
              )}
            </div>
          </div>
        </div>
      </div>
    </div>
  );
};

export default ObjectDetectionApp;

Now we have all of the pieces together. Let's now run our app and see how it works Start the app with npm run dev : Notice how the object detection happens in real-time. Here is the code repository for the final build.

Conclusion

Google MediaPipe provides a way to run object detection right inside the browser and without needing to set up a backend for it or train data for it.

In this tutorial, we covered a brief introduction to the new Mediapipe APIs (MediaPipe Tasks, MediaPipe Model Maker, and MediaPipe Studio.

We then set up an object detection app using MediaPipe. We used the React getUserMedia method to get the user's camera feed, which we fed to the MediaPipe object detector model. We then nicely showed in real-time the results of the object detection on the screen.

Get set up with LogRocket's modern error tracking in minutes:

Visit https://logrocket.com/signup/ to get an app ID.
Install LogRocket via NPM or script tag. LogRocket.init() must be called client-side, not server-side.

NPM:

$ npm i --save logrocket 

// Code:

import LogRocket from 'logrocket'; 
LogRocket.init('app/id');

Script Tag:

Add to your HTML:

<script src="https://cdn.lr-ingest.com/LogRocket.min.js"></script>
<script>window.LogRocket && window.LogRocket.init('app/id');</script>

3.(Optional) Install plugins for deeper integrations with your stack:

Redux middleware
ngrx middleware
Vuex plugin

Get started now.

How to build unified AI interfaces using the Vercel AI SDK

Megan Lee — Thu, 24 Jul 2025 18:00:00 +0000

Written by Ikeh Akinyemi✏️

Integrating AI into modern frontend applications is no small feat. Developers often find themselves juggling multiple APIs from providers like OpenAI, Anthropic, and Google, each with its own quirks around authentication, streaming, error handling, and response formats. What should be a straightforward task quickly becomes a fragmented, hard-to-maintain experience.

The Vercel AI SDK aims to change that. This article explores how the SDK simplifies AI integration by offering a unified interface that abstracts away the differences between providers, without sacrificing the unique strengths of each model.

Instead of wrestling with inconsistent APIs, you’ll use a consistent set of tools to unlock powerful capabilities like streaming responses, multimodal inputs, and adaptive UI generation.

We’ll walk through building a multi-modal chat application that highlights these features: real-time feedback via streamed text, visual content processing through image analysis, and generative UI elements that respond to AI output.

By the end of this tutorial, you’ll have a solid foundation for integrating rich, AI-driven functionality into any frontend experience, with far less overhead.

Core Vercel AI SDK features that transform frontend AI development

The Vercel AI SDK tackles the core challenges of building AI-powered frontends through four key features designed to streamline development. Together, these features eliminate the complexity of working with multiple AI providers while enabling sophisticated user interfaces.

Provider abstraction: Unified API layer

The SDK's provider abstraction eliminates the need to learn multiple AI APIs by offering a consistent interface across all supported models.

Whether you're working with OpenAI's GPT-4, Anthropic's Claude, or Google's Gemini, the same code patterns apply for initialization, configuration, and response handling:

import { openai } from '@ai-sdk/openai';
import { anthropic } from '@ai-sdk/anthropic';
import { google } from '@ai-sdk/google';

// Switch between providers without changing your component logic
const model = openai('gpt-4o'); // or anthropic('claude-3-5-sonnet-20241022')

This abstraction extends beyond simple API calls to normalize response formats, error handling patterns, and streaming behaviors across providers. When business requirements change or new providers emerge, you can switch models with minimal code changes.

While the Vercel AI SDK simplifies cloud-based provider integration, there's also growing interest in on-device inference using browser APIs. For example, Chrome’s window.ai API allows lightweight, privacy-focused AI experiences in the frontend. Learn how this works in Vue apps.

Built for React: Native streaming support

The SDK provides first-class React integration through specialized Hooks that handle complex state management for AI interactions. The useChat Hook manages conversations with built-in loading states, while useCompletion handles single-turn interactions:

import { useChat } from '@ai-sdk/react';

function ChatComponent() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat();

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role === 'user' ? 'User: ' : 'AI: '}</div>
          <div>{message.content}</div>
        </div>
      ))}
      <form onSubmit={handleSubmit}>
        <input 
          value={input} 
          onChange={handleInputChange} 
          disabled={isLoading}
          placeholder="Type a message..."
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}

The React integration improves performance by offloading heavy AI computations to the server while keeping the client-side interface fast and responsive: The Vercel AI SDK enables clean, streaming chat interfaces with minimal setup

Streaming responses: Real-time text generation

Instead of waiting for complete responses, the SDK enables real-time streaming where text appears progressively as the AI generates it. This creates a more engaging user experience and provides immediate feedback:

import { useCompletion } from '@ai-sdk/react';

function StreamingComponent() {
  const { completion, input, handleInputChange, handleSubmit, isLoading } = useCompletion({
    api: '/api/completion',
  });

  return (
    <form onSubmit={handleSubmit}>
      <input
        name="prompt"
        value={input}
        onChange={handleInputChange}
        placeholder="Enter your prompt..."
      />
      <button type="submit" disabled={isLoading}>
        {isLoading ? 'Generating...' : 'Submit'}
      </button>
      <div>{completion}</div> {/* Streams in real-time */}
    </form>
  );
}

// Server-side API route that enables streaming
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export async function POST(request: Request) {
  const { prompt } = await request.json();

  const result = streamText({
    model: openai('gpt-4o'),
    prompt,
  });

  return result.toDataStreamResponse();
}

The streaming implementation handles partial responses, maintains proper formatting, and provides graceful error recovery if connections are interrupted: Text appears progressively as the AI generates it, creating engaging real-time interactions For more on how React handles server-side streaming and Suspense, check out LogRocket’s guide to streaming SSR with React 18.

Function calling: Extending AI capabilities

Function calling transforms static AI responses into dynamic interactions that can perform actions, fetch data, or integrate with external services. The SDK provides structured interfaces for defining custom tools with type safety and validation:

import { generateText, tool } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const { text } = await generateText({
  model: openai('gpt-4o'),
  prompt: 'What is the weather like today in San Francisco?',
  tools: {
    getWeather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
});

The SDK handles the complex orchestration of function calls, parameter validation, and response integration, allowing you to focus on implementing business logic: These four core features work synergistically to create a development experience that prioritizes both ease of implementation and sophisticated AI capabilities.

Advanced frontend patterns with AI SDK

Beyond the fundamentals, the SDK supports advanced use cases like image analysis, voice output, and generative UI. These patterns help developers build rich, multi-modal, and adaptive experiences.

Image upload and analysis integration

The SDK's built-in attachment system makes it straightforward to add image upload capabilities to your chat interface.

The key is leveraging the experimental_attachments parameter in the handleSubmit function, which automatically processes files and sends them to your AI model for analysis. Let's start with the main component structure and state management:

'use client';
import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

interface ImagePreviewsProps {
  files: File[];
  onRemove: (files: File[]) => void;
}
export default function MultiModalChat() {
  const [selectedFiles, setSelectedFiles] = useState<File[]>([]);

  const { 
    messages, 
    input, 
    handleInputChange, 
    handleSubmit, 
    status 
  } = useChat({
    api: '/api/multi-modal',
  });

  const handleFileChange = (event: React.ChangeEvent<HTMLInputElement>) => {
    const files = Array.from(event.target.files || []);
    setSelectedFiles(files);
  };

  const fileToDataURL = (file: File): Promise<string> => {
    return new Promise((resolve, reject) => {
      const reader = new FileReader();
      reader.onload = (e) => resolve(e.target?.result as string);
      reader.onerror = reject;
      reader.readAsDataURL(file);
    });
  };

The custom submit handler converts selected files into the attachment format that the SDK expects. This transformation is crucial because it ensures the AI model receives properly formatted image data:

    const customSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
    event.preventDefault();

    // Convert files to data URLs for API compatibility
    const attachments = await Promise.all(
      selectedFiles.map(async (file) => ({
        name: file.name,
        contentType: file.type,
        url: await fileToDataURL(file)
      }))
    );
    handleSubmit(event, {
      experimental_attachments: attachments
    });

    setSelectedFiles([]);
  };

Now we'll build the UI structure that displays messages and provides file upload functionality. The messages container establishes the conversational layout with proper styling and responsive design.

This component handles the display of both user and AI messages, including attached images and streaming indicators:

  return (
    <div className="max-w-4xl mx-auto p-6">
      <h2 className="text-3xl font-bold mb-8 text-gray-900">Multi-Modal AI Chat</h2>

      {/* Messages container */}
      <div className="bg-white rounded-xl shadow-sm border border-gray-200 mb-6">
        <div className="p-6 space-y-4 max-h-96 overflow-y-auto">
          {messages.length === 0 ? (
            <div className="text-center text-gray-500 py-8">
              <p>Start a conversation by uploading images and asking questions!</p>
            </div>
          ) : (
            messages.map((message) => (
              <div key={message.id} className={`flex ${message.role === 'user' ? 'justify-end' : 'justify-start'}`}>
                <div className={`max-w-[80%] p-4 rounded-lg ${
                  message.role === 'user' 
                    ? 'bg-blue-600 text-white' 
                    : 'bg-gray-100 text-gray-900 border border-gray-200'
                }`}>
                  <div className="text-sm font-medium mb-1 opacity-75">
                    {message.role === 'user' ? 'You' : 'AI Assistant'}
                  </div>
                  <div className="text-sm leading-relaxed">{message.content}</div>

                  {/* Show attached images for user messages */}
                  {message.role === 'user' && message.experimental_attachments && (
                    <div className="mt-2 flex gap-2 flex-wrap">
                      {message.experimental_attachments.map((attachment, index) => (
                        <img
                          key={index}
                          src={attachment.url}
                          alt={attachment.name || 'Uploaded image'}
                          className="w-20 h-20 object-cover rounded border border-white/20"
                        />
                      ))}
                    </div>
                  )}
                </div>
              </div>
            ))
          )}

          {status === 'streaming' && (
            <div className="flex justify-start">
              <div className="bg-gray-100 text-gray-900 border border-gray-200 max-w-[80%] p-4 rounded-lg">
                <div className="text-sm font-medium mb-1 opacity-75">AI Assistant</div>
                <div className="flex items-center space-x-2">
                  <div className="flex space-x-1">
                    <div className="w-2 h-2 bg-gray-500 rounded-full animate-bounce"></div>
                    <div className="w-2 h-2 bg-gray-500 rounded-full animate-bounce" style={{animationDelay: '0.1s'}}></div>
                    <div className="w-2 h-2 bg-gray-500 rounded-full animate-bounce" style={{animationDelay: '0.2s'}}></div>
                  </div>
                  <span className="text-sm text-gray-600">Analyzing images...</span>
                </div>
              </div>
            </div>
          )}
        </div>
      </div>

The file upload interface combines functionality with accessibility, using a hidden input element and a custom-styled label for better control over appearance.

The design provides immediate feedback about selected files and integrates seamlessly with the chat interface:

        {/* Input form */}
      <div className="bg-white rounded-xl shadow-sm border border-gray-200 p-6">
        <form onSubmit={customSubmit} className="space-y-4">
          {/* File upload section */}
          <div className="flex items-center gap-3">
            <input
              type="file"
              accept="image/*"
              multiple
              onChange={handleFileChange}
              className="hidden"
              id="file-upload"
            />
            <label
              htmlFor="file-upload"
              className="inline-flex items-center px-4 py-2 bg-gray-50 hover:bg-gray-100 text-gray-700 font-medium rounded-lg border border-gray-300 cursor-pointer transition-colors"
            >
              <svg className="w-4 h-4 mr-2" fill="none" stroke="currentColor" viewBox="0 0 24 24">
                <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M15.172 7l-6.586 6.586a2 2 0 102.828 2.828l6.414-6.586a4 4 0 00-5.656-5.656l-6.415 6.585a6 6 0 108.486 8.486L20.5 13" />
              </svg>
              Attach Images
            </label>
            {selectedFiles.length > 0 && (
              <span className="text-sm text-gray-600 font-medium">
                {selectedFiles.length} image{selectedFiles.length > 1 ? 's' : ''} selected
              </span>
            )}
          </div>
          {/* File previews */}
          {selectedFiles.length > 0 && (
            <ImagePreviews files={selectedFiles} onRemove={setSelectedFiles} />
          )}
          {/* Input and submit */}
          <div className="flex gap-3">
            <input
              value={input}
              onChange={handleInputChange}
              placeholder="Describe what you want to know about the images..."
              className="flex-1 px-4 py-3 border border-gray-300 rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-transparent text-gray-900 placeholder-gray-500"
              disabled={status === 'streaming'}
            />
            <button
              type="submit"
              disabled={status === 'streaming' || (!input.trim() && selectedFiles.length === 0)}
              className="px-6 py-3 bg-blue-600 hover:bg-blue-700 disabled:bg-gray-300 disabled:cursor-not-allowed text-white font-medium rounded-lg transition-colors"
            >
              {status === 'streaming' ? 'Analyzing...' : 'Send'}
            </button>
          </div>
        </form>
      </div>
    </div>
  );
}

The ImagePreviews component provides a clean interface for managing selected images before they're sent to the AI model:

function ImagePreviews({ files, onRemove }: ImagePreviewsProps) {
  const removeFile = (indexToRemove: number) => {
    onRemove(files.filter((_, index) => index !== indexToRemove));
  };
  return (
    <div className="bg-gray-50 rounded-lg p-4 border border-gray-200">
      <div className="flex gap-3 flex-wrap">
        {files.map((file, index) => (
          <div key={index} className="relative group">
            <img
              src={URL.createObjectURL(file)}
              alt={file.name}
              className="w-20 h-20 object-cover rounded-lg border border-gray-300 shadow-sm"
            />
            <button
              type="button"
              onClick={() => removeFile(index)}
              className="absolute -top-2 -right-2 w-6 h-6 bg-red-500 hover:bg-red-600 text-white rounded-full flex items-center justify-center text-sm font-medium transition-colors shadow-sm"
            >
              ×
            </button>
            <div className="absolute bottom-0 left-0 right-0 bg-black bg-opacity-50 text-white text-xs p-1 rounded-b-lg truncate">
              {file.name}
            </div>
          </div>
        ))}
      </div>
    </div>
  );
}

Multi-modal support makes image upload and analysis straightforward with built-in preview functionality

Voice output handling

The AI SDK's speech capabilities enable applications to generate natural-sounding audio responses, creating more immersive and accessible user experiences. The generateSpeech function transforms text into high-quality audio using advanced speech synthesis models.

In addition to enhancing user engagement, speech synthesis can improve accessibility for users with visual impairments or reading difficulties, making AI-driven applications more inclusive and user-friendly. The core speech generation functionality allows you to convert any text response into audio format using various voice models and configurations:

import { experimental_generateSpeech as generateSpeech } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function createAudioResponse(text: string, voice: string = 'alloy') {
  const audio = await generateSpeech({
    model: openai.speech('tts-1'),
    text: text,
    voice: voice, // Options: alloy, echo, fable, onyx, nova, shimmer
  });

  return audio.audioData; // Returns Uint8Array for audio data
}

The backend API route handles the actual speech generation and returns audio data in a streamable format:

// app/api/generate-speech/route.ts
import { experimental_generateSpeech as generateSpeech } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(request: Request) {
  try {
    const { text, voice = 'alloy' } = await request.json();

    const audio = await generateSpeech({
      model: openai.speech('tts-1'),
      text: text,
      voice: voice,
    });

    // Return audio as blob response
    return new Response(audio.audioData, {
      headers: {
        'Content-Type': 'audio/mpeg',
        'Content-Length': audio.audioData.length.toString(),
      },
    });
  } catch (error) {
    return Response.json({ error: 'Speech generation failed' }, { status: 500 });
  }
}

For more user experience in your applications, you can implement voice selection and audio processing features:

const voiceOptions = [
  { value: 'alloy', label: 'Alloy (Neutral)' },
  { value: 'echo', label: 'Echo (Male)' },
  { value: 'fable', label: 'Fable (British Male)' },
  { value: 'onyx', label: 'Onyx (Deep Male)' },
  { value: 'nova', label: 'Nova (Female)' },
  { value: 'shimmer', label: 'Shimmer (Soft Female)' }
];

function VoiceSettings({ selectedVoice, onVoiceChange }) {
  return (
    <div className="voice-settings">
      <label>AI Voice:</label>
      <select value={selectedVoice} onChange={(e) => onVoiceChange(e.target.value)}>
        {voiceOptions.map(option => (
          <option key={option.value} value={option.value}>
            {option.label}
          </option>
        ))}
      </select>
    </div>
  );
}

This implementation provides a complete voice output system that enhances user engagement through natural speech synthesis, making AI interactions more accessible and humanlike.

Generative user interfaces

The foundation of generative UI lies in tool invocations that trigger specific React components. When the AI determines a user needs a particular interface element, it calls the appropriate tool and passes the results to a corresponding component:

export const formGeneratorTool = createTool({
  description: 'Generate a dynamic form based on user requirements',
  parameters: z.object({
    formType: z.string().describe('Type of form to create'),
    fields: z.array(z.object({
      name: z.string(),
      type: z.enum(['text', 'email', 'select', 'textarea']),
      label: z.string(),
      required: z.boolean(),
      options: z.array(z.string()).optional()
    }))
  }),
  execute: async function ({ formType, fields }) {
    return {
      title: `${formType} Form`,
      fields: fields.map(field => ({
        ...field,
        options: field.options?.map(opt => ({ value: opt, label: opt }))
      }))
    };
  },
});

The chat interface checks for tool invocations and renders appropriate components based on the tool name and state:

const renderMessage = (message) => {
  if (message.toolInvocations?.length > 0) {
    return message.toolInvocations.map((tool) => {
      if (tool.state === 'result') {
        switch (tool.toolName) {
          case 'generateForm':
            return <DynamicForm key={tool.toolCallId} formData={tool.result} />;
          case 'createChart':
            return <DataVisualization key={tool.toolCallId} chartConfig={tool.result} />;
          case 'buildSurvey':
            return <InteractiveSurvey key={tool.toolCallId} surveyData={tool.result} />;
        }
      }
      return <LoadingIndicator key={tool.toolCallId} toolName={tool.toolName} />;
    });
  }
  return <div>{message.content}</div>;
};

Generative UI extends beyond forms to include dynamic chart creation. Users can request visualizations through conversational queries, and the AI generates appropriate chart configurations:

const chartTool = createTool({
  description: 'Create data visualizations from user queries',
  parameters: z.object({
    chartType: z.enum(['line', 'bar', 'pie', 'area']),
    data: z.array(z.record(z.union([z.string(), z.number()]))),
    xAxis: z.string(),
    yAxis: z.string(),
    title: z.string()
  }),
  execute: async function ({ chartType, data, xAxis, yAxis, title }) {
    // In practice, this might fetch real data from APIs
    return { chartType, data, xAxis, yAxis, title };
  }
});

The resulting visualization component adapts to different chart types while maintaining consistent styling and interaction patterns, making data analysis accessible through natural language requests.

For a complementary exploration of AI agents in frontend apps, including how to bridge reasoning with action, check out LogRocket’s tutorial on building an AI agent for your frontend project.

Model context protocol (MCP) integration

MCP integration allows your AI applications to connect with external data sources and services, enabling context-aware applications that can access real-time information and perform complex operations. Note: MCP integration is currently experimental and may change in future SDK releases:

// Server-side MCP client setup
import { experimental_createMCPClient as createMCPClient } from 'ai';

const mcpClient = createMCPClient({
  name: 'data-connector',
  version: '1.0.0',
  transport: {
    type: 'stdio',
    command: 'node',
    args: ['./mcp-servers/database-server.js']
  }
});

const tools = await mcpClient.tools();

const result = await streamText({
  model: openai('gpt-4o'),
  tools,
  messages: 'What is the weather in Brooklyn, New York?',
  onFinish: async () => {
    await mcpClient.close();
  },
});

By integrating MCP (multi-context prompting) capabilities with the standard AI SDK tools, the API route allows your AI to query external systems on demand. This approach makes it possible to build more context-aware applications, ones that can understand and respond to information beyond the immediate user input.

These advanced patterns highlight how the Vercel AI SDK supports rich, dynamic interactions that go well beyond basic text generation. The real power lies in how these features work together to deliver seamless, intelligent user experiences.

Frontend performance considerations for AI features

AI-powered features introduce unique performance challenges that traditional web applications don't face. The Vercel AI SDK provides several built-in optimizations and patterns to ensure your AI features remain fast and responsive.

Streaming optimization: Managing progressive loading and user engagement

Effective streaming hinges on balancing responsiveness with resource efficiency. When the rate of data production outpaces consumption, stream backpressure prevents resource overload by slowing the flow.

The Vercel AI SDK manages this seamlessly using lazy evaluation data is only pulled when the consumer is ready, ensuring efficient, real-time interactions without overwhelming the client:

function createOptimizedStream(generator) {
  return new ReadableStream({
    async pull(controller) {
      const { value, done } = await generator.next();

      if (done) {
        controller.close();
      } else {
        controller.enqueue(value);
      }
    },
  });
}

This mechanism also helps prevent memory leaks when users navigate away or connections drop unexpectedly. Without proper backpressure handling, servers may continue pulling data from AI providers even after the client has disconnected, wasting resources and risking exhaustion.

Cancellation works hand-in-hand with backpressure. When streams are implemented with lazy evaluation, unused resources are cleaned up automatically as soon as the consumer stops reading. Additionally, the SDK allows you to render multiple streamable UI elements in parallel, rather than waiting for a single large response, enabling faster, more interactive user experiences:

export async function generateDashboard() {
  const analyticsUI = createStreamableUI(<Skeleton />);
  const chartsUI = createStreamableUI(<Skeleton />);
  const summaryUI = createStreamableUI(<Skeleton />);

  // These update independently as data becomes available
  Promise.all([
    fetchAnalytics().then(data => analyticsUI.done(<Analytics data={data} />)),
    fetchCharts().then(data => chartsUI.done(<Charts data={data} />)),
    fetchSummary().then(data => summaryUI.done(<Summary data={data} />))
  ]);

  return {
    timestamp: Date.now(),
    analytics: analyticsUI.value,
    charts: chartsUI.value,
    summary: summaryUI.value,
  };
}

This approach reduces perceived loading time by showing progressive updates rather than blocking until all data is ready. Users see immediate feedback while background processes complete asynchronously.

Caching strategies: Response caching and provider switching

Modern AI applications need intelligent caching to balance performance with cost. The AI SDK provides multiple layers of caching that work seamlessly together.

You can utilize server-side response caching. This involves response caching to prevent redundant API calls by storing and replaying previous responses. Using language model middleware, you can cache both streaming and non-streaming responses while preserving the user experience:

export const cacheMiddleware: LanguageModelV1Middleware = {
  wrapStream: async ({ doStream, params }) => {
    const cacheKey = JSON.stringify(params);
    const cached = await redis.get(cacheKey);

    if (cached !== null) {
      return {
        stream: simulateReadableStream({
          chunkDelayInMs: 10,
          chunks: cached,
        }),
      };
    }

    // Cache new responses after streaming completes
    const { stream } = await doStream();
    const fullResponse = [];

    return {
      stream: stream.pipeThrough(new TransformStream({
        transform(chunk, controller) {
          fullResponse.push(chunk);
          controller.enqueue(chunk);
        },
        flush() {
          redis.set(cacheKey, fullResponse);
        },
      })),
    };
  },
};

The simulateReadableStream function replays cached responses chunk-by-chunk, maintaining the streaming experience users expect while eliminating redundant API calls. Also, you can leverage a dynamic LLM provider selection during runtime.

The SDK's provider abstraction enables intelligent model switching based on system load, cost constraints, or response requirements. This allows applications to optimize performance and expenses dynamically:

const getOptimalProvider = (requestType) => {
  const currentLoad = getSystemLoad();
  const isSimpleQuery = requestType === 'simple';

  if (currentLoad > 0.8 || isSimpleQuery) {
    return openai('gpt-4o-mini');
  }
  return openai('gpt-4o');
};

const result = streamText({
  model: getOptimalProvider(queryType),
  messages,
});

This approach lets you route expensive queries to premium models while handling routine requests with cost-effective alternatives, all without changing your application logic.

Cost management: Rate limiting and usage monitoring

AI API costs can escalate quickly without proper controls. The SDK provides several mechanisms to help you monitor and control costs effectively. Rate limiting is essential for preventing abuse and managing costs.

You can implement limits at both the client and server level - client-side for immediate user feedback, and server-side for enforcement. The SDK's built-in request tracking makes it straightforward to implement per-user or per-session limits.

Usage monitoring becomes critical as your application scales. Token usage varies significantly based on conversation length, model choice, and feature complexity. The onFinish callback in useChat provides detailed usage statistics,including prompt tokens, completion tokens, and finish reasons, allowing you to track costs in real-time.

Final thoughts

The Vercel AI SDK reshapes frontend AI development by offering unified APIs, seamless React integration, real-time streaming, function calling, and generative UI capabilities, all in one cohesive toolkit.

It removes the friction of integrating multiple AI providers while enabling advanced, responsive user experiences. With provider abstraction to avoid vendor lock-in, intuitive React Hooks to streamline state management, and streaming for fluid interactions, the SDK makes AI feel like a native part of your app.

Function calling connects AI-driven reasoning with real-world actions, and generative UI unlocks adaptive, dynamic interfaces. Start simple with a basic chat interface, layer in multimodal features as needed, and explore generative UI where it enhances the experience.

The AI-first future of frontend development is already here, and the Vercel AI SDK makes it easier than ever to build for it. You can explore the full sample project and source code on GitHub.

Get set up with LogRocket's modern error tracking in minutes:

Visit https://logrocket.com/signup/ to get an app ID.
Install LogRocket via NPM or script tag. LogRocket.init() must be called client-side, not server-side.

NPM:

$ npm i --save logrocket 

// Code:

import LogRocket from 'logrocket'; 
LogRocket.init('app/id');

Script Tag:

Add to your HTML:

<script src="https://cdn.lr-ingest.com/LogRocket.min.js"></script>
<script>window.LogRocket && window.LogRocket.init('app/id');</script>

3.(Optional) Install plugins for deeper integrations with your stack:

Redux middleware
ngrx middleware
Vuex plugin

Get started now.

Next.js real-time video streaming: HLS.js and alternatives

Megan Lee — Wed, 23 Jul 2025 18:00:00 +0000

Written by Jude Miracle✏️

Real-time video streaming is essential for modern web applications. It supports activities like live broadcasting, video conferencing, and media interaction. For developers using Next.js, finding a reliable streaming solution is important for creating smooth user experiences.

One popular choice is HLS.js, a JavaScript library that enables adaptive bitrate streaming using HTTP Live Streaming (HLS) directly in the browser. In this article, we will show you how to set up real-time video streaming in Next.js using HLS.js.

You'll learn about integration and advanced features like adaptive streaming and token-based authentication. We will also compare HLS.js with other open source options to help you pick the best solution based on cost, latency, features, and scalability.

What is HLS.js?

HLS.js is a free JavaScript library that allows browsers to play HTTP Live Streaming (HLS) content. HLS is a popular video streaming method created by Apple. It helps deliver live and on-demand video smoothly over standard web servers. HLS.js works well even when browsers don’t support HLS natively, which often varies across platforms.

It reads HLS manifest files (usually with the .m3u8 extension), separates the video and audio streams, and then feeds them into a standard HTML5 <video> element. This structure helps adapt to changing network conditions, ensuring the best video quality with little buffering. HLS.js features:

Adaptive Bitrate Streaming (ABR): HLS.js automatically changes video quality based on internet speed and device performance. It switches between quality levels defined in the HLS manifest to give viewers the best experience without interruptions
Live and on-demand streaming: It supports both live broadcasts and video-on-demand (VOD) content, handling various types of HLS streams
Fragmented MP4 (fMP4) and MPEG-2 Transport Stream (TS) support: HLS.js works with common HLS segment formats, allowing different content types
Audio-only streams: It can manage streams that only include audio
Extensive event handling: HLS.js has many events that help developers track the streaming process, handle errors, monitor buffering, and manage playback for a customized experience
Low Latency HLS (LL-HLS) support: HLS.js includes features for Low Latency HLS, which is important for reducing delays during live events like sports
Performance optimization: Users can optimize performance with settings like buffer tuning, pre-loading segments, and managing network requests

Prerequisites for our demo project

Before implementing HLS.js in Next.js, ensure you have:

Node.js (v18+) installed
A Next.js (v14+) project set up
Basic knowledge of React Hooks and JavaScript
Basic understanding of HTML5 Video

Project setup

Start by creating a fresh Next.js project using the official create-next-app command:

npx create-next-app@latest next-video-streaming

During the setup process, you'll be prompted to configure your project. For video streaming applications, we recommend the following selections:

TypeScript: Yes (recommended for better development experience)
ESLint: Yes (helps maintain code quality)
Tailwind CSS: Yes (for styling components)
App Router: Yes (uses the modern App Router)
Import alias: Yes (helps with cleaner imports)

Navigate to your project directory:

cd video-streaming-app

Next, we will install hls.js for our video streaming functionality:

npm install hls.js

If you're using TypeScript, you might also want to install the type definitions:

npm install --save-dev @types/hls.js

Streaming implementation

To start a live broadcast using HLS, you need to create and manage an HLS playlist that keeps updating with new video segments. This section explains how to add live streaming to your Next.js application.

Starting a live broadcast using an HLS playlist

HLS splits video into small segments and creates a playlist file (.m3u8) that lists these segments. For live streaming, this playlist updates regularly with new segments as the broadcast continues.

Setting up a live stream component

Create a new component for live streaming:

"use client";
import { useEffect, useRef, useState } from 'react';
import Hls from 'hls.js';

export default function LiveStream({ playlistUrl }: { playlistUrl: string }) {
  const videoRef = useRef<HTMLVideoElement>(null);
  const hlsRef = useRef<Hls | null>(null);
  const [isLive, setIsLive] = useState(false);
  const [error, setError] = useState<string | null>(null);
  useEffect(() => {
    const video = videoRef.current;
    if (Hls.isSupported() && video) {
      const hls = new Hls({
        // Enable live streaming optimizations
        liveSyncDurationCount: 3,
        liveMaxLatencyDurationCount: 5,
      });
      hlsRef.current = hls;
      // Load the live playlist
      hls.loadSource(playlistUrl);
      hls.attachMedia(video);
      return () => {
        hls.destroy();
      };
    } else if (video?.canPlayType('application/vnd.apple.mpegurl')) {
      // Safari native HLS support
      video.src = playlistUrl;
      setIsLive(true);
    } else {
      setError('Live streaming not supported in this browser');
    }
  }, [playlistUrl]);
  const startBroadcast = () => {
    const video = videoRef.current;
    if (video && isLive) {
      video.play();
    }
  };
  const stopBroadcast = () => {
    const video = videoRef.current;
    if (video) {
      video.pause();
    }
  };
  return (
    <div>
      <div className='mb-2'>
        <span 
        className={`text-sm font-bold ${isLive ? 'text-green-500' : 'text-red-500'}`}>
          {isLive ? 'LIVE' : 'OFFLINE'}
        </span>
      </div>
      {error ? (
        <div className='text-red-500'>Error: {error}</div>
      ) : (
        <video
          ref={videoRef}
          controls
          className='w-full h-auto'
        />
      )}
      <div className='mt-2'>
        <button onClick={startBroadcast} disabled={!isLive} className='bg-blue-500 text-white px-4 py-2 rounded-md'>
          Start Watching
        </button>
        <button onClick={stopBroadcast} className='ml-2 bg-red-500 text-white px-4 py-2 rounded-md'>
          Stop
        </button>
      </div>
    </div>
  );
}

The LiveStream component shows how to stream video. It takes an HLS playlist URL (.m3u8) and uses the Hls class to load the stream into an HTML5 <video> element. Important configurations like liveSyncDurationCount and liveMaxLatencyDurationCount help keep the stream close to real-time, reducing delays while ensuring smooth playback.

For browsers that don’t support HLS.js, such as Safari with its native HLS support, the component directly sets the video source to the playlist URL. This way, the component works well across most modern browsers. Now, import the component into your page.js file:

'use client';
import LiveStream from "@/components/live-stream";
export default function Home() {
  return (
    <div className="flex flex-col justify-center items-center h-screen">
      <h1 className="text-2xl font-bold mb-4">Live Stream</h1>
      <LiveStream playlistUrl="https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8" />
    </div>
  );
}

Start the dev server and go to your browser. You will be able to start and stop the stream.

Managing stream states using HLS.js events

HLS.js has a system of events that helps you manage and control streaming. By listening to these events, you can track the streaming status (like live, buffering, or errors) and improve the user experience with immediate updates. The key events for live streaming are:

Hls.Events.MANIFEST_LOADED: This happens when the main playlist is loaded
Hls.Events.LEVEL_LOADED: This is triggered when a quality level playlist is loaded
Hls.Events.FRAG_LOADED: This occurs when video segments load successfully
Hls.Events.ERROR: This handles different types of streaming errors
Hls.Events.BUFFER_APPENDED: This indicates when video data is added to the buffer

Now, to integrate event handling into the LiveStream component, modify the useEffect Hook:

useEffect(() => {
  const video = videoRef.current;

  if (Hls.isSupported() && video) {
    const hls = new Hls({
      liveSyncDurationCount: 3,
      liveMaxLatencyDurationCount: 5,
    });
    hlsRef.current = hls;

    hls.loadSource(playlistUrl);
    hls.attachMedia(video);

    // Event listeners
    hls.on(Hls.Events.MANIFEST_PARSED, () => {
      setIsLive(true);
    });
    hls.on(Hls.Events.ERROR, (event, data) => {
      if (data.fatal) {
        setError(`Stream error: ${data.details}`);
      }
    });

    return () => hls.destroy();
  } else if (video?.canPlayType('application/vnd.apple.mpegurl')) {
    video.src = playlistUrl;
    setIsLive(true);
  } else {
    setError('Live streaming not supported in this browser');
  }
}, [playlistUrl]);

This code updates the isLive state when the manifest is parsed and sets an error message for fatal errors, improving user feedback.

HLS.js advanced features

As your streaming app becomes more complex and gains more users, it's important to add features that enhance performance, security, and user experience.

This section discusses three key advanced features: 1) adaptive bitrate streaming to provide different video quality levels, 2) secure token-based authentication for user safety, and 3) performance optimization through configuration adjustments.

Handling multiple quality levels for Adaptive Bitrate Streaming

Adaptive Bitrate Streaming (ABR) is important for providing a smooth video experience, no matter how good or bad the internet connection is or what kind of device you are using. HLS.js uses smart ABR methods that automatically change the video quality based on current network speed, how much data is stored, and the device being used.

Understanding ABR in HLS.js

HLS.js determines the best video quality based on several factors:

Network bandwidth: This measures how fast segments download
Buffer health: This shows how much content is stored ahead of what you're watching
Device capabilities: This includes the screen size and processing power of your device
Historical performance: This looks at past decisions for changing quality and their results

In our Next.js application, we can allow users to choose video quality levels for more control. HLS.js automatically adjusts video quality based on the playlist's variant streams. However, you can improve the user experience by letting them select quality levels. Here is an example of how to set this up in the LiveStream component:

"use client";
import { useEffect, useRef, useState } from 'react';
import Hls from 'hls.js';
export default function LiveStream({ playlistUrl }: { playlistUrl: string }) {
  const videoRef = useRef<HTMLVideoElement>(null);
  const hlsRef = useRef<Hls | null>(null);
  const [isLive, setIsLive] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const [qualityLevels, setQualityLevels] = useState<Hls.Level[]>([]);
  const [currentQuality, setCurrentQuality] = useState<number | null>(null);
  useEffect(() => {
    const video = videoRef.current;
    if (Hls.isSupported() && video) {
      const hls = new Hls({
        // Enable live streaming optimizations
        liveSyncDurationCount: 3,
        liveMaxLatencyDurationCount: 5,
      });
      hlsRef.current = hls;
      // Load the live playlist
      hls.loadSource(playlistUrl);
      hls.attachMedia(video);
        // Event listeners
        hls.on(Hls.Events.MANIFEST_PARSED, () => {
            setQualityLevels(hls.levels as unknown as Hls.Level[]); // store available quality levels
            setIsLive(true);
        });
        hls.on(Hls.Events.LEVEL_SWITCHED, (event, data) => {
            console.log(`Switched to quality level: ${data.level}`);
            setCurrentQuality(data.level);
        });
        hls.on(Hls.Events.ERROR, (event, data) => {
            if (data.fatal) {
            setError(`Stream error: ${data.details}`);
            }
        });
      return () => {
        hls.destroy();
      };
    } else if (video?.canPlayType('application/vnd.apple.mpegurl')) {
      // Safari native HLS support
      video.src = playlistUrl;
      setIsLive(true);
    } else {
      setError('Live streaming not supported in this browser');
    }
  }, [playlistUrl]);
  const changeQuality = (levelIndex: number) => {
    if (hlsRef.current) {
      hlsRef.current.currentLevel = levelIndex;
    }
  };
  const startBroadcast = () => {
    const video = videoRef.current;
    if (video && isLive) {
      video.play();
    }
  };
  const stopBroadcast = () => {
    const video = videoRef.current;
    if (video) {
      video.pause();
    }
  };
  return (
    <div>
      <div className='mb-2'>
        <span
        className={`text-sm font-bold ${isLive ? 'text-green-500' : 'text-red-500'}`}>
          {isLive ? 'LIVE' : 'OFFLINE'}
        </span>
      </div>
      {error ? (
        <div className='text-red-500'>Error: {error}</div>
      ) : (
        <div>
          <video ref={videoRef} controls className='w-full h-auto' />
          <div className='mt-2'>
            {qualityLevels.map((level, index) => (
              <button key={index} onClick={() => changeQuality(index)} className={`${currentQuality === index ? 'bg-blue-500 text-white' : 'bg-gray-200 text-gray-800'} px-2 py-1 rounded-md`}>
                {level.height}p
              </button>
            ))}
          </div>
        </div>
      )}
      <div className='mt-2'>
        <button onClick={startBroadcast} disabled={!isLive} className='bg-blue-500 text-white px-4 py-2 rounded-md'>
          Start Watching
        </button>
        <button onClick={stopBroadcast} className='ml-2 bg-red-500 text-white px-4 py-2 rounded-md'>
          Stop
        </button>
      </div>
    </div>
  );
}

The Hls.Events.MANIFEST_PARSED event gathers the available quality levels and saves them in the state. Each quality level includes information like height for resolution. The changeQuality function lets users manually pick a quality level by setting hls.currentLevel.

HLS.js will automatically change quality levels unless the currentLevel is set. You can adjust how quickly it changes quality with settings like abrEwmaFastLive and abrEwmaSlowLive to respond to network changes.

The component shows buttons for each quality level, which let users switch resolutions easily: If you need specific behavior from ABR, you can also create your own switching logic.

Implementing token-based authentication for secure streams

Protecting HLS streams is important for premium content, private broadcasts, and subscription services. HLS.js offers different ways to authenticate users by using a flexible system that lets you add authentication tokens to playlist requests and segment downloads. HLS authentication works in two main steps:

Playlist authentication: This secures access to the main .m3u8 manifest file
Segment authentication: This protects individual video segments, such as .ts or .mp4 files

The typical authentication flow is:

The user logs into your application
The server creates a temporary token
The client uses this token in HLS requests
The server checks the token before delivering content

Implementing custom request headers

HLS.js lets you change HTTP requests using the xhrSetup callback. Here's how to set up token-based authentication:

"use client";
import { useEffect, useRef, useState } from 'react';
import Hls from 'hls.js';
interface AuthResponse {
  authenticated: boolean;
  user?: {
    id: string;
    username: string;
    subscription: string;
  };
  stream?: {
    id: string;
    name: string;
    url: string;
  };
  accessGranted?: boolean;
  error?: string;
}
interface SecureVideoPlayerProps {
  streamId: string;
  authToken: string;
  onAuthError: (error: string) => void;
}
export default function SecureVideoPlayer({ streamId, authToken, onAuthError }: SecureVideoPlayerProps) {
  const videoRef = useRef<HTMLVideoElement>(null);
  const hlsRef = useRef<Hls | null>(null);
  const [isLive, setIsLive] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const [isLoading, setIsLoading] = useState(true);
  const [isAuthenticated, setIsAuthenticated] = useState(false);
  const [user, setUser] = useState<any>(null);
  const [streamUrl, setStreamUrl] = useState<string | null>(null);
  const [qualityLevels, setQualityLevels] = useState<Hls.Level[]>([]);
  const [currentQuality, setCurrentQuality] = useState<number | null>(null);
  // Authenticate with the API
  const authenticate = async () => {
    try {
      setIsLoading(true);
      setError(null);
      const response = await fetch(`/api/auth?token=${authToken}&streamId=${streamId}`, {
        method: 'GET',
        headers: {
          'Content-Type': 'application/json',
        },
      });
      const data: AuthResponse = await response.json();
      if (!response.ok) {
        throw new Error(data.error || 'Authentication failed');
      }
      if (!data.authenticated || !data.accessGranted) {
        throw new Error('Access denied to this stream');
      }
      setIsAuthenticated(true);
      setUser(data.user);
      setStreamUrl(data.stream?.url || null);
    } catch (err) {
      const errorMessage = err instanceof Error ? err.message : 'Authentication failed';
      setError(errorMessage);
      onAuthError(errorMessage);
    } finally {
      setIsLoading(false);
    }
  };
  // Initialize HLS stream after authentication
  useEffect(() => {
    if (!isAuthenticated || !streamUrl) return;
    const video = videoRef.current;
    if (!video) return;
    if (Hls.isSupported() && video) {
      const hls = new Hls({
        xhrSetup: (xhr) => {
          xhr.setRequestHeader('Authorization', `Bearer ${authToken}`);
          xhr.setRequestHeader('X-Custom-Auth', authToken);
          xhr.setRequestHeader('X-Timestamp', Date.now().toString());
          xhr.setRequestHeader('X-Stream-ID', streamId);
        },
        liveSyncDurationCount: 3,
        liveMaxLatencyDurationCount: 5,
        manifestLoadingTimeOut: 10000,
        fragLoadingTimeOut: 20000,
      });
      hlsRef.current = hls;
      hls.loadSource(streamUrl);
      hls.attachMedia(video);
      hls.on(Hls.Events.MANIFEST_PARSED, () => {
        setQualityLevels(hls.levels as unknown as Hls.Level[]);
        setIsLive(true);
        setIsLoading(false);
      });
      hls.on(Hls.Events.LEVEL_SWITCHED, (event, data) => {
        console.log(`Switched to quality level: ${data.level}`);
        setCurrentQuality(data.level);
      });
      hls.on(Hls.Events.ERROR, (event, data) => {
        if (data.fatal) {
          switch (data.type) {
            case Hls.ErrorTypes.NETWORK_ERROR:
              if (data.response?.code === 401 || data.response?.code === 403) {
                setError('Authentication failed. Please log in again.');
                onAuthError('Authentication failed. Please log in again.');
              } else {
                setError('Network error occurred');
              }
              break;
            case Hls.ErrorTypes.MEDIA_ERROR:
              setError('Media error occurred');
              break;
            default:
              setError('An unknown error occurred');
              break;
          }
        }
      });
      return () => {
        hls.destroy();
      };
    } else if (video?.canPlayType('application/vnd.apple.mpegurl')) {
      video.src = streamUrl;
      setIsLive(true);
      setIsLoading(false);
    } else {
      setError('Live streaming not supported in this browser');
    }
  }, [isAuthenticated, streamUrl, authToken, streamId, onAuthError]);
  useEffect(() => {
    authenticate();
  }, [streamId, authToken]);
  const changeQuality = (levelIndex: number) => {
    if (hlsRef.current) {
      hlsRef.current.currentLevel = levelIndex;
    }
  };
  const startBroadcast = () => {
    const video = videoRef.current;
    if (video && isLive) {
      video.play();
    }
  };
  const stopBroadcast = () => {
    const video = videoRef.current;
    if (video) {
      video.pause();
    }
  };
  if (isLoading) {
    return (
      <div className="flex items-center justify-center h-64 bg-gray-100 rounded-lg">
        <div className="text-center">
          <div className="animate-spin rounded-full h-8 w-8 border-b-2 border-blue-500 mx-auto mb-2"></div>
          <p className="text-gray-600">Authenticating and loading stream...</p>
        </div>
      </div>
    );
  }
  if (error) {
    return (
      <div className="bg-red-50 border border-red-200 rounded-lg p-4">
        <div className="text-red-800 font-medium">Error: {error}</div>
        <button 
          onClick={authenticate}
          className="mt-2 bg-red-600 text-white px-4 py-2 rounded-md hover:bg-red-700"
        >
          Retry Authentication
        </button>
      </div>
    );
  }
  return (
    <div className="space-y-4">
      {user && (
        <div className="bg-blue-50 border border-blue-200 rounded-lg p-3">
          <div className="text-sm text-blue-800">
            <span className="font-medium">User:</span> {user.username}
            <span className="ml-2 font-medium">Subscription:</span> {user.subscription}
          </div>
        </div>
      )}
      <div className="flex items-center space-x-2">
        <span className={`text-sm font-bold ${isLive ? 'text-green-500' : 'text-red-500'}`}>
          {isLive ? 'LIVE' : 'OFFLINE'}
        </span>
        {isAuthenticated && (
          <span className="text-xs bg-green-100 text-green-800 px-2 py-1 rounded-full">
            Authenticated
          </span>
        )}
      </div>
      <div className="relative">
        {isLoading && (
          <div className="absolute inset-0 flex items-center justify-center bg-black bg-opacity-50 z-10">
            <div className="text-white">Loading secure stream...</div>
          </div>
        )}
        <video
          ref={videoRef}
          controls
          className="w-full h-auto rounded-lg"
          onLoadStart={() => setIsLoading(true)}
          onCanPlay={() => setIsLoading(false)}
        />
      </div>
      {qualityLevels.length > 0 && (
        <div className="space-y-2">
          <h4 className="text-sm font-medium text-gray-700">Quality Options:</h4>
          <div className="flex flex-wrap gap-2">
            {qualityLevels.map((level, index) => (
              <button
                key={index}
                onClick={() => changeQuality(index)}
                className={`px-3 py-1 rounded-md text-sm font-medium transition-colors ${
                  currentQuality === index
                    ? 'bg-blue-500 text-white'
                    : 'bg-gray-200 text-gray-800 hover:bg-gray-300'
                }`}
              >
                {level.height}p
              </button>
            ))}
          </div>
        </div>
      )}
      <div className="flex space-x-2">
        <button
          onClick={startBroadcast}
          disabled={!isLive}
          className="bg-blue-500 text-white px-4 py-2 rounded-md hover:bg-blue-600 disabled:bg-gray-300 disabled:cursor-not-allowed"
        >
          Start Watching
        </button>
        <button
          onClick={stopBroadcast}
          className="bg-red-500 text-white px-4 py-2 rounded-md hover:bg-red-600"
        >
          Stop
        </button>
      </div>
    </div>
  );
}

In the code above, we used the xhrSetup function in the HLS instance to add an Authorization header with a Bearer token to every HTTP request made by HLS.js, such as for .m3u8 playlists and .ts/fMP4 segments. We pass the authToken as a prop, which we can retrieve after user signup through our authentication service from the backend API.

This process ensures secure access for each user. Additionally, we should listen for the Hls.Events.ERROR event to capture authentication failures, such as an invalid token, so that we can display clear error messages: We have already created a mock user database, a mock stream data, and an authentication form. We kept it short for the purpose of the article. For the full code, visit the GitHub repository.

Optimizing performance with HLS.js configurations

HLS.js offers many configuration options to help improve performance across various situations. By setting it up correctly, you can reduce startup time, decrease buffering, and enhance the overall viewing experience.

Here are some settings for HLS.js to improve performance in the LiveStream component. Add this option to the Hls instance:

  const hls = new Hls({
    liveSyncDurationCount: 3,
    liveMaxLatencyDurationCount: 5,
    maxBufferLength: 30,
    maxMaxBufferLength: 60,
    lowLatencyMode: true,
    backBufferLength: 90,
    abrEwmaFastLive: 3,
    abrEwmaSlowLive: 9,
    enableWorker: true,
    startLevel: -1,
  });

liveSyncDurationCount: 3: This keeps the player three segments behind the live broadcast, balancing latency and stability
liveMaxLatencyDurationCount: 5: If latency exceeds five segments, the stream skips segments to stay close to real-time
lowLatencyMode: true: This activates features that help achieve less than three seconds of latency
maxBufferLength: 30: The buffer is limited to 30 seconds to reduce memory use
maxMaxBufferLength: 60: The buffer size is capped at 60 seconds to prevent too much buffering on fast networks
backBufferLength: 90: This keeps 90 seconds of past content available for DVR features
abrEwmaFastLive: 3: This quickly adjusts to network changes for live streams
abrEwmaSlowLive: 9: This helps stabilize stream quality during small changes
enableWorker: true: This uses a Web Worker to handle processing, improving performance on the main thread
startLevel: -1: This allows HLS.js to automatically choose the best quality based on the network conditions

Hls.js alternatives

HLS.js is a strong option for video streaming in Next.js. However, developers can also choose from several open source alternatives, including Video.js, Stream’s API, Daily.co, Node.js/FFmpeg, and Dolby OptiView.

Each alternative has its own strengths, making them suitable for different use cases, such as live broadcasting, video conferencing, or low-latency streaming.

Video.js

Video.js is a popular open source tool for playing HTML5 videos. It works consistently across different browsers and devices and can use Flash or other technologies when HTML5 video isn’t fully supported.

While Video.js mainly focuses on playing videos, it has many plugins that let you customize it and add more features with special plugins. Video.js also provides a complete user interface with options for ads, analytics, and virtual reality. Key Video.js features include:

Cross-browser compatibility: It works well on many browsers and devices, both desktop and mobile, to provide a consistent playback experience
Customizable UI: It allows developers to easily customize the video player's look using CSS to match their application's style
Plugin architecture: It has a strong and clear plugin system that allows developers to add new features, such as analytics and advertising, or support for advanced streaming protocols. The videojs-contrib-hls plugin is important for HLS support
Adaptive streaming support (via plugins): With the right plugins, it can play HLS and DASH streams, automatically adjusting video quality based on network conditions.
Accessibility: It offers support for captions, subtitles, and audio descriptions, improving the viewing experience for everyone.
Event-driven API: It provides many events for controlling playback, checking status, and working with other application logic

Pros of Video.js

This tool has been developing for over ten years and has a large, active community
Its plugin system allows for many different uses
It provides clear guides and API references
Its large user base means there are many resources for help and best practices

Cons of Video.js

To use HLS playback, you need an extra plugin (videojs-contrib-hls), which adds a small overhead
Compared to HLS.js, which focuses only on HLS parsing, Video.js can be larger because of its wider range of features
Although it supports HLS, setting up and optimizing advanced features like specific ABR strategies might take more work than HLS.js's default setup

Use case

Here’s how to install Video.js via npm or yarn:

npm install video.js @types/video.js

Here’s how to implement it in your project:

import { useEffect, useRef } from 'react';
import videojs from 'video.js';
import 'video.js/dist/video-js.css';

export default function VideoJSPlayer({ options }) {
  const videoRef = useRef(null);
  const playerRef = useRef(null);

  useEffect(() => {
    if (!playerRef.current) {
      const videoElement = videoRef.current;
      if (!videoElement) return;

      playerRef.current = videojs(videoElement, options, () => {
        console.log('Player ready');
      });
    }

    return () => {
      if (playerRef.current) {
        playerRef.current.dispose();
        playerRef.current = null;
      }
    };
  }, [options]);

  return (
    <div data-vjs-player>
      <video ref={videoRef} className="video-js vjs-big-play-centered" />
    </div>
  );
}

Stream API (video)

Stream provides a set of APIs for building real-time applications. Its Video API offers a simple, managed way to handle live streaming and video playback on demand.

Instead of using client-side libraries for playback, Stream’s API manages the entire streaming process from capturing and encoding video to delivering it globally. This approach reduces the technical burden on developers, letting them concentrate on building their applications. Its key features include:

Managed streaming infrastructure: Stream takes care of video uploads, converts videos to different quality levels and formats, stores them, and delivers them globally using a CDN
Live and on-demand: It supports live broadcasts in real-time and also allows for the storage and delivery of pre-recorded videos
Low-latency options: It is designed to provide fast streaming for interactive experiences
Scalability: Stream easily handles a large number of viewers without needing developers to manage servers or networks
Integration with other stream products: You can easily combine it with Stream's chat or activity feed APIs to create engaging, interactive experiences
Recording and archiving: It often includes features to record live streams for later viewing

Pros of using Stream

It reduces the time it takes to launch a product by simplifying complex backend streaming systems
You don’t have to manage FFmpeg, media servers, or content delivery networks (CDNs)
It uses a global network to keep performance consistent for large audiences
It provides a complete set of features for professional streaming right away

Cons of using Stream

It is a paid API service, which means you will incur costs based on usage, like minutes, bandwidth, and storage
It gives developers less control over encoding, packaging, and delivery compared to managing their own system
Using a specific API can create a dependency on that service provider

Use case

Here’s how to install Stream via npm or yarn:

npm install @stream-io/video-react-sdk

Here’s a basic implementation of Stream:

>import {
  StreamCall,
  StreamVideo,
  StreamVideoClient,
  User,
} from "@stream-io/video-react-sdk";

const apiKey = "your-api-key";
const userId = "user-id";
const token = "authentication-token";
const user: User = { id: userId };

const client = new StreamVideoClient({ apiKey, user, token });
const call = client.call("default", "my-first-call");
call.join({ create: true });

export const LiveStream = () => {
  return (
    <StreamVideo client={client}>
      <StreamCall call={call}>
        /* <MyVideoUI /> */
      </StreamCall>
    </StreamVideo>
  );
};

Daily.co

Daily.co offers an easy-to-use API and SDKs that let you add real-time video and audio calls to both web and mobile apps. It uses WebRTC technology to handle video conferencing smoothly, whether it's one-on-one or with many people.

Daily.co provides APIs for peer-to-peer video rooms with features like screen sharing, recording, and end-to-end encryption. This makes it a great choice for live interactions where quick connections and clear communication are essential. Its key features include:

WebRTC focus: It is designed for real-time, low-latency video and audio calls for one-on-one or group communication
Cross-platform SDKs: Daily.co provides SDKs for JavaScript, React, React Native, iOS, and Android to ensure a consistent user experience across devices
Interactive live streaming: It allows up to 100,000 participants to join a real-time session, all able to send and receive video
Recording and compositing: It offers cloud recording options for sessions and can combine multiple video feeds into one output
Screen sharing: It includes built-in tools for sharing screens during calls
AI-ready features: It integrates with AI platforms to provide features like real-time transcription, background blur, and noise cancellation
Global mesh network: It uses a distributed network to reduce latency and improve video quality based on location

Pros of using Daily.co

It offers very fast response times
It simplifies WebRTC, providing an easy-to-use API for developers
It is designed to deliver clear video quality for multiple participants
It works well for apps that focus on live interactions

Cons of using Daily.co

This is a paid service with costs that can be high, especially for large-scale broadcasts
While it can record videos, it excels in real-time communication rather than for large-scale, cost-effective VOD use
The SDKs are open, but the main system is controlled by Daily.co

Use case

daily-react makes it easier to integrate Daily.co into your React/Next applications using npm:

npm i @daily-co/daily-react

To get started with using Daily.co in a Next.js app, include DailyProvider in your app:

import { DailyProvider, useCallObject } from '@daily-co/daily-react';

function App() {
  // Create an instance of the Daily call object
  const callObject = useCallObject();

  return <DailyProvider callObject={callObject}>{children}</DailyProvider>;
}

Node.js/FFmpeg

The combination of Node.js and FFmpeg represents the most flexible and customizable approach to video streaming. This setup gives you complete control over the video streaming process.

You use Node.js as the backend to manage video tasks, and FFmpeg, which is an open source multimedia tool written in C for processing video and audio files. FFmpeg can work with almost any media format. It can transcode, package, segment (for HLS/DASH), and stream videos, among other things. With this combination, you can create your own custom media server. Its features include:

You have complete control over the entire streaming process, from getting the content to delivering it
FFmpeg works with many video and audio formats
You can convert videos into different formats and bitrates for adaptive streaming
You can create HLS (.m3u8 playlists, .ts or fMP4 segments) or DASH manifests and segments
You can process live camera feeds or other sources for real-time streaming
Node.js lets you control FFmpeg commands programmatically, allowing for flexible stream management

Pros of using Node.js/FFmpeg

It lets you customize every part of your streaming solution to meet your specific needs
After the initial setup, your main expenses will be for infrastructure, such as servers and bandwidth. This can be cheaper than using managed services if you have a large scale
You have full ownership and control of your entire streaming system
You can add the latest streaming enhancements or custom features that aren’t available in ready-made solutions

Cons of using Node.js/FFmpeg

You need extensive knowledge about video encoding, streaming protocols, and managing servers
Creating and keeping a strong streaming solution from the ground up is a difficult task
Your team is responsible for scaling, monitoring, and keeping the infrastructure running
To achieve very low latency, you must carefully optimize and manage your FFmpeg and network settings

Basic implementation:

const { spawn } = require('child_process');
const express = require('express');
const app = express();

app.use(express.static('public')); // Serve HLS files

const ffmpeg = spawn('ffmpeg', [
  '-i', 'input-stream-url', // e.g., RTMP or live feed
  '-c:v', 'libx264',
  '-c:a', 'aac',
  '-f', 'hls',
  '-hls_time', '4',
  '-hls_segment_filename', 'public/stream_%03d.ts',
  'public/stream.m3u8',
]);

ffmpeg.stderr.on('data', (data) => console.log(data.toString()));

app.listen(3000, () => console.log('Server running on port 3000'));

Dolby OptiView

Dolby OptiView is a video streaming solution that provides high-quality and fast streaming across different platforms. It combines the technology of Dolby.io with THEOplayer’s efficient HTML5 video player. Dolby OptiView focuses on sports, media, entertainment, and interactive applications such as live sports betting, auctions, and online gaming.

It aims to enhance real-time engagement, ensure consistent performance across platforms, and create opportunities for revenue through features like server-guided ad insertion (SGAI). Features include:

WebRTC-based streaming with Dolby’s audio and video enhancements
Support for low-latency group calls and broadcasts
APIs for video, audio, and real-time communication
Integration with JavaScript frameworks like Next.js
Basic open-source SDKs for custom streaming implementations
Advanced noise cancellation and video optimization

Pros of using Dolby OptiView

Dolby's enhancements provide superior audio and video quality
Open-source WebRTC components are free and can be customized
The low-latency streaming works well for real-time applications
Developers have access to well-documented APIs and SDKs
It supports both web and mobile platforms

Cons of using Dolby OptiView

Open source components have limits. Some advanced features require a paid Dolby platform
Setting up WebRTC can be complicated for large deployments
It is not optimized for traditional streaming protocols like HLS or DASH
Premium features depend on Dolby's ecosystem

Use case

You can integrate Dolby OptiView in your Next.js/React using npm or yarn:

npm install @dolbyio/comms-sdk-web

Here is a basic live stream implementation:

"use client";
import { useEffect, useRef, useState } from 'react';
import VoxeetSDK from '@dolbyio/comms-sdk-web';

function DolbyPlayer({ appKey, appSecret }) {
  const videoRef = useRef(null);
  const [isJoined, setIsJoined] = useState(false);

  useEffect(() => {
    // Initialize Dolby SDK
    VoxeetSDK.initialize(appKey, appSecret);
  }, [appKey, appSecret]);

  const joinStream = async () => {
    try {
      // Open session
      await VoxeetSDK.session.open({ name: 'User' });

      // Create/join conference
      const conference = await VoxeetSDK.conference.create({ alias: 'stream' });
      await VoxeetSDK.conference.join(conference);

      setIsJoined(true);
    } catch (error) {
      console.error('Join failed:', error);
    }
  };

  const leaveStream = async () => {
    try {
      await VoxeetSDK.conference.leave();
      await VoxeetSDK.session.close();
      setIsJoined(false);
    } catch (error) {
      console.error('Leave failed:', error);
    }
  };

  return (
    <div>
      <video 
        ref={videoRef} 
        autoPlay 
        controls 
        style={{ width: '100%', height: '400px', background: '#000' }}
      />

      <div style={{ marginTop: '10px' }}>
        {!isJoined ? (
          <button onClick={joinStream}>Join Stream</button>
        ) : (
          <button onClick={leaveStream}>Leave Stream</button>
        )}
      </div>
    </div>
  );
}

export default function Home() {
  const appKey = 'your-dolby-app-key';
  const appSecret = 'your-dolby-app-secret';

  return (
    <div style={{ padding: '20px' }}>
      <h1>Dolby Stream</h1>
      <DolbyPlayer appKey={appKey} appSecret={appSecret} />
    </div>
  );
}

Comparing the video streaming tools

Tool	Latency	Cost	Best for	Scalability
HLS.js	3–10s	Free	Custom HLS players	High (client-side)
Video.js	3–10s	Free	Multi-format playback	Moderate
Stream API	<500ms	Paid	Interactive apps	Very High (managed)
Daily.co	<1s	Freemium	Video conferencing	High (managed)
Node.js/FFmpeg	Configurable	Free (infrastructure)	DIY streaming pipelines	Self-hosted
Dolby OptiView	<1s	Enterprise	High-quality broadcasts	High (managed)

Conclusion

To implement real-time video streaming in Next.js, consider several key factors: latency, scalability, development resources, and long-term maintenance.

This article discusses HLS.js as a strong choice for HTTP Live Streaming. We demonstrated how to create production-ready streaming applications that include features like adaptive bitrate streaming, secure authentication, and optimized performance.

We also looked at alternatives, including Video.js, Stream, and Daily.co. When choosing your streaming solution, think about your specific needs, use HLS.js or Video.js for reliable and widely compatible streaming, choose Daily.co for real-time interaction, use Stream’s API for quick development with managed infrastructure, apply Node.js/FFmpeg for full customization, or take advantage of Dolby OptiView for high-quality experiences.

With what you've learned from this article, you are ready to create engaging video streaming experiences that grow with your application and adapt to user needs!

LogRocket: Full visibility into production Next.js apps

Debugging Next applications can be difficult, especially when users experience issues that are difficult to reproduce. If you’re interested in monitoring and tracking state, automatically surfacing JavaScript errors, and tracking slow network requests and component load time, try LogRocket.

LogRocket captures console logs, errors, network requests, and pixel-perfect DOM recordings from user sessions and lets you replay them as users saw it, eliminating guesswork around why bugs happen — compatible with all frameworks.

LogRocket's Galileo AI watches sessions for you, instantly identifying and explaining user struggles with automated monitoring of your entire product experience.

The LogRocket Redux middleware package adds an extra layer of visibility into your user sessions. LogRocket logs all actions and state from your Redux stores.

React & TypeScript: 10 patterns for writing better code

Megan Lee — Wed, 16 Jul 2025 14:00:00 +0000

Written by Peter Aideloje✏️

Building a scalable and maintainable React application is often accompanied by a series of challenges, including a lack of type safety, growing pains as projects expand, unreliable prop validation, and brittle DOM manipulation.

While most of these issues can be handled by plain JavaScript, it lacks the guardrails required for long-term confidence in your codebase. That’s where TypeScript comes in to solve these recurring issues in a consistent and scalable way.

In this article, we’ll explore several proven patterns for writing safer, cleaner, and more readable code in React and TypeScript.

Benefits of using TypeScript in React

TypeScript offers several advantages when used with a React application, including code quality and developer productivity:

Maintainability: It makes code more readable and self-documenting, which helps teams manage and scale projects effectively
Early error detection: It identifies bugs at compile time, allowing developers to resolve issues before they even affect end users
Better tooling: It provides superior IDE support with features like auto-completion, refactoring, and code navigation for a smoother development experience
Type safety: Catches type-related errors during development, reducing runtime bugs and improving code reliability
Confidence in refactoring: It enables safer code changes by ensuring that incorrect type usage is flagged immediately

Typed component props and default props

In TypeScript, interfaces are useful when describing component props, especially when you need to extend or implement them in multiple places. Here’s how you can declare and use props with an interface:

   import React from 'react';

   interface MyEmployeeProps {
     name: string;
     age: number;
     isEmployed?: boolean; // Optional property
   }

   const MyEmployee: React.FC<MyEmployeeProps> = ({
         name, 
         age, 
         isEmployed }) => {
     return (
       <div>
         <p>Name: {name}</p>
         <p>Age: {age}</p>
         {isEmployed !== undefined && <p>Employed: {isEmployed ? 'Yes' : 'No'}</p>}
       </div>
     );
   };

   export default MyEmployee;

You can also use type in place of interfaces if you’re composing types with unions or intersections, but for extensibility, interfaces are often preferred:

   import React from 'react';

   type SubmitButtonProps = {
     text: string;
     onClick: () => void;
     variant?: 'primary' | 'secondary'; // Union type
   };

   const UserButton: React.FC<SubmitButtonProps> = ({
           text,
           onClick,
           variant }) => {
     return (
       <button onClick={onClick} className={variant === 'primary' ? 'primary-button' : 'secondary-button'}>
         {text}
       </button>
     );
   };

   export default UserButton;

In TypeScript with React, component props are treated as required unless you add a ? to mark one as optional. That rule holds true whether you use an interface or a type alias to describe the props.

required props:

interface MyEmployeeProps {
  requiredFullName: string;
  requiredAge: number;
}

const MyEmployee: React.FC<MyEmployeeProps> = ({
     requiredFullName,
     requiredAge}) => {
  return (
    <div>
      {requiredFullName} {requiredAge}
    </div>
  );
};

From the above, it is clear that requiredFullName and requiredAge are both required, and TypeScript will enforce this at compile-time by throwing an error.

optional props:

interface MyEmployeeProps {
  requiredFullName: string;
  optionalAge?: number;
}
const MyEmployee: React.FC<MyEmployeeProps> = ({ 
      requiredFullName,
      optionalAge }) => {
  return (
    <div>
      {requiredFullName} {optionalAge}
    </div>
  );
};

The optionalAge is marked with a ?, which indicates that it’s optional and can be safely omitted when using the component without causing any TypeScript errors.

default props and default parameter values (functional components):

    //this is the class component
class UserComponent extends React.Component<UserProps> {
         render(){
          return <div style={{ color: this.props.color, fontSize: this.props.fontSize
          }}>{this.props.title}</div>;
                }
             }

       UserComponent.defaultProps = {
           color: 'blue'
           fontSize: 20,
       };

    //this is the functional component (FC)
    const UserFunctionalComponent: React.FC<UserProps> = ({ 
            title, 
            color = "blue", 
            fontSize = 20 }) => {
               return <div style={{ color: color, fontSize: fontSize }}>{title}</div>;
            };

By using the defaultProps property for class components, you’re able to set default values for props to ensure components behave in a predictable way even when some props are not provided.

While in the functional components, you only need to assign default values directly in the function parameters for any optional props. This will make your code cleaner and also a guaranteed way to prevent runtime bugs from missing props.

Handling children:

    interface UserComponentProps {
        title: string;
        children: React.ReactNode;
    }
    const UserComponent: React.FC<UserComponentProps> = ({title, children}) => {
         return (
              <div>
                   <h1>{title}</h1>
                       {children}
              </div>
            );
        };

As you can see from the above, the children prop lets you pass contents between a component’s opening and closing tags represented by a large range of data types, like text, other components, or even multiple elements.

This allows you to make components more flexible and reusable by letting them “wrap” or display whatever content you put inside them.

Using discriminated unions for conditional rendering

What are discriminated unions, and when should you use them?

When you build an app with TypeScript and React, you often deal with a single piece of data that can be in various states: loading, error, or success. Discriminated unions, sometimes called tagged unions or algebraic data types (ADTs), provide a tidy way to model these different forms.

By grouping related types under one label, you keep type safety while easing the mental load during coding. This clear separation makes it simple to decide which UI to show in your components because each state carries its own signature.

In the examples that follow, we’ll see how this approach helps us write safer, more readable, and still expressive code:

type DataLoadingState = {
  status: 'request loading...';
};

type DataSuccessState<T> = {
  status: 'request success';
  data: T;
};

type DataErrorState = {
  status: 'request error';
  message: string;
};

type DataState<T> = DataLoadingState | DataSuccessState<T> | DataErrorState;

From the code snippet above, each type has a common trait, usually known as a discriminator or tag, that marks its kind, much like a status label. TypeScript leans on this tag whenever the shapes are piled into a union to tell them apart.

Because each shape carries a distinct fixed value for that trait, the language knows exactly which one it is and can trim the type accordingly. Once the shapes are defined, you bundle them with the | operator, allowing you to model complex state in a way that stays safe and predictable.

Exhaustive checking with the `never` type

Exhaustive checking with the never type in TypeScript is a technique that ensures that all possible cases of a discriminated union are explicitly handled in a switch statement or conditional logic, allowing developers to catch unhandled scenarios at compile time through type safety.

It is worth noting that the never type represents a value that never occurs, i.e., unreachable code, and is used in exhaustive checks to ensure all cases of a discriminated union are properly handled. If a new case is added but not addressed, the compiler throws an error, which enhances type safety:

function DisplayData<T>({ state }: { state: DataState<T> }) {
  switch (state.status) {
    case 'loading':
      return <p>Loading Data</p>;
    case 'success':
      return <p>Data: {JSON.stringify(state.data)}</p>;
    case 'error':
      return <p>Error: {state.message}</p>;
    default:
      return <p>Unknown status</p>;
  }
}

The above illustrates the final step in using discriminated unions effectively in React components by employing conditional logic like a switch or an if statement based on the discriminator property (status).

This will allow you to render different UI elements depending on the current state and catch missing branches at compile time, keeping your components both type-safe and error-resistant.

Inferring types from APIs with `ReturnType` and `typeOf`

Two powerful utilities by TypeScript are typeof and ReturnType<T>, which are used to infer types from existing values and extract the return type of functions, respectively, enabling safer and more maintainable code, especially when working with services, APIs, and utility functions.

Using `typeOf` to infer types from functions or constants

For constants, typeOf is used to infer the type of a variable (string) so it’s reusable throughout without the need for hardcoding it, as shown below:

const API_BASE_URL = "https://api.newpayment.com/services/api/v1/transfer";
type ApiBaseUrlType = typeOf API_BASE_URL;

You can also use typeOf to get the function type, which is useful for typing callbacks:

const getEmployeeDetails = (employeeId: number) => ({
    employeeId,
    employeeName: "Peter Aideloje",
    employeeEmail: "aidelojepeter123@gmail.com",
    position: "Software Engineer",
});

// using typeof to get the function type
type GetEmployeeDetailsFnType = typeof getEmployeeDetails;

Leveraging `ReturnType<T>` for function results

This pattern is very useful when a utility/service function returns structured data. This way, you automatically derive the result type using Returntype, ensuring consistency across your code base.

By combining Returntype and typeof, you keep types synchronized with function signatures, avoiding manual duplication and reducing the risk of type mismatch:

// Get the return type of the getUser function
const employeeDetails : EmployeeDetails = {
    employeeId = 3,
    employeeName: "Peter Aideloje",
    employeeEmail: "aidelojepeter123@gmail.com",
    position: "Software Engineer", 
};

type EmployeeDetails = ReturnType<typeof getEmployeeDetails>;

Extracting types from services and utility functions

This helps to derive the result type automatically from the structured data of a utility or service function, thereby ensuring consistency in consuming components, as shown below:

//utility function
function calculateTotalFee (price: number, quantity: number){
  return {
     total: price * quantity,
     currency: "GBP",
  };
}

// extracting return type of utility function
type TotalSummary = ReturnType<typeof calculateTotalFee>;

const summary: TotalSummary = {
  total: 100,
  currency: "GBP",
};

Utility types: `Pick`, `Omit`, `Partial`, `Record`

TypeScript comes with a set of built-in utility types that make it easier to build new types from the ones you’ve already defined flexibly.

These tools help to shape component props, organize state, reduce redundancy, and improve code maintainability in React projects. Below are some practical use cases of the most commonly used utility types you’ll find handy in a React + TypeScript setup.

Practical use cases for each utility type

(i) `Pick<Type, Keys>`

The Pick utility type is used to construct a new type by selecting specific properties from a large Type thereby enhancing type safety and reducing redundancy:

interface Employee{
          employeeId: number,
          employeeName: String,
          employeeEmail: String,
          employeePosition: String,
}
type EmployeePreview = Pick<Employee, 'employeeId' | 'employeeName'>;
const preview : Employeepreview = {
         employeeId: 35,
         employeeName: "Peter Aideloje",
};

This is great for displaying minimal data in a list or component.

(ii) `Omit<Type, Keys>`

The Omit utility type is the direct opposite of Pick type, used to create a new type by excluding specific properties from an existing type:

interface Employee{
          employeeId: number,
          employeeName: String,
          employeeEmail: String,
          employeePosition: String,
}

type EmployeeWithoutEmail = Omit<Employee, 'employeeEmail'>;
const employee : EmployeeWithoutEmail = {
         employeeId: 35,
         employeeName: "Peter Aideloje",
         employeePosition: "Software Engineer",
};

This is great for excluding unnecessary information or sensitive fields such as passwords, email, or database ID.

(iii) `Partial<Type>`

The Partial utility type makes all properties in a type optional. This is useful when you are updating an object when not all properties must be provided:

interface Employee{
          employeeId: number,
          employeeName: String,
          employeeEmail: String,
          employeePosition: String,
}

type PartialEmployee = Partial<Employee>;
const partialEmployee : PartialEmployee = {
         employeeName: "Peter Aideloje",
};

(iv) `Record<Keys, Type>`

The Record utility type creates an object with a specific set of keys and types:

type Roles = "admin" | "employee" | "viewer";

type Permissions = Record<Role, string[]>;

const permissions: Permissions = {
     admin["read", "write", "delete"],
     employee["read", "write"],
     viewer["read"],
};

Utility types in TypeScript help to reduce code repetition when defining props or state by reusing and reshaping existing types. They’re also great for modelling flexible data structures, such as dynamic form inputs or API responses, making your codebase cleaner and easier to maintain.

Generic components and hooks

Writing reusable components with generics

Generics in TypeScript help developers to make reusable UI elements capable of managing multiple data types while keeping strong type safety. They shine better and are more important when used in React to design components that are not tied to a particular data type.

This flexibility makes your React component more dynamic and can fit into various types as required in any part of your application. To achieve this, follow these steps below to set up your project: First, open your terminal or command prompt to run the command to create a new React project using TypeScript:

npx create-react-app react-project --template typescript

Next, this command navigates you to the project directory:

cd react-project

Folder structure:

Next, we’ll create a generic List component that can showcase a list of items of any type using the following code snippet below:

import React from 'react';

// A Generic component
type Props<T> = {
  items: T[];
  renderItem: (item: T) => React.ReactNode;
};

function GenericComponent<T>({ items, renderItem }: Props<T>): JSX.Element {
   return <div>{items.map(renderItem)}</div>;
}

export default GenericComponent;

The GenericComponent defines a reusable generic list component in a React + TypeScript setup. It accepts two props: an array of items and a renderItem function, which determines how each item should be displayed.

The use of generics allows this component to work with any data type, making it a more flexible and type-safe solution for rendering lists across various use cases.

Typing Refs and DOM elements

Using `useRef` with DOM elements

In React development, it’s necessary to leverage the built-in tools that the library provides, such as useRef. When using useRef with DOM elements like HTMLInputElement, you need to specify the ref as shown below:

import React, {useRef, useEffect} from 'react';
const FocusInput: React.FC = () => {
    const nameInputRef = useRef<HTMLInputElement | null>(null);
    useEffect ( () => {
       nameInputRef.current?.focus();
    }, []);
    return (
       <div>
          <label> htmlFor= "name">Name:</label>
          <input id="name" type="text" ref={nameInputRef} />
       </div>
      );
   };   

  export default FocusInput;

Forwarding refs with `React.forwardRef`

In React, the forwardRef is a handy feature that lets you pass a ref from a parent component down to a child component. This comes in useful when the child component wraps a DOM element but does not directly expose it.

Essentially, React.forwardRef allows parent components to directly access the inner DOM node (child component’s DOM) even when it’s hidden or wrapped in other layers of abstraction. When working with TypeScript, you’ll need to define the type of the ref to keep things safe and predictable. It’s a great way to make your components more flexible and maintainable:

import React, {forwardRef, useRef, useImperativeHandle } from 'react';

type ButtonProps = {
  handleClick?: () => void;
};

const CustomerButton = forwardRef<HTMLButtonElement, ButtonProps> ((props, ref) => { 
   const internalRef = useRef<HTMLButtonElement>(null);

   useImperativeHandle(ref, () => ({
      focus: () => {
        internalRef.current?.focus();
      },
    }));

   return (
     <button ref={internalRef} onClick={props.hanldeClick}>
      Press Here
     </button>
    );
  });

const WrapperComponent = () => {
     const refToButton = useRef<HTMLButtonElement>(null);

     const triggerFocus = () => {
         refToButton.current?.focus();
      };
   return (
     <div>
        <customButton ref={refToButton} handleClick={triggerFocus} />
     </div>
    );
 };

export default WrapperComponent;

Avoiding any DOM manipulations

In React, try your best to avoid directly changing the DOM yourself. Instead, adopt a more reliable and maintainable approach to use React’s built-in state system to manage changes.

For instance, rather than using a ref to manually set the value of an input field, you should let React control it through state. This keeps your component predictable and easier to debug:

import React, {useState, useRef, useEffect } from 'react';

function ControlledInput() {
   const [inputValue, setInputValue] = useState('');
   const inputRef = useRef<HTMLInputElement>(null);

   const handleInputChange = (event: React.ChangeEvent<HTMLInputElement>) => {
      setInputValue(event.target.value);
   };

 useEffect(() => {
 if (inputRef.current){
   //Accessing property safely
  console.log(inputRef.current.value);
  // Do not directly manipulate the DOM, use React state instead
 }
}, [inputValue]);
    return <input
                  type ="text" 
                  ref{inputRef} 
                  value{inputValue} 
                  onChange={handleInputChange} 
          />;
     }

Context with strong typing

Creating and consuming context with generic types

When you build an app with React and TypeScript, the createContext method lets you pass things like theme preferences or logged-in user details to distant components without drilling props through every layer. To keep that process type-safe and easy to manage, start by writing a TypeScript type or interface that clearly lists every piece of data the context will hold.

Doing so lets the compiler flag mistakes early and keeps the shape of your context consistent wherever you import it. After you have your type, pass a sensible default value to React.createContext and supply that value as the argument.

The default value makes sure that any component reading the context outside of a Provider gets a safe fallback rather than crashing the app. Introduced in React 16, the Context API has become the go-to method for sharing state globally in a cleaner, more scalable fashion. Below, we’ll walk through three straightforward steps to create the context, provide it, and then consume it in a component.

Define the context with an interface

interface AppContextType{
  currentValue: string;
  updateValue(updated: string) => void;
}

Create the context

import React from 'react';

const AppContext = React.createContext<AppContextType>({
      currentValue: 'default',
      updateValue: () => {}, //Temporary function placeholder

 })

Consume the context

import React, { useContext } from 'react';
import { AppContext } from './AppContextProvider'; //Assume context is defined in a separate file

function infoDisplay() {
     const { currentValue, updateValue} = useContext(AppContext);

return(
     <section>
            <p>Current Context: {currentValue}</p>
            <button onClick ={()=> updateValue('updateContext')}>Change Value</button>
     </section>
  );
}

Using `createContext` with default values and undefined checks

When working with a createContext in a React + TypeScript setup, care must be taken to define default values and account for cases where the context might be undefined. This will help you ensure that your application remains safe, predictable, and less prone to runtime errors.

Default value in `createContext`

When you call createContext in React, you can pass a default value as an argument. That value is what useContext gives back when a component reading the context isn’t inside the correct Provider, or when the Provider itself sets the value to undefined:

interface IThemeContext {
  theme: 'light'|'dark';
  switchTheme: () => void;
}

const ThemeContext = React.createContext<IThemeContext | null>(null);

Handling undefined with `useContext`

When you pull data with the useContext Hook in React but forget to wrap the component in its matching Provider, or if that Provider accidentally sends undefined, the hook just hands you undefined.

To keep TypeScript happy and give your app a safety net against sneaky runtime errors, always add a quick check after you read the context. That way, your component can react calmly when the context is missing instead of crashing loudly:

import { createContext, useContext } from 'react';

interface ContextShape {
   data: string;
}

const customContext = createContext<ContextShape | undefined>(undefined);

export function useCustomContext() {
   const ctx = useContext(CustomContext);
   if (!ctx) {
      throw new Error("useCustomContext must be used within a customProvider");
    }
    return ctx;
  }

export function CustomProvider ({ children }: { children: React.ReactNode }) {
  const contextValue: contextShape = { data: "Shared context data"};
  return (
    <CustomContext.Provider value={contextValue}>
      {children}
    </CustomContext.Provider>
   );
}

Conclusion

We have seen the crucial role that TypeScript plays in modern React development by helping teams to build more scalable, robust, and maintainable applications with increased code readability. Developers can use features like typeof, ReturnType to infer types from APIs, thereby reducing manual duplications and keeping types in sync with actual implementation.

Additionally, when you enable typed component props and default props in your code base, you can catch misuse early and improve code readability, as shown in this article. TypeScript also shines in handling lower-level concerns like type refs and DOM elements, bringing more clarity and safety when working with React context, which enables strong typing across consumer components.

If you’re new to these patterns, don’t feel pressured to adopt everything at once. Adopting TypeScript in React doesn’t have to be overwhelming; you can start small by incorporating it where it brings immediate value, and build it up from there. Over time, these practices will become second nature with a long-term payoff in maintainability, code quality, and making it worth the investment. Happy coding!

Get set up with LogRocket's modern error tracking in minutes:

Visit https://logrocket.com/signup/ to get an app ID.
Install LogRocket via NPM or script tag. LogRocket.init() must be called client-side, not server-side.

NPM:

$ npm i --save logrocket 

// Code:

import LogRocket from 'logrocket'; 
LogRocket.init('app/id');

Script Tag:

Add to your HTML:

<script src="https://cdn.lr-ingest.com/LogRocket.min.js"></script>
<script>window.LogRocket && window.LogRocket.init('app/id');</script>

3.(Optional) Install plugins for deeper integrations with your stack:

Redux middleware
ngrx middleware
Vuex plugin

Get started now.

How API client automation can save you hours in development

Megan Lee — Thu, 10 Jul 2025 17:00:00 +0000

Written by Lewis Cianci✏️

When you opened up this article to read it, your phone or computer sent several HTTP requests to the place where it was hosted. Also, various APIs were sent pieces of data related to user analytics and the like. Pretty much every app today will invoke some sort of API, which is a way for apps and websites to send data around. For APIs, you are responsible for writing what happens on the server, how the database is queried, etc.

Over time, apps increase in complexity, so our API requests become more complex — and so do our responses. Our client-side app needs updates to how these API are invoked. Thus, you are responsible for “balancing the equation” by making all the same updates to the client app.

This can take a lot of time in itself. Worse still, you’re susceptible to making a typo or something in this code, and can be stuck wondering why nothing is working. Wouldn’t it be nice if we could remove ourselves from making a typo in this code, and even from spending hours writing this API invocation? Boom, that’s what OpenAPI-generated clients are for.

The problem with invoking server APIs

In the earlier days of the web, it wasn’t hard to write your own code to invoke an HTTP client, retrieve some data (or post some), and deal with the result. And, for new projects, it can be tempting to roll your own API service that essentially does just this. However, apps are much more complicated these days.

As we add more functionality to our app, our API surface swells as well. Whenever we adjust some code in our web app or add a new HTTP action, we also have to add the same functionality within our API client. This endless “balancing of the equation” takes a lot of time. Things continue to increase in complexity as we introduce Bearer tokens for authentication.

These tokens aren’t set-and-forget, though; they have a short lifespan to ensure application security. When they expire, they get sent to our identity service with a refresh token, and a new Bearer token is returned. It’s not realistic to expect our user to have a failed request that will work again in a moment, so we have to retry the failed operation on the fly.

All of this complexity adds time, and there’s also a chance we could make a mistake. Worse, it’s possible that we could introduce a hard-to-troubleshoot problem in how we invoke our server API, caused by something as simple as a typo in our API client that we wrote.

Solutions that (kinda) solve the problem

The other problem is that, once you have an API that exposes certain actions, you may want to invoke it, so you can test it and see what the response will be. If you’re particularly enterprising, you can do this yourself using terminal commands like cURL, or put together some kind of script to do it for you.

But still, this is more code that is lovingly crafted by yourself, and needs to be maintained. In more recent years, other solutions like Postman have come on the scene to try to give you a one-stop shop for API usage and testing. But, again, having to maintain how Postman interacts with your API is quite a lot of work in itself.

The solution — OpenAPI generated clients

When we think about documentation, we imagine sitting down to write something that describes how our application works. We associate a level of effort with doing that. But when writing web apps, we mostly use HTTP to communicate back to our API’s.

In doing so, we know the HTTP method (such as GET/PATCH/POST etc). We also know the arguments that are being sent to these HTTP actions. These are defined at compile-time. It’s not impossible for something to inspect these actions and produce some documentation on what the actions are called, and what sort of arguments they produce. With a bit more finesse, we can describe what kind of return codes we expect from the API.

Finally, it’s possible to specify which actions require authentication and are protected, and we can even describe polymorphic data in the documentation. There is a bit of a setup cost, and sometimes it takes some nudging for the documentation to work correctly. But the payoff is immense; you can generate ready-made API clients for your app in dozens of different languages and frameworks.

All you have to do is decorate your API with some OpenAPI bits and pieces. It doesn’t matter if someone wants a C#, PowerShell or B ash script client; they can just generate what they want.

Building a sample app — Cat API

To demonstrate, let’s create a simple C# Web API app that has two actions, an HTTP GET and HTTP POST:

app.MapGet("/catbreeds", () =>
{
    var catBreeds = new[]
    {
        "Persian", "Siamese", "Maine Coon", "British Shorthair",
        "Bengal", "Ragdoll", "Sphynx", "Russian Blue",
        "Scottish Fold", "American Shorthair"
    };

    return catBreeds[Random.Shared.Next(catBreeds.Length)];
});

app.MapPost("/reverser", (string payload) =>
{
    var reversed = new string(payload.Reverse().ToArray());
    return reversed;
});

They’re two very simple actions, but they tell us something important:

The HTTP verb (GET/POST)
What parameters the action takes (such as string payload in the second action)
The routing address for the particular action

Then, early in our Program.cs and pipeline setup, we configure OpenAPI to create the document:

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenApi();

var app = builder.Build();

if (app.Environment.IsDevelopment())
{
    // Enable middleware to serve generated Swagger as a JSON endpoint.
    app.MapOpenApi();
}

Finally, we configure Swagger UI so we can interact with our API’s from a handy UI:

app.UseSwaggerUI(x =>
{
    x.SwaggerEndpoint("/openapi/v1.json", "v1");
});

And, with just that small amount of configuration, this is what happens when we go to our swagger/index.html address within our browser: Our actions are given an appropriate name, and we can see that they have the GET and POST verbs appropriately. Expanding an action describes what it does: What I find most useful is the Try it out button. Executing an API action from within the browser directly saves a lot of time and lets you test out what your API does before writing a client-side app that uses the API.

In this case, once we hit Execute, we can see that our app invokes the API and produces a result: It’s the same thing with the reverser action, accepting a query string parameter of payload for our string:

Adding authentication

Most apps use some form of authentication, and OpenAPI/SwaggerUI let us invoke secured APIs in an accurate way. Let’s go ahead and add a (very basic) authentication safeguard for our app:

// Simple auth middleware
app.Use(async (context, next) =>
{
    var path = context.Request.Path.Value?.ToLower();
    // Skip auth for Swagger/OpenAPI endpoints
    if (path != null && (path.StartsWith("/swagger") || path.StartsWith("/openapi")))
    {
        await next();
        return;
    }

    var authHeader = context.Request.Headers.Authorization.ToString();
    if (string.IsNullOrEmpty(authHeader) || authHeader != "Bearer fake_bearer_auth")
    {
        context.Response.StatusCode = 401;
        await context.Response.WriteAsync("Unauthorized");
        return;
    }

    await next();
});

We always want our API documentation to be available, but we want to lock the rest of our API behind a Bearer token. Your implementation will differ, but in our case, it’s enough to just use the simple fake_bearer_auth token. Our AddOpenApi call now becomes:

builder.Services.AddOpenApi(options =>
{
    options.AddDocumentTransformer((document, context, cancellationToken) =>
    {
        document.Components ??= new OpenApiComponents();
        document.Components.SecuritySchemes["Bearer"] = new OpenApiSecurityScheme
        {
            Type = SecuritySchemeType.Http,
            Scheme = "bearer",
            Description = "Enter 'fake_bearer_auth'"
        };

        document.SecurityRequirements.Add(new OpenApiSecurityRequirement
        {
            {
                new OpenApiSecurityScheme
                {
                    Reference = new OpenApiReference { Type = ReferenceType.SecurityScheme, Id = "Bearer" }
                },
                Array.Empty<string>()
            }
        });

        return Task.CompletedTask;
    });
});

Swagger UI reads this configuration, and now our API actions have a lock icon next to them. An Authorize button becomes visible: Our description is available, namely, telling us to enter fake_bearer_auth: And now, we can see that fake_bearer_auth is included in the Authorization header, and our secured API responds appropriately! 🎉:

Generating client code — The pièce de résistance

All of this up until now has been fine. But the real power of this approach comes with its ability to produce clients for pretty much any language or framework we can think of. Seriously, there’s a lot: Admittedly, the OpenAPI generator GitHub does come across a bit cluttered, and it’s hard to work out just how to use it.

Assuming you have the server and client in one Git repository, run npm init in the shared directory to get a package.json
Run npm i @openapitools/openapi-generator-cli --save in this directory
Create a makeapi option in the scripts entity

This is going to look weird. It’s quite lengthy, but there’s a method to my madness. Here’s what mine looks like:

"makeapi": "export JAVA_OPTS=\"-Dio.swagger.parser.util.RemoteUrl.trustAll=true -Dio.swagger.v3.parser.util.RemoteUrl.trustAll=true\" && export NODE_TLS_REJECT_UNAUTHORIZED=0 && openapi-generator-cli generate -i https://localhost:7067/swagger/v1/swagger.json -g typescript-angular -o ./apiclient/client -p npmName=apiclient -p apiModulePrefix=ApiClient --enable-post-process-file --skip-validate-spec"

Obviously, this is a huge command to run and does some scary things (like tanking SSL verification). Naturally, that’s a bad idea. So why do we do it? Typically, in development, we use a lot of self-signed certificates. Because it's a Java app, the OpenAPI generator struggles with this; Java maintains its own repository of trusted certificates.

Frequently, API generation would choke because of this private certificate problem. SSL c ertificates exist to help ensure we’re talking to who we think we are talking to.

They’re a cornerstone of modern-day internet security. Do we introduce security risks by not checking them when talking to localhost? Personally, I don’t think so. You may feel differently. If you do, feel free to configure your self-signed certificates as you see fit. The other commands are:

-g typescript-angular — Create a Typescript API client for use in Angular
-o ./apiclient/client — Where to put the generated client
-p npmName=apiclient — What name the generated package should have
-p apiModulePrefix=ApiClient — What the generated API clients' invocations should be prefixed with. Handy to know for sure that a certain class or interface is definitely generated and not coming from your own app
--enable-post-process-file — Format the generated files to make them more readable
--skip-validate-spec — The OpenAPI specification can be quite strict. Normally, API client generation will fail if your API isn’t conformant. This lets the generator make some concessions to ensure client generation while not technically adhering to the OpenAPI specification

Now, we can just run npm i and then npm run makeapi. We should receive our generated API client! Within our Angular project, our component code looks like this:

cat = signal('unknown...');
reversedString = signal('');
// Obtain a reference to our genered ApiClientsService
apiService = inject(ApiClientsService);
textReverser: string = "";

async getCat(){
  // And just call our API actions as we need to
  let catBreed = await firstValueFrom(this.apiService.apiCatbreedsGet());
  // Deal with the result appropriately
  this.cat.set(catBreed);
}

async reverseText(){
  let reversed = await firstValueFrom(this.apiService.apiReverserPost(this.textReverser));
  this.reversedString.set(reversed);
}

As you can see, we obtain a reference to the service and then call our API actions. The result type and signature are all inferred from our OpenAPI documentation: And that’s it, easy peasy lemon squeezy.

Don’t wait — Try out OpenAPI

Writing your own API clients, apart from being a gigantic waste of time, also means that you are responsible for how specific functionality works. For example, even injecting Bearer authentication can be a bit of a hazard if you’re not entirely sure what you’re doing. When you generate your clients, you essentially generate a contract that states how your app will communicate with the API.

Because it’s generated, all of that is done for you. The other details, like what tokens and authentication to use, can be implemented by yourself. In Angular’s case, that means implementing an HTTP Interceptor that intercepts requests to the /api/ path and injects our fake bearer auth token. Our API invocation code is neatly in its own place; how our tokens get into the request is neatly in another area.

It all makes for a nice separation of concerns. Starting out with a generated API client means that you don’t have to throw out your hand-written one to move to OpenAPI/Swagger in the future. It also means that you’ll save a lot of time by not writing your own API client.

Making that shift later will require a bit of grind, so doing it early in your project will help you in the long run. Don’t forget to bring in the sample project from here. Tell us how you automate API generation in your projects and the benefits you’ve found in the comments below.

Get set up with LogRocket's modern error tracking in minutes:

Visit https://logrocket.com/signup/ to get an app ID.
Install LogRocket via NPM or script tag. LogRocket.init() must be called client-side, not server-side.

NPM:

$ npm i --save logrocket 

// Code:

import LogRocket from 'logrocket'; 
LogRocket.init('app/id');

Script Tag:

Add to your HTML:

<script src="https://cdn.lr-ingest.com/LogRocket.min.js"></script>
<script>window.LogRocket && window.LogRocket.init('app/id');</script>

3.(Optional) Install plugins for deeper integrations with your stack:

Redux middleware
ngrx middleware
Vuex plugin

Get started now.

AI dev tool power rankings & comparison [July 2025 edition]

Megan Lee — Thu, 10 Jul 2025 14:00:00 +0000

Which AI frontend dev tool reigns supreme? This post is here to answer that question. We’ve put together a comparison engine to help you compare AI tools side-by-side, produced an updated power rankings to show off the highest performing tools of the month, and conducted a thorough analysis across 36 features to help spotlight the best tools for every purpose. We’ll discuss the following (click on the links for LogRocket deep dives into select tools):

Claude 4 Sonnet
Claude 4 Opus
GPT-4.1
GitHub Copilot
Cursor
Windsurf
Vercel v0
Bolt.new
Gemini 2.5 Pro
JetBrains AI
Lovable AI
DeepSeek Coder

Let’s dive in!

Comparison tool: Compare up to four AI tools at once

Having a hard time picking one tool over another? Or maybe you have a few favorites, but your budget won’t allow you to pay for all of them.

We’ve built this comparison engine to help you make informed decisions.

How it works

Simply select between two and four AI tools you're considering, and the comparison engine instantly highlights their differences. This targeted analysis helps you identify which tools best match your specific requirements and budget, ensuring you invest in the right combination for your workflow.

The comparison engine analyzes 12 leading AI tools across 36 specific features, helping developers choose based on their exact requirements rather than subjective assessments. Most comparisons rate the AI capabilities in percentage and stars, but this one informs you on specific features each AI has over another. Note: No single tool dominates all categories. Making feature-specific selection is a good choice for workflows.

Power rankings: The top 5 AI tools for July

Our July 2025 power rankings highlight AI tools that either recently hit the scene or released a major update in the past two months. Here’s how they stack up in our eyes:

1. Claude 4 Sonnet ⬆️: The undisputed technical champion

Previous ranking — N/A (new release May 2025)
Performance summary — Claude 4 Sonnet achieves an unprecedented 80.2% on SWE-bench, verified with high-compute parallel reasoning. It’s establishing new performance standards for real-world software engineering tasks.

2. Claude 4 Opus ⬆️: The reasoning genius

Previous ranking — N/A (new release May 2025)
Performance summary — Claude 4 Opus delivers 72.5% SWE-bench Verified performance while excelling at complex architectural challenges requiring deep reasoning. Designed specifically for long-running development tasks, Opus handles multi-day projects with sustained context and strategic decision-making capabilities that smaller models cannot match.

3. Gemini 2.5 Pro ⬆️: The context king

Previous ranking — 1
Performance summary — Gemini 2.5 Pro dominates WebDev Arena with a 1443 Elo rating and leads LMArena leaderboard while offering an industry-leading 1 million token context window (expanding to 2 million). The model excels at 84.8% VideoMME benchmark performance, bringing multimodal capabilities to development workflows.

4. Cursor IDE ⬆️: The breakout candidate

Previous ranking — 7
Performance summary: Cursor transformed from startup to $2.6 billion valuation in 12 months with the fastest SaaS revenue growth ever ($1M to $100M ARR). The AI-native IDE delivers multi-file code generation with real-time codebase understanding, enabling developers to refactor complex systems with unprecedented speed and accuracy.

5. GPT-4.1 ↔️ : The OG

Previous ranking: 5
Performance summary: GPT-4.1 delivers consistent 85%+ accuracy across coding benchmarks while maintaining the strongest general-purpose capabilities in the market. The model excels at cross-domain problem solving, seamlessly transitioning between code generation, documentation, debugging, and architectural planning.

Comparison tables: How these 12 AI tools stack up

If you're more of a visual learner, we've also put together tables that compare these tools across different criteria. Rather than overwhelming you with all 36 features at once, we've grouped them into focused categories that matter most to frontend developers.

Development capabilities and framework support

This table compares core coding features and framework compatibility across AI development tools.

Key takeaway — Claude 4 Sonnet, Gemini 2.5 Pro, and Opus lead in comprehensive framework support. They’re the only tools that excel across React, Vue, Angular, and TypeScript while maintaining strong design-to-code conversion capabilities:

Feature	Claude 4 Sonnet	Claude 4 Opus	GPT-4.1	GitHub Copilot	Cursor IDE	Windsurf	Vercel v0	Bolt.new	Gemini 2.5 Pro	JetBrains AI	Lovable AI	Deepseek Coder
Real-time code completion	✅	✅	✅	✅	✅	✅	❌	✅	✅	✅	✅	✅
Multi-file editing	✅	✅	✅	✅	✅	✅	❌	✅	✅	✅	✅	Limited
Design-to-code conversion	✅	✅	✅	✅	✅	✅	✅	✅	✅	❌	✅	❌
React component generation	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Vue.js support	✅	✅	✅	✅	✅	✅	❌	✅	✅	✅	✅	✅
Angular support	✅	✅	✅	Limited	✅	✅	❌	✅	✅	✅	✅	✅
TypeScript support	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Tailwind CSS integration	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅

Quality and optimization features

This table compares code quality, accessibility, and performance optimization capabilities across tools.

Key takeaway — Only Claude 4 Sonnet, Opus, and GPT-4.1 offer complete bundle size analysis—a critical feature for production applications that most other tools overlook entirely:

Feature	Claude 4 Sonnet	Claude 4 Opus	GPT-4.1	GitHub Copilot	Cursor IDE	Windsurf	Vercel v0	Bolt.new	Gemini 2.5 Pro	JetBrains AI	Lovable AI	Deepseek Coder
Responsive design generation	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Accessibility (WCAG) compliance	✅	✅	✅	✅	✅	❌	✅	❌	✅	❌	Limited	Limited
Performance optimization suggestions	✅	✅	✅	✅	✅	✅	❌	❌	✅	✅	Limited	Limited
Bundle size analysis	✅	✅	✅	❌	❌	❌	❌	❌	❌	❌	❌	❌
SEO optimization	✅	✅	✅	✅	✅	❌	❌	❌	✅	❌	✅	✅
Error debugging assistance	✅	✅	✅	✅	✅	✅	❌	✅	✅	✅	✅	✅
Code refactoring	✅	✅	✅	✅	✅	✅	❌	✅	✅	✅	✅	✅
Browser compatibility checks	✅	✅	✅	❌	❌	❌	❌	❌	❌	❌	Limited	❌

Development workflow integration

This table compares version control, collaboration, and development environment integration features.

Key takeaway — Cursor IDE and Windsurf stand out by combining Git integration with live preview capabilities, making them the strongest choice for developers prioritizing real-time feedback during development:

Feature	Claude 4 Sonnet	Claude 4 Opus	GPT-4.1	GitHub Copilot	Cursor IDE	Windsurf	Vercel v0	Bolt.new	Gemini 2.5 Pro	JetBrains AI	Lovable AI	Deepseek Coder
Git integration	✅	✅	✅	✅	✅	✅	❌	✅	❌	✅	✅	✅
Live preview/hot reload	✅	✅	✅	❌	❌	✅	✅	✅	❌	❌	✅	Limited
Collaborative editing	✅	✅	✅	✅	❌	✅	❌	❌	❌	❌	✅	❌
Custom component libraries	✅	✅	✅	✅	✅	❌	✅	❌	❌	❌	✅	Limited
API integration assistance	✅	✅	✅	✅	✅	✅	❌	✅	✅	✅	✅	✅
Testing code generation	✅	✅	✅	✅	✅	✅	❌	❌	✅	✅	❌	✅
Documentation generation	✅	✅	✅	✅	✅	✅	❌	❌	✅	✅	✅	✅
Semantic/deep search	✅	✅	✅	✅	✅	✅	❌	❌	✅	✅	❌	Limited

Modern web development features

This table compares support for contemporary web standards like PWAs, mobile-first design, and multimedia input.

Key takeaway — Gemini 2.5 Pro is the sole tool offering voice/audio input capabilities, while most tools still lack essential modern features like PWA support and offline capabilities:

Feature	Claude 4 Sonnet	Claude 4 Opus	GPT-4.1	GitHub Copilot	Cursor IDE	Windsurf	Vercel v0	Bolt.new	Gemini 2.5 Pro	JetBrains AI	Lovable AI	Deepseek Coder
Mobile-first design	✅	V	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Dark mode support	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	Limited
Internationalization (i18n)	✅	✅	✅	✅	✅	❌	❌	❌	✅	❌	Limited	✅
PWA features	✅	✅	✅	✅	✅	❌	❌	❌	✅	❌	✅	Limited
Offline capabilities	✅	✅	✅	❌	❌	❌	❌	❌	❌	❌	✅	✅
Voice/audio input	❌	❌	❌	❌	❌	❌	❌	❌	✅	❌	❌	❌
Image/design upload	✅	✅	✅	✅	✅	❌	✅	✅	✅	❌	✅	✅
Semantic/deep search	✅	✅	✅	✅	✅	✅	❌	❌	✅	✅	❌	Limited

Business and deployment considerations

This table compares pricing models, enterprise features, privacy options, and deployment flexibility.

Key takeaway: DeepSeek Coder offers the best value proposition as the only fully open-source solution with self-hosting options, making it ideal for teams with more strict data privacy requirements or budget constraints:

Feature	Claude 4 Sonnet	Claude 4 Opus	GPT-4.1	GitHub Copilot	Cursor IDE	Windsurf	Vercel v0	Bolt.new	Gemini 2.5 Pro	JetBrains AI	Lovable AI	Deepseek Coder
Free tier available	✅	❌	❌	✅	✅	✅	✅	✅	✅	✅	✅	✅
Open source	❌	❌	❌	❌	❌	❌	❌	Partial	❌	❌	❌	✅
Self-hosting option	❌	❌	❌	❌	Privacy mode	❌	❌	✅	❌	❌	Limited	✅
Enterprise features	✅	✅	✅	✅	✅	✅	✅	❌	✅	✅	✅	Limited
Privacy mode	✅	✅	✅	✅	✅	✅	❌	❌	❌	❌	✅	✅
Custom model training	❌	❌	✅	✅	❌	❌	❌	❌	❌	❌	❌	✅
Image/design upload	✅	✅	✅	✅	✅	❌	✅	✅	✅	❌	✅	✅
Semantic/deep search	✅	✅	✅	✅	✅	✅	❌	❌	✅	✅	❌	Limited

Conclusion

I hope you enjoyed this post and tried our comparison app. At this point, you should have a broader scope of which AI tool best serves your needs. See you next month for any new updates. Keep coding!

Get set up with LogRocket's modern error tracking in minutes:

Visit https://logrocket.com/signup/ to get an app ID.
Install LogRocket via NPM or script tag. LogRocket.init() must be called client-side, not server-side.

NPM:

$ npm i --save logrocket 

// Code:

import LogRocket from 'logrocket'; 
LogRocket.init('app/id');

Script Tag:

Add to your HTML:

<script src="https://cdn.lr-ingest.com/LogRocket.min.js"></script>
<script>window.LogRocket && window.LogRocket.init('app/id');</script>

3.(Optional) Install plugins for deeper integrations with your stack:

Redux middleware
ngrx middleware
Vuex plugin

Get started now.

Getting started with Claude 4 API: A developer’s walkthrough

Megan Lee — Wed, 09 Jul 2025 14:00:00 +0000

Written by Andrew Baisden✏️

Claude 4 is Anthropic's latest generation of advanced AI language models. It was designed to provide developers with a more powerful and reliable way to utilize AI in a safe environment. Existing models have been upgraded, and developers now have access to Claude Opus 4 and Claude Sonnet 4, which replace the previous third-generation models.

Opus 4 is the better model for complex tasks because it utilizes maximum intelligence for its reasoning capabilities. Claude Opus 4 excels at completing tasks that need deep understanding and analysis combined with problem-solving. This means that, when using Claude 4, you will have higher performance and better overall results for your projects. On the other hand, Claude Sonnet 4 is better suited for everyday development use cases and provides a balance between performance and efficiency. It works well across the board and is good for chatbots, content creation, coding, and normal AI-related tasks.

Both models are built on Anthropic's Constitutional AI framework, which means they have been trained to be useful, friendly, and truthful. As a result, they are well-suited for production applications where having a reliable and safe platform is important.

These models also have many improvements over the previous generations because of their improved reasoning capabilities, which results in better accuracy, a larger context window, longer conversations, and more cost-effective benefits.

This article will teach you everything you need to know to start building apps with Anthropic's new Claude 4 API. We will explore account setup, environment configuration, core API concepts, and advanced features. At the end of this guide, you should have a basic understanding of how to work with the Claude 4 API and how to integrate it into your applications.

Why choose the Claude 4 API for development?

Claude 4 has so many advantages that solidify it as a favorite, even though developers have many other AI models to choose from. The API has reliable performance, high availability, and uptime, and is good at following prompts. The documentation is also very clear and organized. Claude 4's versatility means that it is suitable for a variety of applications, which can span across all types of environments and use cases.

Code assistance:
Claude 4 is a standout performer when writing, debugging, and optimizing code in different programming languages. The models excel when used with complex algorithms, code improvement suggestions, and even when asked to generate complete functions, which have been based on natural language descriptions from a user’s input.

Content generation:
Claude's creative capabilities make it a good choice for writing blogs, creating marketing copy, doing technical documentation, as well as writing general fiction content. The model has been designed to be good at interpreting and understanding brand, context, tone, and voice, which leads to its high-quality content output.

AI agents and assistants:
Claude is capable of handling context over a long conversation while also following up on complex instructions. These models can handle steps over multiple tasks and are great at integrating with external APIs, which creates a consistent personality.

Customer support:
These models can be integrated with customer support to give intelligent, context-based responses to customer questions. The models are good at understanding questions and providing accurate responses while remaining professional. Data analysis: Claude's analytical capabilities can be used to process datasets, reports, and various insights.

The environment setup and account configuration

Before you start coding with Claude, you first need to create an Anthropic account if you don’t already have one. Then you will be able to access the Anthropic dashboard with your developer account.

The dashboard is the main hub for writing prompts, managing your API keys, accessing the documentation, and monitoring your data usage. A minimum amount of $5 credits is needed before you can start using the API: API keys are the main method for implementing authentication with Claude 4.

Creating an API key is simple once you navigate to the Get API Key section. From there, you can generate a key and give it a descriptive name, which will help you manage all the keys across your environments and projects. Good security measures should always be taken into account when you are working with an API. When working with Claude, it's a good idea to always:

Use HTTPS: When you make API requests, this ensures that your data and API are sent securely
Generate different API keys: Creating a different API key for development, staging, and production environments is a good approach because it leads to security isolation, so you can rotate keys when needed
Store keys in environment variables: This is one of the best methods for security and key management. It is best practice to never hardcode API keys or commit them to different systems using version control; they should be private and not publicly accessible
Use error handling: With good error handling, you can make sure that sensitive information is not accidentally shown
Rate limiting: Applications can often generate many requests, which results in them hitting API limits faster than expected. With rate limits implemented, these risks can be prevented
Rotate API keys: Generate new keys and deactivate old ones to prevent misuse and leaks if there has been a data breach

Setting up API versioning headers

Claude uses versioning headers, which makes it easy to differentiate between the different versions of the API. This ensures that there is consistent behavior as new features become available. You can specify the version in your requests when using the anthorpic-version header.

With this setup, it's easier to guarantee that your application works as expected, regardless of which API version has been released. It's best practice to test new versions in a development environment before you go to production. You can learn more about this process in the Migrating to Claude 4 documentation.

Pricing and usage

Claude 4 uses a token-based pricing model, which means you pay for the number of tokens that are processed in your requests and responses. These tokens are represented as pieces of text and can be comparable to words or parts of words. Understanding how tokenization works makes it easier to estimate your costs and optimize your usage.

Below, you can see a comparison of Anthropic's leading AI models: The input tokens are charged every time you send text to the API, and output tokens are charged for all text that Claude generates in the responses. Different rates are typically charged for input and output tokens; output tokens are slightly more expensive because of the computation costs required for generation.

Counting tokens can be difficult because it doesn’t always translate to word count. You also have to consider punctuation, spaces, and special characters, which also use tokens. Each model has a different price structure because they have different capabilities and computational requirements. When thinking about cost optimization, it’s important to take into account what your goals are and which model will best suit your interests:

Claude Sonnet 4 is better for routine tasks and more affordable
Claude Opus 4 excels when working on complex problems that need advanced reasoning
Optimize prompts to be concise while also having all the context needed
Use caching strategies for repeated requests, as this results in fewer API calls
Utilize streaming responses when working on long-form content to improve the user experience

The development environment

Language and framework selection

Claude 4 supports integration with most programming languages or frameworks when using the standard HTTP request. Anthropic also provides an official SDK for Python and JavaScript/TypeScript, which provides a streamlined developer experience. Python continues to be the most popular choice for AI applications because of its large ecosystem. The Python SDK integrates well with popular frameworks such as Django, Flask, and FastAPI.

JavaScript and TypeScript are good choices for web applications, Node backends, and full-stack development. The JavaScript SDK can be used with frameworks and libraries like React, Vue, Angular, Express, and many others. As for other programming languages, the REST API can be used directly with HTTP clients. This includes languages like Go, Java, C#, Ruby, and more. These languages all have great HTTP libraries that can work seamlessly with Claude 4.

Official SDK installation

Installing the official SDKs is straightforward when using normal package managers. For Python development, it's common to use the following command:

pip install anthropic

With this command, we can install the official Anthropic Python library, which gives us a clean interface to work with the Claude 4 API. The library is capable of handling authentication, error handling, and request formatting. For Node.js and TypeScript development, we would run this command:

npm install @anthropic-ai/sdk

The JavaScript SDK provides TypeScript definitions that work out of the box and seamlessly with JavaScript and TypeScript projects. You can read more in the official Claude documentation.

Alternative tools

The official SDK is the best way to interact with the Claude 4 API because it provides the best development experience. However, there are a few other ways to work with it, as outlined here:

cURL is good when it comes to testing API endpoints and debugging issues. It's helpful when you find yourself in a situation that requires you to inspect raw HTTP requests and responses, and also when working in environments where SDKs can't be used
Postman has a user-friendly interface and is a great tool for doing API testing when checking endpoints. The tool can be used for managing environments and API exploration alongside documentation
REST clients are another solution, especially those that have been built into an IDE or can be used as an extension in VS Code, for example. They offer similar capabilities to external tools like Postman, but in this case, you can run the testing and prototyping inside your development environment

IDE setup and configuration

When using Python for development, I recommend using PyCharm, VS Code, Jupyter notebooks, or a similar Python IDE. Install the Anthropic SDK and configure your API key to be used as an environment variable. It's common practice to create a .env file for local development.

JavaScript/TypeScript development can be achieved in any general-purpose IDE or code editor like VS Code, Cursor, or Windsurf; they all have good support for IntelliSense and debugging features. Setting up your project for TypeScript can be beneficial because it means that you can take advantage of the SDK's type definitions. Regardless of the setup you choose, ensure that you have installed the relevant plugins and extensions so that you are ready for the development process and maintaining code quality.

Core API fundamentals

Authentication methods

It's common practice to use an authentication method when interacting with an API for security purposes. The Claude 4 API is no different, with authentication recommended for implementation. To do so, you can include your API key in the x-api-key header for every request you make.

Fortunately, the official SDKs handle this automatically as soon as you have configured them with your API key. When using the Python SDK, your code might look like this:

import anthropic

client = anthropic.Anthropic(
    api_key="your-api-key-here",
)

This code is self-explanatory and requires you to insert your API key. For the JavaScript SDK, the code is quite similar to the Python version:

import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic({
  apiKey: 'your-api-key-here',
});

As mentioned earlier, you should never hardcode your API keys into the source code. It's best practice to use an environment variable instead of some type of secure configuration management system. The majority of deployment platforms have a secure way to manage sensitive configurations.

Messages API deep dive

The Messages API is the primary interface for interacting with the Claude 4 API. It uses a conversation format that works by having a user send a series of messages, which then receive a response from Claude. With this type of design, it becomes natural to build a conversational application as well as maintain a context across multiple exchanges.

The messages inside the conversation have a role of either "user" or "assistant" alongside the content. The "user" role defines input from your application, while "assistant" is defined as Claude's responses. The Claude API can support different types of content, including plain text and structured data. As with most use cases, plain text is usually enough. But the API's flexibility lets you perform more complex interactions as needed.

Your first API calls

Let's see an overview of what it looks like when running your first Claude 4 API call. In this example, we can see the basic structure and the required parameters. Using the Python SDK:

import anthropic

client = anthropic.Anthropic(api_key="your-api-key")

message = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1000,
    temperature=0,
    messages=[
        {
            "role": "user",
            "content": "Hello, Claude! Can you help me understand how to use your API?"
        }
    ]
)

print(message.content)

Like before, your API key is required for authentication, and you need to define the model you are using as well as the tokens, temperature, and the message. In this example, the message is printed in the logs. Using the JavaScript SDK:

import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic({
  apiKey: 'your-api-key',
});

const msg = await anthropic.messages.create({
  model: 'claude-sonnet-4-20250514',
  max_tokens: 1000,
  temperature: 0,
  messages: [
    {
      role: 'user',
      content: 'Hello, Claude! Can you help me understand how to use your API?',
    },
  ],
});

console.log(msg.content);

The code is much the same in this JavaScript example, the only difference being the syntax. Response handling is an important aspect when building any application. The Claude 4 API can return structured responses that include generated content, the metadata, plus the request.

In the official documentation, you can find the structured error types, which make it easy to handle all of the different error conditions.

New Claude 4 API 2025 capabilities

The code execution tool

An exciting new addition to Claude 4 is its code execution capability, which lets Claude write and run code as it solves problems, analyzes data, and provides more accurate data results. With this feature, Claude can now perform calculations, generate various visualizations, and work with different types of data, all in real time.

This new tool also supports Python, and it can handle different types of libraries for data analysis, visualization, and scientific computing. So any applications that require these types of capabilities can benefit considerably, especially in the case of chart and graph generation.

Files API

The Files API is another feature improvement from Claude 4. It allows users to upload and work with different types of media, including documents, images, and more. With this feature enabled, applications can process uploaded content, analyze it, and extract information for further analysis.

Some of the formats supported by the Files API include PDFs, images, text files, as well as structured data formats. When files have been successfully uploaded, they can be referenced in conversations. This allows Claude to analyze, summarize, and answer questions that are related to them.

Working with the Model Context Protocol (MCP) Connector

The Model Context Protocol (MCP) connector gives Claude 4 a standard way of extending its core functionality by giving it the power to connect to external data sources and tools.

This protocol leads to a seamless way of integrating with databases, APIs, and other similar services while also remaining secure and reliable. MCP gives developers the ability to create custom connectors that give Claude access to certain data sources or tools relevant to their apps.

For example, connectors could be created for a company's customer database, knowledge base, or APIs. The protocol can handle authentication, data formatting, and error handling automatically on its own, and because of this, it becomes easier to extend Claude's capabilities without having to do the manual labor of complex integrations, which can be time-consuming. Standardization also means that connectors can be shared and reused across different types of environments.

Extended prompt caching (Up to one hour)

Caching in Claude 4 works for up to one hour, which greatly improves performance while also reducing costs for applications that use the same types of prompts repeatedly or work with large amounts of context data that doesn’t change often. Prompt caching is very useful for apps that need to process large documents, as well as maintain a consistent system prompt, or when working with template-based interactivity.

Caching the processed prompt means that repeat requests that use the same cached content can be done faster, which results in more cost savings for the user. The cache system is smart and is able to automatically handle cache invalidation and its management. Developers are able to take advantage of this new feature because prompts can now be structured to maximize cache hit rates, which is useful for apps that have a predictable usage pattern.

Advanced features and optimization

Streaming responses

With streaming responses, your application can display Claude's output while it's being generated instead of waiting for the complete response to come through. This feature improves the user experience, especially when working with long content generation, by making applications feel more interactive and responsive. Streaming is fairly easy to implement with the official SDKs because the streaming interface can provide real-time access to tokens when they are generated, which allows you to update your user interface when needed.

When building chatbot interfaces, streaming is very valuable because it creates a more natural conversation, which users have become more used to when using a modern application. Content generation tools and other apps where users have to wait for long outputs can also benefit from streaming.

Vision and multi-modal capabilities

The vision capabilities in Claude allow it to analyze and understand images, which opens up new possibilities for applications with visual content. The new model can describe images and answer questions about visual elements while also extracting text from images and analyzing graphs, charts, and diagrams. Different types of image formats can be processed, including multiple images in a single request.

With this type of capability, applications that require document processing, visual analysis, and accessibility can get good usage. Claude 4 tends to perform best when images are clear, high resolution, and relevant for the task, so it's worth taking this into account when doing data analysis.

System prompts and behavior customization

System prompts let you customize Claude's behavior, personality, and responses. This can be tailored to your specific application. When you give clear instructions on how you want Claude to work, you can create a consistent, branded experience that best aligns with your app's requirements.

Good system prompts should be clear, specific, and provide strong examples of the type of behavior that is needed. Prompts should define the context, role, and constraints needed for Claude's responses. System prompts are important for customer-facing applications where you have to be consistent and maintain brand alignment. Testing is always a good idea as you try to find and work on prompts that offer the value you are looking for.

Batch processing and asynchronous operations

Applications that require large volumes of requests to be processed or handle time-sensitive tasks can also benefit from Claude 4. Batch processing and asynchronous operations allow users to submit a variety of requests efficiently while also handling the responses when they are ready, in contrast to blocking individual requests. Batch processing is especially useful in cases where applications have to analyze large datasets, process multiple documents at the same time, or generate content for many users.

Because it uses an asynchronous method, these operations allow apps to stay responsive as Claude works in the background, making requests. When using batch processing, it's good to consider factors like rate limiting, error handling for requests, and progress tracking for operations that have been running for a long time. Scalability and user experience are enhanced when batch processing has been integrated effectively.

Performance benchmarks and model comparison

AI model comparisons

Many companies, including Google, DeepSeek, and OpenAI, offer leading AI models, each with distinct strengths and capabilities. In this section, we will review how these models compare to Anthropic's Claude 4 models. The performance benchmark and model comparison data were checked in June 2025. As the models improve, the data is likely to change, so bear that in mind. For a more in-depth and up-to-date comparison, I recommend you check out these popular AI LLM leaderboard platforms:

Claude 4 series (Released May 2025):

Claude Opus 4: Anthropic's flagship model scored 72.5% on SWE-bench Verified and 43.2% on Terminal-bench. The model was able to handle hours of multi-step challenges with excellent consistency
Claude Sonnet 4: The model has been significantly improved and excels at coding with a state-of-the-art score of 72.7% on the SWE-bench test

Google Gemini 2.5 Pro (Released March 2025):

The thinking model scored 63.82% on SWE-Bench and was verified with a custom agent setup
It also comes top in the LMArena leaderboard with an Arena Elo score of 1468

DeepSeek R1:

The open source reasoning model has a good overall competitive performance across coding and math tasks
It was able to demonstrate strong performance in competitive programming style tasks (Codeforces) and with mathematical logic (MATH-500)

OpenAI models:

GPT-4.1: The model has enhanced instructions and improved coding capabilities
GPT-4o: This model has an established multimodal model that serves as a performance baseline

This performance data is based on data from LiveBench June 2025 data:

Model	Organization	Global Average	Reasoning	Coding	Agentic coding	Mathematics	Data analysis	Language	IF
[Claude 4 Opus Thinking](https://www.anthropic.com/news/claude-4)	Anthropic	72.93	90.47	73.25	33.33	88.25	70.73	73.72	80.74
[Claude 4 Opus](https://www.anthropic.com/news/claude-4)	Anthropic	65.93	56.44	73.58	31.67	78.79	66.51	76.11	78.38
[Claude 4 Sonnet Thinking](https://www.anthropic.com/news/claude-4)	Anthropic	72.08	95.25	73.58	30.00	85.25	69.84	70.19	80.43
[Claude 4 Sonnet](https://www.anthropic.com/news/claude-4)	Anthropic	63.37	54.86	78.25	25.00	76.39	64.68	67.18	77.25
[Gemini 2.5 Pro Preview (2025-05-06)](https://developers.googleblog.com/en/gemini-2-5-pro-io-improved-coding-performance/)	Google	72.09	88.25	73.58	30.00	88.63	68.85	71.81	83.50
[DeepSeek R1 (2025-05-28)](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528)	DeepSeek	70.10	91.08	71.40	26.67	85.26	71.54	64.82	79.95
[GPT-4.1](https://openai.com/index/gpt-4-1/)	OpenAI	55.90	44.39	73.19	13.33	62.39	66.40	54.55	77.05
[ChatGPT-4o](https://x.com/OpenAIDevs/status/1905335104211185999)	OpenAI	54.74	48.81	77.48	13.33	55.72	66.52	49.43	71.92

This diagram shows how these models compare when it comes to Global Average Score metrics: This diagram shows how these models compare when it comes to Model Performance:

And in this diagram, we can see the model performance data in a heatmap:

As we can see, this data shows that the Claude and Gemini models excel in most categories, although OpenAI models are not too far behind. DeepSeek is good all around, across the board, too.

Performance benchmarks summary

When we take a look at a summary for the benchmark tests, we can see the differences between the models and how the size of the context window can be highly beneficial in many use cases:

Model	SWE-bench Verified	Context Window	Primary strengths
Claude Opus 4	72.5% (79.4% high-compute)	200K	Best coding, sustained performance
Claude Sonnet 4	72.7% (80.2% parallel compute)	200K	Balanced performance/efficiency
Gemini 2.5 Pro	63.2%	1M+	Advanced reasoning, multimodal
DeepSeek R1	36.8%	128K	Open source, reasoning
GPT-4.1	54.6%	1M	Large context, instruction following
GPT-4o	33.2%	128K	Multimodal, speed

_With thinking capabilities enabled _Sources for benchmark data:*

Real-world application performance analysis

Now let's look at the top three LLMs across a few popular categories: AI agent performance, content creation, code assistance, and data analysis.

AI agents performance

1. Claude Opus 4: In tests, this model sets new records with a seven-hour autonomous coding run, where it was able to complete complex software engineering tasks with very little human intervention. The model was good at working on its own and had good memory management across extended workflows 2. Gemini 2.5 Pro: Google's best model has strong agentic capabilities with built-in reasoning and tool use. The fact that the model has a 1 M+ token context window results in a very thorough understanding of complex multi-step tasks 3. Claude Sonnet 4: This model is very reliable when used in production agent deployments as it has an excellent cost-to-effectiveness ratio. It can score 35.5% on Terminal-bench (41.3% with parallel test-time compute)

Content generation

1. Gemini 2.5 Pro: Google's Gemini 2.5 Pro leads the LMArena leaderboard for human preference when it comes to content quality, and it demonstrates top-of-the-line style and creativity in text generation 2. Claude Sonnet 4: Claude Sonnet 4 offers high-quality control with content generation and has reduced hallucination in most tests 3. GPT-4.1: OpenAI's GPT 4.1 model improves upon GPT-4o with better consistency in content generation tasks

Code assistance

1. Claude Opus 4:
Anthropic's Claude Opus 4 is an industry leader when used for coding, with a performance of 72.5% on SWE-bench Verified, and it is capable of outperforming GPT-4o and Gemini 2.5 Pro in multi-step code generation and debugging tests

2. Claude Sonnet 4:
This renowned model has state-of-the-art performance with a 72.7% score on the SWE-bench test, delivering better coding performance than models that are bigger when using parallel test-time compute 3. DeepSeek R1:
DeepSeek R1 is great at outperforming competitors in algorithmic challenges with scores of (87% vs. 82% on LeetCode Hard problems), but it can struggle with some framework-specific patterns

Data analysis

1. Gemini 2.5 Pro:
It's hard to beat Gemini 2.5 Pro in data analysis tests, as this model has superior multimodal data processing capabilities because of the 1 M+ token context for analysing large datasets and detailed document reading

2. Claude Opus 4:
With its extended thinking capabilities, this model can utilise and access memory files for persistent analysis contexts across long-running data processing tasks

3. GPT-4.1:
The 1M token context window means that the processing of large documents and datasets can be done with a better understanding of a long prompt In this table, we can see some of their best use cases:

Model	Context Window	Best use cases
Claude Opus 4	200K	Complex coding, AI agents
Claude Sonnet 4	200K	Production coding, balanced use
Gemini 2.5 Pro	1M+	Content, data analysis, research
DeepSeek R1	128K	Open-source projects
GPT-4.1	1M	Enterprise, long documents
GPT-4o	128K	Multimodal, general purpose

Overall performance leaders

Claude Opus 4: Anthropic's most powerful premium model is a powerhouse for complex tasks
Gemini 2.5 Pro: Versatile and with strong reasoning capabilities, excelling in data analysis
Claude Sonnet 4: An overall balanced performer, which is cost-effective
GPT-4.1: A solid enterprise choice with a large context
DeepSeek R1: A fantastic open source choice for budget-conscious projects

Recommendations

For enterprise development:

Claude Opus 4 is a good choice for complex coding projects that require ongoing performance and basic human supervision
Claude Sonnet 4 is great for production environments and when you need the best performance-to-cost ratio

For content and research:

Gemini 2.5 Pro does a good job when used for content creation and lots of data analysis because of its superior context window
The model's built-in reasoning does well when used in complex research applications

For budget-conscious projects:

DeepSeek R1 has got to be one of the best open source options, as it has strong reasoning capabilities
GPT-4o is good for established workflows that require multimodal capabilities

The benchmarking data confirms that the Claude 4 models still dominate in many coding tasks, and that Gemini 2.5 Pro excels in reasoning and content creation. Context window size and specific capabilities can contrast greatly across models, so making the choice depends on your own tailored use case and requirements.

Conclusion

After reading through this walkthrough guide, it becomes clear that the new Claude 4 API is a significant advancement over the previous generations and offers many new capabilities for developers. Having access to the powerful language, code execution, and vision capabilities means that the platform is very versatile when building AI applications.

The API has an extensive, well-thought-out SDK with detailed documentation. With a transparent token-based pricing model, developers can balance cost and features by finding the right model for their projects. New capabilities like the Files API, MCP connectors, and the extended caching open up doors to new opportunities that were not possible in the past.

Claude's enhanced reasoning capabilities and the improvements made in Claude Sonnet 4 and Claude Opus 4 mean that building AI-powered applications has reached a new level. The official Anthropic documentation provides detailed information about the new API capabilities, best practices, and troubleshooting tips, which are worth exploring as you improve your AI skills.

Get set up with LogRocket's modern error tracking in minutes:

Visit https://logrocket.com/signup/ to get an app ID.
Install LogRocket via NPM or script tag. LogRocket.init() must be called client-side, not server-side.

NPM:

$ npm i --save logrocket 

// Code:

import LogRocket from 'logrocket'; 
LogRocket.init('app/id');

Script Tag:

Add to your HTML:

<script src="https://cdn.lr-ingest.com/LogRocket.min.js"></script>
<script>window.LogRocket && window.LogRocket.init('app/id');</script>

3.(Optional) Install plugins for deeper integrations with your stack:

Redux middleware
ngrx middleware
Vuex plugin

Get started now.

How to properly handle rejected promises in TypeScript

Megan Lee — Mon, 07 Jul 2025 14:00:00 +0000

Written by Lewis Cianci✏️ A long time ago, computing was almost entirely synchronous. You’d call a function, and it would return almost immediately. Then, as time went on, we started depending on resources that would take a little while to give a response.

In the early days, we’d use callbacks for other functions to get called after our asynchronous function had completed, but this caused “callback hell”, where it would be hard to track what code was executing at a given time, and difficult to follow the logical progression of an app's execution. Fortunately, easier asynchronous operations were introduced to JavaScript and TypeScript in the form of promises.

It may seem weird to use a word that conveys a sense of human emotion, ie: “I promise to buy you flowers”, but it certainly captures the temporal aspect. Making a promise is constructing a contract with the recipient that you will do something, but that thing is always in the future.

Promising someone that you’ll stand next to them and talk to them when you are standing next to them talking is uncanny at best, and slightly crazed at worst. But sometimes, we need to break our promises. The flower shop shut early, or a remote API call didn’t work out as we hoped it would. Ah, a pity. But not the end of the world. Let’s see why it’s not.

Learning to cope with rejection

Typically, when we fire off our asynchronous request, there are only a few possible outcomes.

It’s currently running (pending)
It’s completed successfully, or
It’s failed

When a promise resolves successfully, the expected program flow can continue. But when things go wrong, our promise gets rejected. This is obviously a problem because, if we don’t handle our error (or exception), the exception can “bubble up” the stack. If nothing is available to handle the rejection, then execution of our JavaScript will bail out. Let’s see how we can handle promise rejection in a way that works for us.

Promise rejection in TypeScript (Angular)

We’ll use Angular to demonstrate how promise rejection works in TypeScript for client-side libraries. We only use Angular as a way to visualize what’s happening, so you should be able to apply these concepts to any TypeScript client-side app that you are working on. To demonstrate, let's create an array that has ten promises in it, and make every other promise fail:

export class NumbersComponent implements OnInit {

  promises = Array<Promise<any>>();

  ngOnInit(): void {
    this.promises = Array.from({length: 10}, (_, i) => i % 2 === 0 ? this.randomAsyncNumber() : this.willFail());
  }

  willFail(): Promise<void> {
    return new Promise((resolve, reject) => {
      setTimeout(() => {
        reject('error');
      }, 1000);
    });
  }

  randomAsyncNumber(): Promise<number> {
    return new Promise((resolve, reject) => {
      setTimeout(() => {
        resolve(Math.random() * 100);
      }, 1000)
    })
  }

And this is the HTML:

<div style="width: 100%; height: 100%; display: flex; flex-direction: column; gap:5px">
  @for (number of promises; track number) {
    <div style="min-height: 20px; border: 1px solid black;">
      <p>
        {{ number | async }}
      </p>
      <p>
        <i>Object Value: {{ number | json }}</i>
      </p>
    </div>
  }
</div>

We use the json directive so we can see what is happening inside the Promise as it executes. Every other Promise fails with a value of “error”, and no value given: But what if we wanted to go back and re-run those failed promises? Sometimes we might have to, like if we have a list of items and some of them fail. Promises execute and have a result; we can’t repeat them. So instead, we need to splice out the old failed promise, insert a new promise, and re-execute it. To achieve this, let's make a new function that does just that:

fixPromise(index: number){
  const promise = this.randomAsyncNumber();
  this.promises.splice(index, 1, promise);
}

And then, within our for loop, add the button to retry the promise on failure:

@if ((number | json).indexOf("error") > -1){
  <button (click)="fixPromise($index)">Retry Promise</button>
}

The result is that the failed promises are taken out of the array, and a new functional promise is added instead: Promises can specify types of the return. But you can’t specify a return type for the reject path. Instead, you can wrap your resolution in another object that indicates whether a request has succeeded or not.

Specifying a `Result` interface

To achieve this, lets create a new interface that describes this Result type object. It’s very simple: something to hold the result, and something to tell us whether the Promise has succeeded or not:

export interface Result{
  result: any;
  success: boolean;
}

Now, let’s make both of our functions resolve to this new type:

willFail(): Promise<Result> {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve({result: 'fail', success: false});
    }, 1000);
  });
}

randomAsyncNumber(): Promise<Result> {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve({
        result: Math.floor(Math.random() * 100),
        success: true
      });
    }, 1000)
  })
}

Both Promises resolve now, but they resolve to a typed object: The benefit of this approach is that we can be sure that the Promise will always resolve to our expected type, as opposed to calling reject, which can only ever be any.

Uncaught errors in an observable chain

We can’t talk about Promises and asynchronous functionality in TypeScript without also mentioning Observables. Promises that reject can cause our execution to stop, and errors within an Observable pipe can cause our logic to exit earlier than intended. For example, this Observable will yield once with a valid result:

randomAsyncNumber(): Observable<Result> {
  return of(null).pipe(
    delay(1000),
    map(() => ({
      result: Math.floor(Math.random() * 100),
      success: true
    }))
  );
}

But what would this do?:

willThrowError(): Observable<Result> {
  return of(null).pipe(
    delay(1000),
    tap(() => {
      throw new Error('This observable has thrown an error!');
    }),
    map(() => ({ result: 'This should not be visible', success: false })),
    catchError(error => {
      // This will not prevent the observable from entering error state
      // It just transforms the error into a Result object
      return of({ result: error.message, success: false });
    })
  );
}

Normally, we’d expect each operator within our pipe to be evaluated. But within the tap, an error is thrown. This causes the catchError operator to run before the Observable completes. Catching errors within our Observable pipe is always a good idea because if our Observable errors, then it will continue to process events, as opposed to just ditching out.

Conclusion

Things don’t always work out, and when they don’t, that’s okay. Catching exceptions within our Promises and Observables with some good type information can help us degrade gracefully and retry certain operations when it makes sense.

How do you handle asynchronous errors in your applications? Let us know in the comments below. And don’t forget to check out our code samples from this article here.

LogRocket understands everything users do in your web and mobile apps.

LogRocket lets you replay user sessions, eliminating guesswork by showing exactly what users experienced. It captures console logs, errors, network requests, and pixel-perfect DOM recordings — compatible with all frameworks, and with plugins to log additional context from Redux, Vuex, and @ngrx/store.

With Galileo AI, you can instantly identify and explain user struggles with automated monitoring of your entire product experience.

Modernize how you understand your web and mobile apps — start monitoring for free.

How to optimize your Next.js app with after()

Megan Lee — Fri, 04 Jul 2025 14:00:00 +0000

Written by Temitope Oyedele✏️

When building apps with Next.js, optimization stops being just about code-splitting or lazy loading and starts becoming about making sure your server does only what's necessary during a request, and defers everything else until after the response has been sent to the client.

That’s where Next.js 15’s after() comes in. after() is a new API that gives you a native hook in the post-response lifecycle. It lets you run logic after your route has finished rendering, without blocking the client.

No more awkward hacks inside Server Actions, wedging side effects into middleware, or worrying about slowing down your TTFB just to log a DB write or send analytics. In this article, we’ll take a look at how to use after() to make your Next.js app more efficient, cleaner, and easier to scale. We’ll cover where it works and where it doesn’t, how it interacts with Server Components, Actions, and route handlers, and what happens when things go wrong.

What is `after()`?

after() is a new hook in the Next.js routing lifecycle that lets you schedule code to run after the response has been sent to the client. It doesn’t block the main request-response cycle and is ideal for handling non-critical side effects like analytics, logging, background tasks, or cache invalidation.

You can think of after() as similar to Go’s defer in HTTP handlers. It is a tool that runs cleanup or background logic after the main task finishes. But unlike defer, after() is asynchronous, non-blocking, and doesn’t guarantee execution order.

In traditional server logic, if you wanted to perform post-processing, like updating a stats counter or firing a webhook, you’d either do it before the response was sent or use a custom setup like a background job or queue. This method has proven to slow things down, especially if you're planning on moving fast. With after(), you now have a clean, first-class way to handle these post-response tasks inside your route handlers.

Where can you use `after()`?

after() can be used in four different areas:

Server Components
Server Actions
Route handlers
Middleware

But it doesn't behave the same way in all of them!

Server Components

In Server Components, after()'s job is to run after the server has rendered and streamed the HTML to the client. If you need to perform logging, analytics, or track whether a request was made without delaying the actual rendering, this is where after() will shine.

Keep in mind that you can’t use request APIs like cookies() or headers() inside the after() block. This is because Next.js needs to know which part of the component tree accesses these request APIs to support partial prerendering, but after() runs after React's rendering lifecycle.

Let’s say you want to know how often users actually reach the homepage, not just hit the route. So, you decide to track the impressions of the <Hero /> component, a key part of your landing page, and only log it after it's fully rendered and streamed:

// app/page.tsx
import { after } from 'next/server';
import { logHeroImpression } from '@/lib/analytics';

export default function HomePage() {
  const showHero = true; // maybe controlled by AB test or flag

  if (showHero) {
    after(() => {
      logHeroImpression({ path: '/', experiment: 'hero-v2' });
    });
  }

  return (
    <main>
      {showHero && <Hero />}
      <OtherContent />
    </main>
  );
}

//lib/analytics.ts
import fs from 'fs';
import path from 'path';

type HeroImpressionData = {
  path: string;
  experiment: string;
  timestamp?: string;
};

export function logHeroImpression({
  path: routePath,
  experiment,
  timestamp = new Date().toISOString(),
}: HeroImpressionData) {
  const logEntry = `[${timestamp}] Hero impression on route: "${routePath}", experiment: "${experiment}"\n`;

  const logFilePath = path.join(process.cwd(), 'logs', 'hero-impressions.log');

  try {
    fs.mkdirSync(path.dirname(logFilePath), { recursive: true });
    fs.appendFileSync(logFilePath, logEntry);
    console.log('Hero impression logged');
  } catch (err) {
    console.error('Failed to log hero impression:', err);
  }
}

Here, the after() block only runs if the <Hero /> component is actually rendered, which means it follows your render logic. If the component doesn't show up, nothing gets logged because the logging occurs after the HTML is streamed to the client, ensuring that the user experience is not compromised.

With this approach, the need for client-side tracking scripts or useEffect hacks is eliminated. You're handling impression tracking on the server, where it's cleaner and more reliable than in-browser JavaScript, which can break or get blocked.

One thing to watch out for is that if your page is statically rendered, for example, using generateStaticParams, after() will run at build time or during revalidation, not per user. If you need real-time logs tied to actual requests, make sure your route is using dynamic rendering by setting export const dynamic = 'force-dynamic'.

Server Actions

When working with Server Actions, after() runs after the action finishes executing and the response is sent back to the client.

This makes it ideal for background tasks that don’t slow down form submissions or UI updates, and tasks like sending emails, logging metrics, or syncing with a third-party API. Unlike Server Components, Server Actions have access to the request context at the time of execution.

You can use request APIs like cookies() and headers() inside the after() block, but you need to use them with proper async/await syntax. Say you’re building a registration form. After a user signs up, you want to send a welcome email, but you don’t want that email process to delay the form’s response. With after(), you can decouple the email logic and handle it after the response is out:

'use server';

import { after } from 'next/server';
import { sendWelcomeEmail } from '@/lib/email';
import { db } from '@/lib/db';

export async function registerUser(formData: FormData) {
  const email = formData.get('email') as string;

  const user = await db.user.create({
    data: { email },
  });

  after(async () => {
    await sendWelcomeEmail(email);
  });

  return { success: true, userId: user.id };
}

Once the response is sent back to the client, the sendWelcomeEmail() function will run in the background, and the user will see a fast UI response without having to wait for the email logic. You can think of after() as registering a unit of work that Next.js will run on the server after the Server Action finishes, but still within the same server process. It separates the background task from the core logic, so it won’t delay the user’s response.

Route handlers

This is where after() hits its full stride. In route handlers like app/api/xyz/route.ts, where you’re working directly with the Request and Response objects and need to handle things like logging, webhooks, or analytics, you can use after() to run that logic. In these use cases, after() gives you a clean way to offload any non-critical background tasks without blocking the response cycle.

This means no more cramming side effects into the main request logic or worrying about response delays. Let’s take a look at how to use the after() function to log a checkout event with metadata.

Imagine a typical checkout API route. When a user completes a purchase, you want to log the event for analytics, which includes additional details such as their IP address and user agent. But this kind of logging isn’t something the client needs to wait for. By using after(), you can handle all of that logging after the response has already been sent:

// app/api/checkout/route.ts
import { after } from 'next/server';
import { headers } from 'next/headers';
import { logCheckoutEvent } from '@/lib/logging';

export async function POST(request: Request) {
  const body = await request.json();
  const order = await createOrder(body);

  after(async () => {
    const userAgent = (await headers()).get('user-agent') || 'unknown';
    const ip = (await headers()).get('x-forwarded-for') || 'unknown';

    await logCheckoutEvent({
      orderId: order.id,
      userAgent,
      ip,
      timestamp: new Date().toISOString(),
    });
  });

  return Response.json({ success: true, orderId: order.id });
}

This way, the user gets a fast response, while your server quietly logs the event in the background, avoiding any impact on the user experience. This helps ensure that you’re not sacrificing performance just to collect logs or fire off analytics.

If you tried to handle that logging inline, you’d end up holding the response hostage for something the user doesn’t even see. And that cost only grows if the work involves slow third-party services, large payloads, or multiple async operations like sending emails or pinging webhooks. Offloading that work to after() would help you maintain a fast user experience while still providing a full audit trail, cleanly and reliably.

Middleware

In middleware, after() runs after the response is sent to the client. It’s designed for lightweight, non-blocking side effects that relate to routing or request inspection, such as logging, tagging, or background monitoring.

Unlike with route handlers, you don’t have access to the full response body here, and you can’t modify the response inside the after() block. It’s purely for side effects, not for shaping what gets returned. Say you want to log every request that hits your app. Instead of embedding this logic directly in every route, you can use middleware to log once, globally, without affecting the request flow:

// middleware.ts
import { NextResponse } from 'next/server';
import { after } from 'next/server';
import { logRequest } from '@/lib/traffic';

export function middleware(request: Request) {
  const response = NextResponse.next();

  const url = request.url;
  const ip = request.headers.get('x-forwarded-for') || 'unknown';
  const userAgent = request.headers.get('user-agent') || 'unknown';

  after(async () => {
    await logRequest({ url, ip, userAgent, timestamp: new Date().toISOString() });
  });

  return response;
}

What this does is that as each request comes in, the middleware inspects it, lets it continue with NextResponse.next(), and then uses after() to log metadata, such as the URL, IP address, and user agent, in the background.

The logging happens separately and doesn't affect the request or the page response in any way. Because middleware runs on every request, it’s an ideal place to use after() for capturing global side effects without duplicating logic across routes. Below is a table showing how their usages differ:

Context	When it runs	Access to request info	Use cases	Notes
Server Components	After server renders and streams HTML to client	No (cannot use cookies/headers in after() due to PPR requirements)	Render-based logging, analytics, impressions	Runs only if component is rendered; be cautious with static rendering
Server Actions	After the Server Action completes and the response is sent	Yes (can use headers/cookies with async/await)	Background tasks like sending emails, syncing APIs	after() runs in same process, non-blocking for UI
Route Handlers	After request handling is done and response is sent	Yes (can read headers and metadata with async/await)	Event logging, analytics, third-party calls	Ideal for offloading heavy background work without blocking user
Middleware	After the response is sent to the client	Yes (limited, headers only; can't modify response in after)	Global request logging, monitoring, tagging	Only for side effects; can't shape or modify the actual response

How `after()` functions behave when nested

after()can be nested inside other after() calls. This has real implications when you’re working with layouts, pages, or components stacked on top of each other. If you're wondering what happens when multiple components each register their own after() call, the answer is that they all run, and they run in the reverse order of their rendering.

In other words, the most deeply nested component runs first, then its parent, then the next one up. It's a last-in, first-out model. This is the opposite of how middleware works, where logic flows top-down. With after(), it's more like a cleanup stack where each layer pushes its task, and those tasks get executed after the response, from the inside out. Say you have got a product page wrapped in a product section layout, then wrapped again in a root layout:

The page logs a product view
The product layout logs the current A/B test variant
The root layout logs a general page view

You want all of these logs to run, but you want the detailed logs to come first. That’s exactly what after() gives you:

// app/layout.tsx
import { after } from 'next/server';

export default function RootLayout({ children }) {
  after(() => console.log('[AFTER] root layout'));
  return <html><body>{children}</body></html>;
}```





```typescript
// app/product/layout.tsx
import { after } from 'next/server';

export default function ProductLayout({ children }) {
  after(() => console.log('[AFTER] product layout'));
  return <section>{children}</section>;
}

// app/product/page.tsx
import { after } from 'next/server';

export default function ProductPage() {
  after(() => console.log('[AFTER] product page'));
  return <h1>Product</h1>;
}

If a user hits this page, you’ll see this in your logs:

[AFTER] product page  
[AFTER] product layout  
[AFTER] root layout

So, yes, after() calls a cascade, but not top-down, like middleware. It runs bottom-up, starting from the deepest rendered component. Please note that it is not the same as multiple after()s in one file.

Don’t confuse this with calling after() multiple times inside the same component. Those are also stacked and run in reverse, but they’re scoped locally, meaning they don’t care about layout boundaries.

How `after()` behaves during errors

A big question developers will have before trusting after() for anything critical, like logging, analytics, or background tasks, is: what happens when things go wrong? Will it still run if:

The page crashes?
A 500 error is sent?
The route is thrown before rendering starts?

According to its documentation, after() will be executed even if the response didn't complete successfully, including when an error is thrown. This is a major trust point.

If you're using after() for observability, such as logging failed checkouts, tracking what crashed, or saving request metadata, you need it to always run, not just on clean responses. Here’s a simple route that throws an error during execution but still defines an after() callback:

// app/api/error-test/route.ts
import { NextResponse } from 'next/server';
import { after } from 'next/server';

export async function GET() {
  const userId = 'abc123';

  after(() => {
    console.log('[AFTER] Logging userId:', userId);
  });

  throw new Error('Simulated failure');
}

When you hit /api/error-test, you’ll get a 500 response, but your server logs will still print:

[AFTER] Logging userId: abc123

So, how is this important? In real-world apps, things break: routes crash, APIs fail, and components blow up. When that happens, you want to know what led up to it, who the user was, what they were doing, or which input triggered the failure.

With after(), you can log all that even when your route throws. Without it, you'd have to wrap every route in a try...catch, handle logging manually, and remember to rethrow the error, which is brittle and easy to mess up. You can centralize this kind of failure logging in one place. It gives you a reliable safety net that runs whether the request finishes cleanly or explodes halfway through.

One thing to keep in mind, though, is that just because after() runs during a failure doesn’t mean it magically has access to everything. If your route throws before you read the request headers or cookies, that data is gone, and after() won’t be able to access it. So, if you need request-specific info like IPs or user agents in your after() logic, extract it early before anything crashes:

const ip = request.headers.get('x-forwarded-for');
after(() => logErrorWithIP(ip));

This way, even if the rest of the handler fails, your log still has the info you need.

Important things to note about the `after()` function

Before using after, there are a few important things to keep in mind: 1. after() is not a dynamic API This is because it’s not a hook or reactive runtime. after() doesn’t re-run when props or state change, and it doesn't track dependencies like React H ooks do.

It runs once during the server-side render, after the response is sent. That’s it. If you need to conditionally run logic, you have to manually gate it, like wrapping the call in an if statement based on render conditions. 2. You can use React.cache() to deduplicate work If you’re calling expensive functions (like DB reads or API calls) inside after(), wrap those calls in React.cache(). This gives you per-request memoization, so multiple after() calls won’t duplicate work:

import { cache } from 'react';

const getUser = cache(async (id) => {
  return db.user.findUnique({ where: { id } });
});

This is especially useful if after() is declared in multiple nested components, and each tries to fetch the same data. 3. It’s not reactive, it’s static after() is a static export, meaning it runs once per request during the server render cycle. It doesn’t respond to state changes, props updates, or anything dynamic.

If you want conditional logic in after(), you need to wrap it in an if or place it inside a conditional render branch. You can’t use useState, useEffect, or expect it to respond to user interactions — this is server-side code that runs after the response is sent, and it’s tied to the render lifecycle, not the browser. 4. It only works on app routes This function is only supported in the app directory, not the old pages directory.

That means if you’re still using pages/api, getServerSideProps, or other legacy Next.js APIs, you’re out of luck. Also, after() only runs on the server. There’s no client equivalent. It won't run inside client components, and trying to use it there will throw an error.

Alternatives to `after()`

There are a few alternative tools built for post-response logic. Which one to use depends entirely on where you're writing your code. The main alternatives are waitUntil() and plain old await. If you need to run something after the response is sent, without slowing the user down, then after() is your best bet. It’s ideal for:

Logging
Analytics
Sending emails
Syncing with third-party APIs
Any side effect that doesn’t need to block the response

But if you're working inside Edge Middleware, then waitUntil() is what you want. It lets you run background tasks after the middleware returns a response.

This is very similar to after(), but only works in the Edge runtime. On the other side, if the task must finish before sending a response, like saving to a database, verifying input, or processing a payment, then just use await. This will delay the response until the task completes, but that’s necessary in those cases.

Conclusion

Next.js’ after() function represents a significant step forward in building performant, scalable web applications. It provides you with a native way to defer non-critical operations until after the response is sent, and this eliminates one of the most common performance bottlenecks in server-side applications: blocking the user while handling side effects.

The key to using after() effectively is understanding its limitations and choosing the right context for your use case. Remember that it's not reactive, requires proper async/await patterns for request APIs, and has different capabilities depending on where you use it.

But when used correctly, after() can dramatically improve your application's perceived performance and user experience.

LogRocket: Full visibility into production Next.js apps

LogRocket's Galileo AI watches sessions for you, instantly identifying and explaining user struggles with automated monitoring of your entire product experience.

The LogRocket Redux middleware package adds an extra layer of visibility into your user sessions. LogRocket logs all actions and state from your Redux stores.

Modernize how you debug your Next.js apps — start monitoring for free.

Your AI has agency — here’s how to architect its frontend

Megan Lee — Thu, 03 Jul 2025 17:00:00 +0000

Written by Rosario De Chiara✏️

As more teams rush to integrate AI agents into their products, frontend developers often face a frustrating gap: the back-end intelligence is evolving fast, but the UI patterns to support it are lagging behind.

Traditional input-output interfaces struggle to keep up with agents that can reason, revise, and act autonomously. Agentic AI refers to AI systems capable of reasoning, acting autonomously, and adapting to their environment or task. As these agents shift from passive tools to dynamic collaborators, the design patterns underpinning them become increasingly critical.

This article outlines six emerging architectural patterns that define how these agents are built and operated. These patterns enable powerful applications in search, automation, coding, and more. We also explore how these back-end capabilities shape UI/UX on the front end — highlighting opportunities and challenges for frontend developers working with intelligent systems.

Agentic AI architecture patterns

These six core design patterns form the technical backbone of agentic AI systems. Each offers a different take on how intelligent agents reason, act, collaborate, or retrieve information — and they shape how front-end teams will need to support, visualize, and work with these capabilities.

ReAct agent — think, act, repeat

The ReAct (Reasoning and Acting) model is a foundational pattern where agents alternate between reasoning steps — using large language models (LLMs) — and taking actions, such as querying a tool or external service. This looped process of “thinking and doing” mimics human problem-solving and serves as the blueprint for most current AI agents.

It’s also the baseline architecture for using LLM-based agents — the LLM “reasons” about how to solve a problem and then executes that plan via a tool. In a sense, this serves as the abstract blueprint for implementing all the various agentic architectures described below.

CodeAct agent — Python as the execution engine

Developed by Manus AI, this architecture enables agents to directly generate and execute Python code — rather than relying on rigid JSON-based actions. This opens the door to more complex reasoning and execution.

While JSON merely conveys structured data, Python allows for both data representation and manipulation. This architecture enables more complex reasoning and dynamic task execution. For example, when you ask ChatGPT to create a PowerPoint presentation, it may generate and execute Python code to build the file, which you can then download. This ability to operate beyond pre-defined functions creates more flexible, autonomous workflows.

Modern tool use — cloud-scale orchestration

In this pattern, agents interact with cloud infrastructure through lightweight execution standards such as MCP (Multi-Cloud Platform). MCP allows agents to access and orchestrate tools across hundreds of cloud platforms, enabling massive scalability and integration without requiring deep infrastructure expertise.

From a dev standpoint, this allows developers to build thin layers of orchestration logic on top of powerful, distributed services — making the agent far more extensible with minimal overhead:

Self-reflection — agents with a conscience

Inspired by metacognitive strategies, self-reflective agents can review and critique their own outputs to improve performance over time. Typically, this involves a second LLM acting as a kind of internal reviewer or conscience. The agent evaluates its own proposed actions, iteratively refining them until they meet a quality threshold.

This not only boosts accuracy but introduces a mechanism for learning from failure — moving closer to human-like problem-solving loops:

Multi-agent workflow — many agents, one goal

Rather than relying on a single generalist agent, this pattern involves multiple specialized agents collaborating on a task. Each agent brings a distinct skillset — often through tailored prompts or tools — and contributes to a shared output. This is especially valuable during development and debugging, as responsibilities are clearly divided.

For instance, when a specific output seems off, it’s easier to isolate which agent handled that step. This pattern also opens the door to more modular, composable agent systems — much like microservices in software architecture.

Agentic RAG — memory-enhanced generation

This pattern combines retrieval mechanisms with generative capabilities. Agents access external knowledge — typically from a vector database or API — then synthesize that data into meaningful responses.

By blending search with generation, Agentic RAG systems produce more accurate and informed responses. We have a detailed article on RAG that explores its various flavors in depth.

Designing UI for agentic architectures

Designing user interfaces for Agentic AI systems requires rethinking the typical request-response model. These agents don’t just answer queries — they explain, revise, and coordinate with other systems.

That means the front end must be transparent, adaptive, and collaborative. Below are practical examples of how different Agentic patterns shape front-end design and user interaction flows:

ReAct agents

Use a dual-pane conversational interface:

One pane displays the agent’s internal reasoning process (e.g., plans, hypotheses)
The other pane shows external actions taken (e.g., API calls, file generation)

This creates a clear, step-by-step user journey and avoids a “black-box” experience. Including animated “thinking” states or progress indicators enhances transparency and sets user expectations.

CodeAct agents

CodeAct UIs resemble live coding environments:

A top panel shows auto-generated Python code
A bottom panel displays execution results (graphs, logs, etc.)

Include contextual buttons like “Explain this code” or “Edit and rerun” to preserve user control. Because the architecture depends on code as the core execution format, code visibility and editability become central to the UX — especially for technical users.

Modern tool use agents

These agents require modular, task-based layouts:

A sidebar lists all integrated tools, showing real-time status or usage metrics
The main panel shows a timeline of actions taken by the agent

This allows users to trace decisions and understand how tools are leveraged. The challenge lies in representing a wide variety of tools without cluttering the interface.

Self-reflective agents

Here, versioning and revision history are critical:

Use timeline-based views to show drafts, critiques, and final outputs
Inline annotations (like Google Docs comments) can show what changed and why

Let users accept, reject, or comment on revisions — creating a feedback loop that mirrors how humans co-edit and iterate.

Multi-agent workflows

UIs must visualize agent collaboration and task flow:

Use process diagrams or activity maps to show which agent did what, when
Expandable cards or accordions can help users dive deeper into each agent’s output

This transparency empowers users to interact with specific agents — e.g., reassigning tasks or retrying underperforming steps.

Agentic RAG interfaces

Best served by split views:

One pane shows retrieved context (e.g., vector search results)
The other displays the generated response, with footnotes or tooltips linking to source material

Include a “Show Sources” toggle to help users distinguish between factual grounding and hallucination. This is especially useful in technical Q&A, tutoring, or research scenarios.

A final note on Chat UIs

Altogether, these UI patterns show how the front-end must evolve in tandem with agentic back-end architectures — not only to support functionality, but also to create a collaborative and transparent relationship between humans and intelligent systems.

Keep in mind that all these variants can ultimately boil down to a chat-based UI, but we think that using a chat-based UI can create inertia that we might want to avoid.

Quick comparison of agentic patterns

For your quick reference, I’ve built this table comparing all six architectures:

PATTERN	CORE CAPABILITY	WHAT MAKES IT DIFFERENT	IDEAL USE CASE	UI IMPLICATIONS
ReAct	Reason + act loop	Alternates thinking with tool use	Search agents, email assistants	Two-pane view to expose reasoning and actions
CodeAct	Live code execution	Uses code generation as action	Report generation, API interaction	Console-style output, editable code cells
Modern Tool Use	Cloud orchestration	Agent-to-tool abstraction across services	Enterprise workflow automation	Modular dashboard with real-time status
Self-Reflection	Output self-critique	Agent uses feedback loops to improve	Content validation, QA	Versioned views with inline annotations
Multi-Agent	Specialized agent collaboration	Multiple agents with focused roles	Product design, document generation	Process flow with agent activity highlights
Agentic RAG	Contextual retrieval	Combines memory search with generation	Knowledge assistants, tutoring	Split view with sources and generated output

Actual tradeoffs and limitations

Agentic architectures are powerful, but they’re not magic. With great autonomy comes great complexity. Here’s what you should keep an eye on:

Latency — More reasoning means more steps, which means more waiting. Users don’t love watching your AI think out loud unless it’s worth the wait
Observability — Multi-step agents can be a nightmare to debug without clear logs, traceability, or visibility into the toolchain
Security — Giving agents the keys to your APIs or file system is risky business. Guardrails, scopes, and runtime restrictions are non-negotiable
Cost and compute — Self-reflection, multi-agent workflows, and memory retrieval often stack multiple LLM calls. That invoice adds up — fast
User trust — If users can’t tell what the agent did or why, they won’t just be confused — they’ll be gone

Designing around these tradeoffs isn’t optional — it’s part of the job. Think of it as agentic gravity. You can’t defy it, but you can build smartly within it.

Conclusion

These six design patterns represent the current frontier in Agentic AI development. From reflective loops to multi-agent collaboration, they’re reshaping how frontend and backend systems interact.

As these systems evolve, we expect to see more autonomous agents with long-term memory, agent-specific GUIs with persistent state, and real-time feedback loops where users and agents co-adapt.

UI systems that allow for granular transparency, reversibility, and human override will likely define successful implementations in production. And as these patterns mature, expect to see hybrid models emerge — blending reasoning, retrieval, execution, and critique into tightly integrated systems.

For developers, especially those working on the front end, understanding these patterns isn’t just helpful — it’s essential to building the next generation of intelligent, user-aligned applications.

Get set up with LogRocket's modern error tracking in minutes:

Visit https://logrocket.com/signup/ to get an app ID.
Install LogRocket via NPM or script tag. LogRocket.init() must be called client-side, not server-side.

NPM:

$ npm i --save logrocket 

// Code:

import LogRocket from 'logrocket'; 
LogRocket.init('app/id');

Script Tag:

Add to your HTML:

<script src="https://cdn.lr-ingest.com/LogRocket.min.js"></script>
<script>window.LogRocket && window.LogRocket.init('app/id');</script>

3.(Optional) Install plugins for deeper integrations with your stack:

Redux middleware
ngrx middleware
Vuex plugin

Get started now.

JavaScript loops explained, and best practices

Megan Lee — Thu, 03 Jul 2025 15:00:00 +0000

Written by Rahul Padalkar✏️

Loops, the most commonly used construct in programming languages, are used to help run a piece of code multiple times. In JavaScript, where everything is an object, there are many options available to loop over objects. With so many options available, it can be confusing to know which loop to use where.

This article aims to simplify this confusion by providing a walkthrough of the most popular loops in JavaScript, along with some real-world examples. Let's dive in!

`for` loop

The for loop is the most commonly used loop to iterate over arrays, strings, or any array-like objects is the for loop. Everyone learns for on their first day of learning JavaScript! A for loop in JavaScript follows this format:

for (initialization; condition; final-expression) {
  // body
}

Here’s an example of the loop in action:

let numbers = [1,2,3,4,5]

for(let i=0;i<numbers.length;i++) {
  // do something...
  console.log(numbers[i]);
}

You can use for when you want low-level control over a loop. Though it’s fairly basic, there are a few things a developer should know about the for loop: Using a break keyword in a for loop breaks the execution, and the code will exit the for loop:

let numbers = [1,2,3,4,5]

for(let i=0 ; i < numbers.length ;i++) {
  if(i === 3) break;
  console.log(numbers[i]);
}```



The above `for` loop will exit when iterating over the third number in the array. The `break` statement is useful when you want to stop processing elements in an array or array-like data structures when a certain condition is met. **Using the `continue`keyword in a `for` loop skips to the next element:**



```javascript
>let numbers = [1,2,3,4,5]

for(let i=0 ; i < numbers.length ;i++) {
  if(i === 3) continue;
  console.log(numbers[i]);
}```



The above `for` loop will skip logging `3` to the console because the `continue` keyword skips to the next element in an array. All statements below the `continue` keyword are skipped. `continue` is useful when you selectively process elements in an array or array-like data structure. **All expressions in the loop are optional:** Both `for` loops below are valid `for` loops in JavaScript:



```javascript
let i = 0;
for (; i < 3;) {
  console.log(i++); // 0 1 2
}

for(;;;) {
  console.log("This will print forever.....")  //infinite loop
}```



## `for...of` loop

This loop is a variation of a `for` loop that was recently added to JavaScript. It is useful for iterating over arrays, strings, sets, maps, typed arrays, etc., — basically any object that implements the `[Symbol.iterator]` function. Following the same example, this is how you would use a `for…of` loop:



```javascript
const numbers = [1,2,3,4,5]
for(const num of numbers) {
    console.log(num) // 1 2 3 4 5
}

The break and continue keywords work similarly for the for…of loop. Here are a few things to keep in mind when using this loop: The iterated values are copies of the original ones, not references:

const numbers = [1,2,3,4]

for(let num of numbers) {
  num = num + 2; // this doesn't change the values in the numbers array
}

console.log(numbers) // 1,2,3,4```
{% endraw %}


**{% raw %}`for...of`{% endraw %} can't access the index:** Unlike the {% raw %}`for` loop, the `for…of` loop doesn’t pass the index of the current item in the collection. You would have to manually keep track of the index:



```javascript
const numbers = [1,2,3,4]
let index = 0
for(let num of numbers) {
  index++;
  console.log(num);
}

for...of can be used in async functions: The for…of loop also allows you to iterate over async data sources. Here’s an example:

async function load() {
  for await (const res of urls.map(fetch)) {
    const data = await res.json();
  }
}

Here, the loop iterates over an array of promises returned by urls.map(fetch). Once each of the promises resolves, it then iterates over the response object returned by the fetch call.

`for…in` loop

This is another variation of the for loop that can be used to iterate over the keys of an object:

const user = {id: 12,first_name: "John",last_name: "Doe",age: 27, gender: "male"}

for(const key in user) {
  console.log(key) // id first_name last_name age gender
  console.log(user[key]) // 12 John Doe 27 male
}

One thing to understand is that for...of iterates over values, whereas for...in iterates over keys. Under the hood, thefor...in loop calls Object.keys(object) . A few things to note: for…in iterates over inherited properties:

Object.prototype.foo = 123;
const obj = { bar: 456 };

for (const key in obj) {
  console.log(key); // 'bar', 'foo'
}```



To avoid this, we can use `hasOwnProperty` to check if the property belongs to the object or someone in the prototype chain:



```javascript
Object.prototype.foo = 123;
const obj = { bar: 456 };
for (const key in obj) {
  if(obj.hasOwnProperty(key)) {
    console.log(key); // 'bar'
  }
}

Don’t use for...in with arrays: As mentioned earlier, for...in iterates over the keys of an object. Internally, arrays are objects with an index as the key. Therefore, when used with arrays, for...in will return the index of the values and not the actual values:

const nums = [101, 102];

for (const key in nums) {
  console.log(key); // '0', '1'
}

break and continue work as expected with for...in: Using break exits the loop and continue skips to the next iteration. Both these keywords work as expected:

const user = {
  name: 'Alice',
  age: 30,
  role: 'admin',
  banned: false,
  gender: 'female' 
};

for (const key in user) {
  if (key === 'role') continue; // skip 'role'
  if (key === 'banned') break;  // stop before 'banned'

  console.log(key, user[key]); // name Alice age 30
}

`do...while` loop

The do…while loop isn’t common in real-world use cases. It is a post-tested loop, which means the loop runs at least once before the condition is evaluated to either true or false:

const nums = [1,2,3,4,5]
let i=0;
do {
console.log(nums[i]) // 1 2 3
i++
} while(i < 3)

Here are a few things to note about the do...while loop: do...while runs at least once before the condition is evaluated:

const nums = [1,2,3,4,5]
let i=0;
do {
console.log(nums[i]) // 1
i++
} while(i < 0)```



The loop will run once, even though the `while` condition is always going to be false. **The loop works with `break` and `continue`:** Similar to the other loops, with `do...while`, you can use `break` to exit the loop or `continue` to skip to the next iteration:



```javascript
let i = 0;
do {
  i++;
  if (i === 2) continue;
  if (i === 4) break;
  console.log(i);   // 1 3
} while (i < 5);

`while` loop

A while loop runs as long as the condition evaluates to true or a true-like value. Unlike do..while, while doesn’t ensure the loop runs at least once. It’s also very easy to run into infinite loops, so you need to be very careful when writing one:

const nums = [1,2,3,4,5]
let i =0;
while(i < 6) {
  console.log(nums[i]) // 1 2 3 4 5
  i++;
}

Below are a few things to note about the while loop: You can easily create infinite loops: The code below creates an infinite loop, so you need to be extremely cautious when using while:

let i = 0;
while (i < 3) {
  console.log(i) // 0 0 0 0 0 0 .....
  // forgot i++;
}

while works with break and continue: Like we saw for most other loops, while works with these statements:

let i = 0;
while (i < 5) {
  i++;
  if (i === 2) continue;
  if (i === 4) break;
  console.log(i);        // 1 3
}

`forEach` loop

forEach is a higher-order function that takes in a callback as its argument. This callback function is called for each element in the array. The callback function has the signature (element, index, array). Below is an example of how to use the forEach loop:

const nums = [1,2,3,4,5]

nums.forEach((num, index, nums) => {
  // do something
})

Things to note about this loop: forEach is not ideal for async functions: The forEach loop doesn’t wait for an async function to finish before going to the next element in the array. So, if you plan on async processing array elements one after the other, you should probably stick to the simple for loop:

const nums = [1,2,3,4,5]

nums.forEach(async (num, index, nums) => {
  const data = await fetch(`https://someAPI.com/${num}`) // control won't stop here
})

Keywords like continue and break don’t work as expected with forEach: When using forEach, you can’t quit the loop early with break and skip an iteration with continue. Remember, forEach is a higher-order function:

const nums = [1, 2, 3]

nums.forEach((n) => {
  if (n === 2) return; // Only exits this iteration, not the loop
  console.log(n);
});

forEachalways runs to completion: A forEach loop runs for all elements. You can skip processing for certain elements with a return statement:

const nums = [1, 2, 3]

nums.forEach((n) => {
  console.log(n); // 1 2 3
  return;
});

N.B., forEach also exists for other data structures like Set, Maps, and TypedArrays. They behave more or less in the same way.

`map` loop

map is generally used when we need to transform the values in an array and return the transformed values in an array. A common mistake is to use map to iterate over an array instead of a simple for loop or forEach function. map is also a higher-order function that takes in a callback and returns an array of transformed values.

Remember: to run side effects on elements in an array, use forEach, and to perform transformations on array elements and return the transformed array, use map:

const nums = [1, 2, 3];

const doubled = nums.map((num, index, nums) { return num * 2 } );

console.log(doubled); // [2, 4, 6]

Here are a few things one should keep in mind when using the map function on an array: Always return from the map callback function: The value returned by the callback function is returned as an element of an array from the map function, so never forget to return a value from the map callback function:

const nums = [1, 2, 3];

const result = nums.map(item => {
  return item * 2;
});

console.log(result) // [2,4,6]

map is not async-friendly: Just like the forEach loop, map is a higher-order function. It doesn’t wait for any async operation to complete in the callback function:

const nums = [1, 2, 3];

const resultWithPromises = nums.map(num => {
  return fetch(`https://someapi.com/${num}`)
});

console.log(resultWithPromises) // [Promise, Promise, Promise]

const result = await Promise.all(nums.map(num => {
  return fetch(`https://someapi.com/${num}`)
}));

console.log(result) // 123 456 789

`some` loop

some is a lesser-known array function that checks if at least one element in an array passes the given test. You can think of this method as asking, “Do some item(s) in this array meet this condition?”

const nums = [1,2,3,4,5]

// check if atleast one number is even
const containsEven = nums.some((num, index, nums) => {
  return num%2 === 0 
})

console.log(containsEven) // true

Here, if at least one element (2 and 4) passes the test, the some function will return a truthy value. You would use some in real-world use cases like:

Checking if a user has a particular role (like an admin role)
Checking if a product is out of stock

A few things to note about the some function: some always returns a Boolean: some always returns a Boolean value. Looking at the function name, some might think that it would return an array of elements that pass the test.

But instead, using some returns a Boolean value of true or false immediately after it encounters the appropriate number. For example, if I have an array [1,2,3,4,5] and I pass a condition that checks if a number is even, as soon as 2 is encountered, the other numbers will be skipped and the function will “early return” with a true value.

In this way, some can be thought of as a shortcut; it returns early if an element passes the test. It doesn’t support await in callback: Because some is a higher-order function, if there is an await in the callback, the callback won’t wait for the promise to resolve:

await arr.some(async x => await isValid(x)); // ❌ Doesn’t work

The callback returns a Boolean: The callback passed to some should always return a Boolean. This Boolean signals if the element passed the test or not:

arr.some(item => item === 3);

`every` loop

every is an array method that checks if all elements in an array pass the test or not. It takes in a callback and expects it to return a Boolean value. The callback signature is similar to other array methods:

const nums = [2, 4, 6];
const allEven = nums.every(n => n % 2 === 0);
console.log(allEven); // true

The code above checks if all the elements in the array are even. Though rarely used, every can be used in the following use cases:

Checking if all users are active → users.every(u => u.isActive)
Validating items → items.every(i => validate(i))

A few things to note about every: It short-circuits on the first false: Because the function checks if all elements in the array pass the test, if any element fails the test (meaning it returns false), the function will short-circuit. It's not async-friendly: Because every is a higher-order function, if there is an await in the callback, the callback won’t wait for the promise to resolve:

arr.every(async item => await isValid(item)); // ❌ Doesn't work as expected

TL;DR: Comparing the loops

We covered a lot of loops, and it might take time to digest this information. Here’s a quick overview of the loops we explored:

Feature	for	for...in	for...of	while	do...while	map()	forEach()	every()	some()
Iterates over values	✅ (manual)	❌ (keys only)	✅	✅ (manual)	✅ (manual)	✅	✅	✅	✅
Target	Indexable items	Object keys	Iterables	Any condition	Any condition	Arrays	Arrays	Arrays	Arrays
Returns a value	❌	❌	❌	❌	❌	✅ New array	❌ undefined	✅ Boolean	✅ Boolean
Can break/continue	✅	✅	✅	✅	✅	❌	❌	❌	❌
Executes at least once	❌	❌	❌	❌	✅	❌	❌	❌	❌
Supports await	✅ (manually)	❌	✅ (for await)	✅ (manually)	✅ (manually)	❌	❌	❌	❌

JavaScript loops are some of the most commonly used constructs. Knowing the difference between each of them will help you write cleaner and more performant code that enhances readability. Thanks for reading!

LogRocket: Debug JavaScript errors more easily by understanding the context

Debugging code is always a tedious task. But the more you understand your errors, the easier it is to fix them.

LogRocket allows you to understand these errors in new and unique ways. Our frontend monitoring solution tracks user engagement with your JavaScript frontends to give you the ability to see exactly what the user did that led to an error.

LogRocket records console logs, page load times, stack traces, slow network requests/responses with headers + bodies, browser metadata, and custom logs. Understanding the impact of your JavaScript code will never be easier!

Try it for free.

How to design apps with Apple Intelligence in mind

Megan Lee — Tue, 01 Jul 2025 20:00:00 +0000

Written by Murat Yüksel✏️

In its recent WWDC Conference, Apple introduced some serious new advancements to Apple Intelligence. One of the world’s largest software companies is evolving how it deploys AI within its products.

With this evolution comes a host of new considerations for frontend developers and UX designers. Hundreds of millions of users globally access their digital lives daily through iPhones, iPads, MacBooks, and other Apple devices.

With such a vast and engaged user base, understanding and leveraging Apple's unique approach to AI is paramount for creating future-forward, relevant, and compelling app experiences. Now this shift requires appropriate reflection in the mindset when it comes to designing new modern web applications. In this article, we will:

Explore the core features of the Apple Intelligence update
Discuss do’s and don’ts for designing with Apple Intelligence in mind
Reflect on what Apple Intelligence means for the future of AI design

Let’s dive in!

What exactly is Apple Intelligence?

Before going further, let’s first understand what Apple Intelligence is and what kind of AI it is. We’re used to chatbots. They’re almost synonymous with the idea of artificial intelligence in the minds of many people.

Many have suggested that Apple was lagging behind in the AI race. Instead of coming up with a new chatbot, Apple is positioning AI as more of an ambient entity: an assistant that’s present all the time.

Apple Intelligence is not a chatbot app that you open, ask questions, and get responses; it’s directly integrated into your Apple device. It’s context-aware and can assist you with tasks. Now, AI can help you reply to emails directly, record conversations and give summaries, react and operate on what’s seen on the phone’s screen, etc.

Apple Intelligence does this directly on-device most of the time. We say most of the time, because there are other options like extending the capabilities via using Private Cloud Compute or directly interacting with ChatGPT. With these advancements come privacy questions. On-device AI usage is private, and Apple claims that Private Cloud Compute is private too.

How Apple Intelligence is shifting UX

Now, with the latest release, we’ve learned that using the Foundation Models framework, developers can use embedded intelligence even offline. This is big, and it requires a shift in how we should approach UX.

AI is now embedded and can do things it hadn’t before; read the screen, access different apps, and chain together actions. Developers will need to approach their design in a more intent-based way. Now, apps might have to take into account what the user intends to do, rather than what is predefined by the dev team:

Apple’s architecture and philosophy: Moving past chatbots

Apple’s AI feels more like a personal assistant than an answer-spitting AI we’re so used to now. Privacy is a prime concern here. In using the on-device AI, user data does not leave the device. Apple also stressed that the Private Compute Cloud functionality was private, as well.

This functionality can also tap into ChatGPT if required. Here, you have to think about whether you want to present that specific information to the external LLM or not.

This flexibility between private and more general approaches is a win for user choice and experience. Genmojis and image generation are another big selling point, as Apple continues to pursue personal exploration and expression through AI. Users are incentivized to have and create unique, personalized experiences. Here are three lessons that frontend developers can take from Apple's new innovations:

Apple is flexible – You can choose either very private, private, or 3rd party (not private)
AI is not only about chatbots – It can act as a personal assistant, a helper, and someone who knows the context and can do complex tasks
Generative AI is here to stay – Apple strives to give users more room for creativity and personal expression (E.g. Genmojis)

What are the key features of Apple Intelligence?

Apple Intelligence allows developers to benefit from Writing Tools, Genmoji, and Image Playground. Writing Tools help users rewrite, proofread, and summarize text, but it can also be customized according to the app’s needs.

Genmoji allows users to generate a new emoji, which enables them to express their emotions more thoroughly. With Image Playground, apps can allow users to play with images using AI.

For example, users can remove unwanted background elements. You can also go further with creating your own models. Here are a few of the highlights of the Apple Intelligence release:

Foundation Models Framework

With Foundation Models Framework, you get direct access to the on-device LLM at the code of Apple Intelligence. By using Foundation Models, you can summarize text, extract text, classify it, generate new text, call specific tools in your app, and so on. One important thing to note is that this feature is available offline; it's free of charge for developers to use, and it has native support for Swift.

ChatGPT improvements

Now with direct ChatGPT integration, users can interact with ChatGPT inside their apps. Each supporting Apple product comes with the ChatGPT free tier, so users don’t need to open an account for it. For premium models, users would need to log in to their accounts.

The power of ChatGPT comes with a cost, though. While on-device processing and Private Cloud Computing stress privacy, giving user data to a third party is a whole different act. When designing applications, the developer team should inform the user what is shared with third parties, and when.

Visual intelligence

Now AI can “see” the user’s screen and react to it. For example, a user might be browsing social media and they see a comedy show that they’d like to attend. Now, instead of taking a screenshot of the event, saving the details, and updating their calendar, the user will be able to ask AI to do that for them.

This ubiquitousness of AI brings different design implications with it. Now, everything on the screen is subject to AI. It can look at images, alter them, find specific items in them, assess location based on background etc. To sum it up, Apple Intelligence can:

Remove unwanted elements from a photo’s background
Create unique Genmojis
Create new, custom images in seconds
Summarize long emails
Rewrite text
Tap into ChatGPT
Use the camera to get info and create appointments
Read and react to the device’s screen

'Intent-based app design': Building with Apple Intelligence in mind

Building with Apple Intelligence in mind requires a shift in how we think of application design. We can describe this new way of thinking as “intent-based app design.” Instead of following a predefined set of actions, the user will express their intents in myriad ways and expect the applications to behave accordingly. To help illustrate the idea, let’s go over some hypothetical examples.

Inter-app communication

Old thinking

The user opens your app to perform a task.

New thinking

User expresses an intent (verbally, through text, or even implicitly through context), and Apple Intelligence (via Siri or other system-level features) either fulfills that intent directly or suggests your app as the best tool for the job, with the relevant part of your app pre-loaded. An example could be a user taking a picture of their kid’s dance school schedule, and Apple Intelligence would take that picture and use the information it gathers from there to add the performance date to the user’s calendar.

It’ll then send them reminder notifications before the event date. Another example could be Apple Intelligence making sure that the user never forgets their anniversary. The user’s intent could be something like “look through my phone, gather all information, and arrange my schedule accordingly.” In developing such applications, developers will be required to approach user privacy with extra care. Sensitive information should never be shared without the user’s consent.

AI anticipates user needs before they arise

Old thinking

Apps wait for user input.

New thinking

Apps, powered by Apple Intelligence, anticipate user needs based on context (location, time, calendar, communication, photos, etc.) and proactively offer relevant functionalities or information. Developers will have to consider how the app can "listen" to the user's environment and suggest relevant features before the user even thinks to open the app.

This requires thinking about data privacy and user control. Like the previous example, the user may or may not be happy with the contents of their images being run through an LLM.

On top of that, devs should be even more careful to inform their user about whether they want to make use of Private Cloud Computing, or tap into ChatGPT. For another example, think of a weather app that sends notifications to the user when it’s going to rain to remind them to bring their umbrellas with them.

AI across the ecosystem

Old thinking

Your app's value is primarily contained within its own UI.

New thinking

Your app's valuable features are surfaced across the entire Apple ecosystem: in Siri, Spotlight, Messages, Mail, Notes, Photos, and even new "Smart Overlays" or "Visual Intelligence" capabilities. Example: Your app suggests Genmoji as the user types.

Where will it work? Will it also work when writing emails as it does in chat messages? What kind of content can it suggest? Will there be contextual differences regarding different platforms and situations? Generative AI can be dangerous and create confusion.

Gen AI brings unique design scenarios

Old thinking

Users create all content from scratch or select from pre-defined options.

New thinking

Apple Intelligence provides powerful generative capabilities (text, images, emojis, summaries, etc.) that users can leverage directly within your app or through system-wide tools. Think of a mobile game where the environment changes depending on where you are located.

In this case, the game would create new backgrounds. Or, when interacting with characters, they wouldn’t say the same things all the time, but change their behavior depending on, say, current world events.

In such a scenario, since the tone of the app is subject to change, developers will have to adapt the styling accordingly. Just adding a dark mode will not be enough anymore. Now that we have an understanding of how the design could shift for this new era of AI that connects and assists, let’s talk about some best practices and pitfalls to avoid.

Best practices: Do’s and don’ts for designing with Apple Intelligence in mind

The following best practices are not just good design principles; they represent crucial considerations for developers getting ready for this new era of the omnipresent, embedded, generative AI.

Now that deeply integrated AI is at hand, with its on-device, offline processing and absolute privacy when needed, developers will face rather unique challenges. These recommendations strive to address the challenges and opportunities of the new era of omnipresent AI.

Everyone who’ll dabble with the Foundation Models Framework, experiment with Visual Intelligence, and tap into ChatGPT features should take these into account:

Expect the unexpected – Remember that users can give practically any prompt, something you’ve never considered before, and cause your AI app to behave in a very unexpected way.
Help users maintain control – You are developing for the user, and the user should be in control. They should have the ability to dismiss, revert, or retry AI-produced content as they wish.
Minimize cultural bias - Keep in mind that AI models learn from data, and they tend to be in favor of the most common information, whether it represents the truth for all or not. The data that the model has been fed could create images about certain things that a certain group of people may not experience or think of as reality. Give people the ability to fine-tune their requests in these situations, and try to minimize assumptions about cultural subjects.
Offer opt-out choices - Do not force your users to use generative AI. They should have the choice to opt out. A concrete example is Genmoji and emojis. Users are not required to use Genmoji all the time; they can use good ol’ emojis whenever they want.
Be transparent - Let your user know and choose when they want to interact with AI. Never trick someone into thinking they’re interacting with or viewing content authored by a human if they’re actually interacting with AI.
Set clear expectations - Do not overpromise. Make sure your users understand what the feature can and cannot do. If they assume X will happen, but X is not a feature of your app, the user may get frustrated and lose confidence. This stems from a miscommunication between the developer and the user. Keep in mind that you should set expectations right.
Favor on-device models - On-device models are strong enough; favor them over server-based models. Why? Pros of on-device models are as follows:
- They’re private; computations stay on your device
- They can be run offline
- They’re faster
Respect data privacy - Always ask for the user’s permission when using their data. Let the user have full control over what they want to share and what they want to keep private. If they agree to share, give them the possibility to revoke that permission.
Understand your model deeply - In developing your app, make sure you understand the underlying model well. If you don’t, your app might give subpar results.
Educate your user - AI is still confusing for many. Remind your users that AI can hallucinate. Try to avoid using your apps to deal with factual information. New dialog generation would be way safer than a news notification app in terms of factuality.
Handle permissions with care - Tread lightly on permissions. Your app should try to avoid committing changes that are hard to take back, like deleting pictures. Think about how you’ll implement and how you’d turn things around in the worst-case scenario.
Implement robust safeguards - Remember that some people will try to break your app or get harmful content out of it. Try to sanitize your app as best as you can.
Plan for AI failure - Think about what would happen if AI fails at a given task. Will it just stop doing anything, or will it try to guide the user to re-prompt for better results?

Big-picture reflections: What Apple Intelligence signals about the future of interface design

Like with many great innovations, we’re still in “wait and see” mode. However, Apple's WWDC announcement wasn't just another tech update; it was a foundational shift, signaling clear directions for the future of interface design that developers must understand and adapt to now.

To understand the shift, we might need to think of the difference between the old, static websites of the 90s and today’s modern applications.

Their content and capabilities foresaw the design mindset that is used towards them. A similar thing is happening with Apple’s understanding of AI. We can identify several key points that directly interact with our design approach:

Chatbots are old news; Ambient Intelligence is the new standard

Apple decided to shift its attention from the chatbot race to integrating AI as an omnipresent, context-aware personal assistant. AI is not seen as an interactive encyclopedia but a helper that reads information, anticipate, and addresses pain points. This shift signals that future interfaces must prioritize intent-based interactions. In that way of thinking, the user’s goal drives the system.

AI can be private, too

From on-device processing, where the data never leaves the device, to Private Cloud Compute, Apple shows that it takes privacy seriously. New design thinking should follow this decision.

Personal expression and creativity with AI

With features like Genmoji and Image Playground, Apple brings generative AI into daily life. The easier it is to use something, the more people will use it. Since users can now create images that represent their input, developers should think about designing interfaces that beckon the user to experiment with AI: more creativity and more uniqueness. The more personal, the better.

AI is integrated into the OS

Considering that via the Foundation Models Framework, AI can be used even offline, we are witnessing a shift in how AI is perceived. Apple is directly integrating the AI into their OS, which means that core features like Siri, Photos, Messages etc. can be used with AI, even offline!

One thing that bears repeating is the interaction between AI (the robot), and the human. Now, in building applications, you’ll have to think about all the prompts the user (human) can give, and all the output the AI (the robot) provides in return.

Things will go awry in the most unexpected ways, and developers and designers will need to think in broader terms. But overall, it’s nice to see we’re moving past the “only chatbots” stage of AI usage. Now AI can do more than just answer questions and write buggy code; it can be everywhere at once.

Conclusion: Embracing the future of intelligent design

Apple Intelligence marks a pivotal moment, shifting AI from a siloed tool to an ambient, privacy-first assistant woven into the fabric of Apple platforms.

For front-end developers and UX designers, this isn't just a new set of features; it's an invitation to redefine how users interact with technology.

With direct access to on-device foundation models, powerful Writing Tools, and innovative Image Playground APIs, the opportunities to create truly intelligent, personalized, and intuitive experiences are immense and immediately available.

Now is the time to explore and test these groundbreaking capabilities. Dive into the developer documentation, experiment with the new frameworks, and begin crafting the next generation of intelligent applications that seamlessly assist users while respecting their privacy. The future of interface design is here, and it's intelligent.

Get set up with LogRocket's modern error tracking in minutes:

Visit https://logrocket.com/signup/ to get an app ID.
Install LogRocket via NPM or script tag. LogRocket.init() must be called client-side, not server-side.

NPM:

$ npm i --save logrocket 

// Code:

import LogRocket from 'logrocket'; 
LogRocket.init('app/id');

Script Tag:

Add to your HTML:

<script src="https://cdn.lr-ingest.com/LogRocket.min.js"></script>
<script>window.LogRocket && window.LogRocket.init('app/id');</script>

3.(Optional) Install plugins for deeper integrations with your stack:

Redux middleware
ngrx middleware
Vuex plugin

Get started now.