Forem: Akash Thakur

PageIndex vs Vector RAG: Why Reasoning-Based Retrieval Achieves 98.7% Accuracy (Not 31%)

Akash Thakur — Thu, 26 Feb 2026 19:51:26 +0000

The Complete Guide to Vectorless RAG, Hierarchical Indexing, and Why Vector Databases Are Becoming Obsolete for Professional Documents

PageIndex, a vectorless RAG framework, achieved 98.7% accuracy on financial documents compared to GPT-4o's 31%. Instead of chunking documents into vectors, it builds a hierarchical tree index that preserves structure and enables LLM reasoning. This is how document retrieval should have worked from the start.

The Problem We All Know Too Well

You've been there. You ask your RAG system a specific question about a 50-page financial report. It confidently returns a chunk that's "semantically similar" to your query. But it's the wrong section. The terminology matches. The keywords align. But the meaning? Completely off.

This is what happens when your retrieval system confuses similarity with relevance.

Traditional vector databases are optimized for one thing: finding text that sounds like your question. They're essentially performing semantic karaoke—matching vibes, not meanings. For short documents and broad searches, that's fine. But for professional documents where structure matters and precision is non-negotiable? Vector-based RAG is increasingly looking like a band-aid on a broken system.

Enter PageIndex: a vectorless, reasoning-based RAG framework that fundamentally rewires how AI retrieves information from documents. And spoiler alert? It's absolutely crushing the benchmarks.

Vector-Based RAG's Original Sin: The Chunking Problem

Before we talk about the revolution, let's understand what we're revolting against.

Traditional vector-based RAG works in three steps:

Chunk the document into ~500-token pieces
Embed each chunk into vector space using an embedding model
Search by similarity when a query comes in

Sounds reasonable, right? It's not. Here's what gets destroyed in step one:

Hierarchical structure — A 300-page report becomes 600 orphaned chunks with no sense of how they relate
Context windows — The relationship between a header and its subsections evaporates
Semantic integrity — A footnote and a key finding might use identical terminology but mean completely different things
Reference precision — Good luck telling someone exactly where in the document you found something

In financial documents, legal contracts, technical manuals, and research papers, this structural destruction is catastrophic. A balance sheet's header and a footnote reference can have nearly identical embeddings. Both get high similarity scores. Only one actually answers your question.

The kicker? This isn't a bug—it's a feature of how vector similarity works. Semantic closeness and factual relevance are not the same thing.

What If We Just... Didn't Chunk?

PageIndex's core insight is deceptively simple: what if we organized documents the way humans do?

Instead of destroying document structure through chunking, PageIndex builds a hierarchical tree index—essentially an intelligent table of contents optimized for LLM reasoning. The framework operates in two elegant steps:

Step 1: Generate a Tree Structure Index

Transform a 200-page document into a semantic tree where:

Nodes represent sections, not arbitrary chunks
Hierarchy is preserved, maintaining the relationship between chapters, subsections, and details
Summaries are generated for each node, giving the LLM context without bloat
No vectors are computed (hence "vectorless")

Example: PageIndex Tree for a Financial Report

Key insight: When you ask "What are the company's main risks?", PageIndex navigates to Section 3 (Risk Factors), not 2.3 (Profitability). Structure guides reasoning.

Instead of similarity matching, the LLM reasons its way through the tree:

Read the table of contents
Identify potentially relevant sections based on the question
Decide whether to dive deeper or explore another branch
Extract and synthesize information
Return a traceable, explainable answer with specific page references

This is how you or I would navigate a complex document. We don't turn it into vectors and find the "most similar" parts. We think about structure, logic, and relevance. We navigate. We reason. We decide.

PageIndex makes LLMs do exactly that.

Traditional Vector RAG vs PageIndex: Side-by-Side

The Benchmark That Changed Everything: 98.7% Accuracy

Here's where PageIndex stopped being theoretical and started being undeniable.

VectifyAI's reasoning-based RAG system, powered by PageIndex, achieved 98.7% accuracy on FinanceBench—a challenging benchmark for financial document question-answering. For context:

The breakdown:

GPT-4o (vanilla): ~31% accuracy
Perplexity: ~45% accuracy
Traditional vector RAG solutions: ~60% (estimated)
PageIndex-powered Mafin 2.5: 98.7% accuracy 🚀

This isn't a marginal improvement. This is the difference between a system that sometimes works and a system that's reliably, auditably, enterprise-grade correct.

The why? Document structure matters. Professional documents aren't random collections of semantically similar passages. They're carefully organized hierarchies where a subsection under "Risk Factors" means something fundamentally different from the exact same phrase under "Financial Overview."

PageIndex understands this. Vector similarity doesn't.

Why This Matters (The Use Cases That Can't Wait)

PageIndex isn't a solution looking for a problem. It's solving real pain points for:

Financial Services

SEC filings, earnings reports, and investment documents are where vector RAG falls apart hardest. The same phrase might appear in risk disclosures, footnotes, and executive summaries—each with vastly different implications. PageIndex's reasoning-based approach ensures you get the section that actually matters.

Legal & Compliance

Contracts and regulatory documents are hierarchically dense. Structure is meaning. A clause nested under "Termination" implies something entirely different from the same text under "Definitions." Chunking destroys this precision.

Technical Documentation

Industrial manuals, system guides, and technical specifications repeat terminology obsessively. "Pressure" appears in operating procedures, safety guidelines, and specifications. Vector similarity will gleefully return all of them. Reasoning-based retrieval asks: "What is the context of this question?" and navigates accordingly.

Research & Academia

Academic papers have rigorous hierarchies: abstract, introduction, methodology, results, discussion, conclusion. Each section plays a specific role. A statement in the methodology section (how we tested) is fundamentally different from the same statement in the discussion (what it means). PageIndex preserves this semantic role.

The Technical Magic: How It Actually Works

If you're curious about the mechanics, here's the elegant bit.

Vision-Native Retrieval

Unlike traditional RAG that relies on OCR (which introduces its own errors), PageIndex includes vision-native capabilities. It can "see" document structure directly from page images. Complex financial tables, layout-critical diagrams, and formatted content are understood without intermediate text extraction.

Agentic Reasoning

The retrieval process is agentic—the LLM dynamically decides which sections to explore based on the evolving conversation. This isn't a static "retrieve top-k chunks" operation. It's an interactive thinking process.

Complete Explainability

Every answer comes with a full reasoning trace: which nodes were explored, why certain sections were selected, and exactly which pages and sections the answer came from. This is invaluable for audits, compliance, and verification.

No Vector Infrastructure Required

No vector databases. No embedding model maintenance. No approximate similarity calculations introducing subtle errors. Just hierarchical reasoning and LLM intelligence.

The Different Flavors: One Tool, Multiple Strategies

PageIndex recognizes that different documents and use cases need different retrieval strategies:

Document Search by Metadata

When you have multiple documents that can be distinguished by metadata (source, date, author), search by metadata first, then apply PageIndex reasoning within the selected document.

Document Search by Semantics

For diverse documents covering different topics, use semantic understanding to select the right document, then use PageIndex for precise retrieval within it.

Document Search by Description

For smaller document collections, provide descriptions of each document. The LLM reads the descriptions and chooses intelligently.

This flexibility means PageIndex scales from single-document precision to multi-document scenarios without losing its reasoning-based advantages.

How to Actually Use This Thing

For Developers

pip install pageindex
python run_pageindex.py --pdf_path your_document.pdf

The open-source repo on GitHub gives you everything you need to generate tree structures from PDFs and markdown. Full Python SDK support. MCP integration ready.

For Teams Without Engineering Resources

PageIndex Chat is a ChatGPT-like platform where you upload documents and get human-like analysis. No code required. Upload a 200-page report, ask specific questions, get reasoning-based answers with full traceability.

For Enterprise

MCP (Model Context Protocol) integration lets you connect PageIndex to Claude, other LLMs, and agentic frameworks. Private deployment options available.

The Architecture: How PageIndex Components Work Together

PageIndex represents a fundamental shift in how we think about retrieval in the LLM era.

From similarity-based to reasoning-based: We're moving away from "what text sounds like your question?" toward "what text actually answers your question?"

From black-box search to traceable reasoning: Vector similarity is opacity by design. PageIndex gives you a full reasoning chain. In regulated industries, this isn't a feature—it's table stakes.

From arbitrary chunking to structural preservation: We've internalized that chunking documents is normal. It's not. It's destructive. PageIndex proves that maintaining structure dramatically improves retrieval quality.

From infrastructure-heavy to inference-light: You don't need specialized vector infrastructure. You need an LLM that can reason. The bar for deployment just dropped significantly.

This is why PageIndex appeared on Hacker News with hundreds of upvotes. Why it reached 5.7k stars on GitHub in months. Why financial and legal firms are quietly deploying it. It solves a problem everyone in the RAG space has been ignoring: document structure matters, and vector similarity is the wrong tool for structured retrieval.

The Vision-Only Angle: OCR-Free Retrieval

Here's a detail that deserves its own section: PageIndex supports vision-native retrieval without OCR.

This matters because OCR is a lossy, error-prone bottleneck. Layout information gets destroyed. Complex tables break. Structured content becomes unstructured text.

PageIndex's vision approach allows the LLM to "see" the actual visual structure of documents. Complex financial tables are understood as tables, not as mangled text. Diagrams remain diagrams. The visual hierarchy informs the reasoning.

For documents where layout is meaning (financial statements, technical blueprints, architectural diagrams), this is a game-changer.

When Should You Actually Use PageIndex vs. Traditional Vector RAG?

Let's be honest: PageIndex isn't always the right answer. Here's the decision matrix:

The Ecosystem: It's Not Just a Framework Anymore

PageIndex has evolved beyond a single research paper:

Open-source repo with full Python SDK and cookbooks
Chat platform for non-technical users
MCP integration to plug into Claude, Cursor, and other agents
API for developers building custom applications
Enterprise support for organizations with deployment requirements
Specialized OCR for converting PDFs to markdown while preserving structure

The breadth suggests this isn't a flash-in-the-pan research project. It's becoming infrastructure.

What's Next? The Trajectory

If you're paying attention to the RAG space, a few patterns are becoming clear:

Vector databases are under pressure — For structured retrieval tasks, similarity search is losing to reasoning-based approaches
Document structure is staging a comeback — We spent five years ignoring hierarchy in favor of bags of vectors. That era is ending
Explainability is becoming non-negotiable — Regulatory requirements and enterprise standards demand transparent retrieval paths
Vision-native understanding matters — OCR-free, vision-based reasoning is the future for document processing

PageIndex sits at the intersection of all these trends. It's not predicting the future; it's implementing it right now.

The Bottom Line

For years, we've assumed that vector databases were the "right" way to do RAG. They're not. They're convenient. They're fast for certain use cases. But for the retrieval tasks that matter most—the ones where accuracy is non-negotiable and structure is semantic—they're fundamentally limited.

PageIndex proves that a different approach works better. Not just marginally—dramatically better.

If you're building systems that retrieve from professional documents, financial reports, legal contracts, technical specifications, or research papers, PageIndex isn't just an option anymore. It's becoming the default.

The era of "vibe-based search" is ending. The era of reasoning-based retrieval is here.

Resources to Go Deeper

GitHub: VectifyAI/PageIndex
Docs: docs.pageindex.ai
Chat Platform: chat.pageindex.ai
Blog & Research: pageindex.ai/blog
Discord Community: Join the community
Framework Paper: PageIndex: Next-Generation Vectorless, Reasoning-based RAG

The revolution in document retrieval isn't coming. It's already here. The only question is whether you'll adopt it before your competitors do.

Temporal + AI Agents: The Missing Piece for Production-Ready Agentic Systems

Akash Thakur — Wed, 26 Nov 2025 18:07:54 +0000

Introduction

In the world of distributed systems and agentic AI, managing complex, long-running workflows reliably is a significant challenge. Traditional approaches often struggle with failures, state management, and ensuring that processes complete successfully even when systems crash or network issues occur.

Temporal is an open-source platform that solves these problems by providing durable, reliable workflow orchestration. It abstracts away the complexities of distributed systems, allowing developers to focus on business logic while Temporal handles failures, retries, and state management automatically.

This article explores Temporal's core concepts, workflow patterns, and how it's revolutionizing the development of agentic AI systems.

What is Temporal?

Temporal is a workflow orchestration platform that provides:

Durable Execution: Workflows that can run for days, months, or even years without interruption
Automatic Failure Recovery: Built-in retry mechanisms and state persistence
Time-Travel Debugging: Complete event history for every workflow execution
Scalability: Horizontal scaling through distributed workers
Language Support: SDKs for Go, Java, Python, TypeScript, .NET, PHP, and Ruby

Unlike traditional task queues or workflow engines, Temporal maintains a complete event history of every workflow execution, enabling it to resume workflows from any point after failures.

Core Concepts

1. Workflows

Workflows are the heart of Temporal. They represent the business logic of your application as code. Workflows are:

Deterministic: They must produce the same result when replayed
Durable: State is persisted automatically
Long-running: Can execute for extended periods
Versioned: Support safe deployments and migrations

from temporalio import workflow
from temporalio.common import RetryPolicy
from datetime import timedelta

@workflow.defn
class TradingWorkflow:
    """Example workflow for automated trading analysis."""

    @workflow.run
    async def run(self, symbol: str, strategy: str) -> dict:
        # Workflow logic here
        result = await workflow.execute_activity(
            analyze_market_activity,
            args=[symbol, strategy],
            start_to_close_timeout=timedelta(seconds=180),
            retry_policy=RetryPolicy(
                initial_interval=timedelta(seconds=1),
                maximum_interval=timedelta(seconds=30),
                maximum_attempts=3
            )
        )
        return result

2. Activities

Activities are individual units of work within workflows. They:

Interact with external systems (APIs, databases, services)
Can be retried automatically on failure
Support timeouts and cancellation
Execute in a separate process from workflows

from temporalio import activity

@activity.defn
async def fetch_market_data_activity(symbol: str, timeframe: str) -> dict:
    """
    Activity that fetches market data from external APIs.
    This can fail and will be automatically retried by Temporal.
    """
    # Import external dependencies inside activity
    import requests

    # Fetch market data from API
    response = requests.get(
        f"https://api.marketdata.com/v1/quote/{symbol}",
        params={"timeframe": timeframe},
        timeout=30
    )
    response.raise_for_status()

    return {
        "symbol": symbol,
        "price_data": response.json(),
        "timestamp": response.headers.get("timestamp")
    }

3. Workers

Workers are processes that execute workflow and activity code. They:

Poll the Temporal service for tasks
Execute workflows and activities
Can be scaled horizontally
Support multiple task queues

from temporalio.client import Client
from temporalio.worker import Worker

async def run_worker():
    # Connect to Temporal
    client = await Client.connect(
        "temporal-server:7233",
        namespace="default"
    )

    # Create worker
    worker = Worker(
        client,
        task_queue="trading-queue",
        workflows=[TradingWorkflow],
        activities=[fetch_market_data_activity]
    )

    # Run worker (blocks until stopped)
    await worker.run()

Workflow Patterns

Temporal supports various workflow patterns to handle complex business processes. Each pattern has specific use cases, performance characteristics, and architectural considerations. Let's explore the most important ones in detail:

1. Sequential Execution

Overview: The simplest and most common pattern where activities execute one after another, with each step depending on the previous step's output. This creates a linear pipeline where data flows sequentially through the workflow.

Architecture:

Characteristics:

Execution Time: Sum of all activity execution times (no parallelism)
Dependencies: Each activity depends on the previous one
Error Handling: Failure in any step stops the workflow
State Management: State accumulates as data passes through each step
Use When: Steps have strict dependencies, data must be processed in order, or when parallel execution isn't possible

Performance Considerations:

Total execution time = sum of all activity durations
No parallelism benefits
Simple to understand and debug
Best for pipelines where each step transforms data for the next

@workflow.defn
class SequentialTradingWorkflow:
    @workflow.run
    async def run(self, trade_request: dict) -> dict:
        # Step 1: Fetch Market Data
        market_data = await workflow.execute_activity(
            fetch_market_data_activity,
            args=[trade_request["symbol"], trade_request["timeframe"]],
            start_to_close_timeout=timedelta(seconds=180)
        )

        # Step 2: Technical Analysis (uses market data)
        trade_request["market_data"] = market_data["price_data"]
        analysis_result = await workflow.execute_activity(
            technical_analysis_activity,
            args=[trade_request],
            start_to_close_timeout=timedelta(seconds=120)
        )

        # Step 3: Risk Assessment (uses analysis result)
        trade_request["analysis"] = analysis_result["signals"]
        risk_result = await workflow.execute_activity(
            risk_assessment_activity,
            args=[trade_request],
            start_to_close_timeout=timedelta(seconds=120)
        )

        # Step 4: Execute Trade (uses all previous results)
        trade_request["risk_score"] = risk_result["risk_score"]
        execution_result = await workflow.execute_activity(
            execute_trade_activity,
            args=[trade_request],
            start_to_close_timeout=timedelta(seconds=120)
        )

        return execution_result

Use Case: When each step depends on the previous step's output, like a data processing pipeline, trading workflows where analysis depends on market data, or ETL processes.

Best Practices:

Use appropriate timeouts for each activity
Implement retry policies for transient failures
Pass only necessary data between activities to reduce state size
Log intermediate results for debugging
Consider breaking into sub-workflows if the sequence becomes too long

2. Parallel Execution

Overview: Execute multiple independent activities concurrently to maximize throughput and minimize total execution time. This pattern is essential for performance optimization when activities don't depend on each other.

Architecture:

Characteristics:

Execution Time: Maximum of all activity execution times (not the sum)
Dependencies: Activities are independent and can run simultaneously
Error Handling: Can use return_exceptions=True to handle partial failures
State Management: Results are collected and aggregated after all complete
Use When: Activities are independent, you need to fetch data from multiple sources, or when performance is critical

Performance Considerations:

Total execution time ≈ max(activity durations) + aggregation time
Significant performance improvement over sequential execution
Resource usage scales with number of parallel activities
Best for I/O-bound operations (API calls, database queries)

@workflow.defn
class ParallelWorkflow:
    @workflow.run
    async def run(self, data_sources: list) -> dict:
        # Define activity options
        activity_options = {
            "start_to_close_timeout": timedelta(seconds=120),
            "retry_policy": RetryPolicy(maximum_attempts=3)
        }

        # Execute multiple activities in parallel
        futures = []
        for source in data_sources:
            future = workflow.execute_activity(
                fetch_data_activity,
                args=[source],
                **activity_options
            )
            futures.append(future)

        # Wait for all activities to complete
        results = await asyncio.gather(*futures)

        # Aggregate results
        aggregated = await workflow.execute_activity(
            aggregate_data_activity,
            args=[results],
            start_to_close_timeout=timedelta(seconds=60)
        )

        return aggregated

Use Case: Fetching data from multiple sources, processing independent tasks, batch operations, multi-exchange trading data collection, or parallel analysis of different market segments.

Real-World Example: In a trading workflow, we could parallelize data collection from multiple exchanges:

# Parallel market data fetching from different exchanges
market_data_futures = [
    workflow.execute_activity(
        fetch_market_data_activity,
        args=[{"symbol": symbol, "exchange": "NYSE"}]
    ),
    workflow.execute_activity(
        fetch_market_data_activity,
        args=[{"symbol": symbol, "exchange": "NASDAQ"}]
    ),
    workflow.execute_activity(
        fetch_market_data_activity,
        args=[{"symbol": symbol, "exchange": "LSE"}]
    )
]

# Wait for all market data activities to complete
market_data_results = await asyncio.gather(*market_data_futures)

Best Practices:

Use asyncio.gather() with return_exceptions=True for error handling
Set appropriate timeouts for each parallel activity
Consider rate limiting when calling external APIs in parallel
Aggregate results efficiently to avoid large state objects
Monitor resource usage when scaling parallel activities

3. Sub-Workflows (Child Workflows)

Overview: Break complex workflows into smaller, reusable sub-workflows (child workflows). This pattern promotes modularity, reusability, and separation of concerns. Child workflows can be independently versioned, monitored, and scaled.

Architecture:

Characteristics:

Modularity: Each child workflow is a self-contained unit
Reusability: Child workflows can be reused across multiple parent workflows
Isolation: Child workflow failures don't necessarily fail the parent
Versioning: Child workflows can be versioned independently
Monitoring: Each child workflow has its own execution history
Use When: You need to modularize complex processes, create reusable components, or implement hierarchical workflows

Performance Considerations:

Child workflows execute independently and can be distributed across workers
Parent workflow waits for child completion (can be made async with signals)
Each child workflow maintains its own state and history
Useful for breaking down large workflows into manageable pieces

@workflow.defn
class OrderExecutionWorkflow:
    @workflow.run
    async def run(self, order_id: str) -> dict:
        # Execute child workflow for order validation
        validation_result = await workflow.execute_child_workflow(
            OrderValidationWorkflow.run,
            args=[order_id],
            id=f"validation-{order_id}"
        )

        if validation_result["status"] != "valid":
            return {"status": "failed", "reason": "validation_failed"}

        # Execute child workflow for risk check
        risk_result = await workflow.execute_child_workflow(
            RiskCheckWorkflow.run,
            args=[order_id],
            id=f"risk-{order_id}"
        )

        # Execute child workflow for trade execution
        execution_result = await workflow.execute_child_workflow(
            TradeExecutionWorkflow.run,
            args=[order_id],
            id=f"execution-{order_id}"
        )

        return {
            "order_id": order_id,
            "validation": validation_result,
            "risk_check": risk_result,
            "execution": execution_result
        }

@workflow.defn
class OrderValidationWorkflow:
    @workflow.run
    async def run(self, order_id: str) -> dict:
        # Order validation logic
        result = await workflow.execute_activity(
            validate_order_activity,
            args=[order_id]
        )
        return result

@workflow.defn
class RiskCheckWorkflow:
    @workflow.run
    async def run(self, order_id: str) -> dict:
        # Risk assessment logic
        result = await workflow.execute_activity(
            assess_risk_activity,
            args=[order_id]
        )
        return result

Use Case: Modularizing complex processes, creating reusable workflow components, implementing saga patterns, or when you need to orchestrate multiple independent workflows.

Best Practices:

Use descriptive workflow IDs for child workflows
Handle child workflow failures gracefully
Consider using signals for async child workflow communication
Version child workflows independently for safe deployments
Monitor child workflows separately for better observability

4. Conditional Execution

Overview: Execute different workflow paths based on runtime conditions, state, or external factors. This pattern enables dynamic workflow routing and adaptive behavior.

Architecture:

Characteristics:

Dynamic Routing: Workflow path determined at runtime
Conditional Logic: Based on activity results, external state, or user input
Branching: Multiple execution paths from a single decision point
Convergence: Paths may converge back to common activities
Use When: You need adaptive behavior, A/B testing, different processing based on data characteristics, or user-driven workflows

Performance Considerations:

Only one path executes, so performance depends on chosen path
Condition evaluation adds minimal overhead
Useful for avoiding unnecessary work in alternative paths
Can optimize by evaluating conditions early

@workflow.defn
class AdaptiveTradingWorkflow:
    @workflow.run
    async def run(self, trade_request: dict) -> dict:
        # Initial market analysis
        market_analysis = await workflow.execute_activity(
            analyze_market_activity,
            args=[trade_request]
        )

        # Conditional branching based on market conditions
        if market_analysis["volatility"] == "high":
            # Use conservative strategy
            result = await workflow.execute_activity(
                conservative_trading_activity,
                args=[trade_request, market_analysis]
            )
        elif market_analysis["volatility"] == "medium":
            # Use standard strategy
            result = await workflow.execute_activity(
                standard_trading_activity,
                args=[trade_request, market_analysis]
            )
        else:
            # Use aggressive strategy
            result = await workflow.execute_activity(
                aggressive_trading_activity,
                args=[trade_request, market_analysis]
            )

        return result

Use Case: Dynamic workflow routing, A/B testing, adaptive processing based on data characteristics, market condition-based trading strategies, or user preference-driven workflows.

Best Practices:

Keep condition evaluation deterministic
Document all possible execution paths
Handle all branches in error scenarios
Use clear, descriptive condition names
Consider using enums for condition values

5. Fan-Out/Fan-In Pattern

Overview: Distribute work across multiple activities (fan-out), then collect and aggregate results (fan-in). This pattern is ideal for processing large datasets, batch operations, or map-reduce scenarios.

Architecture:

Characteristics:

Fan-Out: Distribute work to multiple parallel activities
Fan-In: Collect results from all activities
Error Handling: Can handle partial failures gracefully
Scalability: Can process large numbers of tasks
Use When: Batch processing, map-reduce operations, processing large datasets in chunks, or parallel task execution

Performance Considerations:

Execution time ≈ max(task durations) + aggregation time
Scales well with number of tasks
Can process thousands of tasks in parallel
Resource usage scales with task count
Best for CPU-bound or I/O-bound batch operations

@workflow.defn
class FanOutFanInWorkflow:
    @workflow.run
    async def run(self, task_list: list) -> dict:
        # Fan-out: Distribute tasks across multiple activities
        futures = []
        for task in task_list:
            future = workflow.execute_activity(
                process_task_activity,
                args=[task],
                start_to_close_timeout=timedelta(seconds=300)
            )
            futures.append(future)

        # Fan-in: Wait for all tasks to complete
        results = await asyncio.gather(*futures, return_exceptions=True)

        # Process results (handle exceptions)
        successful_results = []
        failed_tasks = []

        for i, result in enumerate(results):
            if isinstance(result, Exception):
                failed_tasks.append({"task": task_list[i], "error": str(result)})
            else:
                successful_results.append(result)

        # Aggregate results
        summary = await workflow.execute_activity(
            aggregate_results_activity,
            args=[successful_results, failed_tasks]
        )

        return summary

Use Case: Batch processing, map-reduce operations, processing large datasets in chunks, processing multiple orders simultaneously, or parallel portfolio analysis.

Best Practices:

Use return_exceptions=True to handle partial failures
Implement retry logic for transient failures
Consider batching tasks to avoid overwhelming workers
Monitor task completion rates
Implement graceful degradation for partial failures

6. Saga Pattern (Compensating Transactions)

Overview: Execute a series of transactions with compensating actions (rollback operations) if any step fails. This pattern ensures data consistency in distributed systems where traditional ACID transactions aren't possible.

Architecture:

Characteristics:

Forward Path: Execute transactions in sequence
Compensation: Each transaction has a compensating action
Rollback: On failure, execute compensations in reverse order
Consistency: Ensures system remains in consistent state
Use When: Distributed transactions, order processing, multi-step processes requiring rollback, or financial operations

Performance Considerations:

Execution time includes both forward and potential compensation paths
Compensation adds overhead but ensures consistency
Critical for financial and transactional systems
Can be optimized by minimizing compensation complexity

@workflow.defn
class TradeExecutionSagaWorkflow:
    @workflow.run
    async def run(self, trade_order: dict) -> dict:
        completed_steps = []

        try:
            # Step 1: Reserve capital
            capital_result = await workflow.execute_activity(
                reserve_capital_activity,
                args=[trade_order]
            )
            completed_steps.append(("capital", capital_result))

            # Step 2: Place order
            order_result = await workflow.execute_activity(
                place_order_activity,
                args=[trade_order]
            )
            completed_steps.append(("order", order_result))

            # Step 3: Confirm execution
            execution_result = await workflow.execute_activity(
                confirm_execution_activity,
                args=[trade_order]
            )
            completed_steps.append(("execution", execution_result))

            return {"status": "success", "steps": completed_steps}

        except Exception as e:
            # Compensate: Undo completed steps in reverse order
            for step_name, step_result in reversed(completed_steps):
                try:
                    await workflow.execute_activity(
                        compensate_trade_activity,
                        args=[step_name, step_result]
                    )
                except Exception as comp_error:
                    workflow.logger.error(f"Compensation failed for {step_name}: {comp_error}")

            raise e

Use Case: Distributed transactions, order processing, trade execution with rollback, or any multi-step process requiring rollback.

Best Practices:

Design compensating actions to be idempotent
Store completed steps in workflow state
Execute compensations in reverse order
Handle compensation failures gracefully
Log all compensation actions for audit trails

7. Polling Pattern

Overview: Poll an external system or check a condition repeatedly until a desired state is reached or a timeout occurs. This pattern is useful when waiting for external processes or asynchronous operations to complete.

Architecture:

Characteristics:

Polling Loop: Repeatedly check condition with intervals
Timeout: Maximum time to wait before giving up
Interval: Time between polling attempts
Condition Check: Determines if polling should continue
Use When: Waiting for external processes, monitoring long-running tasks, checking resource availability, or waiting for async operations

Performance Considerations:

Execution time depends on how quickly condition is met
Polling interval affects responsiveness vs. resource usage
Use appropriate timeouts to avoid infinite loops
Consider exponential backoff for polling intervals
Best for waiting on external systems with unknown completion times

@workflow.defn
class PollingWorkflow:
    @workflow.run
    async def run(self, job_id: str) -> dict:
        max_attempts = 60  # Poll for up to 5 minutes (60 * 5 seconds)
        attempt = 0

        while attempt < max_attempts:
            # Check job status
            status = await workflow.execute_activity(
                check_job_status_activity,
                args=[job_id],
                start_to_close_timeout=timedelta(seconds=10)
            )

            if status["state"] == "completed":
                # Fetch results
                results = await workflow.execute_activity(
                    fetch_job_results_activity,
                    args=[job_id]
                )
                return results

            elif status["state"] == "failed":
                raise Exception(f"Job {job_id} failed: {status.get('error')}")

            # Wait before next poll
            await workflow.sleep(timedelta(seconds=5))
            attempt += 1

        raise TimeoutError(f"Job {job_id} did not complete within timeout")

Use Case: Waiting for external processes, monitoring long-running tasks, checking for resource availability, waiting for trade execution confirmation, or polling for order status.

Best Practices:

Use exponential backoff for polling intervals
Set reasonable timeouts to avoid infinite loops
Make condition checks efficient (avoid heavy operations)
Consider using signals instead of polling when possible
Log polling attempts for debugging

8. Event-Driven Pattern

Overview: Workflows that respond to external events or signals, enabling reactive and interactive workflows. This pattern allows workflows to wait for external input or events before proceeding.

Architecture:

Characteristics:

Reactive: Workflows respond to external events
Interactive: Can wait for human input or external systems
Flexible: Workflow behavior adapts based on received events
Use When: Human-in-the-loop workflows, waiting for external approvals, event-driven systems, or interactive processes

Example:

@workflow.defn
class EventDrivenTradingWorkflow:
    @workflow.run
    async def run(self, trade_request: dict) -> dict:
        # Initial analysis
        analysis = await workflow.execute_activity(
            analyze_market_activity,
            args=[trade_request]
        )

        # Wait for approval signal (can come from user or system)
        approval = await workflow.wait_condition(
            lambda: self.approved is not None,
            timeout=timedelta(minutes=5)
        )

        if self.approved:
            # Execute trade
            result = await workflow.execute_activity(
                execute_trade_activity,
                args=[trade_request, analysis]
            )
            return result
        else:
            return {"status": "rejected", "reason": "not_approved"}

    @workflow.signal
    def approve_trade(self, approved: bool):
        self.approved = approved

9. Chained Workflows Pattern

Overview: Execute workflows in a chain where the output of one workflow triggers or provides input to the next. This creates a pipeline of workflows, each handling a specific stage of processing.

Architecture:

Characteristics:

Pipeline: Workflows execute in sequence
Data Flow: Output of one workflow becomes input to next
Modularity: Each workflow handles a specific stage
Use When: Multi-stage processing, data transformation pipelines, or when you need to separate concerns across workflow boundaries

Pattern Comparison Summary

Pattern	Execution Time	Complexity	Use Case
Sequential	Sum of all activities	Low	Dependent steps, pipelines
Parallel	Max of activities	Medium	Independent tasks, performance
Sub-Workflows	Depends on children	Medium-High	Modularity, reusability
Conditional	Single path	Medium	Dynamic routing, adaptation
Fan-Out/Fan-In	Max + aggregation	Medium-High	Batch processing, map-reduce
Saga	Forward + compensation	High	Transactions, rollback
Polling	Variable (until condition)	Low-Medium	External waits, monitoring
Event-Driven	Variable (until event)	Medium	Interactive, reactive
Chained	Sum of workflows	Medium	Multi-stage pipelines

Temporal in Agentic AI

Agentic AI systems involve autonomous agents that make decisions and perform tasks over extended periods. Temporal provides an ideal foundation for building these systems.

Why Temporal for Agentic AI?

1. Reliability and Resilience

AI agents operate in unpredictable environments and may encounter:

Network failures
API rate limits
Model inference errors
Resource constraints

Temporal's durable execution ensures agent workflows recover automatically:

@workflow.defn
class AIAgentWorkflow:
    @workflow.run
    async def run(self, user_query: str) -> dict:
        # Agent reasoning step
        reasoning = await workflow.execute_activity(
            agent_reasoning_activity,
            args=[user_query],
            retry_policy=RetryPolicy(
                initial_interval=timedelta(seconds=2),
                maximum_interval=timedelta(seconds=60),
                maximum_attempts=5
            )
        )

        # Agent action step
        action_result = await workflow.execute_activity(
            agent_action_activity,
            args=[reasoning],
            retry_policy=RetryPolicy(maximum_attempts=3)
        )

        return action_result

2. Complex Decision-Making Orchestration

AI agents often need to coordinate multiple decision-making processes. Temporal's support for parallel execution and sub-workflows enables agents to:

Perform concurrent analysis
Delegate subtasks efficiently
Make decisions based on aggregated results

@workflow.defn
class MultiAgentOrchestrationWorkflow:
    @workflow.run
    async def run(self, task: dict) -> dict:
        # Parallel agent execution
        agent_futures = [
            workflow.execute_activity(
                research_agent_activity,
                args=[task]
            ),
            workflow.execute_activity(
                analysis_agent_activity,
                args=[task]
            ),
            workflow.execute_activity(
                planning_agent_activity,
                args=[task]
            )
        ]

        # Wait for all agents to complete
        agent_results = await asyncio.gather(*agent_futures)

        # Coordinator agent synthesizes results
        final_result = await workflow.execute_activity(
            coordinator_agent_activity,
            args=[agent_results]
        )

        return final_result

3. State Management Across Long-Running Processes

AI agents need to maintain context over extended periods. Temporal's event history provides:

Complete state persistence
Ability to resume from any point
Time-travel debugging
Audit trails

@workflow.defn
class LongRunningAgentWorkflow:
    def __init__(self):
        self.conversation_history = []
        self.agent_state = {}

    @workflow.query
    def get_conversation_history(self) -> list:
        """Query method to get current conversation state."""
        return self.conversation_history

    @workflow.run
    async def run(self, initial_message: str) -> dict:
        self.conversation_history.append({
            "role": "user",
            "content": initial_message
        })

        # Agent processes message
        response = await workflow.execute_activity(
            agent_process_activity,
            args=[self.conversation_history]
        )

        self.conversation_history.append({
            "role": "assistant",
            "content": response["content"]
        })

        # Workflow can continue indefinitely, maintaining state
        return {"status": "completed", "history": self.conversation_history}

4. Scalability for AI Workloads

As AI systems scale, Temporal's architecture supports:

Horizontal scaling through multiple workers
Load distribution across task queues
Efficient resource utilization

# Multiple workers can process different trading agent types
worker_market_data = Worker(
    client,
    task_queue="market-data-agents-queue",
    workflows=[MarketDataAgentWorkflow],
    activities=[market_data_activities]
)

worker_trading = Worker(
    client,
    task_queue="trading-agents-queue",
    workflows=[TradingAgentWorkflow],
    activities=[trading_activities]
)

Real-World Example: Automated Trading Agent System

Let's examine a real-world implementation that demonstrates Temporal's power in agentic AI for trading:

@workflow.defn
class AutomatedTradingWorkflow:
    """
    Automated Trading Workflow - Orchestrates multiple AI agents
    for comprehensive trading analysis and execution.
    """

    def __init__(self):
        self.progress_events = []
        self.trade_state = {}

    @workflow.query
    def get_progress(self) -> list:
        """Query method for real-time progress tracking."""
        return self.progress_events

    @workflow.query
    def get_trade_state(self) -> dict:
        """Query method for current trade state."""
        return self.trade_state

    @workflow.run
    async def run(
        self,
        symbol: str,
        strategy: str,
        user_id: str = "default-user"
    ) -> str:
        """
        Execute automated trading workflow with 4 sequential AI agents.
        """
        trade_request = {
            "symbol": symbol,
            "strategy": strategy,
            "user_id": user_id
        }
        all_results = []

        # Define retry policy for reliability
        retry_policy = RetryPolicy(
            initial_interval=timedelta(seconds=1),
            maximum_interval=timedelta(seconds=30),
            maximum_attempts=3,
            non_retryable_error_types=["ValueError", "InsufficientFundsError"]
        )

        # Agent 1: Market Data Collection
        self.progress_events.append({
            "event": "node_executing",
            "node": "market_data_collection",
            "type": "data_fetch"
        })

        result1 = await workflow.execute_activity(
            fetch_market_data_activity,
            args=[symbol, "1h"],
            start_to_close_timeout=timedelta(seconds=180),
            retry_policy=retry_policy
        )

        trade_request["market_data"] = result1["price_data"]
        self.trade_state["market_data"] = result1
        all_results.append({"agent": "Market Data Collector", "result": result1})

        # Agent 2: Technical Analysis
        self.progress_events.append({
            "event": "node_executing",
            "node": "technical_analysis",
            "type": "analysis"
        })

        result2 = await workflow.execute_activity(
            technical_analysis_activity,
            args=[trade_request],
            start_to_close_timeout=timedelta(seconds=120),
            retry_policy=retry_policy
        )

        trade_request["signals"] = result2["signals"]
        self.trade_state["analysis"] = result2
        all_results.append({"agent": "Technical Analyst", "result": result2})

        # Agent 3: Risk Assessment
        self.progress_events.append({
            "event": "node_executing",
            "node": "risk_assessment",
            "type": "risk_analysis"
        })

        result3 = await workflow.execute_activity(
            risk_assessment_activity,
            args=[trade_request],
            start_to_close_timeout=timedelta(seconds=120),
            retry_policy=retry_policy
        )

        trade_request["risk_score"] = result3["risk_score"]
        self.trade_state["risk"] = result3
        all_results.append({"agent": "Risk Analyst", "result": result3})

        # Agent 4: Trade Execution (only if risk is acceptable)
        if result3["risk_score"] < 0.7:  # Risk threshold
            self.progress_events.append({
                "event": "node_executing",
                "node": "trade_execution",
                "type": "execution"
            })

            result4 = await workflow.execute_activity(
                execute_trade_activity,
                args=[trade_request],
                start_to_close_timeout=timedelta(seconds=120),
                retry_policy=retry_policy
            )

            self.trade_state["execution"] = result4
            all_results.append({"agent": "Trade Executor", "result": result4})
        else:
            all_results.append({
                "agent": "Trade Executor",
                "result": {"status": "skipped", "reason": "risk_too_high"}
            })

        return json.dumps({
            "summary": "Trading workflow completed",
            "agents": all_results,
            "workflow_id": workflow.info().workflow_id,
            "final_state": self.trade_state
        })

Key Features Demonstrated:

Sequential Agent Execution: Each agent uses the previous agent's output
Progress Tracking: Real-time progress events via workflow queries
State Queries: Ability to query current trade state at any time
Automatic Retries: Retry policy ensures reliability
Conditional Execution: Trade execution only if risk is acceptable
Timeout Management: Appropriate timeouts for each activity type
State Persistence: Workflow state maintained across failures

Enhanced Version: Parallel Agent Execution

We can enhance the workflow to use parallel execution for independent tasks:

@workflow.defn
class EnhancedTradingWorkflow:
    @workflow.run
    async def run(self, symbol: str, strategy: str) -> dict:
        # Parallel data collection from multiple exchanges
        market_data_futures = [
            workflow.execute_activity(
                fetch_market_data_activity,
                args=[{"symbol": symbol, "exchange": "NYSE"}]
            ),
            workflow.execute_activity(
                fetch_market_data_activity,
                args=[{"symbol": symbol, "exchange": "NASDAQ"}]
            ),
            workflow.execute_activity(
                fetch_market_data_activity,
                args=[{"symbol": symbol, "exchange": "LSE"}]
            )
        ]

        # Wait for all market data to complete
        market_data_results = await asyncio.gather(*market_data_futures)

        # Parallel analysis on different timeframes
        analysis_futures = [
            workflow.execute_activity(
                technical_analysis_activity,
                args=[{"data": result, "timeframe": "1h"}]
            ) for result in market_data_results
        ]

        analysis_results = await asyncio.gather(*analysis_futures)

        # Aggregate signals and execute trade
        aggregated_signals = await workflow.execute_activity(
            aggregate_signals_activity,
            args=[{"analyses": analysis_results}]
        )

        # Execute trade based on aggregated signals
        trade_result = await workflow.execute_activity(
            execute_trade_activity,
            args=[{"symbol": symbol, "signals": aggregated_signals}]
        )

        return trade_result

Advanced Patterns for Agentic AI

1. Agent Decision Trees

Use sub-workflows to create complex decision trees for AI agents:

@workflow.defn
class TradingStrategyRouterWorkflow:
    @workflow.run
    async def run(self, trade_request: dict) -> dict:
        # Initial market analysis
        market_analysis = await workflow.execute_activity(
            analyze_market_activity,
            args=[trade_request]
        )

        # Route to appropriate trading strategy based on market conditions
        if market_analysis["market_condition"] == "trending":
            return await workflow.execute_child_workflow(
                TrendFollowingStrategyWorkflow.run,
                args=[trade_request, market_analysis]
            )
        elif market_analysis["market_condition"] == "ranging":
            return await workflow.execute_child_workflow(
                MeanReversionStrategyWorkflow.run,
                args=[trade_request, market_analysis]
            )
        else:
            return await workflow.execute_child_workflow(
                AdaptiveStrategyWorkflow.run,
                args=[trade_request, market_analysis]
            )

2. Agent Collaboration

Orchestrate multiple agents working together:

@workflow.defn
class CollaborativeTradingWorkflow:
    @workflow.run
    async def run(self, trade_request: dict) -> dict:
        # Agent 1: Market Data Collection
        market_data = await workflow.execute_activity(
            market_data_agent_activity,
            args=[trade_request]
        )

        # Agent 2: Technical Analysis (uses market data)
        technical_analysis = await workflow.execute_activity(
            technical_analysis_agent_activity,
            args=[market_data]
        )

        # Agent 3: Risk Assessment (uses both)
        risk_assessment = await workflow.execute_activity(
            risk_assessment_agent_activity,
            args=[market_data, technical_analysis]
        )

        # Agent 4: Trade Decision (uses all)
        trade_decision = await workflow.execute_activity(
            trade_decision_agent_activity,
            args=[market_data, technical_analysis, risk_assessment]
        )

        return trade_decision

3. Adaptive Agent Workflows

Adjust workflow behavior based on intermediate results:

@workflow.defn
class AdaptiveTradingAgentWorkflow:
    @workflow.run
    async def run(self, trade_request: dict) -> dict:
        max_iterations = 5

        for iteration in range(max_iterations):
            # Agent processes trade request
            result = await workflow.execute_activity(
                trading_agent_process_activity,
                args=[trade_request]
            )

            # Check if trade is complete
            if result["status"] == "executed" or result["status"] == "rejected":
                return result

            # Adapt based on result
            if result["needs_more_data"]:
                # Fetch additional market data
                additional_data = await workflow.execute_activity(
                    fetch_additional_data_activity,
                    args=[result["missing_data"]]
                )
                trade_request["additional_data"] = additional_data

            # Adjust strategy based on market feedback
            if result["strategy_adjustment_needed"]:
                trade_request["strategy"] = result["recommended_strategy"]

            # Update trade request for next iteration
            trade_request["iteration"] = iteration + 1
            trade_request["previous_result"] = result

        return {"status": "max_iterations_reached", "final_result": trade_request}

Benefits of Temporal for Agentic AI

1. Reliability

Automatic retries on failures
State persistence across crashes
Guaranteed execution completion
No data loss

2. Observability

Complete execution history
Real-time progress tracking
Time-travel debugging
Built-in monitoring

3. Scalability

Horizontal worker scaling
Load distribution
Efficient resource utilization
Support for high-throughput systems

4. Developer Experience

Write workflows as code
Familiar programming constructs
Strong typing support
Comprehensive SDKs

5. Production-Ready

Battle-tested at scale
Used by major companies
Active community
Regular updates and improvements

Best Practices

1. Activity Design

Keep activities focused and single-purpose
Import external dependencies inside activities
Use appropriate timeouts
Handle errors gracefully

2. Workflow Design

Keep workflows deterministic
Avoid random number generation (use activities)
Don't use system time directly (use workflow.now())
Use queries for read-only state access

3. Error Handling

Define appropriate retry policies
Use non-retryable error types for permanent failures
Implement compensation logic for sagas
Log errors appropriately

4. Performance

Use parallel execution when possible
Set appropriate timeouts
Monitor workflow execution times
Optimize activity execution

5. Testing

Test workflows in isolation
Mock activities for unit testing
Use Temporal's test framework
Test failure scenarios

Conclusion

Temporal provides a robust foundation for building reliable, scalable agentic AI systems. Its durable execution model, comprehensive workflow patterns, and excellent developer experience make it an ideal choice for orchestrating complex AI agent workflows.

Key takeaways:

Temporal abstracts complexity: Focus on business logic, not infrastructure
Multiple workflow patterns: Sequential, parallel, sub-workflows, and more
Perfect for agentic AI: Reliability, state management, and scalability
Production-ready: Battle-tested and widely adopted
Rich ecosystem: Multiple language SDKs and active community

Whether you're building simple sequential workflows or complex multi-agent systems, Temporal provides the reliability and observability needed for production AI applications.

Resources

Code Execution with MCP: Building More Efficient AI Agents

Akash Thakur — Mon, 17 Nov 2025 05:40:00 +0000

How integrating code execution with the Model Context Protocol is revolutionizing AI agent efficiency and scalability

Introduction

As AI agents become increasingly sophisticated, they're connecting to more tools, APIs, and data sources than ever before. The Model Context Protocol (MCP) has emerged as a powerful open standard for connecting AI agents to external systems. However, traditional MCP implementations face significant challenges around context window management and token efficiency.

In this article, we'll explore how integrating code execution with MCP addresses these challenges, enabling more efficient, scalable, and capable AI agents. We'll dive into real-world examples, implementation patterns, and best practices.

Understanding the Model Context Protocol (MCP)

The Model Context Protocol (MCP) is an open standard that enables AI agents to seamlessly connect to external systems, tools, and data sources. Think of it as a universal adapter that allows your AI agent to interact with:

Cloud Services: Google Drive, Salesforce, AWS, Azure
Databases: PostgreSQL, MongoDB, Redis
APIs: REST APIs, GraphQL endpoints
File Systems: Local and remote file storage
Development Tools: GitHub, Jira, Slack

MCP provides a standardized way for agents to discover, invoke, and interact with these resources, making it easier to build powerful, multi-tool AI applications.

The Problem: Challenges with Traditional MCP

Challenge 1: Context Window Overload

The Issue:
When an AI agent connects to multiple tools via MCP, all tool definitions must be loaded into the model's context window. As the number of tools grows, this consumes increasingly large amounts of context space.

Real-World Impact:

An agent connecting to 50 tools might need 20,000+ tokens just for tool definitions
This leaves less room for actual conversation and task execution
Response times increase as the model processes more tokens
Costs escalate with every additional tool

Example Scenario:

Agent Context Window (128K tokens):
├── Tool Definitions: 25,000 tokens (50 tools × 500 tokens avg)
├── Conversation History: 15,000 tokens
├── System Instructions: 5,000 tokens
└── Available for Work: 83,000 tokens (65% of capacity)

Challenge 2: Excessive Token Consumption

The Issue:
Every intermediate result from tool calls must pass through the model's context window. For large datasets or multi-step operations, this creates a token consumption spiral.

Real-World Impact:

Downloading a 10,000-row spreadsheet consumes tokens for the entire dataset
Processing results requires the data to be in context multiple times
Complex workflows can easily exceed context limits
Costs multiply with each intermediate result

Example Scenario:

Task: Process sales data from Google Drive and update Salesforce

Without Code Execution:
1. Download spreadsheet: 50,000 tokens (entire file in context)
2. Model processes data: 50,000 tokens (still in context)
3. Update Salesforce: 5,000 tokens (results in context)
Total: 105,000 tokens consumed

The Solution: Code Execution with MCP

Code execution with MCP allows agents to run code in a secure, sandboxed environment. Instead of passing all data through the model's context, agents can:

Execute code to process data locally
Filter and transform results before returning
Maintain state across operations
Build reusable functions and workflows

Key Benefits with Real-World Examples

1. Progressive Disclosure

What It Means:
Agents can load tool definitions on-demand rather than loading everything upfront. Tools are presented as code on a filesystem, allowing the model to read definitions only when needed.

Example:

# Traditional Approach (All tools loaded)
tools = [
    "gdrive.read_file",
    "gdrive.write_file", 
    "gdrive.list_files",
    "salesforce.get_record",
    "salesforce.update_record",
    "salesforce.create_lead",
    # ... 44 more tools
]
# Result: 25,000 tokens consumed upfront

# With Code Execution (Progressive Disclosure)
# Agent searches for what it needs:
search_tools("salesforce")
# Result: Only loads Salesforce tools (2,000 tokens)

# Later, when needed:
search_tools("gdrive")
# Result: Loads Google Drive tools (1,500 tokens)

Benefits:

80% reduction in initial context load
Faster agent startup times
Lower costs per interaction
More context available for actual work

2. Efficient Data Handling

What It Means:
Agents can filter, transform, and process large datasets within the execution environment before returning results to the model.

Example: Sales Data Processing

Without Code Execution:

# Agent downloads entire spreadsheet
spreadsheet = gdrive.get_sheet("sales_data_2024.xlsx")
# Result: 10,000 rows × 20 columns = 200,000 data points
# Tokens consumed: ~50,000 tokens

# Model must process all data in context
# Agent asks: "Find all sales over $10,000 in Q4"
# Model processes 10,000 rows in context
# Additional tokens: ~30,000 tokens
# Total: 80,000 tokens

With Code Execution:

# Agent executes code to filter data
code = """
import pandas as pd

# Load spreadsheet
df = pd.read_excel('sales_data_2024.xlsx')

# Filter: Q4 sales over $10,000
q4_sales = df[
    (df['Quarter'] == 'Q4') & 
    (df['Amount'] > 10000)
]

# Return only filtered results
return q4_sales.to_dict('records')
"""

result = execute_code(code)
# Result: Only 150 rows returned
# Tokens consumed: ~5,000 tokens
# Savings: 75,000 tokens (94% reduction)

Real-World Use Case:
An agent processing customer feedback from a CSV file with 100,000 rows can:

Filter by sentiment (positive/negative)
Extract only relevant columns
Aggregate statistics
Return a summary instead of raw data

Result: 95% token reduction, faster processing, lower costs.

3. Enhanced Control Flow

What It Means:
Agents can use familiar programming constructs (loops, conditionals, error handling) to execute complex workflows in a single step.

Example: Multi-Step Data Pipeline

Without Code Execution:

# Step 1: Get data from API
api_data = api_client.get_data()
# Tokens: 5,000

# Step 2: Process in model context
# Model decides: "I need to transform this"
transformed = transform_data(api_data)
# Tokens: 5,000 (data still in context)

# Step 3: Save to database
db.save(transformed)
# Tokens: 2,000

# Step 4: Update status
status.update("completed")
# Tokens: 1,000

# Total: 13,000 tokens, 4 round trips

With Code Execution:

code = """
import requests
import json
from database import save_to_db
from status import update_status

try:
    # Step 1: Get data
    response = requests.get('https://api.example.com/data')
    data = response.json()

    # Step 2: Transform
    transformed = [
        {
            'id': item['id'],
            'name': item['name'].upper(),
            'value': item['value'] * 1.1
        }
        for item in data
    ]

    # Step 3: Save to database
    save_to_db(transformed)

    # Step 4: Update status
    update_status("completed")

    return {"status": "success", "records": len(transformed)}

except Exception as e:
    update_status("failed")
    return {"status": "error", "message": str(e)}
"""

result = execute_code(code)
# Total: 2,000 tokens (only result returned)
# Savings: 11,000 tokens (85% reduction)
# Single round trip instead of 4

Benefits:

85% reduction in token usage
75% reduction in latency (fewer round trips)
Better error handling
More reliable workflows

4. Privacy Preservation

What It Means:
Intermediate results remain within the execution environment by default. Sensitive data only enters the model's context if explicitly returned.

Example: Processing Sensitive Customer Data

Without Code Execution:

# Agent needs to process customer PII
customers = database.get_customers()
# All customer data (names, emails, SSNs) enters context
# Tokens: 20,000
# Privacy Risk: HIGH - All PII in model context

# Model processes data
anonymized = anonymize(customers)
# Tokens: 20,000 (still in context)

With Code Execution:

code = """
from database import get_customers
from anonymize import hash_pii

# Get customers (stays in execution environment)
customers = get_customers()

# Anonymize within execution environment
anonymized = [
    {
        'id': hash_pii(c['id']),
        'segment': c['segment'],
        'value': c['lifetime_value']
    }
    for c in customers
]

# Only return anonymized data
return anonymized
"""

result = execute_code(code)
# Only anonymized data enters context
# Tokens: 3,000 (only summary data)
# Privacy Risk: LOW - No PII in model context

Use Cases:

Healthcare data processing (HIPAA compliance)
Financial data analysis (PCI compliance)
Personal information handling (GDPR compliance)

5. State Persistence and Skill Development

What It Means:
Agents can maintain state across operations by writing to files, and can save reusable code implementations for future tasks.

Example: Building a Reusable Data Analysis Function

# Agent creates a reusable function
code = """
import pandas as pd
import json

def analyze_sales_data(file_path, filters):
    '''
    Reusable function for sales data analysis
    '''
    df = pd.read_excel(file_path)

    # Apply filters
    for key, value in filters.items():
        df = df[df[key] == value]

    # Calculate metrics
    metrics = {
        'total_sales': df['amount'].sum(),
        'average_sale': df['amount'].mean(),
        'count': len(df),
        'top_product': df.groupby('product')['amount'].sum().idxmax()
    }

    return metrics

# Save function for future use
with open('utils/sales_analyzer.py', 'w') as f:
    f.write(inspect.getsource(analyze_sales_data))
"""

execute_code(code)

# Later, agent can reuse the function
code = """
from utils.sales_analyzer import analyze_sales_data

result = analyze_sales_data(
    'sales_2024.xlsx',
    {'quarter': 'Q4', 'region': 'North'}
)

return result
"""

result = execute_code(code)

Benefits:

Agents build a library of reusable functions
State persists across sessions
Agents become more capable over time
Reduced token usage for repeated operations

Implementation Examples

Example 1: E-commerce Data Pipeline

Scenario: An agent needs to sync product data from Shopify to a local database, filter for high-value items, and generate a report.

code = """
import requests
import sqlite3
from datetime import datetime

# Step 1: Fetch products from Shopify API
shopify_url = "https://your-store.myshopify.com/admin/api/2024-01/products.json"
headers = {"X-Shopify-Access-Token": "your_token"}
response = requests.get(shopify_url, headers=headers)
products = response.json()['products']

# Step 2: Filter high-value products (price > $100)
high_value = [
    p for p in products 
    if float(p.get('variants', [{}])[0].get('price', 0)) > 100
]

# Step 3: Save to local database
conn = sqlite3.connect('products.db')
cursor = conn.cursor()

for product in high_value:
    cursor.execute('''
        INSERT OR REPLACE INTO products 
        (id, title, price, updated_at)
        VALUES (?, ?, ?, ?)
    ''', (
        product['id'],
        product['title'],
        product['variants'][0]['price'],
        datetime.now().isoformat()
    ))

conn.commit()
conn.close()

# Step 4: Generate summary
summary = {
    'total_products': len(products),
    'high_value_count': len(high_value),
    'sync_timestamp': datetime.now().isoformat()
}

return summary
"""

result = execute_code(code)
# Returns: {"total_products": 5000, "high_value_count": 1200, ...}
# Only summary in context, not 5000 products!

Token Savings:

Without code execution: ~150,000 tokens (all products in context)
With code execution: ~500 tokens (only summary)
99.7% reduction

Example 2: Multi-Source Data Aggregation

Scenario: An agent needs to aggregate data from multiple sources (Google Sheets, Airtable, and a REST API) and create a unified report.

code = """
import pandas as pd
import requests
from gdrive import get_sheet
from airtable import get_records

# Step 1: Get data from Google Sheets
sheet_data = get_sheet('1ABC123', 'Sheet1')
df_sheets = pd.DataFrame(sheet_data)

# Step 2: Get data from Airtable
airtable_data = get_records('app123', 'table1')
df_airtable = pd.DataFrame(airtable_data)

# Step 3: Get data from REST API
api_response = requests.get('https://api.example.com/data')
api_data = api_response.json()
df_api = pd.DataFrame(api_data)

# Step 4: Merge and deduplicate
df_merged = pd.concat([df_sheets, df_airtable, df_api])
df_merged = df_merged.drop_duplicates(subset=['id'])

# Step 5: Calculate aggregated metrics
report = {
    'total_records': len(df_merged),
    'sources': {
        'google_sheets': len(df_sheets),
        'airtable': len(df_airtable),
        'api': len(df_api)
    },
    'date_range': {
        'earliest': df_merged['date'].min(),
        'latest': df_merged['date'].max()
    },
    'summary_stats': {
        'total_value': df_merged['value'].sum(),
        'average_value': df_merged['value'].mean()
    }
}

# Save merged data to file for future use
df_merged.to_csv('merged_data.csv', index=False)

return report
"""

result = execute_code(code)
# Complex multi-source aggregation in one step
# Only aggregated report in context, not raw data

Example 3: Intelligent File Processing

Scenario: An agent needs to process a directory of PDF files, extract text, summarize each, and create an index.

code = """
import os
from PyPDF2 import PdfReader
from pathlib import Path

def extract_and_summarize_pdf(file_path):
    '''Extract text and create summary for a PDF'''
    reader = PdfReader(file_path)
    text = ""
    for page in reader.pages:
        text += page.extract_text()

    # Simple summary: first 200 chars
    summary = text[:200] + "..." if len(text) > 200 else text

    return {
        'file': os.path.basename(file_path),
        'pages': len(reader.pages),
        'char_count': len(text),
        'summary': summary
    }

# Process all PDFs in directory
pdf_dir = Path('./documents')
pdf_files = list(pdf_dir.glob('*.pdf'))

index = []
for pdf_file in pdf_files:
    try:
        info = extract_and_summarize_pdf(pdf_file)
        index.append(info)
    except Exception as e:
        index.append({
            'file': pdf_file.name,
            'error': str(e)
        })

# Save index to file
import json
with open('pdf_index.json', 'w') as f:
    json.dump(index, f, indent=2)

# Return summary
return {
    'total_files': len(pdf_files),
    'processed': len([i for i in index if 'error' not in i]),
    'total_pages': sum(i.get('pages', 0) for i in index),
    'index_file': 'pdf_index.json'
}
"""

result = execute_code(code)
# Processed 50 PDFs, extracted text, created summaries
# Only summary statistics in context, not all PDF text

Security Considerations

Implementing code execution with MCP requires careful attention to security:

1. Sandboxing

# Example: Docker-based sandbox
sandbox_config = {
    'image': 'python:3.11-slim',
    'memory_limit': '512m',
    'cpu_limit': '1.0',
    'network': 'none',  # No network access
    'read_only': True,  # Read-only filesystem
    'timeout': 30  # 30 second timeout
}

2. Resource Limits

Memory: Limit to prevent memory exhaustion
CPU: Throttle CPU usage
Time: Set execution timeouts
Disk: Limit disk space usage

3. Code Validation

# Example: Validate code before execution
def validate_code(code):
    # Check for dangerous operations
    dangerous_patterns = [
        'import os',
        'subprocess',
        '__import__',
        'eval(',
        'exec('
    ]

    for pattern in dangerous_patterns:
        if pattern in code:
            raise SecurityError(f"Dangerous pattern detected: {pattern}")

    return True

4. Monitoring and Logging

Log all code executions
Monitor resource usage
Alert on suspicious patterns
Audit trail for compliance

Best Practices

1. Start Small

Begin with simple operations and gradually add complexity as you gain confidence in your security setup.

2. Use Type Hints and Validation

code = """
def process_data(data: list[dict]) -> dict:
    # Type hints help the model understand expected inputs/outputs
    return {'count': len(data)}
"""

3. Error Handling

Always include error handling in executed code:

code = """
try:
    result = risky_operation()
    return {'status': 'success', 'data': result}
except Exception as e:
    return {'status': 'error', 'message': str(e)}
"""

4. Document Functions

Well-documented functions help agents understand and reuse code:

code = """
def calculate_metrics(data):
    '''
    Calculate key metrics from sales data.

    Args:
        data: List of dictionaries with 'amount' and 'date' keys

    Returns:
        Dictionary with 'total', 'average', and 'count'
    '''
    # Implementation...
"""

When to Use Code Execution with MCP

Use code execution when:

✅ Processing large datasets (>1000 rows)
✅ Working with multiple data sources
✅ Need for complex data transformations
✅ Privacy-sensitive operations
✅ Repetitive operations that can be automated
✅ Multi-step workflows

Stick with traditional MCP when:

❌ Simple, single-step operations
❌ Small datasets (<100 rows)
❌ One-off queries
❌ Security requirements prohibit code execution

Conclusion

Integrating code execution with MCP represents a significant leap forward in AI agent efficiency and capability. By enabling:

Progressive disclosure of tools
Efficient data processing within execution environments
Enhanced control flow with familiar programming constructs
Privacy preservation through local processing
State persistence and skill development

We can build AI agents that are not only more powerful but also more cost-effective, scalable, and privacy-conscious.

The key is to balance the benefits against the implementation complexity and security considerations. With proper sandboxing, resource limits, and monitoring, code execution with MCP can transform how we build AI applications.

As AI agents continue to evolve, this pattern will become increasingly important for building production-ready, scalable AI systems that can handle real-world complexity while maintaining efficiency and security.

TOON vs JSON: The New Format Designed for AI

Akash Thakur — Sun, 09 Nov 2025 03:39:07 +0000

How a novel data format is saving developers 30-60% on LLM token costs

If you've been working with Large Language Models, you've probably noticed something: feeding data to AI isn't free. Every JSON object you pass through an API costs tokens, and those tokens add up fast. Enter TOON (Token-Oriented Object Notation), a new serialization format designed specifically to solve this problem.

The Token Tax Problem

Let's start with a real example. Imagine you're building an app that sends employee data to an LLM for analysis:

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin", "salary": 75000 },
    { "id": 2, "name": "Bob", "role": "user", "salary": 65000 },
    { "id": 3, "name": "Charlie", "role": "user", "salary": 70000 }
  ]
}

This JSON snippet consumes 257 tokens. Now look at the same data in TOON:

users[3]{id,name,role,salary}:
1,Alice,admin,75000
2,Bob,user,65000
3,Charlie,user,70000

Just 166 tokens — a 35% reduction. For this small example, the savings might seem trivial. But scale that to hundreds of API calls with thousands of records, and suddenly you're looking at real cost reductions.

What Makes TOON Different?

TOON borrows the best ideas from existing formats and optimizes them for LLM consumption:

1. Tabular Arrays: Declare Once, Use Many

The core insight behind TOON is simple: when you have uniform arrays of objects (same fields, same types), why repeat the keys for every single object?

JSON's Approach (repetitive):

[
  { "sku": "A1", "qty": 2, "price": 9.99 },
  { "sku": "B2", "qty": 1, "price": 14.50 }
]

TOON's Approach (efficient):

[2]{sku,qty,price}:
A1,2,9.99
B2,1,14.5

The schema is declared once in the header {sku,qty,price}, then each row is just CSV-style values. This is where TOON shines brightest.

2. Smart Quoting

TOON only quotes strings when absolutely necessary:

hello world → No quotes needed (inner spaces are fine)
hello 👋 world → No quotes (Unicode is safe)
"hello, world" → Quotes required (contains comma delimiter)
" padded " → Quotes required (leading/trailing spaces)

This minimal-quoting approach saves tokens while keeping the data unambiguous.

3. Indentation Over Brackets

Like YAML, TOON uses indentation instead of curly braces for nested structures:

JSON:

{
  "user": {
    "id": 123,
    "profile": {
      "name": "Ada"
    }
  }
}

TOON:

user:
  id: 123
  profile:
    name: Ada

Cleaner, more readable, and fewer tokens.

4. Explicit Array Lengths

TOON includes the array length in brackets ([N]), which actually helps LLMs understand and validate the structure:

tags[3]: admin,ops,dev

This explicit metadata reduces parsing errors when LLMs are generating or interpreting structured data.

Real-World Benchmarks

The TOON project ran comprehensive benchmarks across different data types and LLM models. Here's what they found:

Token Savings by Dataset

Dataset	JSON Tokens	TOON Tokens	Savings
GitHub Repos (100 records)	15,145	8,745	42.3%
Analytics (180 days)	10,977	4,507	58.9%
E-commerce Orders	257	166	35.4%

The sweet spot? Uniform tabular data — records with consistent schemas across many rows. The more repetitive your JSON keys, the more TOON can optimize.

LLM Comprehension

But token efficiency doesn't matter if the LLM can't understand the format. The benchmarks tested 4 different models (GPT-5 Nano, Claude Haiku, Gemini Flash, Grok) on 154 data retrieval questions:

TOON accuracy: 70.1%
JSON accuracy: 65.4%
Token reduction: 46.3%

TOON not only saves tokens but actually improves LLM accuracy. The explicit structure (array lengths, field declarations) helps models parse and validate data more reliably.

When Should You Use TOON?

TOON isn't meant to replace JSON everywhere. Think of it as a specialized tool for a specific job.

✅ Use TOON When:

Sending large datasets to LLMs (hundreds or thousands of records)
Working with uniform data structures (database query results, CSV exports, analytics)
Token costs are a significant concern
You're making frequent LLM API calls with structured data

❌ Stick With JSON When:

Building traditional REST APIs
Storing data in databases
Working with deeply nested or non-uniform data
You need universal compatibility with existing tools

As the TOON documentation puts it: "Use JSON programmatically, convert to TOON for LLM input."

How to Get Started

TOON is available as an npm package with a simple API:

import { encode, decode } from '@toon-format/toon'

const data = {
  items: [
    { sku: 'A1', qty: 2, price: 9.99 },
    { sku: 'B2', qty: 1, price: 14.5 }
  ]
}

// Convert to TOON
const toon = encode(data)
console.log(toon)
// items[2]{sku,qty,price}:
// A1,2,9.99
// B2,1,14.5

// Convert back to JSON
const restored = decode(toon)

There's also a CLI tool for quick conversions:

# Encode JSON to TOON
npx @toon-format/cli data.json -o data.toon

# Decode TOON to JSON
npx @toon-format/cli data.toon -o data.json

# Show token savings
npx @toon-format/cli data.json --stats

Alternative Delimiters

For even more token efficiency, you can use tab or pipe delimiters instead of commas:

// Tab-separated (often more token-efficient)
encode(data, { delimiter: '\t' })

// Pipe-separated
encode(data, { delimiter: '|' })

The Growing Ecosystem

While TOON is relatively new, the community is already building implementations across multiple languages:

Official: JavaScript/TypeScript, Python (in dev), Rust (in dev)
Community: PHP, Ruby, Go, Swift, Elixir, C++, Java, and more

The project maintains a comprehensive specification and conformance test suite to ensure compatibility across implementations.

The Bottom Line

TOON represents a shift in thinking about data formats. For decades, we've optimized for human readability and machine interoperability. Now, with LLMs consuming massive amounts of structured data, we need formats optimized for token efficiency and AI comprehension.

Is TOON going to replace JSON? No. But for the specific use case of feeding structured data to LLMs, it offers compelling advantages:

30-60% token savings on uniform tabular data
Better LLM accuracy thanks to explicit structure
Drop-in conversion from existing JSON workflows
Growing ecosystem with multi-language support

If you're building AI-powered applications that consume significant amounts of structured data, TOON is worth exploring. Your token budget will thank you.

Resources:

Have you tried TOON in your projects? What kind of token savings are you seeing? Share your experience in the comments.

React 19.2: The Game-Changing Update Every Developer Should Know About

Akash Thakur — Thu, 16 Oct 2025 05:32:03 +0000

React just dropped another bombshell! 🚀 React 19.2, released on October 1, 2025, is packed with features that will fundamentally change how we build React applications. As someone who's been diving deep into these new capabilities, I'm excited to share what makes this release so special and why you should care.

If you've ever struggled with managing component state during navigation, dealt with annoying effect dependency issues, or wanted better performance insights, React 19.2 has answers. Let's explore these game-changing features together!

What Makes React 19.2 Special?

This isn't just another incremental update. React 19.2 introduces features that solve real problems we face every day:

🎯 Smarter Component Management with the new <Activity /> component
🔧 Cleaner Effects with useEffectEvent hook
⚡ Better Performance Insights with Chrome DevTools integration
🚀 Faster Loading with Partial Pre-rendering
🛠️ Enhanced Developer Experience across the board

Let's dive into each of these and see how they can transform your React development experience.

The Big Four: Features That Will Change Your React Game

Before we dive deep, let me give you the TL;DR of what's coming:

🎯 `<Activity />` Component

Think of it as "smart conditional rendering" that keeps your components alive but hidden, perfect for pre-loading next pages or maintaining form state.

🔧 `useEffectEvent` Hook

Finally! A solution to the "my effect re-runs when it shouldn't" problem that's been plaguing React developers forever.

⚡ Performance Tracks

Chrome DevTools now shows you exactly what React is doing internally - no more guessing why your app is slow.

🚀 Partial Pre-rendering

The holy grail of web performance: serve static content instantly while streaming dynamic content as it's ready.

Now, let's explore each of these with real examples you can use today!

🎯 The `<Activity />` Component: Smart Component Management

Have you ever built a multi-step form and lost all the user's input when they navigated back? Or created a tab interface where switching tabs felt sluggish? The <Activity /> component is here to solve these exact problems!

The Problem We've All Faced

Picture this: You're building an e-commerce checkout flow. The user fills out their shipping information, moves to payment, then realizes they made a mistake and goes back. Boom! All their form data is gone because you used conditional rendering.

Or maybe you've built a dashboard with multiple tabs. Each tab loads data from an API, but switching between tabs feels slow because you're re-fetching data every time.

Sound familiar? Here's how <Activity /> changes the game:

// 😞 The old way - data and state lost on navigation
function CheckoutFlow() {
  const [step, setStep] = useState(1);

  return (
    <div>
      {step === 1 && <ShippingForm />}     {/* State lost when hidden */}
      {step === 2 && <PaymentForm />}      {/* Re-renders from scratch */}
      {step === 3 && <ReviewOrder />}      {/* API calls repeated */}
    </div>
  );
}

// 🚀 The new way - intelligent state management
function CheckoutFlow() {
  const [step, setStep] = useState(1);

  return (
    <div>
      <Activity mode={step === 1 ? 'visible' : 'hidden'}>
        <ShippingForm />     {/* State preserved when hidden */}
      </Activity>

      <Activity mode={step === 2 ? 'visible' : 'hidden'}>
        <PaymentForm />      {/* Renders once, stays in memory */}
      </Activity>

      <Activity mode={step === 3 ? 'visible' : 'hidden'}>
        <ReviewOrder />      {/* Data cached, no re-fetching */}
      </Activity>
    </div>
  );
}

Understanding the Two Modes

visible Mode - Your component behaves exactly like normal:

✅ Renders and displays to users
✅ Effects run as expected
✅ Updates happen immediately
✅ Fully interactive

hidden Mode - This is where the magic happens:

🔄 Component stays in memory (state preserved!)
🚫 Hidden from view (no DOM rendering)
⏸️ Effects are paused (cleanup functions run)
⏳ Updates are deferred until React has free time
🚫 No user interaction possible

Real-World Example 1: Netflix-Style Navigation

Ever notice how Netflix feels instant when you browse? Here's how you could build something similar:

function VideoApp() {
  const [currentCategory, setCurrentCategory] = useState('trending');

  return (
    <div>
      {/* Always show current category */}
      <Activity mode="visible">
        <CategoryView category={currentCategory} />
      </Activity>

      {/* Pre-load popular categories users often visit */}
      <Activity mode="hidden">
        <CategoryView category="action" />
      </Activity>

      <Activity mode="hidden">
        <CategoryView category="comedy" />
      </Activity>

      <CategoryNavigation 
        current={currentCategory}
        onChange={setCurrentCategory} 
      />
    </div>
  );
}

When users click "Action", it appears instantly because it's already loaded!

Real-World Example 2: Social Media Feed

Here's how you might implement Instagram-style story navigation:

function StoryViewer({ stories, currentIndex }) {
  return (
    <div className="story-container">
      {/* Current story */}
      <Activity mode="visible">
        <Story story={stories[currentIndex]} />
      </Activity>

      {/* Pre-load next story for smooth transitions */}
      {stories[currentIndex + 1] && (
        <Activity mode="hidden">
          <Story story={stories[currentIndex + 1]} />
        </Activity>
      )}

      {/* Keep previous story in memory for back navigation */}
      {stories[currentIndex - 1] && (
        <Activity mode="hidden">
          <Story story={stories[currentIndex - 1]} />
        </Activity>
      )}
    </div>
  );
}

Real-World Example 3: Complex Dashboard Tabs

Perfect for admin dashboards where each tab has expensive computations:

function AdminDashboard() {
  const [activeTab, setActiveTab] = useState('analytics');

  return (
    <div>
      <TabNavigation active={activeTab} onChange={setActiveTab} />

      <Activity mode={activeTab === 'analytics' ? 'visible' : 'hidden'}>
        <AnalyticsDashboard />  {/* Expensive charts and calculations */}
      </Activity>

      <Activity mode={activeTab === 'users' ? 'visible' : 'hidden'}>
        <UserManagement />      {/* Large data tables */}
      </Activity>

      <Activity mode={activeTab === 'reports' ? 'visible' : 'hidden'}>
        <ReportsPanel />        {/* Heavy data processing */}
      </Activity>
    </div>
  );
}

When Should You Use `<Activity />`?

✅ Perfect for:

Multi-step forms (preserve user input)
Tab interfaces (instant switching)
Navigation pre-loading (faster page loads)
Modal dialogs (keep expensive modals ready)
Dashboard widgets (maintain state between views)

❌ Avoid for:

Simple show/hide components (use regular conditional rendering)
Components that are rarely revisited
Very lightweight components (overhead isn't worth it)

🔧 `useEffectEvent`: Finally, Clean Effects!

If you've been using React for a while, you've definitely hit this frustrating scenario: you have an effect that needs to access some props or state, but you don't want the effect to re-run every time those values change. Enter useEffectEvent - the hero we've been waiting for!

The Frustrating Problem We've All Faced

Let me paint a picture that'll make you nod in recognition. You're building a chat application:

function ChatRoom({ roomId, theme, userId }) {
  useEffect(() => {
    const connection = createConnection(serverUrl, roomId);

    connection.on('connected', () => {
      // You want to show a notification with the current theme
      showNotification('Connected!', theme);

      // And maybe log some analytics
      analytics.track('chat_connected', { userId, theme });
    });

    connection.connect();
    return () => connection.disconnect();
  }, [roomId, theme, userId]); // 😱 This is a nightmare!
}

What happens? Every time the user changes their theme (dark/light mode) or any user data updates, your chat disconnects and reconnects! That's terrible UX.

The dilemma:

✅ Include theme and userId in dependencies → Chat reconnects unnecessarily
❌ Don't include them → Linter yells at you, and you might use stale values
🤷‍♂️ Disable the linter → You're living dangerously and might introduce bugs

Sound familiar? This is where useEffectEvent saves the day!

The `useEffectEvent` Solution

Here's how useEffectEvent elegantly solves this problem:

function ChatRoom({ roomId, theme, userId }) {
  // 🎉 Create "Effect Events" that always see the latest values
  const onConnected = useEffectEvent(() => {
    showNotification('Connected!', theme);
    analytics.track('chat_connected', { userId, theme });
  });

  const onMessage = useEffectEvent((message) => {
    // Always uses latest theme for message styling
    displayMessage(message, theme);
  });

  useEffect(() => {
    const connection = createConnection(serverUrl, roomId);

    connection.on('connected', onConnected);
    connection.on('message', onMessage);

    connection.connect();
    return () => connection.disconnect();
  }, [roomId]); // 🎯 Only roomId! Theme and userId changes won't reconnect
}

The magic: onConnected and onMessage always have access to the latest theme and userId, but they're not dependencies of the effect. Theme changes won't cause reconnections!

More Real-World Examples

Let me show you some scenarios where useEffectEvent will save your sanity:

Example 1: E-commerce Product Tracking

function ProductPage({ productId, userId, theme, currency }) {
  const trackProductView = useEffectEvent(() => {
    // Always uses latest user preferences without refetching product
    analytics.track('product_viewed', {
      productId,
      userId,
      theme,
      currency,
      timestamp: Date.now()
    });
  });

  const trackAddToCart = useEffectEvent((product) => {
    analytics.track('add_to_cart', {
      ...product,
      userId,
      currency // Always current currency
    });
  });

  useEffect(() => {
    // Only refetch when productId changes, not when theme/currency changes
    fetchProductData(productId).then(product => {
      setProduct(product);
      trackProductView(); // Track with latest user preferences
    });
  }, [productId]); // 🎯 Theme and currency changes won't refetch!
}

Example 2: Real-time Notifications

function NotificationSystem({ userId, preferences, theme }) {
  const showNotification = useEffectEvent((notification) => {
    // Always respects current user preferences and theme
    if (preferences.enabled) {
      toast(notification.message, {
        theme: theme,
        position: preferences.position,
        duration: preferences.duration
      });
    }
  });

  useEffect(() => {
    // Connect to WebSocket only once
    const ws = new WebSocket(`/notifications/${userId}`);

    ws.onmessage = (event) => {
      const notification = JSON.parse(event.data);
      showNotification(notification); // Uses latest preferences!
    };

    return () => ws.close();
  }, [userId]); // Only reconnect when user changes, not preferences!
}

Example 3: Auto-save with User Preferences

function DocumentEditor({ documentId, content, autoSaveInterval, userId }) {
  const saveDocument = useEffectEvent(() => {
    // Always uses latest content and user context
    api.saveDocument({
      id: documentId,
      content,
      userId,
      timestamp: Date.now()
    });
  });

  useEffect(() => {
    // Auto-save timer only restarts when interval changes
    const timer = setInterval(saveDocument, autoSaveInterval);
    return () => clearInterval(timer);
  }, [autoSaveInterval]); // Content changes won't restart the timer!
}

The Rules of `useEffectEvent`

✅ Perfect for:

Event handlers inside effects
Analytics and logging functions
Callbacks that need latest values
Notification systems
Any function that's "event-like" but fired from an effect

❌ Don't use for:

Core effect logic that should re-run
Functions that are part of the effect's main purpose
Just to silence the linter (that's dangerous!)

🚨 Important Rules:

Never include Effect Events in dependency arrays
Only use in the same component as the effect
Think of them as "events" not "effects"

⚡ Performance Tracks: X-Ray Vision for Your React App

Ever wondered what React is actually doing when your app feels slow? The new Performance Tracks in Chrome DevTools are like having X-ray vision into React's internal workings. No more guessing why your app is sluggish!

What You Get

React 19.2 adds two powerful new tracks to Chrome DevTools:

🔄 Scheduler Track - Shows React's priority system in action:

Blocking Priority: User interactions (clicks, typing) - highest priority
Transition Priority: Updates inside startTransition - can be interrupted
Background Priority: Low-priority updates - runs when nothing else is happening

🧩 Components Track - Shows component-level performance:

Mount: When components are first created
Update: When components re-render
Effects: When useEffect hooks run
Blocked: When rendering is paused (this is normal!)

Real Example: Debugging a Slow Search

Let's say you built a search feature that feels sluggish:

function SearchApp() {
  const [query, setQuery] = useState('');
  const [results, setResults] = useState([]);

  // 😞 This makes typing feel slow
  const handleSearch = (newQuery) => {
    setQuery(newQuery);
    // Heavy search operation blocks typing
    performExpensiveSearch(newQuery).then(setResults);
  };

  return (
    <div>
      <SearchInput onChange={handleSearch} />
      <SearchResults results={results} />
    </div>
  );
}

What Performance Tracks would show:

⚛ Scheduler
├─ Blocking: User typing (2ms) ← This should be fast
├─ Blocking: Search operation (150ms) ← This is blocking typing!
└─ Background: Update results (20ms)

⚛ Components  
├─ SearchInput (Update: 2ms)
├─ SearchResults (Blocked: waiting for search) ← Blocking other work
└─ SearchResults (Mount: 45ms) ← Heavy rendering

The fix using startTransition:

function SearchApp() {
  const [query, setQuery] = useState('');
  const [results, setResults] = useState([]);

  // 🚀 This keeps typing responsive
  const handleSearch = (newQuery) => {
    setQuery(newQuery); // Immediate update for input

    // Move expensive work to transition priority
    startTransition(() => {
      performExpensiveSearch(newQuery).then(setResults);
    });
  };

  return (
    <div>
      <SearchInput onChange={handleSearch} />
      <SearchResults results={results} />
    </div>
  );
}

Now Performance Tracks shows:

⚛ Scheduler
├─ Blocking: User typing (2ms) ← Fast and responsive!
├─ Transition: Search operation (150ms) ← Can be interrupted
└─ Transition: Update results (20ms) ← Non-blocking

⚛ Components
├─ SearchInput (Update: 2ms) ← Always responsive
└─ SearchResults (Update: 20ms) ← Smooth updates

How to Use Performance Tracks

Open Chrome DevTools (F12)
Go to Performance tab
Click Record (or Ctrl+E)
Use your React app (click, type, navigate)
Stop recording
Look for the ⚛ tracks in the timeline

What to Look For

🚨 Red Flags:

Long blocking tasks during user interactions
Components that take >16ms to render (causes frame drops)
Effects running too frequently
Background work interfering with user interactions

✅ Good Signs:

User interactions complete in <16ms
Heavy work happens in transition priority
Effects run only when necessary
Smooth 60fps animations

🚀 Partial Pre-rendering: The Holy Grail of Web Performance

Imagine if you could serve your website's shell instantly (like a static site) while streaming in dynamic content as it becomes ready. That's exactly what Partial Pre-rendering does - it's like having the best of Static Site Generation AND Server-Side Rendering!

The Problem with Current Approaches

Static Site Generation (SSG):

✅ Lightning fast initial load
❌ Can't handle user-specific content
❌ Requires rebuild for any changes

Server-Side Rendering (SSR):

✅ Handles dynamic content
❌ Slow initial load (waits for all data)
❌ One slow database query blocks everything

Client-Side Rendering (CSR):

✅ Highly interactive
❌ Terrible initial load time
❌ Poor SEO

How Partial Pre-rendering Changes Everything

Partial Pre-rendering lets you have your cake and eat it too:

Pre-render the shell (navigation, layout, static content)
Serve it instantly from a CDN
Stream dynamic parts as they become available

Real-World Example: E-commerce Product Page

function ProductPage({ productId }) {
  return (
    <div>
      {/* ⚡ Static parts - pre-rendered and served instantly */}
      <Header />
      <Navigation />
      <Breadcrumbs />

      {/* 🔄 Dynamic parts - streamed in as ready */}
      <Suspense fallback={<ProductSkeleton />}>
        <ProductDetails productId={productId} />
      </Suspense>

      <Suspense fallback={<ReviewsSkeleton />}>
        <ProductReviews productId={productId} />
      </Suspense>

      <Suspense fallback={<RecommendationsSkeleton />}>
        <ProductRecommendations productId={productId} />
      </Suspense>

      {/* ⚡ Static parts - pre-rendered */}
      <Footer />
    </div>
  );
}

What happens:

0ms: User sees header, navigation, breadcrumbs, and loading skeletons
50ms: Product details stream in (fast database query)
200ms: Reviews appear (slower API call)
500ms: Recommendations load (complex ML computation)

Implementation Example

Step 1: Pre-render the shell

import { prerender } from 'react-dom/server';

async function buildProductPageShell() {
  const controller = new AbortController();

  const { prelude, postponed } = await prerender(
    <ProductPage productId="123" />, 
    { signal: controller.signal }
  );

  // Save postponed state for later
  await saveToCache('product-123-postponed', postponed);

  // Serve this shell from CDN
  return prelude;
}

Step 2: Resume with dynamic content

import { resume } from 'react-dom/server';

app.get('/api/product/:id/resume', async (req, res) => {
  const postponed = await getFromCache(`product-${req.params.id}-postponed`);

  // Resume rendering with fresh data
  const stream = await resume(<ProductPage productId={req.params.id} />, postponed);

  stream.pipe(res);
});

Advanced Example: User Dashboard

function UserDashboard({ userId }) {
  return (
    <div>
      {/* ⚡ Pre-rendered shell */}
      <DashboardHeader />
      <Sidebar />

      <main>
        {/* 🔄 User-specific content streams in */}
        <Suspense fallback={<WelcomeSkeleton />}>
          <WelcomeMessage userId={userId} />
        </Suspense>

        <div className="dashboard-grid">
          <Suspense fallback={<StatsSkeleton />}>
            <UserStats userId={userId} />
          </Suspense>

          <Suspense fallback={<ActivitySkeleton />}>
            <RecentActivity userId={userId} />
          </Suspense>

          <Suspense fallback={<NotificationsSkeleton />}>
            <Notifications userId={userId} />
          </Suspense>
        </div>
      </main>
    </div>
  );
}

The magic: Anonymous users see the shell instantly. Logged-in users see their personalized content stream in progressively!

🛠️ Server Components: `cacheSignal` for Smart Resource Management

If you're using React Server Components, cacheSignal is a game-changer for managing expensive operations. It's like having a smart cleanup crew that knows when to stop work that's no longer needed.

The Problem: Wasted Server Resources

Picture this: You're building a server component that fetches user data. A user starts loading a page, but then navigates away before it finishes. Without cacheSignal, your server keeps working on that abandoned request, wasting resources.

import { cache } from 'react';

// This caches fetch requests
const cachedFetch = cache(fetch);

async function UserProfile({ userId }) {
  // 😞 This keeps running even if user navigates away
  const response = await cachedFetch(`/api/users/${userId}`);
  const user = await response.json();

  return <div>{user.name}</div>;
}

The Solution: Smart Cleanup with `cacheSignal`

import { cache, cacheSignal } from 'react';

const cachedFetch = cache(fetch);

async function UserProfile({ userId }) {
  try {
    // 🚀 This automatically aborts if cache lifetime ends
    const response = await cachedFetch(`/api/users/${userId}`, {
      signal: cacheSignal()
    });

    const user = await response.json();
    return <div>{user.name}</div>;
  } catch (error) {
    if (error.name === 'AbortError') {
      // Request was cleanly aborted - no problem!
      return null;
    }
    throw error; // Re-throw other errors
  }
}

Real-World Example: E-commerce Recommendations

import { cache, cacheSignal } from 'react';

const cachedRecommendations = cache(async (userId, productId) => {
  // This could be an expensive ML computation
  const signal = cacheSignal();

  const response = await fetch('/api/recommendations', {
    method: 'POST',
    signal,
    body: JSON.stringify({ userId, productId }),
    headers: { 'Content-Type': 'application/json' }
  });

  return response.json();
});

async function ProductRecommendations({ userId, productId }) {
  const recommendations = await cachedRecommendations(userId, productId);

  return (
    <div className="recommendations">
      <h3>You might also like</h3>
      {recommendations.map(product => (
        <ProductCard key={product.id} product={product} />
      ))}
    </div>
  );
}

Advanced Example: Database Queries with Cleanup

import { cache, cacheSignal } from 'react';

const cachedQuery = cache(async (sql, params) => {
  const signal = cacheSignal();

  // Set up cleanup for database connection
  const connection = await database.connect();

  signal.addEventListener('abort', () => {
    connection.close(); // Clean up database connection
  });

  try {
    return await connection.query(sql, params);
  } finally {
    if (!signal.aborted) {
      connection.close();
    }
  }
});

async function OrderHistory({ userId }) {
  const orders = await cachedQuery(
    'SELECT * FROM orders WHERE user_id = ? ORDER BY created_at DESC LIMIT 10',
    [userId]
  );

  return (
    <div>
      {orders.map(order => (
        <OrderCard key={order.id} order={order} />
      ))}
    </div>
  );
}

How to Use `cacheSignal`

import { cache, cacheSignal } from 'react';

const cachedFetch = cache(fetch);

async function UserProfile({ userId }) {
  try {
    // The signal will abort the request if the cache lifetime ends
    const response = await cachedFetch(`/api/users/${userId}`, {
      signal: cacheSignal()
    });

    const user = await response.json();
    return <div>{user.name}</div>;
  } catch (error) {
    if (error.name === 'AbortError') {
      // Request was aborted because cache lifetime ended
      return null;
    }
    throw error;
  }
}

Real-World Example: Database Queries

import { cache, cacheSignal } from 'react';

const cachedQuery = cache(async (query, params) => {
  const controller = new AbortController();

  // Listen for cache signal
  cacheSignal().addEventListener('abort', () => {
    controller.abort();
  });

  return await database.query(query, params, {
    signal: controller.signal
  });
});

async function ProductList({ category }) {
  const products = await cachedQuery(
    'SELECT * FROM products WHERE category = ?',
    [category]
  );

  return (
    <div>
      {products.map(product => (
        <ProductCard key={product.id} product={product} />
      ))}
    </div>
  );
}

Benefits

Resource Efficiency: Stops unnecessary work when results aren't needed
Better Performance: Reduces server load and improves response times
Cleaner Code: Automatic cleanup without manual signal management

Important Notes

Only works with React Server Components
Must be used with the cache function
Automatically handles cleanup when cache lifetime ends

🔄 Notable Changes: The Polish That Makes Everything Better

React 19.2 also includes several improvements that make the overall experience smoother. Let me highlight the ones that will impact your day-to-day development:

Smoother Suspense Loading

The Problem: Previously, when you had multiple Suspense boundaries, they would reveal content one by one as it became ready. This created a jarring, sequential loading experience.

The Fix: React 19.2 now batches Suspense boundary reveals for a short time, so more content appears together.

function ProductPage() {
  return (
    <div>
      {/* These now reveal together instead of one-by-one */}
      <Suspense fallback={<ProductSkeleton />}>
        <ProductInfo />
      </Suspense>

      <Suspense fallback={<ReviewsSkeleton />}>
        <ProductReviews />
      </Suspense>

      <Suspense fallback={<RecommendationsSkeleton />}>
        <ProductRecommendations />
      </Suspense>
    </div>
  );
}

Before: Content appeared sequentially, causing layout shifts

After: Content appears in coordinated batches, creating smoother loading

Better ESLint Support

The ESLint plugin now supports flat config (the new standard) and includes React Compiler-powered rules:

// New flat config (recommended)
import reactHooks from 'eslint-plugin-react-hooks';

export default [
  {
    plugins: { 'react-hooks': reactHooks },
    rules: reactHooks.configs.recommended.rules
  }
];

Bonus: The linter now understands useEffectEvent and won't complain when you don't include it in dependency arrays!

Web Streams in Node.js

You can now use Web Streams APIs in Node.js environments (though Node Streams are still recommended for performance):

// Now works in Node.js!
import { renderToReadableStream } from 'react-dom/server';

const stream = await renderToReadableStream(<App />);

🚀 Getting Started: Your Action Plan

Ready to dive in? Here's how to start using React 19.2 today:

Step 1: Upgrade Your Dependencies

npm install react@19.2 react-dom@19.2
npm install eslint-plugin-react-hooks@latest

Step 2: Start Small with `useEffectEvent`

Look for effects that have this pattern and convert them:

// 🔍 Look for this pattern in your codebase
useEffect(() => {
  // Some setup code
  const handler = () => {
    // Uses props/state but shouldn't cause re-runs
    doSomething(prop1, prop2);
  };

  // Event listener or similar
  element.addEventListener('event', handler);
  return () => element.removeEventListener('event', handler);
}, [prop1, prop2]); // These cause unnecessary re-runs

// ✅ Convert to this
const handleEvent = useEffectEvent(() => {
  doSomething(prop1, prop2);
});

useEffect(() => {
  const handler = () => handleEvent();
  element.addEventListener('event', handler);
  return () => element.removeEventListener('event', handler);
}, []); // Clean dependencies!

Step 3: Experiment with `<Activity />`

Try it in a tab interface or multi-step form:

function TabInterface() {
  const [activeTab, setActiveTab] = useState('tab1');

  return (
    <div>
      <TabButtons active={activeTab} onChange={setActiveTab} />

      {/* Convert your conditional rendering */}
      <Activity mode={activeTab === 'tab1' ? 'visible' : 'hidden'}>
        <ExpensiveTab1 />
      </Activity>

      <Activity mode={activeTab === 'tab2' ? 'visible' : 'hidden'}>
        <ExpensiveTab2 />
      </Activity>
    </div>
  );
}

Step 4: Profile with Performance Tracks

Open Chrome DevTools
Go to Performance tab
Record while using your app
Look for the ⚛ tracks
Identify performance bottlenecks

Step 5: Consider Partial Pre-rendering

If you're using SSR, explore pre-rendering your app shell:

// Identify what can be pre-rendered vs. what needs to be dynamic
function App() {
  return (
    <div>
      {/* Static - can be pre-rendered */}
      <Header />
      <Navigation />

      {/* Dynamic - render later */}
      <Suspense fallback={<ContentSkeleton />}>
        <DynamicContent />
      </Suspense>
    </div>
  );
}

🎯 Key Takeaways

React 19.2 isn't just another update - it's a collection of thoughtful improvements that solve real problems we face every day:

🎯 `<Activity />` → Smart component lifecycle management

Pre-load content without performance impact
Preserve state during navigation
Perfect for tabs, forms, and navigation

🔧 `useEffectEvent` → Clean, predictable effects

Access latest values without re-running effects
No more dependency array headaches
Cleaner, more maintainable code

⚡ Performance Tracks → X-ray vision for performance

See exactly what React is doing
Identify bottlenecks with precision
Optimize with confidence

🚀 Partial Pre-rendering → Best of all worlds

Instant static content delivery
Progressive dynamic content loading
Better Core Web Vitals scores

🔮 What's Next?

React 19.2 sets the stage for even more exciting developments. The Activity component hints at more sophisticated lifecycle management, Performance Tracks will likely get more detailed insights, and Partial Pre-rendering opens up new architectural possibilities.

My recommendation? Start experimenting with these features in side projects or non-critical parts of your main application. The patterns you learn now will give you a significant advantage as these features mature and become standard practice.

The React team continues to push the boundaries of what's possible in web development, and React 19.2 is a testament to their commitment to developer experience and application performance.

What feature are you most excited to try? Let me know in the comments below!

📚 Additional Resources

Thanks for reading! If you found this helpful, consider sharing it with your fellow React developers. Happy coding! 🚀

Supercharge Your Development Workflow: A Complete Guide to MCP Integration in Cursor AI

Akash Thakur — Tue, 19 Aug 2025 16:12:03 +0000

Introduction

The development landscape is rapidly evolving, and AI-powered coding assistants are becoming indispensable tools for modern developers. Cursor AI has emerged as a powerful IDE that seamlessly integrates AI capabilities into your development workflow. But what if you could extend its capabilities even further?

Enter the Model Context Protocol (MCP) – a revolutionary framework that enables AI models to interact with external tools, databases, APIs, and services in real-time. This integration transforms Cursor AI from a smart code editor into a comprehensive development ecosystem that can perform complex tasks like file operations, database queries, web scraping, and much more.

What is the Model Context Protocol (MCP)?

The Model Context Protocol is an open standard that enables AI models to securely connect to and interact with external data sources and tools. Think of it as a bridge that allows your AI assistant to:

📁 Access and manipulate files across your system
🗃️ Query databases and retrieve real-time data
🌐 Perform web scraping and API calls
🛠️ Execute system commands and scripts
📊 Process and analyze data from multiple sources
🔧 Integrate with third-party services and tools

Why MCP Matters for Developers

Enhanced Context Awareness: Your AI assistant gains access to real-time, relevant data
Streamlined Workflows: Automate repetitive tasks without leaving your IDE
Extended Capabilities: Access tools and services beyond the AI model's training data
Secure Integration: Controlled access to external resources with proper authentication
Customizable Experience: Add only the tools and data sources you need

Setting Up MCP in Cursor AI

Prerequisites

Before we begin, ensure you have:

Cursor AI installed and running
Python 3.8+ installed on your system
Administrative privileges for installing packages

Step 1: Access MCP Settings

To configure MCP servers in Cursor AI:

Open Cursor Settings:
- Navigate to File → Preferences → Cursor Settings
- Look for the "MCP" section in the settings panel

Understanding Configuration Types:
- Global Configuration: Applies to all projects (~/.cursor/mcp.json)
- Local Configuration: Project-specific (.cursor/mcp.json in project root)

Step 2: Your First MCP Server - Time Server

Let's start with a simple but useful example - adding a time server that can provide current time information.

Installation

# Install the MCP time server
pip install mcp-server-time

Configuration

Create or update your mcp.json file:

{
  "mcpServers": {
    "mcp_server_time": {
      "command": "python",
      "args": ["-m", "mcp_server_time", "--local-timezone=America/New_York"]
    }
  }
}

Step 3: Adding More Powerful MCP Servers

File System Operations Server

For file operations across your system:

pip install mcp-server-filesystem

{
  "mcpServers": {
    "filesystem": {
      "command": "python",
      "args": ["-m", "mcp_server_filesystem", "/path/to/your/project"]
    },
    "mcp_server_time": {
      "command": "python", 
      "args": ["-m", "mcp_server_time", "--local-timezone=America/New_York"]
    }
  }
}

SQLite Database Server

For database operations:

pip install mcp-server-sqlite

{
  "mcpServers": {
    "sqlite": {
      "command": "python",
      "args": ["-m", "mcp_server_sqlite", "/path/to/your/database.db"]
    },
    "filesystem": {
      "command": "python",
      "args": ["-m", "mcp_server_filesystem", "/path/to/your/project"]
    },
    "mcp_server_time": {
      "command": "python",
      "args": ["-m", "mcp_server_time", "--local-timezone=America/New_York"]
    }
  }
}

Real-World Example: MUI Documentation Server

Let's look at a practical example that many React developers would find useful - integrating MUI (Material-UI) documentation directly into Cursor AI.

Setting Up MUI MCP Server

Your configuration might look like this:

{
  "mcpServers": {
    "mui-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@mui/mcp@latest"]
    }
  }
}

This setup provides:

✅ 2 tools enabled for MUI component documentation
📚 Direct access to MUI component examples
🎨 Real-time component API reference
💡 Usage patterns and best practices

Using MCP Tools in Cursor AI

Step 1: Open the Chat Interface

Press Ctrl + L (or Cmd + L on Mac) to open Cursor AI's chat window.

Step 2: Invoke MCP Tools

You can now ask questions that leverage your MCP servers:

// Example queries you can now make:

"What's the current time in Tokyo?"
"Create a new React component using MUI's Button component"
"Show me the files in my project directory"
"Query the users table in my SQLite database"
"What are the latest MUI DataGrid props?"

Step 3: Execute and Observe

When Cursor AI detects that it needs to use an MCP tool:

It will show a "Run tool" button
Click the button to execute the MCP server
View the real-time results in the chat interface

Advanced MCP Configurations

Environment-Specific Configurations

Development Environment

{
  "mcpServers": {
    "filesystem": {
      "command": "python",
      "args": ["-m", "mcp_server_filesystem", "./src"],
      "env": {
        "ENV": "development"
      }
    },
    "dev-database": {
      "command": "python",
      "args": ["-m", "mcp_server_sqlite", "./dev.db"]
    }
  }
}

Production Environment

{
  "mcpServers": {
    "prod-monitor": {
      "command": "python",
      "args": ["-m", "mcp_server_monitoring"],
      "env": {
        "API_KEY": "${PROD_API_KEY}",
        "ENV": "production"
      }
    }
  }
}

Custom MCP Server Example

Create a custom weather server:

# weather_server.py
import asyncio
import requests
from mcp import Server
from mcp.types import Tool, TextContent

app = Server("weather-server")

@app.list_tools()
async def list_tools():
    return [
        Tool(
            name="get_weather",
            description="Get current weather for a city",
            inputSchema={
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name"}
                },
                "required": ["city"]
            }
        )
    ]

@app.call_tool()
async def call_tool(name: str, arguments: dict):
    if name == "get_weather":
        city = arguments["city"]
        # Your weather API logic here
        weather_data = get_weather_data(city)
        return [TextContent(type="text", text=f"Weather in {city}: {weather_data}")]

async def main():
    from mcp.server.stdio import stdio_server
    async with stdio_server() as server_context:
        await app.run()

if __name__ == "__main__":
    asyncio.run(main())

Configuration for your custom server:

{
  "mcpServers": {
    "weather": {
      "command": "python",
      "args": ["weather_server.py"]
    }
  }
}

Popular MCP Servers and Use Cases

1. Database Servers

PostgreSQL: mcp-server-postgres
SQLite: mcp-server-sqlite
MongoDB: mcp-server-mongo

2. Cloud Services

AWS: mcp-server-aws
Google Cloud: mcp-server-gcp
Azure: mcp-server-azure

3. Development Tools

Git: mcp-server-git
Docker: mcp-server-docker
Kubernetes: mcp-server-k8s

4. API Integrations

GitHub: mcp-server-github
Slack: mcp-server-slack
Jira: mcp-server-jira

Troubleshooting Common Issues

Issue 1: MCP Server Not Starting

Problem: Server fails to start or connect

Solutions:

# Check if the package is installed
pip list | grep mcp-server

# Verify Python path
which python

# Test the server manually
python -m mcp_server_time --help

Issue 2: Permission Denied Errors

Problem: Server can't access files or directories

Solutions:

Check file permissions: ls -la /path/to/file
Use absolute paths instead of relative paths
Ensure Cursor AI has necessary permissions

Issue 3: Tools Not Appearing

Problem: MCP tools don't show up in Cursor AI

Solutions:

Restart Cursor AI after configuration changes
Check mcp.json syntax with a JSON validator
Verify server logs for error messages

Issue 4: Environment Variables Not Working

Problem: Environment variables not passed correctly

Solution:

{
  "mcpServers": {
    "api-server": {
      "command": "python",
      "args": ["-m", "my_api_server"],
      "env": {
        "API_KEY": "${API_KEY}",
        "DEBUG": "true"
      }
    }
  }
}

Best Practices for MCP Integration

1. Security Considerations

✅ Use environment variables for sensitive data
✅ Limit file system access to specific directories
✅ Regularly update MCP server packages
❌ Never hardcode API keys in configuration files

2. Performance Optimization

✅ Use local configurations for project-specific servers
✅ Implement caching in custom servers
✅ Monitor server resource usage
❌ Don't run unnecessary servers globally

3. Configuration Management

✅ Version control your mcp.json files
✅ Document your MCP server purposes
✅ Use descriptive server names
✅ Group related servers logically

4. Development Workflow

✅ Test MCP servers independently before integration
✅ Use different configurations for different environments
✅ Keep server logs for debugging
✅ Regularly review and clean up unused servers

Real-World Use Cases

1. Full-Stack Development

{
  "mcpServers": {
    "database": {
      "command": "python",
      "args": ["-m", "mcp_server_postgres", "postgresql://localhost/myapp"]
    },
    "filesystem": {
      "command": "python", 
      "args": ["-m", "mcp_server_filesystem", "./src"]
    },
    "git": {
      "command": "python",
      "args": ["-m", "mcp_server_git", "."]
    }
  }
}

2. Data Science Projects

{
  "mcpServers": {
    "jupyter": {
      "command": "python",
      "args": ["-m", "mcp_server_jupyter"]
    },
    "datasets": {
      "command": "python",
      "args": ["-m", "mcp_server_filesystem", "./data"]
    },
    "database": {
      "command": "python",
      "args": ["-m", "mcp_server_sqlite", "./analysis.db"]
    }
  }
}

3. DevOps and Infrastructure

{
  "mcpServers": {
    "docker": {
      "command": "python",
      "args": ["-m", "mcp_server_docker"]
    },
    "aws": {
      "command": "python",
      "args": ["-m", "mcp_server_aws"],
      "env": {
        "AWS_PROFILE": "default"
      }
    },
    "monitoring": {
      "command": "python",
      "args": ["-m", "mcp_server_prometheus"]
    }
  }
}

The Future of MCP and AI Development

The integration of MCP with Cursor AI represents a significant step forward in AI-assisted development. As the ecosystem grows, we can expect:

🚀 More specialized servers for different domains and technologies
🔗 Better integration patterns and standardized configurations
🛡️ Enhanced security features and access controls
📈 Performance improvements and optimizations
🌐 Community-driven development of custom servers

Conclusion

The Model Context Protocol transforms Cursor AI from a smart code editor into a comprehensive development ecosystem. By following this guide, you've learned how to:

Set up and configure MCP servers in Cursor AI
Implement practical examples for common development tasks
Troubleshoot common issues and optimize performance
Apply best practices for security and maintainability

Whether you're building web applications, analyzing data, or managing infrastructure, MCP integration empowers you to work more efficiently and effectively. The combination of AI assistance with real-time access to tools and data sources creates a development experience that's both powerful and intuitive.

Start with simple servers like the time or filesystem servers, then gradually add more sophisticated integrations as your needs grow. The MCP ecosystem is rapidly expanding, and there's never been a better time to enhance your development workflow with these powerful tools.

Additional Resources

This comprehensive guide provides everything you need to get started with MCP integration in Cursor AI. Happy coding!

Found this article helpful? Share it with your fellow developers and help them supercharge their development workflow too!

From Promise.all() to asyncio.gather(): The Complete Guide to JavaScript-Style Async Patterns in Python

Akash Thakur — Mon, 09 Jun 2025 11:14:00 +0000

If you're a JavaScript developer diving into Python, you've probably wondered: "Where are the Promises?" Python's approach to asynchronous programming is different, but just as powerful. This comprehensive guide will show you how to translate every JavaScript Promise pattern into Python's asyncio equivalents.

The Fundamental Difference

JavaScript uses Promises as objects representing future values, while Python uses coroutines with the async/await syntax directly. Here's the basic comparison:

JavaScript Promise

function fetchData() {
    return new Promise((resolve) => {
        setTimeout(() => resolve("Data fetched!"), 1000);
    });
}

fetchData().then(result => console.log(result));

Python Coroutine

import asyncio

async def fetch_data():
    await asyncio.sleep(1)
    return "Data fetched!"

async def main():
    result = await fetch_data()
    print(result)

asyncio.run(main())

Promise.all() → asyncio.gather()

Use case: Wait for multiple asynchronous operations to complete simultaneously.

JavaScript Promise.all()

async function fetchMultipleData() {
    const urls = [
        'https://api.example.com/users',
        'https://api.example.com/posts',
        'https://api.example.com/comments'
    ];

    const promises = urls.map(url => fetch(url).then(r => r.json()));

    try {
        const results = await Promise.all(promises);
        console.log('All data fetched:', results);
        return results;
    } catch (error) {
        console.error('One or more requests failed:', error);
        throw error;
    }
}

Python asyncio.gather()

import asyncio
import aiohttp

async def fetch_data(session, url):
    async with session.get(url) as response:
        return await response.json()

async def fetch_multiple_data():
    urls = [
        'https://api.example.com/users',
        'https://api.example.com/posts',  
        'https://api.example.com/comments'
    ]

    async with aiohttp.ClientSession() as session:
        try:
            results = await asyncio.gather(
                *[fetch_data(session, url) for url in urls]
            )
            print('All data fetched:', results)
            return results
        except Exception as error:
            print('One or more requests failed:', error)
            raise error

# Alternative with error handling per task
async def fetch_multiple_data_safe():
    urls = [
        'https://api.example.com/users',
        'https://api.example.com/posts',  
        'https://api.example.com/comments'
    ]

    async with aiohttp.ClientSession() as session:
        results = await asyncio.gather(
            *[fetch_data(session, url) for url in urls],
            return_exceptions=True  # Don't fail on first exception
        )

        # Process results and exceptions
        for i, result in enumerate(results):
            if isinstance(result, Exception):
                print(f'Request {i} failed: {result}')
            else:
                print(f'Request {i} succeeded: {len(result)} items')

        return results

Key Differences:

JavaScript: Promise.all([...promises])
Python: asyncio.gather(*coroutines) (note the unpacking with *)
Error Handling: Python's return_exceptions=True lets you handle individual failures

Promise.race() → asyncio.wait() with FIRST_COMPLETED

Use case: Return the result of whichever operation completes first.

JavaScript Promise.race()

async function raceExample() {
    const slowPromise = new Promise(resolve => 
        setTimeout(() => resolve('Slow result'), 3000)
    );
    const fastPromise = new Promise(resolve => 
        setTimeout(() => resolve('Fast result'), 1000)
    );
    const timeoutPromise = new Promise((_, reject) => 
        setTimeout(() => reject(new Error('Timeout')), 2000)
    );

    try {
        const result = await Promise.race([slowPromise, fastPromise, timeoutPromise]);
        console.log('First result:', result); // "Fast result"
        return result;
    } catch (error) {
        console.error('First to complete was an error:', error);
        throw error;
    }
}

Python asyncio.wait() with FIRST_COMPLETED

async def slow_task():
    await asyncio.sleep(3)
    return 'Slow result'

async def fast_task():
    await asyncio.sleep(1)
    return 'Fast result'

async def timeout_task():
    await asyncio.sleep(2)
    raise Exception('Timeout')

async def race_example():
    tasks = [
        asyncio.create_task(slow_task()),
        asyncio.create_task(fast_task()),
        asyncio.create_task(timeout_task())
    ]

    try:
        done, pending = await asyncio.wait(
            tasks, 
            return_when=asyncio.FIRST_COMPLETED
        )

        # Get the first completed result
        first_task = list(done)[0]
        result = first_task.result()  # This will raise if the task failed

        print('First result:', result)  # "Fast result"

        # Cancel remaining tasks
        for task in pending:
            task.cancel()

        return result

    except Exception as error:
        print('First to complete was an error:', error)
        # Cancel all remaining tasks
        for task in tasks:
            if not task.done():
                task.cancel()
        raise error

# Alternative: Using asyncio.wait_for() for timeout patterns
async def race_with_timeout():
    try:
        result = await asyncio.wait_for(slow_task(), timeout=2.0)
        return result
    except asyncio.TimeoutError:
        print("Operation timed out")
        raise

Promise.allSettled() → asyncio.wait() with ALL_COMPLETED

Use case: Wait for all operations to complete, regardless of success or failure.

JavaScript Promise.allSettled()

async function allSettledExample() {
    const promises = [
        Promise.resolve('Success 1'),
        Promise.reject(new Error('Error 1')),
        Promise.resolve('Success 2'),
        new Promise(resolve => setTimeout(() => resolve('Delayed success'), 1000))
    ];

    const results = await Promise.allSettled(promises);

    results.forEach((result, index) => {
        if (result.status === 'fulfilled') {
            console.log(`Promise ${index} succeeded:`, result.value);
        } else {
            console.log(`Promise ${index} failed:`, result.reason);
        }
    });

    return results;
}

Python asyncio.wait() with ALL_COMPLETED

async def success_task_1():
    return 'Success 1'

async def error_task():
    raise Exception('Error 1')

async def success_task_2():
    return 'Success 2'

async def delayed_task():
    await asyncio.sleep(1)
    return 'Delayed success'

async def all_settled_example():
    tasks = [
        asyncio.create_task(success_task_1()),
        asyncio.create_task(error_task()),
        asyncio.create_task(success_task_2()),
        asyncio.create_task(delayed_task())
    ]

    done, pending = await asyncio.wait(tasks, return_when=asyncio.ALL_COMPLETED)

    results = []
    for i, task in enumerate(tasks):
        try:
            result = task.result()
            print(f'Task {i} succeeded: {result}')
            results.append({'status': 'fulfilled', 'value': result})
        except Exception as error:
            print(f'Task {i} failed: {error}')
            results.append({'status': 'rejected', 'reason': str(error)})

    return results

# Python 3.11+ TaskGroup approach (Recommended)
async def all_settled_with_task_group():
    results = []

    # Handle successful tasks
    async def safe_task(task_func, task_name):
        try:
            result = await task_func()
            return {'status': 'fulfilled', 'value': result, 'name': task_name}
        except Exception as error:
            return {'status': 'rejected', 'reason': str(error), 'name': task_name}

    tasks = [
        safe_task(success_task_1, 'task1'),
        safe_task(error_task, 'task2'),
        safe_task(success_task_2, 'task3'),
        safe_task(delayed_task, 'task4')
    ]

    results = await asyncio.gather(*tasks)

    for result in results:
        if result['status'] == 'fulfilled':
            print(f"{result['name']} succeeded: {result['value']}")
        else:
            print(f"{result['name']} failed: {result['reason']}")

    return results

Promise.any() → Custom Implementation

Use case: Return the first successful result, or fail if all operations fail.

JavaScript Promise.any()

async function anyExample() {
    const promises = [
        Promise.reject(new Error('Error 1')),
        Promise.reject(new Error('Error 2')),
        new Promise(resolve => setTimeout(() => resolve('Success!'), 1000)),
        Promise.reject(new Error('Error 3'))
    ];

    try {
        const result = await Promise.any(promises);
        console.log('First success:', result); // "Success!"
        return result;
    } catch (aggregateError) {
        console.error('All promises failed:', aggregateError.errors);
        throw aggregateError;
    }
}

Python Custom Implementation

class AllFailedError(Exception):
    def __init__(self, errors):
        self.errors = errors
        super().__init__(f"All operations failed: {errors}")

async def any_successful(*coroutines):
    """Python equivalent of Promise.any()"""
    if not coroutines:
        raise ValueError("At least one coroutine required")

    tasks = [asyncio.create_task(coro) for coro in coroutines]
    exceptions = []

    try:
        while tasks:
            done, tasks = await asyncio.wait(
                tasks, 
                return_when=asyncio.FIRST_COMPLETED
            )

            for task in done:
                try:
                    result = task.result()
                    # Cancel remaining tasks
                    for remaining_task in tasks:
                        remaining_task.cancel()
                    return result
                except Exception as error:
                    exceptions.append(error)

        # All tasks failed
        raise AllFailedError(exceptions)

    finally:
        # Ensure all tasks are cancelled
        for task in tasks:
            if not task.done():
                task.cancel()

# Usage example
async def failing_task_1():
    await asyncio.sleep(0.5)
    raise Exception('Error 1')

async def failing_task_2():
    await asyncio.sleep(0.3)
    raise Exception('Error 2')

async def success_task():
    await asyncio.sleep(1)
    return 'Success!'

async def failing_task_3():
    await asyncio.sleep(0.8)
    raise Exception('Error 3')

async def any_example():
    try:
        result = await any_successful(
            failing_task_1(),
            failing_task_2(), 
            success_task(),
            failing_task_3()
        )
        print('First success:', result)  # "Success!"
        return result
    except AllFailedError as error:
        print('All operations failed:', error.errors)
        raise error

Modern Python: TaskGroup (Python 3.11+)

Python 3.11 introduced TaskGroup, which provides a more structured way to handle concurrent tasks:

# TaskGroup automatically handles cleanup and error propagation
async def modern_concurrent_example():
    async with asyncio.TaskGroup() as tg:
        task1 = tg.create_task(fetch_data('https://api.example.com/users'))
        task2 = tg.create_task(fetch_data('https://api.example.com/posts'))
        task3 = tg.create_task(fetch_data('https://api.example.com/comments'))

    # All tasks completed successfully if we reach here
    return [task1.result(), task2.result(), task3.result()]

# TaskGroup with exception handling
async def modern_with_error_handling():
    try:
        async with asyncio.TaskGroup() as tg:
            task1 = tg.create_task(success_task_1())
            task2 = tg.create_task(error_task())  # This will cause TaskGroup to fail
            task3 = tg.create_task(success_task_2())

        # Won't reach here if any task fails
        return [task1.result(), task2.result(), task3.result()]

    except* Exception as eg:  # Exception groups (Python 3.11+)
        for error in eg.exceptions:
            print(f"Task failed: {error}")
        raise

Performance Comparison

Here's a practical example showing the performance benefits of concurrent execution:

import time
import asyncio
import aiohttp

async def time_comparison():
    urls = [
        'https://httpbin.org/delay/1',
        'https://httpbin.org/delay/1', 
        'https://httpbin.org/delay/1'
    ]

    # Sequential execution
    start_time = time.time()
    async with aiohttp.ClientSession() as session:
        results_sequential = []
        for url in urls:
            async with session.get(url) as response:
                results_sequential.append(await response.json())
    sequential_time = time.time() - start_time

    # Concurrent execution with asyncio.gather()
    start_time = time.time()
    async with aiohttp.ClientSession() as session:
        async def fetch(url):
            async with session.get(url) as response:
                return await response.json()

        results_concurrent = await asyncio.gather(*[fetch(url) for url in urls])
    concurrent_time = time.time() - start_time

    print(f"Sequential execution: {sequential_time:.2f} seconds")
    print(f"Concurrent execution: {concurrent_time:.2f} seconds") 
    print(f"Speedup: {sequential_time/concurrent_time:.2f}x")

# Expected output:
# Sequential execution: 3.15 seconds
# Concurrent execution: 1.12 seconds  
# Speedup: 2.81x

Quick Reference Cheat Sheet

JavaScript	Python	Use Case
`Promise.all(promises)`	`asyncio.gather(*coroutines)`	Wait for all, fail fast
`Promise.allSettled(promises)`	`asyncio.wait(tasks, return_when=ALL_COMPLETED)`	Wait for all, collect results/errors
`Promise.race(promises)`	`asyncio.wait(tasks, return_when=FIRST_COMPLETED)`	Return first to complete
`Promise.any(promises)`	Custom `any_successful()` function	Return first success
`new Promise((resolve, reject) => ...)`	`async def coroutine(): ...`	Create async operation
`promise.then().catch()`	`try: await coroutine() except:`	Handle results/errors
`Promise.resolve(value)`	`return value` (in async function)	Return immediate value
`Promise.reject(error)`	`raise error` (in async function)	Return immediate error

Best Practices

Use asyncio.gather() for the most common concurrent patterns - it's the closest equivalent to Promise.all()
Prefer TaskGroup (Python 3.11+) for better error handling and resource cleanup
Use return_exceptions=True with gather() when you want to handle individual failures
Always cancel pending tasks when using asyncio.wait() to prevent resource leaks
Use aiohttp instead of requests for HTTP operations in async code
Consider asyncio.wait_for() for simple timeout scenarios instead of implementing race conditions

Conclusion

While Python doesn't have Promises, its asyncio patterns are equally powerful and often more explicit about error handling and resource management. The key is understanding that Python's async/await syntax works directly with coroutines, eliminating the need for Promise wrapper objects.

Whether you're migrating from JavaScript or just learning async patterns, these equivalents will help you write efficient, concurrent Python code that scales beautifully.

Automate React App Deployment to AWS S3 with a Simple Bash Script

Akash Thakur — Wed, 29 May 2024 16:14:28 +0000

Deploying your web applications can often be a tedious and error-prone process, especially when done manually. However, with the power of automation, you can streamline this process and ensure consistent and reliable deployments every time. In this blog post, we'll explore a simple Bash script that automates the deployment of a React app to an AWS S3 bucket, making it publicly accessible as a static website.

Prerequisites
Before you begin, make sure you have the following:

AWS CLI installed and configured with the appropriate permissions.
Node.js and npm installed.
A React application ready to deploy.

Here's a breakdown of the Bash script that automates the deployment of a React app to an AWS S3 bucket:

#!/bin/bash
AWS_REGION="us-east-1"
# Build React app
npm install
npm run build
# Ask user for the bucket name
echo "Enter a unique bucket name: "
read  BUCKET_NAME
# Check if bucket exists
BUCKET_EXISTS=$(aws s3api head-bucket --bucket $BUCKET_NAME --query 'Code' --output text 2>/dev/null)
if [ "$BUCKET_EXISTS" = "200" ]; then
    echo "Bucket $BUCKET_NAME already exists. Do you want to sync the code? (y/n)"
    read SYNC_OPTION
    if [ "$SYNC_OPTION" == "y" ]; then
        # Upload build to S3
        aws s3 sync build/ s3://$BUCKET_NAME
        echo "Code synced to S3"
    fi
    echo "Your React app is deployed at: http://$BUCKET_NAME.s3-website.$AWS_REGION.amazonaws.com"
    echo "Open default browser"
    # Open default browser
    if [[ "$OSTYPE" == "darwin"* ]]; then
        open "http://$BUCKET_NAME.s3-website.$AWS_REGION.amazonaws.com"
    elif [[ "$OSTYPE" == "msys"* ]]; then
        start "http://$BUCKET_NAME.s3-website.$AWS_REGION.amazonaws.com"
    fi
else
    # Set bucket policy for public access
    BUCKET_POLICY='{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "PublicReadGetObject",
                "Effect": "Allow",
                "Principal": "*",
                "Action": [
                    "s3:GetObject"
                ],
                "Resource": [
                    "arn:aws:s3:::'$BUCKET_NAME'/*"
                ]
            }
        ]
    }'
    # Create S3 bucket
    aws s3api create-bucket --bucket $BUCKET_NAME --region $AWS_REGION
    echo "bucket created $BUCKET_NAME"
    # Enable static website hosting
    aws s3 website s3://$BUCKET_NAME/ --index-document index.html
    echo "Enable static website hosting"
    # Disable "Block all public access"
    aws s3api put-public-access-block --bucket $BUCKET_NAME --public-access-block-configuration "BlockPublicAcls=false,IgnorePublicAcls=false,BlockPublicPolicy=false,RestrictPublicBuckets=false"
    echo "Disable 'Block all public access'"
    # Set bucket policy for public access
    aws s3api put-bucket-policy --bucket $BUCKET_NAME --policy "$BUCKET_POLICY"
    # Upload build to S3
    aws s3 sync build/ s3://$BUCKET_NAME
    echo "Upload build to S3"
    echo "Your React app is deployed at: http://$BUCKET_NAME.s3-website.$AWS_REGION.amazonaws.com"
    echo "Open default browser"
    # Open default browser
    if [[ "$OSTYPE" == "darwin"* ]]; then
        open "http://$BUCKET_NAME.s3-website.$AWS_REGION.amazonaws.com"
    elif [[ "$OSTYPE" == "msys"* ]]; then
        start "http://$BUCKET_NAME.s3-website.$AWS_REGION.amazonaws.com"
    fi
fi

Here's what the script does:

Set the AWS Region: The script sets the AWS region to us-east-1, but you can modify this to your desired region.
Build the React App: The script runs npm install and npm run build to install dependencies and build the React app.
Prompt for Bucket Name: The user is prompted to enter a unique bucket name, which is stored in the BUCKET_NAME variable.
Checking for Existing Bucket: Before proceeding with the deployment, the script checks if the specified bucket already exists using the AWS CLI command aws s3api head-bucket. If the bucket exists, the user is prompted to confirm whether they want to sync the code with the existing bucket or not.
Syncing with Existing Bucket: If the user chooses to sync the code with an existing bucket, the script uses aws s3 sync to upload the built React app to the S3 bucket.
Creating a New Bucket: If the specified bucket doesn't exist, the script creates a new S3 bucket with the provided name and region using aws s3api create-bucket.
Set the Bucket Policy: A bucket policy is defined to allow public read access (s3:GetObject) for all objects (/*) in the specified bucket.
Create the S3 Bucket: The script creates an S3 bucket with the provided name and region using the AWS CLI.
Enable Static Website Hosting: Static website hosting is enabled for the bucket, and index.html is set as the index document.
Disable "Block All Public Access": The "Block all public access" setting is disabled to allow public access to the bucket.
Set the Bucket Policy: The previously defined bucket policy is applied to the bucket, granting public read access.
Upload the Build Files: The built React app files are uploaded to the S3 bucket using the aws s3 sync command.
Print the Deployed App URL: The script prints the URL of the deployed React app, which is accessible publicly.
Open the Deployed App: Finally, the script attempts to open the deployed app in the default browser based on the user's operating system (macOS or Windows).

Usage
To use the script, follow these steps:

Save the script to a file (e.g., deploy-react-app.sh).
Make the script executable by running chmod +x deploy-react-app.sh.
Run the script with ./deploy-react-app.sh.
Enter a unique bucket name when prompted.
Wait for the script to complete the deployment process.
Access your deployed React app at the provided URL.

Please find the github link below:-
https://github.com/akki907/aws_learning/blob/main/scripts/static-website-s3-deploy.sh

Conclusion

Automating deployment processes can significantly improve the efficiency and reliability of your development workflow. With the provided Bash script, you can easily deploy your React app to an AWS S3 bucket, making it publicly accessible as a static website. Feel free to modify the script to suit your specific needs or integrate it into your existing deployment pipeline.
Remember, automation is a powerful tool that can save you time, reduce errors, and ensure consistent deployments. By leveraging scripts like this, you can focus more on writing code and less on the deployment process itself.

Deploying Your Next.js App on Cloudflare Pages

Akash Thakur — Sun, 05 May 2024 05:05:31 +0000

Cloudflare Pages is a powerful platform that makes it incredibly easy to deploy your Next.js applications. With its tight integration with Git repositories, built-in global content delivery network (CDN), and advanced features, Cloudflare Pages is an excellent choice for hosting your Next.js projects.
In this article, we'll walk through the steps to deploy your Next.js app on Cloudflare Pages, ensuring a smooth and seamless deployment process.

Step 1: Create a Cloudflare Account
If you haven't already, create a Cloudflare account by visiting https://dash.cloudflare.com/sign-up. Follow the prompts to set up your account.

Step 2: Initialize a Git Repository
Cloudflare Pages works by connecting to your Git repository. If you haven't already, initialize a Git repository for your Next.js app. You can use popular platforms like GitHub, GitLab, or Cloudflare's own Git service.

git init
git add .
git commit -m "Initial commit"

Step 3: Create a New Project on Cloudflare Pages

In the Cloudflare dashboard, navigate to the "Pages" section.
Click on the "Create a Project" button.
You'll be prompted to connect your Git repository. Choose the provider (e.g., GitHub, GitLab) and select the repository containing your Next.js app.

Step 4: Configure Build Settings
Cloudflare Pages will automatically detect that you're deploying a Next.js app and set the appropriate build settings. However, you can customize these settings if needed.

-In the "Configuration" section, review the build settings.
-If necessary, adjust the build command and build output directory to match your Next.js app's configuration.

Step 5: Set Environment Variables (optional)
If your Next.js app requires environment variables, you can set them in the "Environment Variables" section.

-Click on "Environment Variables".
-Add your environment variables by specifying the variable name and value.

Optional if you facing this error while deployment

then add YARN_VERSION in Environment variables and match it with your current version

Step 6: Deploy your App
After configuring the settings, click the "Save and Deploy" button to start the build and deployment process.

Step 7: Wait for Deployment
Cloudflare Pages will now build and deploy your Next.js app. You can monitor the deployment progress in the dashboard.

Step 8: You need to make a few more changes after deployment is complete if you see this screen

Then go to setting -> functions and add nodejs_compact to the compatibility section

Do the same for this and

Redeploy the app again

Step 9: Access your Deployed App
Once the deployment is complete, you'll find the URL of your deployed Next.js app in the "Deployment Details" section.

Additional Notes
-Cloudflare Pages automatically builds and deploys your app whenever you push changes to your connected Git repository.
-It supports preview deployments, allowing you to preview changes before deploying to production.
-Cloudflare Pages provides a global content delivery network (CDN) for your Next.js app, ensuring fast load times worldwide.
-You can configure custom domains, SSL certificates, and other advanced settings in the Cloudflare dashboard.

Cloudflare Pages is a powerful and convenient platform for deploying your Next.js applications. With its streamlined deployment process, built-in CDN, and advanced features, you can rest assured that your Next.js app will be available to your users with exceptional performance and reliability.