Forem: Daniel Gattringer

Model Context Protocol

Daniel Gattringer — Wed, 30 Apr 2025 05:17:36 +0000

Advanced Large Language Models (LLMs) often operate in isolation, limited by their training data and disconnected from live information. To perform meaningful tasks, they must connect to external data, APIs, and business software. This usually requires building complex, custom integrations for each service, a process that is difficult to scale and maintain.

The Model Context Protocol (MCP) is an open standard designed to solve this challenge. Released in late 2024, MCP provides a universal, secure, and two-way communication channel for AI applications to interact with external systems. It replaces fragmented, custom code with a single, standardized connection method, fundamentally changing how AI agents access and use tools.

Why is the Model Context Protocol (MCP) Important?

MCP offers clear benefits for developers creating sophisticated AI agents and automating complex workflows. It provides a more robust and scalable alternative to traditional integration methods.

Accelerated Tool Integration: MCP enables developers to add new capabilities to AI applications with plug-and-play simplicity. If an MCP server exists for a service, any MCP-compatible AI app can immediately use its functions without custom code. This speeds up development, allowing an AI agent to query a database or make an API call just by connecting to the right server.
Enhanced AI Autonomy: By enabling direct interaction with external systems, MCP supports more autonomous AI agent behavior. An agent can actively retrieve information from a CRM, send an email, or query a database as part of a multi-step workflow. This moves beyond simple Q&A and toward true automated task completion
Reduced Integration Complexity: As a universal interface, MCP significantly lowers the effort needed to integrate new APIs or data sources. An application only needs to support one protocol MCP to connect to countless services. This lets development teams focus on core application logic instead of building and maintaining repetitive integration code
Improved Consistency and Interoperability: MCP uses a standardized request/response format based on JSON-RPC, which simplifies debugging and ensures future compatibility. The interface to external tools remains consistent even if the underlying LLM is swapped out, protecting your development investment
Rich, Two-Way Contextual Interaction: Traditional tool use is often a one-way street: the AI calls a tool and gets a result. MCP creates a richer, two-way dialogue. A server can do more than just offer executable Tools; it can proactively provide Resources (like a file's content) to give the model context before it acts. Furthermore, it can offer Prompts, which are structured templates that guide the AI through a complex workflow. This creates a true partnership between the AI and its tools, enabling more sophisticated and reliable automation

Understanding the MCP Architecture

MCP operates on a flexible client-server model designed for security and extensibility.

Hosts: The primary LLM applications that initiate MCP connections, such as an AI assistant like Claude Desktop or an IDE plugin
Clients: Protocol clients that run inside the host application and manage connections to MCP servers. They are embedded directly into chatbots, coding assistants, or automation agents
Servers: Lightweight programs that expose specific capabilities (Resources, Tools, Prompts) via the MCP standard. Servers act as secure bridges between the AI application and external systems
Local Data Sources: Data stored on a user's machine, like files or local databases, that MCP servers can access with permission
Remote Services: External systems accessible over a network, such as web APIs or cloud databases, that MCP servers connect to

In this setup, the AI application (Host) includes an MCP Client. Each external service connects via a dedicated MCP Server. The server lists its available capabilities, and the client connects to use them. The LLM never communicates directly with the external API, instead, all communication is structured and secured through the standardized MCP client-server process.

Key MCP Concepts Explained

MCP is based on several core ideas that structure how clients and servers interact

Resources: Represent file-like data or content provided by a server. An LLM reads resources to gain context for a task. Access is controlled by the host application. Resources have unique URIs (e.g., file:///path/to/file, postgres://db/table) and can contain text or binary data. Servers can also notify clients when a resource changes
Tools: Executable functions offered by a server that allow an LLM to interact with external systems. Tool use is typically controlled by the AI model for automatic execution. Each tool includes a name, a description for the LLM, and an input schema (using JSON Schema)
Prompts: Pre-defined templates or instructions provided by a server. Prompts help users or LLMs complete specific tasks more efficiently by guiding them through a workflow
Roots: URIs suggested by a client to a server to define relevant locations for resources, such as a filesystem path or an HTTP URL. Servers should generally operate within the scope defined by these roots
Transports: The underlying communication mechanisms that define how MCP messages are exchanged. MCP uses JSON-RPC 2.0 as its standard message format

How MCP Servers and Clients Talk: Transports and Messages

All communication in MCP uses a standard language: JSON-RPC 2.0. This means every message is one of four types: a Request asking for something, a Result returning the answer, an Error if something went wrong, or a Notification for one-way information.

But how these messages travel between the client (in the AI app) and the server (exposing the tool) is defined by a transport. MCP defines two standard transports for two different scenarios.

Transport 1: The Direct Line (Stdio Transport)

Think of this as the simplest, most direct connection possible. It's designed for when the client and server are running on the same machine, which is perfect for local development and testing.

How it Works: The client application launches the MCP server as a subprocess. They then communicate directly using standard input (stdin) and standard output (stdout). The client sends a request to the server's stdin, and the server writes its response back to stdout.
Formatting: Each JSON-RPC message must be a single line of text
Logging: The server can write debugging logs to standard error (stderr), which the client can choose to display or ignore
Use Case: Ideal for building and debugging a new MCP server on your local computer before deploying it

Transport 2: The Network Connection (Streamable HTTP Transport)

When your AI application needs to connect to a server over a network, whether on a different machine in your infrastructure or a public service on the internet, you need a more robust transport.

This transport uses standard HTTP POST and GET requests, allowing a single server to handle connections from multiple clients. To enable the rich, two-way communication MCP promises (like progress updates or server-initiated messages), it cleverly uses Server-Sent Events (SSE) to create a persistent stream.

Here’s how it manages this more complex interaction:

Connection & Session:
1. The client sends an initial POST request to connect
2. The server responds with a unique mcp-session-id in the HTTP header
3. The client must include this session ID in all future requests to maintain the conversation
Two-Way Communication: To receive messages from the server (like notifications), the client makes a separate GET request. The server keeps this connection open, streaming messages back via SSE. This solves HTTP's traditional request-response limitation
Resilience: If the connection breaks, the server can assign an ID to each event. The client can use the Last-Event-ID header to tell the server where to resume, preventing lost messages
Security: For remote connections, security is critical. Implementers must validate the Origin header and use proper authentication to prevent common web attacks. When running locally, the server should only bind to localhost (127.0.0.1) for safety
Configuration Flags: Two flags can alter this behavior for specific needs:
- stateless_http=True: This mode removes the need for session IDs, which is essential for environments that require high availability and load balancing across multiple server instances (horizontal scaling). In a stateless setup, any server instance can handle any client request without needing shared session memory. However, this comes at a cost: the server gives up its ability to send unsolicited requests or notifications to the client, making the interaction strictly client-initiated
- json_response=True: This disables streaming for POST responses, returning only a single, final JSON object. This is useful for simpler tools that don't need to send intermediate progress updates

Custom Transports

While Stdio and HTTP are standard, MCP is extensible. Teams can implement their own transports (e.g., using WebSockets or gRPC) as long as they adhere to the core JSON-RPC message structure and lifecycle.

Implementation Resources: SDKs and Examples

To simplify development, MCP provides Software Development Kits (SDKs) for popular programming languages:

You can find quickstart guides and examples for building both client and server applications in the official documentation.

How to Debug and Troubleshoot MCP Integrations

Effective debugging is essential when working with MCP. The ecosystem provides tools and guidance:

MCP Inspector: An interactive tool built specifically for debugging MCP servers. It provides a detailed, real-time view of messages exchanged between a client and server, making it easier to identify issues.

Secure Authentication with OAuth 2.0

A key improvement in MCP is adopting OAuth 2.0 for secure authentication. Earlier versions lacked a standard method, making secure remote connections difficult and often requiring manual credential handling.

Using OAuth 2.0 provides a reliable and standard framework for secure interactions, particularly for remote servers and multi-user applications. Key advantages are:

Standardized Security: Uses the established OAuth 2.0 protocol.
Dynamic Client Registration (DCR): Allows clients to register with authorization servers automatically.
Automatic Endpoint Discovery: Simplifies setup by allowing clients to find necessary OAuth endpoints.
Secure Token Management: Ensures safe handling of access tokens, limiting client access to authorized resources only.

This standardized approach significantly improves security and ease of use.

How MCP Compares to Other Integration Approaches

MCP offers a distinct approach compared to other ways of connecting LLMs to external systems:

Custom Integrations: MCP removes the need for building fragile, labor-intensive, single-use integrations for each service by providing a reusable, standard protocol.
ChatGPT Plugins: While an early step, ChatGPT plugins were proprietary, linked to OpenAI, focused on single interactions, and lacked a universal authentication standard. MCP is an open, model-neutral standard supporting persistent connections and standard OAuth 2.0 security.
LLM Tool Frameworks (e.g., LangChain, LlamaIndex): It's best to think of these as operating at different levels of abstraction.
- LangChain and LlamaIndex are high-level application development frameworks. They provide the scaffolding to build an agent's internal logic—its memory, planning capabilities, and decision-making loops.
- MCP is a low-level communication protocol. It standardizes the "last mile" connection between the agent and the external tool itself.

They are not competitors; they are complementary. Think of it this way:

Scenario: Building a GitHub Issue Manager Agent with LangChain

Without MCP: Your LangChain agent needs a tool to fetch issues. You would write custom Python code, handles authentication with a personal access token, makes the specific API call to get the issues, and then parses the JSON response. If you later want to add a tool for Slack, you repeat the process for Slack's API.
With MCP: Your LangChain agent needs the same tool. Instead of writing custom code, it makes a standardized call to the Github MCP server. The agent's logic doesn't need to know anything about REST APIs, Python libraries, or token authentication. The MCP server handles all the specific implementation details.

MCP standardizes the connection interface, allowing high-level frameworks like LangChain to focus on the agent's internal logic and behavior.

Real-World MCP Applications and Community Engagement

MCP allows AI systems to securely access and utilize live data and perform actions. Developers are already building MCP servers for common tools:

These examples show MCP's value in connecting AI agents to company data, developer tools, communication platforms, and essential business systems.

Continue exploring related content → my notes

Step-by-Step Guide: Loading a HuggingFace ControlNet Dataset from a Local Path

Daniel Gattringer — Thu, 15 Aug 2024 04:56:05 +0000

Huggingface provides different options for loading a dataset. When loading a local image dataset for your ControlNet, it's important to consider aspects such as dataset structure, file paths, and compatibility with Huggingface's data handling tools.

Let's assume you've already created your conditioning images and you have the following folder structure:

my_dataset/
├── README.md
└──data/
   ├── captions.jsonl
   ├── conditioning_images
   │   ├── 00001.jpg
   │   └── 00002.jpg
   └── images
       ├── 00001.jpg
       └── 00002.jpg

In this structure, the conditioning_images folder stores your conditioning images, while the images folder contains the target images for your ControlNet. The captions.jsonl file holds the captions linked to these images.

{"image": "images/00001.jpg", "text": "This is the caption of the first image."}
{"image": "images/00002.jpg", "text": "This is the caption of the second image."}

Note
The caption file (or the following metadata file) can also be a csv file. However, if you opt for CSV, be careful of the value separator, as the text may contain commas, which could lead to parsing issues.

Create Metadata File

A metadata file is a good way to provide additional information about your dataset. It can include various types of data, such as bounding boxes, categories, text, or in our case, a path to the conditioning image.

Let's create the metadata.jsonl file:

import json
from pathlib import Path

def create_metadata(data_dir, output_file):
    metadata = []
    try:
        with open(f"{data_dir}/captions.jsonl", "r") as f:
            for line in f:
                data = json.loads(line)
                file_name = Path(data["image"]).name
                metadata.append(
                    {
                        "image": data["image"],
                        "conditioning_image": f"conditioning_images/{file_name}",
                        "text": data["text"],
                    }
                )

        with open(f"{data_dir}/metadata.jsonl", "w") as f:
            for line in metadata:
                f.write(json.dumps(line) + "\n")
    except (FileNotFoundError, json.JSONDecodeError) as e:
        print(f"Error processing data: {e}")

# Example usage:
data_dir = "my_dataset/data"
create_metadata(data_dir)

This will create a metadata.jsonl containing all the information we need for our ControlNet. Each line in the file corresponds to an image, a conditioning image, and the associated text caption.

{"image": "images/00001.jpg", "conditioning_image": "conditioning_images/00001.jpg", "text": "This is the caption of the first image."}
{"image": "images/00002.jpg", "conditioning_image": "conditioning_images/00002.jpg", "text": "This is the caption of the second image."}

Once you've created the metadata.jsonl file, your file structure should look like this:

my_dataset/
├── README.md
└──data/
   ├── captions.jsonl
   ├── metadata.jsonl
   ├── conditioning_images
   │   ├── 00001.jpg
   │   └── 00002.jpg
   └── images
       ├── 00001.jpg
       └── 00002.jpg

Create a Loading Script

Finally, we must create a loading script that handles all the data in the metadata.jsonl file. The script should be located in the same directory as the dataset and should have the same name.

Your directory structure should look like this:

my_dataset/
├── README.md
├── my_dataset.py
└──data/
   ├── captions.jsonl
   ├── metadata.jsonl
   ├── conditioning_images
   │   ├── 00001.jpg
   │   └── 00002.jpg
   └── images
       ├── 00001.jpg
       └── 00002.jpg

For the script, we need to implement a class that inherits from GeneratorBasedBuilder and contains these three methods:

_info stores information about your dataset.
_split_generators defines the splits.
_generate_examples generates the images and labels for each split.

import datasets

class MyDataset(datasets.GeneratorBasedBuilder):

    def _info(self):

    def _split_generators(self, dl_manager):

    def _generate_examples(self, metadata_path, images_dir, conditioning_images_dir):

Adding Dataset Metadata

There are many options for specifying information about your dataset, but the most important ones are:

features specify the dataset column types.
- image is an Image feature
- conditioning_image is an Image feature
- text is a string value
supervised_keys that specify the input features.

# Global variables
_DESCRIPTION = "TODO"
_HOMEPAGE = "TODO"
_LICENSE = "TODO"
_CITATION = "TODO"

_FEATURES = datasets.Features(
    {
        "image": datasets.Image(),
        "conditioning_image": datasets.Image(),
        "text": datasets.Value("string"),
    },
)

As you can see above, I've set some variables to "TODO". These options are for informational purposes only and do not affect loading.

def _info(self):
    return datasets.DatasetInfo(
                description=_DESCRIPTION,
                features=_FEATURES,
                supervised_keys=("conditioning_image", "text"),
                homepage=_HOMEPAGE,
                license=_LICENSE,
                citation=_CITATION,
            )

Define the Dataset Splits

dl_manager is used to download a dataset from a Huggingface repo but here we use it to get the data directory path which is passed in the load_dataset function.

Here we define the local paths to our data

metadata_path path of the metadata.jsonl file
images_dir path to the images
conditioning_images_dir path to the conditioning images

Note
If you've chosen different names for your folder structure, you may need to adjust the metadata_path, images_dir, and conditioning_images_dir variables.

def _split_generators(self, dl_manager):
    base_path = Path(dl_manager._base_path).resolve()
    metadata_path = base_path / "data" / "metadata.jsonl"
    images_dir = base_path / "data"
    conditioning_images_dir = base_path / "data"

    return [
        datasets.SplitGenerator(
            name=datasets.Split.TRAIN,
            # These kwargs will be passed to _generate_examples
            gen_kwargs={
                "metadata_path": str(metadata_path),
                "images_dir": str(images_dir),
                "conditioning_images_dir": str(conditioning_images_dir),
            },
        ),
    ]

The last method loads the matadata.jsonl file and generates the image and its associated conditioning image and text.

@staticmethod
def load_jsonl(path):
    """Generator to load jsonl file."""
    with open(path, "r") as f:
        for line in f:
            yield json.loads(line)

def _generate_examples(self, metadata_path, images_dir, conditioning_images_dir):
    for row in self.load_jsonl(metadata_path):
        text = row["text"]

        image_path = row["image"]
        image_path = os.path.join(images_dir, image_path)
        image = open(image_path, "rb").read()

        conditioning_image_path = row["conditioning_image"]
        conditioning_image_path = os.path.join(
            conditioning_images_dir, row["conditioning_image"]
        )
        conditioning_image = open(conditioning_image_path, "rb").read()
        yield row["image"], {
            "text": text,
            "image": {
                "path": image_path,
                "bytes": image,
            },
            "conditioning_image": {
                "path": conditioning_image_path,
                "bytes": conditioning_image,
            },
        }

Following these steps, you can load a ControlNet dataset from a local path.

# with the loading script, we can load the dataset
ds = load_dataset("my_dataset")

# (optional)
# pass trust_remote_code=True to avoid the warning about custom code
# ds = load_dataset("my_dataset", trust_remote_code=True)

If you have any questions, feel free to leave a comment below.

Full code for the loading script:

import os
import json
import datasets
from pathlib import Path

_VERSION = datasets.Version("0.0.2")

_DESCRIPTION = "TODO"
_HOMEPAGE = "TODO"
_LICENSE = "TODO"
_CITATION = "TODO"

_FEATURES = datasets.Features(
    {
        "image": datasets.Image(),
        "conditioning_image": datasets.Image(),
        "text": datasets.Value("string"),
    },
)

_DEFAULT_CONFIG = datasets.BuilderConfig(name="default", version=_VERSION)

class MyDataset(datasets.GeneratorBasedBuilder):
    BUILDER_CONFIGS = [_DEFAULT_CONFIG]
    DEFAULT_CONFIG_NAME = "default"

    def _info(self):
        return datasets.DatasetInfo(
            description=_DESCRIPTION,
            features=_FEATURES,
            supervised_keys=("conditioning_image", "text"),
            homepage=_HOMEPAGE,
            license=_LICENSE,
            citation=_CITATION,
        )

    def _split_generators(self, dl_manager):
        base_path = Path(dl_manager._base_path)
        metadata_path = base_path / "data" / "metadata.jsonl"
        images_dir = base_path / "data"
        conditioning_images_dir = base_path / "data"

        return [
            datasets.SplitGenerator(
                name=datasets.Split.TRAIN,
                # These kwargs will be passed to _generate_examples
                gen_kwargs={
                    "metadata_path": metadata_path,
                    "images_dir": images_dir,
                    "conditioning_images_dir": conditioning_images_dir,
                },
            ),
        ]

    @staticmethod
    def load_jsonl(path):
        """Generator to load jsonl file."""
        with open(path, "r") as f:
            for line in f:
                yield json.loads(line)

    def _generate_examples(self, metadata_path, images_dir, conditioning_images_dir):
        for row in self.load_jsonl(metadata_path):
            text = row["text"]

            image_path = row["image"]
            image_path = os.path.join(images_dir, image_path)
            image = open(image_path, "rb").read()

            conditioning_image_path = row["conditioning_image"]
            conditioning_image_path = os.path.join(
                conditioning_images_dir, row["conditioning_image"]
            )
            conditioning_image = open(conditioning_image_path, "rb").read()
            yield row["image"], {
                "text": text,
                "image": {
                    "path": image_path,
                    "bytes": image,
                },
                "conditioning_image": {
                    "path": conditioning_image_path,
                    "bytes": conditioning_image,
                },
            }