Forem: Stephen Nwankwo

The State of Security Protocols in Agent 2 Agent(A2A) Systems.

Stephen Nwankwo — Sun, 05 Oct 2025 05:54:12 +0000

If you are reading this, it's probably because you've heard or read something about Agentic AI sytems(and you care about security). Just so we are on the same page, Agentic AI sytems are autonomous entities capable of perceiving their environment, making decisions, and executing actions with minimal human intervention, typically with the help of an LLM(Large Languauge Model). And as for agent-2-agent agentic ai systems, you guessed it, it means exactly what it sounds like: multi-agent interaction and interoperability. As these intelligent agents increasingly interact and compose workflows across organizational, geographical, and trust boundaries(health and legal spaces), the demand for secure, standardized interoperability becomes paramount. Without shared protocols governing identity, authentication, task exchange, and auditability, agent interactions are prone to fragmentation, redundancy, and critical security vulnerabilities such as impersonation, data exfiltration, and unauthorized privilege escalation.

As we go, I will leave short side notes translating a few things to native SWE lingua, or general lingua.

The cornerstone of modern Agent 2 Agent (A2A) security currently revolves around protocols that emphasize structured communication and identity awareness. One such foundational protocol is Google’s Agent-to-Agent (A2A) specification, which provides a declarative, identity-aware framework designed for composability and trust.

The Foundational A2A Protocol and Secure Interoperability

The A2A protocol allows autonomous agents to discover each other via standardized AgentCards [side note: kind of like a business card containing details about yourself, how to reach you and your role in the business. more on this later], authenticate using modern cryptographic protocols, and exchange tasks in a declarative, auditable manner [side note: I want to be abe to tell you what i want you to do, and i want to be able to tell when you do it]. The protocol facilitates communication between a Client Agent (formulating tasks) and a Remote Agent (acting on tasks). It is built on established web standards like HTTPS, JSON-RPC, and Server-Sent Events (SSE), prioritizing compatibility while adhering to a security-first approach.

A core mechanism is the AgentCard, a structured JSON metadata file located at the standardized path /.well-known/agent.json. This card functions as a machine-readable "business card," containing identification, the A2A endpoint URL (a2aEndpointUrl), detailed function catalogs, capability descriptions, and required authentication methods utilizing OpenAPI 3.x Security Scheme objects (e.g., oauth2, http bearer).

The standard A2A communication flow involves:

Discovery: The client fetches the remote agent's AgentCard via HTTPS to learn its capabilities and authentication requirements. [side note: this is similar to a prefech done by web apps]
Initiation: The client authenticates (using the method specified in the AgentCard) and sends a JSON-RPC request over HTTPS via tasks.send (synchronous) or tasks.sendSubscribe (streaming updates using SSE). [side note: this is like sending a Bearer token in the header or X-API-Key along with your request, depending on the API specs]
Processing & Interaction: The server processes the task and returns the resulting Task object or streaming updates (TaskStatusUpdateEvent, TaskArtifactUpdateEvent).

Assessing the Agentic Threat Landscape

The autonomous nature of agentic AI required specialized security analysis, an entire system was birthed because of this. The MAESTRO framework (Multi-Agent Environment, Security, Threat, Risk, and Outcome) is often leveraged for this purpose, providing a structured, granular, and proactive methodology tailored to the complexities of systems like those built using A2A. MAESTRO recognizes that security must be addressed across seven layers, from Foundation Models (Layer 1) up to the Agent Ecosystem (Layer 7). [side note: these layer methodology is similar to the 7 network layers in a way, but for agentic ai, just in case you're familiar with that.]

The following image is a illustration of the 7 layers:

Key A2A multi-agent system threats identified through MAESTRO include:

Threat	Description	MAESTRO Layers	Mitigation Focus
Agent Card Spoofing	An attacker publishes a forged Agent Card at a malicious domain, leading to task hijacking and data exfiltration.	3 (Agent Frameworks), 4 (Deployment & Infrastructure)	Digital Signatures, Secure Resolution
Poisoned AgentCard	Prompt injection techniques are embedded within AgentCard fields (like descriptions), causing another agent processing the card to execute hidden instructions.	1 (Foundation Model), 2 (Data Operations)	Input Sanitization, Schema Checks
A2A Task Replay	An attacker captures and replays a valid `tasks/send` request, leading to duplicate or unauthorized actions.	3 (Agent Frameworks), 2 (Data Operations)	Nonces, Timestamp Verification, Idempotency
Cross-Agent Task Escalation	A malicious agent attempts to escalate privileges by submitting tasks with forged credentials, breaching trust boundaries.	7 (Agent Ecosystem), 3 (Agent Frameworks)	Strict Authorization, Least Privilege

If you are a software engineer or a CS student, then the threats mentioned above probalbly sound like something you've heard before, e.g spoofing, SQL injection, CSRF, e.g..

Enhancing Security and Governance: Layered Architectures

To address these vulnerabilities, robust protocols must integrate security deep within the system design right from the get-go. The goal is to develop a layered architecture that embeds governance and security across all system layers.

Zero-Trust for Agent Interaction
The principle of Zero-Trust architecture—"never trust, always verify"—is not new to software engineering and is essential for multi-agent systems, requiring continuous validation of identity and authorization for every agent, communication, and action. Implementation strategies include:

Implementing strong authentication mechanisms, ensuring each agent has a unique, cryptographically verifiable identity.
Applying fine-grained authorization controls so agents receive only the minimum necessary permissions for their specific tasks [side note: if you are familiar with the Principle of Least Privilege, this is exactly what it entails).
Setting up encrypted communications end-to-end (TLS/HTTPS) to prevent eavesdropping and message tampering, complemented by integrity checks.

Now, with MAESTRO we are able to identify layers in A2A systems and the various angle of attacks by malicious users, the next obvious question would be how do we prevent these issues. The following paragraph outlines the security model thta have been agreed upon and put in place to curb these loop holes.

SAGA: Granular Access Control and Oversight
While the A2A protocol facilitates decentralized identity and interoperability, it has been noted to lack policy enforcement mechanisms and runtime mediation of agent interactions.

The SAGA (Security Architecture for Governing Agentic systems) architecture addresses this by enhancing governance and security through a Provider-mediated approach that gives users control and oversight over their agents’ lifecycles. SAGA enables users to define an Access Contact Policy (CP) for their agents, governing permissible incoming communication.

SAGA integrates with A2A by protecting Agent Cards under user-specified access control policies and encapsulating A2A messages within its secure communication layer. Key to SAGA’s security model is the use of cryptographic access control tokens. These tokens are generated by the receiving agent and encrypted using a dynamically derived shared key, providing fine-grained control over inter-agent communication and limiting the vulnerability window.

This shared key derivation relies on One-Time Keys (OTKs) obtained from the Provider, which enforces the user's Contact Policy during the initial request. The token includes an expiration timestamp and a limit on the number of permitted requests (Qmax), which allows tokens to be reused for subsequent communication without involving the centralized Provider, here bybalancing security and performance overhead.

Workflow Illustration: Secure A2A Task Delegation using SAGA/Policy Enforcement

The following diagram illustrates the flow required to establish secure communication for a delegated A2A task, incorporating the cryptographic policy enforcement mechanisms found in SAGA.

Secure Implementation: Code Controls

Secure implementation of an A2A Server requires rigorous validation of inputs, tokens, and authorization against threats like replay attacks and token misuse. Agent frameworks should apply controls such as input sanitization to all AgentCard content to mitigate poisoned AgentCard risks. Furthermore, strong token and task validation is crucial.

The following pseudo-code illustrates essential server-side logic for handling an incoming request that carries an Access Control Token, incorporating checks recommended for mitigating replay and misuse threats:

# Function to validate the Access Control Token (ACT) attached to an A2A request

def validate_access_token ( token , initiating_agent_id , request_nonce ):
    """
    Validates the cryptographic token against expiry, quota, and identity.
    Mitigates: A2A Task Replay, Authentication & Identity Threats, Token Misuse.
    """

    # 1. Decrypt token using shared key (SDHK, derived during initial handshake)
    try :
        decrypted_claims = decrypt_token ( token , SDHK )
    except DecryptionError :
        return False , " Invalid token encryption. "

    # 2. Check Expiration Time (Texpire)
    if decrypted_claims [ ' Texpire ' ] < current_timestamp ():
        return False , " Token has expired. "

    # 3. Check Initiating Agent Identity (PACB)
    # Ensures the token wasn't issued to a different agent (mitigates A5)
    if decrypted_claims [ ' PACB ' ] != initiating_agent_id :
        return False , " Token issued for different agent identity. "

    # 4. Check Usage Quota (Qmax)
    # Requires backend storage to track remaining requests against Qmax
    if get_token_usage ( token ) >= decrypted_claims [ ' Qmax ' ]:
        # If quota is exceeded, the token is invalid (A3)
        return False , " Token usage quota exceeded. "

    # 5. Check for Nonce/Task Replay (Mitigation for A2A Task Replay)
    # If the request used a nonce, ensure it hasn't been seen recently.
    if request_nonce and is_nonce_replayed ( request_nonce , initiating_agent_id ):
        return False , " Replayed request nonce detected. "

    # 6. If all checks pass, increment usage and proceed.
    increment_token_usage ( token )
    return True , " Token Valid. "

By adopting security frameworks like MAESTRO, layering Zero-Trust principles, and implementing specialized protocols like A2A, augmented by fine-grained control systems such as SAGA, developers can build robust and trustworthy security protocols essential for the next generation of autonomous agent systems.

Thank you for reading, and may the Force be with you.

Resources/References:

https://cloudsecurityalliance.org/blog/2025/02/06/agentic-ai-threat-modeling-framework-maestro

https://arxiv.org/html/2504.21034v2#S3

https://a2a-protocol.org/dev/specification/

https://a2aprotocol.ai/docs/guide/a2a-protocol-specification-python

https://www.youtube.com/watch?v=56BXHCkngss

Where to find me:

https://www.linkedin.com/in/stephen-nwankwo-9876b4196/

https://x.com/Sage_Sten

https://dev.to/sten

https://github.com/stenwire

Building an MCP Server with RAG Capabilities for Developer Tools

Stephen Nwankwo — Sun, 06 Jul 2025 19:34:06 +0000

As the demand for intelligent, context-aware assistants grows, the need for flexible and modular agent protocols has never been more critical. With this article and project, I hope to demonstrate how easy it is to combine MCP + RAG to aid knowledge-grounded responses whiile working across various development environments such as VS Code, Claude, Cursor, etc..

This article is a very high-level overview of the project, the architecture, and how this hybrid system can be embedded into developer tools seamlessly.

What is MCP?

MCP (Model Context Protocol), invented by a team at Anthropic is an emerging standard for enabling consistent interaction between AI agents and their environments. Designed to separate environment logic from model behavior, MCP allows tools like IDEs and terminal clients to interact with LLMs in a structured way, similar to how the LSP (Language Server Protocol) standardized code intelligence or how REST standardized Client-API interaction. A lot of people like to refer to MCP as the USB-C of Agent - to - external tool interaction, I hope you get the idea.

There are two major parts that make up an MCP, the first is the Server and the second is the Host(aka Client), the former houses the interface that connects to all the external/internal tools, while the Host houses the interfaces responsible for connecting to the server and translating user input to commands to be sent to the server. The image below is an illustration of what a basic MCP architecture looks like.

figure(1)

Connecting the Server to the Host typically involves writing a JSON data to the host MCP configuration or settings.json in the case of VsCode(what I used). The typical structure of the Json configuration looks like this:

"name-of-mcp-server": {
    "type": "protocol",
    "command": "run_command",
    "args": [
        "url_or_file_path_to_server"
    ]
}

What is RAG?

Retrieval-Augmented Generation (RAG) combines a language model with an external knowledge retriever. Instead of relying solely on the model's training data, RAG systems pull relevant documents or facts in real-time to augment the prompt. This helps particularly to improve response accuracy, reduces hallucinations, and makes models useful in specialized domains.

Project Overview

This project involves building an MCP server that wraps a RAG pipeline. Any client or host that supports MCP can interface with this server to ask questions, perform searches, or generate context-aware completions.

The goal is to provide local or self-hosted intelligence that can adapt to your documents, codebase, or domain knowledge without relying on external APIs.

Architecture

figure(2)

The MCP server handles structured requests, transforms them into a context-aware prompt using the RAG system, and returns enriched responses back to the client.

Implementation

For this project, I built an MCP-RAG server that helps to answer questions related to documenting Django Rest Framework(DRF) APIs

Here's a breakdown of the tech stack:

Major Library: Google ADK, langchain, python mcp sdk, chroma(for vectore store).
MCP Protocol: Custom implementation based on official spec from Anthropic.
RAG Stack:
Google ADK with gemini is used to handle embedding and retrieval, langchain is used for chuking text and the vector store used is Chroma DB
Client Interface: VsCode configured using the standard MCP JSON config.

Here's what the file structure looks like:

figure(3)

Here’s an example exchange inside VS Code using the MCP RAG server:

figure(4)

Use Cases

Ask domain-specific questions, get code explanations, or retrieve documentation instantly.
Query internal knowledge bases while chatting.
Integrate organization-specific coding guidelines and FAQs into Cursor's/CoPilot LLM responses. All of these, while in your code editor.

This project is a step toward plug-and-play intelligence across your developer stack. By combining the modular power of MCP with the contextual strength of RAG, you can build assistants that actually understand your codebase, your docs, your workflows.

Resources

Link to Code: https://github.com/stenwire/MCP/tree/main/django-mcp
MCP Spec
Intro to RAG
Google ADK
LangChain & LlamaIndex (for RAG)

Optimizing SQL queries for your AI Agents.

Stephen Nwankwo — Sun, 11 May 2025 19:51:47 +0000

So, you're building the next generation of Agentic AI, a smart, autonomous system ready to understand, reason, and act. Fantastic! But where does this digital brain get its crucial "proprietary knowledge" or the specific context it needs to make intelligent decisions? Often, it's from your trusty SQL database, queried to pull in everything from user history and product catalogs to real-time sensor data that forms the bedrock of its understanding.

You've designed your agent, given it goals, and it's ready to interact. But then... the pause. That moment your agent needs to "think" to fetch that vital piece of information from the database to build context and it feels like an eternity. Your sophisticated agent, designed for dynamic action, is suddenly hamstrung by a slow SQL query, turning its "real-time" reasoning into "just-a-minute" frustration. This, my friends, is where SQL query optimization becomes absolutely critical for potent Agentic AI.

It's like tasking a brilliant detective with solving a case, but every time they need a clue (a piece of data), they have to manually sift through mountains of unsorted files in a dusty archive. The detective is smart, but the information retrieval is crippling their speed and effectiveness. Our agents are no different.
How do we ensure our AI agents get their context fast and efficiently? Drawing from alrady existing solid optimization principles, let's look at the game plan:

Become a Data Profiler with `EXPLAIN`:

Before your agent can act intelligently, you need to understand how it's getting its intelligence. EXPLAIN (or its equivalent) unveils the database's strategy for fetching data.
Red Flag for Agents: Is it scanning millions of rows for a small piece of context your agent needs right now? That's a bottleneck that directly impacts agent responsiveness.
Costly SORT Operations: If the agent needs sorted data for its logic, these can be memory hogs, slowing down the entire context-building process.
Full Table Scans: The arch-nemesis! Your agent shouldn't wait for the DB to read an entire table to understand a specific situation.

Sharpen the Agent's Focus (Tune the Query Itself!):

This is often where the biggest wins for agent performance lie.
Filter with Precision (WHERE clause): Equip your agent's queries to be surgical. Only fetch the exact data needed for the current task or decision. The more targeted the WHERE clause, the faster the context is built.

Efficient JOINs: If your agent needs to combine information from multiple tables (e.g., user profile + interaction history), ensure those JOINs are lean and mean.
Concise IN Lists: If your agent is checking against a set of known items, keep that list tight.

Create Fast-Access Lanes with Indexes:

Think of indexes as creating a super-efficient filing system for the data your agent frequently needs.

Identify columns your agent often uses for lookups in its WHERE clauses or to ORDER BY for its reasoning. Indexing these is like giving your agent a direct line to the information.
Agent Benefit: Faster context retrieval = quicker decision-making = a more responsive and seemingly "smarter" agent.
Caution: Don't just slap indexes everywhere. They speed up reads (good for agents fetching context) but can slow down writes. Use EXPLAIN to validate their impact.

Major Overhauls for Complex Agent Needs:

When agents deal with massive, evolving datasets:

Table Partitioning: If your agent reasons over time-series data (e.g., daily user activity), partitioning by date can mean it only queries the relevant recent slice of data, dramatically speeding up context gathering.
Data Structure Redesign: Sometimes, the way data is structured needs to align better with how your agent thinks and accesses information. This is a deep dive, but for complex agents, it can be transformative.

If you dev with python like myself, here's how to how to use EXPLAIN to inspect potential performance bottlenecks like full table scans and costly sorts, which can directly affect AI agent responsiveness.

Suppose you have postgres setup somewhere and your table schema looks like this:

CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    title TEXT,
    content TEXT,
    created_at TIMESTAMP DEFAULT NOW()
);

Create a python file and enter this:

import psycopg2

# Connect to your PostgreSQL database
conn = psycopg2.connect(
    dbname="your_db",
    user="your_user",
    password="your_password",
    host="localhost",
    port="5432"
)

cur = conn.cursor()

# The actual query the AI agent might run
query = """
SELECT * FROM documents
WHERE content ILIKE '%recycling%'
ORDER BY created_at DESC
LIMIT 10;
"""

# Use EXPLAIN ANALYZE to see what PostgreSQL does under the hood
cur.execute(f"EXPLAIN (ANALYZE, BUFFERS, VERBOSE) {query}")
plan = cur.fetchall()

print("Query Plan:\n")
for row in plan:
    print(row[0])

cur.close()
conn.close()

Save and run the file, then observe the result. And if you are not understanding what you are seeing, you know what to do.

My Takeaway for AI Devs:

For Agentic AI, the speed and efficiency of data retrieval for context building are not just "database concerns", they are core to agent performance. A slow query means a slow-thinking agent, leading to poor user experience or ineffective automation. Regularly optimizing the SQL queries that feed your agents is like sharpening their cognitive tools. Use EXPLAIN religiously, prioritize query tuning and intelligent indexing.

How are you ensuring your agents get the data they need, when they need it, without the lag? Share your strategies for performant context-building! 👇

Connect with me on:

X(Twitter)

GitHub

Learn Google ADK: Build a farming agent.

Stephen Nwankwo — Sun, 20 Apr 2025 05:02:32 +0000

Meet NaijaFarmHand: Your AI Farming Buddy for Nigeria

Ever felt overwhelmed starting a small farm? Wondering when the best time to plant your maize is, what you actually need to start raising chickens, or what the weather will be like tomorrow in your specific town? Getting practical, localized farming information can sometimes feel like searching for a needle in a haystack, especially when you're just starting out.

Imagine having a helpful assistant, powered by Artificial Intelligence (like the technology behind ChatGPT or Google Gemini), specifically designed to answer your beginner farming questions within the Nigerian context. This project isn't about replacing experienced farmers or extension agents, but about providing a quick, accessible first point of contact for common queries.

Built using Google's AI Development Kit (ADK) and running within a Kaggle Notebook environment (a popular platform for data science and AI projects), NaijaFarmHand aims to bridge the information gap for aspiring and early-stage farmers focusing on some of Nigeria's key agricultural products.

Before we dive in, here’s a bit of background: this project was completed as part of the 5-Day Gen AI Intensive course by Google.

If you want to jump straight to the code, find link to the kaggle notebook at the end of the article.

What Can NaijaFarmHand Actually Do?

Think of NaijaFarmHand as having a few specialized skills, like different tools in a farmer's toolkit. It's specifically focused on:

Crops: Maize, Cassava, Yam, Groundnut, and Guinea Corn.
Livestock: Poultry (Broiler and Layer Chickens).

If you ask about growing strawberries or raising snails, it will politely tell you that's outside its current expertise but might try to find general information using Google Search.

Here are its main "tools":

WeatherTool - The Local Forecaster:
- What it does: Gets you the current weather conditions for a specific Nigerian location. Need to know if it's likely to rain in Ibadan today before planting? Ask NaijaFarmHand, and it will use this tool.
- Why it's important: Weather is critical for day-to-day farming decisions. This tool uses the OpenWeatherMap API (powered by a secret API key safely stored using Kaggle Secrets) to fetch real-time data.
FarmPlannerTool - The Starter Checklist Generator:
- What it does: Provides a basic checklist of things to consider when starting a specific type of farm (from its focus list) at a particular scale (e.g., "small scale maize farm" or "50 broiler chickens").
- Why it's important: Gives beginners a rough idea of requirements like land, basic capital estimates (with a big disclaimer that costs vary!), simple tools, and initial inputs. It helps structure initial planning.
TimingAdvisorTool - The Seasonal Guide:
- What it does: Offers general advice on typical planting seasons and harvest times for its focus crops in different broad regions of Nigeria (North, Middle Belt, South).
- Why it's important: Helps understand the usual agricultural calendar. Crucially, NaijaFarmHand is programmed to always remind you that this is general advice and you MUST check the current local weather (using the WeatherTool) and talk to local farmers for precise timing. Climate varies!
google_search_grounding - The Researcher:
- What it does: For questions that the other tools can't answer directly, NaijaFarmHand uses Google Search. This is used for finding details on specific crop varieties suitable for Nigeria, identifying pests/diseases, getting an idea of market prices (which fluctuate wildly!), or learning about government agricultural programs.
- Why it's important: Provides a way to access the vast information online, guided by the agent's instructions to find relevant answers for the user.

How Does It All Work Together?

You don't need to be an AI expert to understand the basics:

The Brain: The project uses a powerful language model from Google ("gemini-1.5-pro") as the core intelligence.
The Instructions: We give the AI very specific instructions (the agent_instructions block you saw) telling it who it is (NaijaFarmHand), what its job is, what topics it should cover, and how to use its tools.
The Framework (Google ADK): This is like the plumbing and wiring. It provides the structure (Agent, Runner, SessionService) to define the agent, connect the tools, manage the conversation flow, and remember what was said previously in the chat (InMemorySessionService).
The Interaction:
- You type a question (e.g., "What do I need for a small backyard poultry farm?").
- The Runner passes this to the Agent (NaijaFarmHand).
- The Agent, guided by its instructions, decides which tool is best. In this case, probably the FarmPlannerTool.
- It calls the tool, gets the checklist back.
- It formats the checklist into a friendly response.
- The Runner delivers the response back to you.
- If you asked "What's the weather in Kano?", it would use the WeatherTool. If you asked "Tell me about Fall Armyworm control in Maize," it would likely use the google_search_grounding tool.

Why Build Something Like NaijaFarmHand?

This project demonstrates how modern AI tools can be tailored to solve specific, localized problems. It's about:

Accessibility: Making basic farming information readily available through a simple chat interface.
Localization: Focusing specifically on Nigerian crops, conditions, and context.
Practicality: Integrating tools like weather and planning checklists directly into the conversation.
Learning: It's a great example of building agents with multiple tools using the Google ADK framework.

While NaijaFarmHand is a prototype running in a notebook, it showcases the potential for AI to support agriculture and empower individuals starting their farming journey in Nigeria. It emphasizes providing helpful guidance while always encouraging users to seek detailed, local expert advice for critical decisions.

Link to Kaglle Notebook: https://www.kaggle.com/code/stephennwankwo/naijafarmhand-agent

More about Google ADK: https://google.github.io/adk-docs/

Performance Testing: Is Your API Ready for Real-World Traffic?

Stephen Nwankwo — Mon, 06 Jan 2025 08:25:57 +0000

Have you ever had the thought of simulating real-life users on your API or service, or have you ever wanted to put your API to the test, to see how much it can and can not handle, to make informed decisions, like optimize your code better or consider scaling your server, horizontally or vertically? Well, I have, on multiple occasions, but apart from writing unit and e2e tests, I have never really gone ahead to run actual performance tests, the reasons being we usually had a QA engineer on the team, and for personal projects, I felt nonchalant and convinced myself it wasn’t necessary.

But here's the issue: It's not about whether I need it in my personal project that will never get up to 5 active users; it's about giving myself a chance to learn something new. I believe having this knowledge/skill will really make you stand out as a software engineer or backend engineer

Knowing how to test your API performance will help solidify your knowledge of whatever technology or framework you're using to build. When you run a test, you notice an issue, and then you go back to your IDE to figure out if it's a query, a join statement, or data manipulation logic that's the issue, you start to ask yourself questions and dig deeper, you select better data structures, you improve your SQL statements, you consider implementing caching, you realize you're making multiple calls to the DB for the same items, and you realize your code structure is looking like a plate of spaghetti from the Bear movie. At the end of all these, you come out a bad-ass engineer. What you learn from performance testing is endless.

I am a backend-heavy software engineer. If you ask me who a backend engineer is in 2025, my answer would be someone who does backend work, understands systems design, is knowledgeable about best dev-ops practices and performance testing, and has a good background in data structures and algorithms. So you need this stuff if you really want to skill up, scale up, and take up senior roles.

What are the types of performance testing?

Before I go on to the types of performance testing, I want to mention that the first step to actually testing your API's performance is by writing proper unit, integration, and e2e tests, and making sure to cover as many edge cases as possible. I know it's for functionality, but come on, you don't have those ones present and you're bothered about performance?

Now, to address the elephant in the room, there are various types of performance testing as shown below. For easy understanding and brevity, I used the explain-example-result format.

So, let's start with load testing, which is primarily carried out to evaluate how a system performs under expected user loads or usage.

Scenario:

You are launching an e-commerce website, and you expect 500 users to browse and shop simultaneously during peak hours.

A load test simulates 500 concurrent users browsing the site, adding items to carts, and checking out.
Metrics: Response time, database query times, and server resource utilization under this expected load.
Goal: By identifying slow database queries or underperforming endpoints during load testing, you can optimize these areas to provide a seamless user experience, preventing potential revenue loss during high-traffic periods.

We have tested the system's capability in handling a normal scenario, or expected usage level. Now we want to find out the system's breaking point, by applying a load beyond its capability. That's where Stress Testing comes in.

Scenario:

You want to understand how your social media platform handles traffic spikes, such as during a viral event.

A stress test simulates 50,000 users simultaneously posting, liking, and commenting—far exceeding normal user activity.
Metrics: System stability, error rates, and recovery time after failure.
Goal: By conducting stress tests, you gain insights into failure thresholds and recovery strategies, which are crucial for maintaining user trust and business continuity during high-demand scenarios.

There are cases where we might be expecting the system usage or traffic to increase, maybe double or triple, and you want to make sure you have everything in place, that's what Scalability Testing is for, to test the system's ability to scale up or down with increased load or resource availability.

Scenario:

Your video streaming app plans to add 1 million users next year, doubling the current base.

A scalability test increases the number of concurrent users incrementally (e.g., 10,000 to 50,000) and observes system performance.
Metrics: Server response times, database performance, and horizontal or vertical scaling efficiency.
Goal: Scalability testing helps validate whether your infrastructure investments, such as autoscaling or database sharding, are effective and ensure the app remains competitive as user demands grow.

And now, Volume Testing ---If you're expecting a large number of users, or your application is data intensive, you probably want to make sure your system can handle a high volume of data and still maintain optimal performance.

Scenario:

Your financial application must process and store 10 million daily transactions.

A volume test loads the database with a large dataset (10 million transaction records) and tests retrieval, updates, and queries.
Metrics: Query execution time, disk I/O, and database index performance.
Goal: By validating database performance under high data loads, volume testing ensures seamless operations for end-users and reduces the risk of downtime during critical transactions.

Lastly, we have Endurance Testing (Soak Testing) which is usually carried out to check the system's stability and performance over an extended period.

Scenario:

Your online gaming platform needs to handle 20,000 concurrent players for 48 hours without downtime or memory leaks.
An endurance test simulates continuous gameplay by 20,000 users, including saving progress and multiplayer interactions, over two days.

Metrics: Memory usage, CPU utilization, and error rates over time.
Goal: By uncovering resource leaks or degradation, endurance testing helps maintain user satisfaction and avoids costly downtime, especially in critical applications requiring high availability."

Now that you have discovered some issues in your project, resolving them should be the next thing on your agenda. There are various ways to resolve these issues, some of which are:

Query Optimization:
Here, you want to go back to IDE to improve the efficiency of your DB queries, and access patterns which may have resulted in an ~N+1 selects problem~.
Bottleneck Analysis:
Bottleneck Analysis is usually the go-to first step because it identifies specific parts of a system (hardware or software) that limit performance, and suggests targeted optimizations like indexing, query restructuring, or load balancing. During bottleneck analysis, you're able to pin-point low database queries, overloaded threads, or network latency
Call Tree:
A call tree analysis will help you resolve issues that arise from a hierarchical representation of function calls, such as situations where functions call others and for some reason that process takes a long time to resolve.
Resource Contention Analysis:
Resource Contention Analysis involves examining how shared resources (e.g., CPU, memory, database connections) are utilized and if contention arises. It helps to identify race conditions or deadlocks under high loads and to optimize thread pools.
Diagnostics:
Perhaps you have some diagnostic services like Sentry, New Relic, Datadog, or Prometheus set up in your server, diagnostics involve collecting and analyzing logs, traces, and metrics from these services to identify the root cause of issues during performance testing

Platforms and Tools for performance testing

There are a lot of these tools available, but in other to keep this article short, I will have to mention a few.

In order to run performance testing, you have to leverage tools like PyCharm Profiler / Django Debug Toolbar to pinpoint performance issues in code, k6(Great for APIs and microservices), Load Runner, Locust(for Python), BlazeMeter(Can be integrated in CI/CD pipelines), HammerDB and Database Benchmark for volume testing.

To select which is best for you, consider your technology stack, the scale of the project, ease of use, and your budget(K6, JMeter, and Locust are the go-tos because they are cost-effective and open-source).

Each type of testing ensures your service is prepared for different real-world scenarios, from handling normal traffic to extreme and prolonged conditions.

If you found this article useful, please consider leaving a like or a comment to share your thoughts, opinions, or experience in performance testing.

Understanding Row-level locking in databases.

Stephen Nwankwo — Wed, 16 Oct 2024 16:43:46 +0000

Recently, on an ongoing side project, I was tasked with implementing endpoints to manage API Keys, one of the main requirements is to enforce a limit to the number of keys that can be allocated to a user. The obvious way to approach this would be creating a global variable carrying the maximum number of requestable keys and then checking for the number of user keys in the database against the value of that variable when the create endpoint is queried, then a key is allocated to the user or not. That should be about it, right? No! I would later find out; that enforcing that API key limit would take more than that. This is where row-level locking comes in.

Assume that you are a boring person (you are, and you know it) and only have two friends. One of your friends (let's call him Samuel) who traveled had told you and Ahmed (your second friend) that he had an item delivered to him awaiting pickup, but only one of you is to take his place and pick up the item. Because you're a boring and annoying person, you and Ahmed had a heated argument and stopped talking two days before one of you was supposed to go pick up the item. Unsure as to who will go, because you two had failed to communicate that because of the argument, you both choose to act independently even though you knew that only one of you should go as instructed by Joseph. You go out to do it yourself because you think your friend won't pick it up. Simultaneously, your friend departs with an identical thought process.

You are both surprised to encounter each other at the pickup station, as well as perplexed. Even worse, the station's attendant is unsure about who to give the item to.

Since the terms of the agreement were unclear, this has disrupted the process of obtaining the item. You two have no idea what the other is doing, and if you had coordinated and communicated earlier, this uncertainty would have been averted.

In database terms, this is what happens when row-level locking isn't used properly during critical operations. If multiple requests (like you and your friend) try to access or update the same piece of data (pick up the item), without row-level locking, there's no coordination or restriction to prevent them from working on the same record simultaneously. This can lead to race conditions, data inconsistency, or even conflicts that disrupt the intended workflow.

The case between you and your friend, with the lack of communication, can be tied to the need for setting up row-level locking on the endpoint that handles creating an API key, so as to not allow un-intentional or malicious concurrent request try to bypass the limit.

What is row locking?

By now, you probably have an overview as to what row-level locking is, row-level locking is one of three levels of data locking that involves managing access to the modification or retrieval of a particular item or row, whilst allowing other rows that do not need to be accessed. Other levels of data locking are table locking and page locking. Row-level locking is widely praised over the other two because it increases concurrency and availability by allowing more transactions to access and modify the same table concurrently, without blocking or waiting for each other.

Types of row locking

Row-level locking differs based on the condition and operation we are trying to enforce. They are two most commonly used types of row-level locking, namely:

1. Shared Lock (S Lock)

S-lock allows multiple transactions to read a row but prevents any from writing to it and is useful in cases when transactions only need to read data without making any modification. For example, multiple users can view the same data row, but none can update it until the shared lock is released.

2. Exclusive Lock (X Lock)

X lock is used to prevent any other transaction from reading or writing to the locked row and usually comes into play when a transaction needs to both read and write to a row, ensuring that no other transaction can modify the row until the lock is released. In my case, when a user is requesting to generate an API key, an exclusive lock ensures that the row can't be accessed by another request until the operation is complete.

Other types of row locking include:

Update Lock
Intent Lock
Key-Range Lock
Deadlock Avoidance Locks

Pitfalls of Row-level locking

While row-level locking provides fine-grained control and helps maintain high concurrency, it comes with trade-offs, some of which are:

Deadlocks
Deadlocks happen when two or more transactions are waiting for each other to release locks, which causes a circular wait that cannot be resolved without intervention (e.g., transaction rollbacks). Due to many row-level locks being held by different transactions, the likelihood of deadlocks increases.
Increased Overhead
When managing row-level locks, it often requires more system resources (memory and CPU) than table-level or page-level locks, because the database needs to track each individual row being locked, which can become resource-intensive, especially with high-concurrency environments.
Potential for Lock Escalation
In some databases, if a transaction locks too many rows, the database system might escalate the row-level locks to page-level or even table-level locks to conserve resources which leads to the blocking of other transactions that should have only been blocked on specific rows. Lock escalation can reduce concurrency and cause performance issues in some cases.

Implementation of Row-level locking

Imagine you have a banking app where users can transfer money between accounts. Without row-level locking, two concurrent requests could lead to incorrect balances (e.g., both transactions reading the same balance before updating it, which would cause the account to become overdrawn).

We'll implement row-level locking with Django Rest Framework transaction module and select_for_update() ORM method in other to lock the user account row while updating balances.

After having set up your DRF(Django Rest Framework) project.

Models:

Firstly, we'll create the schema for our database table, named Account, with three columns to carry the owner of the account, and the balance in the account.

from django.db import models

class Account(models.Model):
    user = models.ForeignKey('auth.User', on_delete=models.CASCADE)
    balance = models.DecimalField(max_digits=12, decimal_places=2)

    def __str__(self):
        return f"Account of {self.user.username} with balance {self.balance}"

Serializer:
Then we will setup our serializer to handle three fields that will be passed along with the request.

from rest_framework import serializers
from .models import Account

class TransferSerializer(serializers.Serializer):
    from_account_id = serializers.IntegerField()
    to_account_id = serializers.IntegerField()
    amount = serializers.DecimalField(max_digits=12, decimal_places=2)

    def validate_amount(self, value):
        if value <= 0:
            raise serializers.ValidationError("Transfer amount must be greater than zero.")
        return value

View:

After we have our models in place, our view should look like the below snippet. Here we utilize the transaction module and DRFs select_for_update() ORM method.

from rest_framework import status, views
from rest_framework.response import Response
from django.db import transaction
from .models import Account
from .serializers import TransferSerializer

class InsufficientFundsException(Exception):
    """Custom exception for insufficient funds."""
    pass

class TransferFundsView(views.APIView):

    def post(self, request, *args, **kwargs):
        serializer = TransferSerializer(data=request.data)
        serializer.is_valid(raise_exception=True)

        from_account_id = serializer.validated_data['from_account_id']
        to_account_id = serializer.validated_data['to_account_id']
        amount = serializer.validated_data['amount']

        # Perform the transfer inside a transaction with row-level locking
        with transaction.atomic():
            # Lock the rows for both the sender's and receiver's accounts
            from_account = Account.objects.select_for_update().get(id=from_account_id)
            to_account = Account.objects.select_for_update().get(id=to_account_id)

            # Check if the sender has sufficient balance
            if from_account.balance < amount:
                raise InsufficientFundsException("Insufficient funds in the sender's account.")

            # Perform the transfer by deducting from sender and adding to receiver
            from_account.balance -= amount
            to_account.balance += amount

            # Save both account updates
            from_account.save()
            to_account.save()

        return Response({'status': 'success', 'message': f'Transferred {amount} successfully'}, status=status.HTTP_200_OK)

    def handle_exception(self, exc):
        """Handle custom exceptions."""
        if isinstance(exc, InsufficientFundsException):
            return Response({'detail': str(exc)}, status=status.HTTP_400_BAD_REQUEST)
        return super().handle_exception(exc)

My Advice

Before you utilize any form of data locking, know what you're doing and take into consideration multiple edge cases.

Resources

Row-level database locks explained by Hussein Nasser

Understanding deadlocks by Microsoft

Understanding database concurrency control and race conditions

Connect with me on:

X(Twitter)

GitHub

Understanding the Basics of Optimization Loops in PyTorch

Stephen Nwankwo — Wed, 25 Sep 2024 11:45:38 +0000

cover image was generated with google's gemini.

PyTorch is a powerful, easy-to-use Python deep learning library, mainly used in areas of computer vision and natural language processing, two areas I have a huge interest in. With this article, I hope to help you understand the usage of optimization loops in PyTorch.

When creating a model, we typically divide the dataset into three parts: one for training, which carries a majority of the data, the second for evaluation, and the last for testing/inference. However, in some cases, we only need just the training set and testing set, depending on the project specification. After obtaining these sets, they undergo a series of operations with the goal of minimizing discrepancies between predictions and actual outcomes.

With this foundation, we can determine the loss function used to compute the loss value, followed by the optimizer. Both of which are vital for the optimization loop.

What is the optimization loop?

The optimization loop is an operation that involves performing two loops, the training loop; an operation that involves learning the relationships or patterns between the internal parameters by passing the training data through the model, and the testing loop, which involves passing the testing data through the trained model and evaluating how good the patterns are that the model learned on the training data. These processes, as the title of this article suggests are called "loops" because we want our model to look (loop through) at each sample in each dataset.

We now have a basic understanding of what the training and testing loops are, but how do we achieve them? There are a few steps involved in each of these processes, with the test loop containing fewer steps for reasons we will see later on.

Steps involved in the training loop:

Forward pass: here, we initiate a function that performs a one-time calculation as all of the training dataset is passed through the model at once.
Loss value: using the loss function obtained earlier, compare the training set with the predictions.
Zero gradient: set the gradients of the model parameters to zero, so they can be updated in the following steps.
Backpropagation: during the loop, a number of model parameters to be updated were obtained, now we have to go back and compute the gradient of the loss using those parameters.
Optimizer step: After performing the backpropagation and obtaining the loss gradients, we can then go on to update the model parameters using those results.

To help visualize what this might look like, take a look at Figure 1 below, which is an overview of what the training loop could look like in code. It won't always look like this, it is dependent on the project.

Figure 1: PyTorch training loop [image source]

Steps involved in the testing loop:

Forward pass: same as in step 1 above, except this time we use the test dataset.
Loss value: compute the loss value in the same manner as in the training loop, using test data.
Other evaluation computations: we often need to obtain other metrics to evaluate our model better, some of which are accuracy, precision, recall, F1, etc.

You may have noticed that the testing loop omits the backpropagation and optimizer step. This is because no model parameters are altered during testing; instead, they have already been determined. We are solely interested in the output of the forward run through the model for testing purposes. The annotated image(Figure 2) below illustrates the steps involved in the testing loop.

Figure 2: pytorch testing loop[image source]

Code example

If we added our understanding of the training loop and testing loop, below is what our optimization loop would look like in a basic project.

link to code:
https://github.com/stenwire/playground/blob/main/pytorch/01_pytorch_workflow_exercises.ipynb

code snippet:

# Optimization loop

# Train model for 300 epochs
epochs = 300

# Send data to target device
X_train = X_train.to(device)
y_train = y_train.to(device)
X_test = X_test.to(device)
y_test = y_test.to(device)

for epoch in range(epochs):
  def test():
   ### Perform testing every 20 epochs
    if epoch % 20 == 0:

      # Put model in evaluation mode and setup inference context 
      model.eval()
      # 1. Forward pass
      test_pred = model(X_test)
      # 2. Calculate test loss
      test_loss = loss_fn(test_pred, y_test)

  def train():
      ### Training

      # Put model in train mode
      model.train()

      # 1. Forward pass
      y_pred = model(X_train)

      # 2. Calculate loss
      loss = loss_fn(y_pred, y_train)

      # 3. Zero gradients
      optimizer.zero_grad()

      # 4. Backpropagation
      loss.backward()

      # 5. Step the optimizer
      optimizer.step()

The above code sample covers a PyTorch model that learns the pattern of a straight line and matches it, I would love to see other use cases, so feel free to reach out to me on any of the social platforms I added below, let's learn together.

Connect with me on LinkedIn:
https://www.linkedin.com/in/stephen-nwankwo-9876b4196/

Connect with me on X(Twitter): https://x.com/Sage_Sten

Checkout my Github: https://github.com/stenwire

References:

https://www.learnpytorch.io/01_pytorch_workflow/

https://www.dataquest.io/blog/pytorch-for-beginners/

OpenAI engineers have cooked once more.

Stephen Nwankwo — Fri, 16 Feb 2024 00:14:03 +0000

If you think ChatGPT was revolutionary, get ready to get your mind blown.

Remember the ruckus between openAI's Sam Altman and OpenAI, that was one occasion that placed OpenAI in the headlines for the wrong reasons. Did you happen to hear the gist? I wrote about it, check it out here. Back to the main topic, OpenAI is here once more with something new, different, original, and revolutionary. I know I sound like an excited fanboy right now, truth is I really am excited, perhaps you've seen a couple of deep fake videos, with the camera fixed in one position and a couple of realistic facial features and ear, eye and mouth movement, this is way more than that.

OpenAi has just given us an insight into a new AI they've been working on, an AI that generates a 60-second-long video from user prompts. Here's why it is worth the hype, to name a few:

Hyper-realistic characters
Hyper-realistic motion/movement of different characters and objects/particles
Hyper-realistic lightning
Hyper-realistic camera movements
Hyper-realistic character emotions

You tell me, what makes an amazing video if not all of the above listed, let's all remember we're talking about AI-generated video here, the characters, motion, and lightning look like something straight out of Unreal Engine 5 on steroids, being run on Microsoft supercomputer, it's that amazing. In case you're still in doubt, check the below screenshots from videos created by Sora(name of AI).

Prompt: A beautiful homemade video showing the people of Lagos, Nigeria in the year 2056. Shot with a mobile phone camera.

Prompt: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.

Prompt: Aerial view of Santorini during the blue hour, showcasing the stunning architecture of white Cycladic buildings with blue domes. The caldera views are breathtaking, and the lighting creates a beautiful, serene atmosphere.

Unfortunately, I can't attach a video without hosting it on sites like YouTube or Vimeo, These screenshots are but an abstraction of the divinity created by the company; OpenAI.

The purpose of this post is to get you excited, I hope you are, and get ready, for openAI engineers are still cooking, and this is nothing compared to what is to come.

The Saga of Sam Altman and OpenAI: The story of a controversial ouster and its aftermath.

Stephen Nwankwo — Tue, 21 Nov 2023 13:36:45 +0000

Perhaps, you have not paid attention to the events with OpenAI and Sam Altman that has happened recently, or maybe you need clearity. Let me share with you what happened this weekend.

Who is Sam Altman?

Sam Altman is one of the prominent people in the tech market who co-founded OpenAI and served as president of Y-combinator between 2014-2019. He now leads another team in applied AI research department at Microsoft(more on this later on).

The Unexpected Layoff

On Friday November 17, 2023; OpenAI made a strange move. The firm renowned for its consideration toward AI ethics and safety terminated Sam Altman who is the most passionate supporter of AIs positive impacts on humanity.

This move surprised and disappointed a lot of people as it was brought about by a resolution passed by a board meeting of OpenAIs directors. It also had its intriguing aspect as very few people knew about the identity of most of the members of the board.

The Reason Behind the Layoff

The board made that decision with the claim that it had lost confidence in Sam Altman’s communication style. Some said that he had failed to be “honest” with the board, thus, it was difficult for them to perform their duties as members of the board.

The confusion was further compounded by the fact that the OpenAI investors were allegedly not informed of the activities and projects carried out at the company. However, it could not be determined with certainty what exactly was the cause of Altman’s failure in communication; it was most likely, though, the reason behind was a deficiency of openness.

During these speculations one Reddit user under the user name Anxious_Bandicoot126 offered a more detailed explanation behind the firing of Altman. Altman has been laid-off due to his focus on profits. They also went on to say that Altman prematurely released tech without proper safety assessment for the sake of fame and fortune rather than considering the company’s core values.

The Attempted Reconciliation

After the strong protests from many parties, even from distinguished experts in technology, the OpenAI board somehow comprehended the magnitude of their choice. The board tried to have the layoff rescinded and brought Sam Altman back to the business.

A little bit of history repeated itself with some irony in this instance, as it bore a striking resemblance to another such event that took place in years past involving Apple and Steve Jobs.

At first, it seemed like Altman could return since he believed totally in the goals set for OpenAI. He, however, stipulated that he should only go back after the current OpenAi Board had been completely purged.

However, negotiations collapsed as Open AI refused to accept the condition proposed by Sam Altman.

The Ripple Effect

Altman’s layoff had far-reaching effects that went beyond the boardroom. Moreover, OpenAI received a lot of criticism from the public, thus losing several important executives namely Greg Brockman and Ilya Sutskever who publicly opposed the action.

An ill-fated layoff saw Ilya Sutskever, one of their board members publicly apologies as well as promise re-unification with the company. However, Greg Brockman reacted saying that he was in great shock with the news but will work towards discovering what happened.

Financial implications have been seen in the loss of billions of dollars in the company’s secondary share, and a foreseen 90% drop in valuation in 2024.

The Aftermath and Next Steps

At present, only the names of Sam Altman and Greg Brockman could be mentioned from among several colleagues who just entered into the “Advanced AI research team” of Microsoft, whereas other representatives may take the decision to resign from OpenAI and join Altman.

The present CEO of OpenAI is Emmett Shear (major shareholder in Anthropic – OpenAI’s foremost competitor). He considers a possibility for combining Anthropic with OpenAI, the leading competing organization. Nevertheless, signs of trouble in negotiations have been evident.

Personally, I think that the first thing that OpenAI must do is take care of the uneasiness inside the company. It is imperative for OpenAI to move with speed following employee dissatisfaction and other companies like Salesforce making attractive offers so as to ensure the firm’s internal morale and stability.

OpenAI's Justified Actions

The question remains: WAS SAM ALTMAN JUSTIFIED TO BE FIRED BY OPENAI? It is a multifaceted and complicated response.

Communication is of utmost importance towards building an ethics-oriented company and therefore the necessity of clearly defined policies in place is critical.

On the other hand, Altman's contributions to OpenAI and his unwavering commitment to AI's positive impact on humanity are undeniable.His dismissal is bound to have a great effect on the company and the tech community in general. This decision by openAI raises questions on their leadership and value system. Although the realities of the matter still are not known, there is no doubt that this would cast a shadow on OpenAI’s future.

Is OpenAI action justified? What do you think?

🙋♂️ 𝑰'𝒎 𝒂 𝒔𝒐𝒇𝒕𝒘𝒂𝒓𝒆 𝒆𝒏𝒈𝒊𝒏𝒆𝒆𝒓 𝒘𝒊𝒕𝒉 𝒊𝒏𝒕𝒆𝒓𝒆𝒔𝒕 𝒊𝒏 𝑨𝑰, 𝙨𝙤𝙛𝙩𝙬𝙖𝙧𝙚 𝒅𝒆𝒗𝒆𝒍𝒐𝒑𝒎𝒆𝒏𝒕, 𝒂𝒏𝒅 𝒓𝒐𝒃𝒐𝒕𝒊𝒄𝒔.

𝑭𝒐𝒍𝒍𝒐𝒘 𝒎𝒆 𝒐𝒏 𝑳𝒊𝒏𝒌𝒆𝒅𝑰𝒏: https://www.linkedin.com/in/stephen-nwankwo-9876b4196

HATEOAS Principle - Generating Full Paths for Objects in Django Rest Framework.

Stephen Nwankwo — Wed, 28 Jun 2023 16:48:55 +0000

Back-story: Pssft - Please move closer.
So I'm in this particular developers WhatsApp group, where various topics are discussed, and pair-debugging is carried out, amongst other things. Most precious of them all are the infos that are shared - from job openings to latest frameworks and best practices.
On a accursed day, a link was shared. It was a link to a blog post by MIcrosoft, its title was "RESTful web API design". I decided to read through and I came across a concept, which is "HATEOAS"(chill! more info later on). I could have sworn I have seen it somewhere before, but I have never really considered it until now. Fast-forward, it tingled my fancy ;) So I decided to try to implement it using Django Rest Framework(DRF) and I will be showing you how.

--------------- But firstly, here's what I learned about HATEOAS:

What is HATEOAS principle?

HATEOAS (Hypermedia as the Engine of Application State) is a principle in API design that aims to make APIs more discoverable and self-descriptive. In the simplest terms, it means that an API response should include links or references to related resources and actions that can be performed.

Imagine you're browsing a website and you come across a product page. On that page, you not only see information about the product but also links to related actions like "Add to Cart," "View Reviews," or "Similar Products." These links provide you with a way to navigate the website without having to manually construct URLs or know the entire structure of the website beforehand.

HATEOAS applies the same concept to APIs. Instead of just receiving data in an API response, the response also contains links or references to related resources or actions. These links can guide you on how to interact with the API and discover other relevant information. For example, if you receive data about a particular user, the response may include links to update their profile, retrieve their orders, or view their followers.

By including these links in the API responses, clients consuming the API can navigate and interact with the API more easily. They don't need to rely on prior knowledge or hard-coded URLs but can follow the links provided by the API to access different resources or perform actions.

In summary, HATEOAS simplifies API usage by including links or references within API responses, allowing clients to navigate and interact with the API more dynamically and discover its capabilities without extensive prior knowledge.

Enough talk, Here's what a HATEOAS response looks like:

In this example, we have a response for a book resource. The response includes the book's ID, title, and the author's information. The author is represented as a nested object, including their ID and name. Additionally, both the book and the author objects contain a "links" field.

The "links" field within the book object provides references to related resources and actions. In this case:

The "self" link points to the URL for the book resource itself:
"https://api.example.com/books/1".
The "reviews" link points to the URL where the client can access the reviews for the book: "https://api.example.com/books/1/reviews".
The "similar_books" link points to the URL where the client can find a list of similar books: "https://api.example.com/books/1/similar".

Similarly, the "links" field within the author object includes a "self" link, which provides the URL for the author's resource: "https://api.example.com/authors/1".

These links allow clients to navigate the API by following the provided URLs, without needing to construct them manually or have prior knowledge of the API's structure. Clients can easily access related resources like reviews or find similar books by simply following the appropriate links in the API response.

Basic Implementation in DRF

The implementation here is rather quick and easy, but might not always be the case depending on project scale.

I would assume you already have knowledge in python, Django and REST, so I won't be talking about all the nitty-gritty of things, only when needed.

Now we know what HATEOAS is mostly about - which is attaching hypermedia or resource links to your API json response. With this article I will be showing you a simple approach to creating such link/path.

Requirements:

-- setup virtual environment(for windows):

$> cd project_folder

$> pip install virtualenv

$> python -m venv venv

$> .\venv\Scripts\activate

Set up a Django project:
Create a new Django project by running the following command in your terminal:

   pip install django

   pip install djangorestframework

   django-admin startproject library_project

Create a new Django app:
Navigate to the project directory and create a new Django app, which will handle the logic for books and authors:

   cd library_project
   python manage.py startapp library

Define the models: In the library/models.py file, define the models for books and authors. For example:

   from django.db import models

class Author(models.Model):
    name = models.CharField(max_length=100)

    def __str__(self):
        return self.name

    def get_absolute_url(self):
        return f"/authors/{self.id}/"

class Book(models.Model):
    title = models.CharField(max_length=100)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)

    def __str__(self):
        return self.title

    def get_absolute_url(self):
        return f"/books/{self.id}/"

Create serializers: In the library/serializers.py file, create serializers for the models defined earlier. Include the SerializerMethodField to generate the full path for the id field.

   from rest_framework import serializers
from library.models import Author, Book

class AuthorSerializer(serializers.ModelSerializer):
    detail = serializers.SerializerMethodField()

    def get_detail(self, obj):
        request = self.context.get('request')
        if request:
            return request.build_absolute_uri(obj.get_absolute_url())
        return obj.id

    class Meta:
        model = Author
        fields = '__all__'

class BookSerializer(serializers.ModelSerializer):
    detail = serializers.SerializerMethodField()
    author = AuthorSerializer()

    def get_detail(self, obj):
        request = self.context.get('request')
        if request:
            return request.build_absolute_uri(obj.get_absolute_url())
        return obj.id

    class Meta:
        model = Book
        fields = '__all__'

Define URLs: In the library/urls.py file, define the URLs for the API endpoints.

   from django.urls import path
from library.views import BookListCreateView, BookRetrieveUpdateDestroyView, AuthorListCreateView

urlpatterns = [
    path('books/', BookListCreateView.as_view(), name='book-list-create'),
    path('books/<int:pk>/', BookRetrieveUpdateDestroyView.as_view(), name='book-retrieve-update-destroy'),
    path('authors/', AuthorListCreateView.as_view(), name='book-list-create'),
]

Create views: In the library/views.py file, create views for handling the API endpoints.

   from rest_framework import generics
from library.models import Book, Author
from library.serializers import BookSerializer, AuthorSerializer

class BookListCreateView(generics.ListCreateAPIView):
    queryset = Book.objects.all()
    serializer_class = BookSerializer

class BookRetrieveUpdateDestroyView(generics.RetrieveUpdateDestroyAPIView):
    queryset = Book.objects.all()
    serializer_class = BookSerializer

class AuthorListCreateView(generics.ListCreateAPIView):
    queryset = Author.objects.all()
    serializer_class = AuthorSerializer

Include the app in the project settings: In the library_project/settings.py file, include the library app and configure the REST framework.

   INSTALLED_APPS = [
       # Other apps...
       'library',
       'rest_framework',
   ]

   REST_FRAMEWORK = {
       'DEFAULT_RENDERER_CLASSES': [
           'rest_framework.renderers.JSONRenderer',
       ]
   }

Include a link to the app urls in the project urlr.py file: In the library_project/urls.py file, include:

from django.contrib import admin
from django.urls import path, include # import include

urlpatterns = [
    path('admin/', admin.site.urls),
    # include the library app urls module
    path('', include('library.urls')),
]
   }

Run migrations: Apply the database migrations to create the necessary tables for the models:

   python manage.py makemigrations
   python manage.py migrate

Start the development server: Launch the development server to test the API:

   python manage.py runserver

Here's what the request and response will look like:

Request:

curl --location 'http://127.0.0.1:8000/books/'

Response:

[
    {
        "id": 1,
        "detail": "http://127.0.0.1:8000/books/1/",
        "author": {
            "id": 1,
            "detail": "http://127.0.0.1:8000/authors/1/",
            "name": "Stephen Nwankwo"
        },
        "title": "Tales of Ogun"
    },
    {
        "id": 2,
        "detail": "http://127.0.0.1:8000/books/2/",
        "author": {
            "id": 1,
            "detail": "http://127.0.0.1:8000/authors/1/",
            "name": "Stephen Nwankwo"
        },
        "title": "Tales of Ogun"
    }
]

Do we really need HATEOAS?

I have browsed through a number of APIs, and didn't actually come across HATEOS implementation. So on the issue of whether one needs it or not is really up to you the developer or project manager.

To help you make that decision, depending on the specific requirements and goals of your project. Here are a few points to consider:

Enhanced discoverability: HATEOAS provides a self-descriptive nature to APIs, allowing clients to discover and navigate the available resources dynamically. It eliminates the need for clients to rely on prior knowledge or hard-coded URLs, making the API more intuitive and easier to use.
Reduced coupling: By including links or references to related resources in API responses, HATEOAS reduces the coupling between clients and servers. Clients can follow the provided links to access different resources and perform actions without needing to know the entire API structure in advance. This promotes loose coupling and enables more flexible and extensible systems.
Improved scalability: HATEOAS enables the server to evolve and add new resources or actions without breaking existing clients. Clients that follow links provided in API responses can adapt to changes in the API without requiring modifications to their code. This makes the API more scalable and facilitates versioning and backward compatibility.
Client development simplicity: HATEOAS simplifies client development by providing a guided approach to API interaction. Clients can rely on the links provided in the API responses instead of constructing URLs manually or hard-coding them. This reduces the complexity and potential errors in client code.
Consistency and documentation: HATEOAS promotes consistency and provides a form of self-documentation within the API itself. The links included in the responses serve as a reference to the available actions and resources, making it easier for developers to understand and utilize the API effectively.

However, it's important to note that implementing HATEOAS adds complexity to the API design and development process. It requires careful consideration of the links to include, balancing between providing enough information and avoiding overwhelming clients with excessive links. Additionally, HATEOAS may not be necessary or appropriate for all API use cases, especially when the API's scope is small or the client-server interaction is straightforward.

Ultimately, the decision to use HATEOAS should be based on factors such as the complexity of your API, the desired level of discoverability and flexibility, and the needs of your clients.

Please leave a like if you found the article useful, also feel free to share your opinions in the comment section.

References:

https://learn.microsoft.com/en-us/azure/architecture/best-practices/api-design

https://www.geeksforgeeks.org/hateoas-and-why-its-needed-in-restful-api/

https://www.w3schools.in/restful-web-services/rest-apis-hateoas-concept#google_vignette

CPU and GPU...like I'm 5

Stephen Nwankwo — Wed, 06 Apr 2022 21:38:36 +0000

I will stray away from the technical aspects as I intend to make this as basic as possible.

A computer is a lot like an earth person. The two most important components of the human being are the Heart and the brain. The computer also possesses just like humans its own heart and brain and that is what we'll be learning about today.

Welcome to computational biology 101...(coined that up right now)

What is the CPU?

We all know the CPU, the central processing unit to be the brain of the computer as we were told in grade five, and that is essentially what a CPU is. A CPU is capable of handling all of the affairs of the computer ranging from writing documents, listening to music, watching videos, and being a nuisance on our favorite social media.

What is the GPU?

GPU stands for Graphics Processing Unit... I like to refer to GPU as the heart of the computer. Just like in human beings where the activities performed by the heart are completely independent of the brain, the same goes for the GPU.
The primary activity performed by the heart is the pumping of blood to the brain and the rest of the body, which we all know is a vital activity for the sustenance of an earth person. Just like the heart, the primary purpose of the GPU is to support the CPU in certain areas which will be discussed later on.

Why have a CPU and a GPU?

As mentioned earlier I will try to stray away from the technical aspects as I intend to make this as basic as possible.
I would like to start with the activities carried out by the CPU.
The CPU controls everything in a computer, some of these common tasks carried out by the CPU includes copying a file from the flash drive to the computer, watching an episode of family guy on your video player, trying to calculate the sum of 67822 and 9370, listening to your favorite artist album, typing a document, joining a meeting online, ... the list goes on.
Moving on to why you may need a GPU. to go straight to the point, one does not really need a GPU if you don't play any game from 5years ago, you don't do any video editing, 3d modeling, artificial intelligence, etc.
If you happen to be like me, a gamer and a dev. I would advise you to go for a pc with a CPU and a GPU.
To sum it all up, the GPU does the real heavy lifting for the pc and the CPU handles all other affairs that do not require too much graphic processing.

Why not use a standalone GPU?

The reason why no pc should operate only on a GPU is that the GPU as mighty as it may sound can not really handle a lot of activity from different system applications all at the same time as the CPU.

The major manufacturer of CPUs is Intel and for GPU we have Nvidia and AMD...example of a CPU is the intel 5500 and a GPU is the NVIDIA 3060TI.

I would leave you with a piece of advice; never buy a PC with a low-grade AMD GPU instead go for a Nvidia 800-series if funding might be an issue.

Legend:

I didn't cover a lot, please use the links above.

Forem: Stephen Nwankwo

The State of Security Protocols in Agent 2 Agent(A2A) Systems.

The Foundational A2A Protocol and Secure Interoperability

Assessing the Agentic Threat Landscape

Enhancing Security and Governance: Layered Architectures

Workflow Illustration: Secure A2A Task Delegation using SAGA/Policy Enforcement

Secure Implementation: Code Controls

Building an MCP Server with RAG Capabilities for Developer Tools

What is MCP?

What is RAG?

Project Overview

Architecture

Implementation

Use Cases

Resources

Optimizing SQL queries for your AI Agents.

Become a Data Profiler with EXPLAIN:

Sharpen the Agent's Focus (Tune the Query Itself!):

Create Fast-Access Lanes with Indexes:

Major Overhauls for Complex Agent Needs:

My Takeaway for AI Devs:

Connect with me on:

Learn Google ADK: Build a farming agent.

Meet NaijaFarmHand: Your AI Farming Buddy for Nigeria

What Can NaijaFarmHand Actually Do?

How Does It All Work Together?

Why Build Something Like NaijaFarmHand?

Performance Testing: Is Your API Ready for Real-World Traffic?

What are the types of performance testing?

Platforms and Tools for performance testing

Understanding Row-level locking in databases.

What is row locking?

Types of row locking

Pitfalls of Row-level locking

Implementation of Row-level locking

My Advice

Resources

Connect with me on:

Understanding the Basics of Optimization Loops in PyTorch

What is the optimization loop?

Steps involved in the training loop:

Steps involved in the testing loop:

Code example

References:

OpenAI engineers have cooked once more.

The Saga of Sam Altman and OpenAI: The story of a controversial ouster and its aftermath.

HATEOAS Principle - Generating Full Paths for Objects in Django Rest Framework.

What is HATEOAS principle?

Basic Implementation in DRF

Do we really need HATEOAS?

CPU and GPU...like I'm 5

What is the CPU?

What is the GPU?

Why have a CPU and a GPU?

Why not use a standalone GPU?

Become a Data Profiler with `EXPLAIN`: