Forem: Tarek Oraby

End-to-end testing of Gen AI Apps

Tarek Oraby — Sat, 11 Oct 2025 12:09:39 +0000

The shift to building applications on top of Large Language Models (LLMs) fundamentally breaks traditional software testing. Methodologies designed for predictable, deterministic systems are ill-equipped to handle the non-deterministic nature of LLM-based apps, creating a critical gap in quality assurance. This blog post dives into the specific challenges of end-to-end testing for these apps and introduces an open-source framework, SigmaEval, designed to provide a statistically rigorous solution.

The Challenges of Testing Gen AI Apps

Testing Gen AI applications is fundamentally different from testing traditional software. The core difficulty stems from two interconnected problems that create a cascade of complexity.

The Infinite Input Space: The range of possible user inputs is basically endless. You cannot write enough static test cases to cover every scenario or every subtle variation in how a user might phrase a question. This problem is compounded in conversational AI, where multi-turn chats branch out fast. Even small wording changes in an early turn can dramatically shift how later turns unfold, making the state space of a conversation explode.
Non-Deterministic & "Fuzzy" Outputs: Unlike a traditional function where 2 + 2 is always 4, a Gen AI model can produce a wide variety of responses to the same prompt. Outputs aren't consistent, so a single clean run doesn’t mean much. On top of this, quality itself is fuzzy. There’s rarely a single right answer, and you’re often juggling competing priorities like helpfulness, safety, and tone all at once. Even human judges don’t always agree on what constitutes a "good" response.

These two fundamental challenges are amplified by the complex systems we build around the models. The integration of tools, memory, and RAG adds more moving parts, each a potential point of failure. External APIs or vector indexes drift over time, and even the underlying models can change quietly in the background. When you add operational concerns like costs, rate limits, and flaky integrations, it’s easy to think you’ve tested enough when you really haven’t.

This reality means we must move beyond simple pass/fail checks and adopt a more robust, statistical approach to evaluation. The process becomes less like traditional software testing and more like a clinical trial. The goal isn't to guarantee a specific outcome for every individual interaction, but to ensure the application is effective for a significant portion of users, within a defined risk tolerance.

Introducing SigmaEval: A Statistical Approach to Gen AI Evaluation

SigmaEval is an open-source Python framework designed specifically for the statistical evaluation of Gen AI apps, agents, and bots. It helps you move from "it seems to work" to making statistically rigorous statements about your AI's quality.

With SigmaEval, you can set and enforce objective quality bars, making statements like:

"We are confident that at least 90% of user issues coming into our customer support chatbot will be resolved with a quality score of 8/10 or higher."

SigmaEval addresses the challenges of Gen AI testing through a novel approach that combines AI-driven user simulation, an AI judge, and inferential statistics.

How SigmaEval Works

SigmaEval uses two AI agents to automate the evaluation process: an AI User Simulator and an AI Judge.

Define "Good": You start by defining a test scenario in plain language, describing the user's goal and the successful outcome you expect. This becomes your objective quality bar.
Simulate and Collect Data: The AI User Simulator acts as a test user, interacting with your application based on your scenario. It runs these interactions multiple times to collect a robust dataset.
Judge and Analyze: The AI Judge scores each interaction against your definition of success. SigmaEval then applies statistical methods to these scores to determine if your quality bar has been met with a specified level of confidence.

This process transforms subjective assessments into quantitative, data-driven conclusions.

Getting Started with SigmaEval

Here’s a simple example of how to use SigmaEval to test a customer support chatbot's refund process:

from sigmaeval import SigmaEval, ScenarioTest, assertions
import asyncio
from typing import List, Dict, Any

# 1. Define a realistic, multi-turn ScenarioTest
scenario = (
    ScenarioTest("Customer Support Refund Process")
    .given(
        "A user is interacting with a customer support chatbot for an "
        "e-commerce store called 'GadgetWorld'."
    )
    .when("The user wants to get a refund for a faulty product they received.")
    .expect_behavior(
        "The chatbot should first empathize with the user's situation, then ask for "
        "the order number, and finally explain the next steps for the refund process. "
        "It should not resolve the issue immediately but guide the user effectively.",
        # We want to be confident that at least 90% of interactions will score an 8/10 or higher.
        criteria=assertions.scores.proportion_gte(min_score=8, proportion=0.9)
    )
    .max_turns(10) # Allow for a longer, more complex back-and-forth conversation
)

# 2. Implement the app_handler to connect SigmaEval to your application
async def app_handler(messages: List[Dict[str, str]], state: Any) -> str:
    # This handler is the bridge between SigmaEval and your application.
    # It's responsible for taking the conversation history from the AI User Simulator
    # and getting a response from your app, enabling multi-turn conversation testing.

    # `messages` contains the full conversation history, e.g.:
    # [
    #   {'role': 'user', 'content': 'My new headphones are broken.'},
    #   {'role': 'assistant', 'content': 'I am sorry to hear that. What is your order number?'}
    #   {'role': 'user', 'content': 'It's GADGET-12345.'}
    # ]

    # In a real test, you would make an API call to your application's endpoint here.
    # For example:
    # response = await http_client.post(
    #     "https://your-chatbot-api.com/chat",
    #     json={"messages": messages, "session_id": state}
    # )
    # return response.json()["content"]

    # This part must be implemented to return the actual response from your app.
    raise NotImplementedError("Connect this handler to your application.")


# 3. Initialize SigmaEval and run the evaluation
async def main():
    sigma_eval = SigmaEval(
        judge_model="gemini/gemini-2.5-flash",
        sample_size=20,  # The number of times to run the test
        significance_level=0.05  # Corresponds to a 95% confidence level
    )
    result = await sigma_eval.evaluate(scenario, app_handler)

    # Assert that the test passed for integration with testing frameworks like pytest
    assert result.passed

if __name__ == "__main__":
    # To run this, you would need to implement the app_handler
    # and handle the potential NotImplementedError.
    # asyncio.run(main())
    pass

When this code is executed (after implementing the app_handler), SigmaEval automates the entire end-to-end testing process:

Rubric Generation: The AI Judge first interprets the .expect_behavior() description to generate a detailed 1-10 scoring rubric that defines success for this specific scenario.
User Simulation: The AI User Simulator then runs 20 (sample_size) independent, multi-turn conversations with your application via the app_handler.
AI-Powered Judging: Each of the 20 conversation transcripts is scored against the rubric by the AI Judge.
Statistical Conclusion: SigmaEval performs a hypothesis test on the collected scores.

The final output is a result object. Its result.passed attribute will be True only if the test can conclude, with 95% confidence, that your chatbot meets the quality bar: at least 90% of interactions will score an 8 or higher. The result object also contains a wealth of data for inspection, including the full conversation logs, individual scores, the generated rubric, and the statistical analysis summary.

Core Concepts

SigmaEval uses a fluent builder API with a Given-When-Then pattern:

.given(): Establishes the context for the AI User Simulator.
.when(): Describes the goal or action the user will take.
.expect_behavior() or .expect_metric(): Specifies the expected outcomes, with statistical criteria.

Key Features

Statistical Assertions: Go beyond pass/fail with assertions like proportion_gte (is the performance good enough, most of the time?) and median_gte (is the typical user experience good?).
User Simulation with Various Writing Styles: The AI User Simulator can adopt different writing styles to ensure your app is robust to real-world user communication.
Built-in Metrics: Measure quantitative aspects like response latency and turn count out of the box.
Compatibility with Testing Libraries: Integrates seamlessly with pytest and unittest.

Practical Applications

SigmaEval is designed to integrate into a standard engineering workflow, providing a structured approach to Gen AI quality assurance. Its primary applications are:

CI/CD Integration: The framework's programmatic interface and clear pass/fail criteria (based on statistical confidence) allow it to be used as a quality gate in CI/CD pipelines. This prevents regressions in application behavior that might otherwise go unnoticed.
Objective A/B Testing: When comparing two versions of a prompt, an agent, or an underlying model, SigmaEval can provide statistically significant results on which version performs better against a defined quality bar.
Regression Testing: By running a suite of ScenarioTests, you can create a regression suite to ensure that changes to one part of your system don't negatively impact the holistic user experience.

Resources

SigmaEval is an open-source project. The repository contains the full source code and documentation: https://github.com/SigmaEval/SigmaEval. A dedicated documentation site is also available at docs.sigmaeval.com.

From Code to Cash: Monetizing Python AI Agents ⚡

Tarek Oraby — Fri, 09 May 2025 15:15:02 +0000

So, you've poured your heart into crafting the perfect AI agent. It runs flawlessly on your machine, and you know it solves a real market need. But what's next? Bridging the gap between a locally successful agent and a revenue-generating product is a journey fraught with challenges. Building the agent was just the first step. Now, the real work begins: convincing clients, navigating complex deployments, ensuring reliability, and, ultimately, getting paid for your hard work.

This post is your practical guide to transforming your AI agent into a profitable venture. By the end, you'll understand how to leverage Stripe, GitHub, and other tools to turn your AI creation into a product that generates a steady stream of revenue. Specifically, we will cover:

Implementing outcome-based pricing for your agent.
Deploying your agent as an API service.
Managing customer payments and subscriptions (using Stripe, GitHub, and other tools).

Why Embrace Outcome-Based Pricing?

Outcome-based pricing is a model where customers pay based on the tangible results they achieve. It's an increasingly popular strategy for AI agents because it perfectly aligns your interests as the provider with those of your customer. Clients pay only when they receive measurable business value, and you earn only when your agent delivers that value. With an agreed-upon unit price for each resolved ticket, secured appointment, or completed transaction, both you and your customer can instantly quantify the return on investment (ROI).

Growing evidence suggests that outcome-based pricing can lead to shorter sales cycles and higher conversion rates. This is likely because it removes the uncertainty surrounding your agent's value proposition, making it an easier "yes" for potential customers. Furthermore, it reassures clients of your commitment to ongoing maintenance and improvement, as they know your revenue stream depends on the agent's continued performance.

Compared to bespoke development projects, fixed-price contracts, or traditional subscription models, outcome-based pricing offers greater transparency and fairness. The primary challenge lies in its technical implementation, which requires precise metering, attribution, and analytics. In this post, we'll demonstrate how to implement this pricing model using Stripe and integrate it with a deployment platform.

Example: A Lead Enrichment Agent

To illustrate the process, let's consider an AI agent designed to monitor a Google Sheet of incoming leads and automatically enrich each one with additional information sourced from the web. (You can find a reference implementation of this agent in this GitHub repository). At a high level, the agent functions as follows:

A Google Apps Script Web app monitors the Google Sheet where leads are stored.
When a new lead is added, the Apps Script sends a POST request containing the lead's email to the agent's API service (which we will set up).
The agent, a Python script utilizing Gemini's Grounding with Google Search and structured outputs, scours the web for information about the lead and their company. It then sends a response back to the Apps Script's web app URL.
The Apps Script updates the Google Sheet with the enriched lead data.

Here's a simplified snippet illustrating the agent's core functionality (the complete code is available in the GitHub repository):

def process_and_callback(request_data):
    try:
        email = request_data.rowData.Email
        if not email:
            # Simplified error handling for brevity
            return {"status": "Error: Email not found"}, 400

        enriched_data = lead_enrichment_agent(email)
        processed_data = enriched_data.__dict__
        callback_status = enriched_data.ProcessingStatus or "Completed"

        callback_payload = {
            "callbackInfo": dataclasses.asdict(request_data.callbackInfo),
            "processedData": processed_data,
            "status": callback_status,
        }
        callback_url = os.environ.get("CALLBACK_URL")
        # Assume CALLBACK_URL is always set for this simplified version
        requests.post(callback_url, json=callback_payload, headers={"Content-Type": "application/json"})
        return {"status": "success"}, 200
    except Exception as e:
        # Simplified error callback for brevity
        error_payload = {"status": f"Error: {str(e)}"}
        # Attempt to send minimal error info
        if callback_url: # Check if callback_url was retrieved
             requests.post(callback_url, json=error_payload)
        return {"status": "error"}, 500

Now, imagine you've successfully demoed this agent to a potential customer. They're convinced it's a perfect fit for their needs and are comfortable with the outcome-based pricing model. To deploy the agent for them, we need to:

Create a Stripe subscription for the customer.
Package the agent code to accept API requests.
Deploy the agent as an API service.
Enable the agent to be triggered and ensure the customer is billed for usage.

Let's delve into each of these steps.

Step 1: Setting Up Stripe

First, you'll need a Stripe account. If you don't have one, you can create it here. Once your account is active, navigate to the Stripe dashboard and find the "Products" section:

Click "Create Product". Provide a name (e.g., "Lead Enrichment Agent") and an optional description.
In the pricing section, click "More pricing options". Ensure "Recurring" is selected, then choose "Usage-based pricing" and "Per unit". Set a price per unit for your product (e.g., $1.00 USD). This will be the price per successfully enriched lead.
Under the "Usage" section, click the add button next to the "Meter" field. This opens a dialog to define how usage is tracked and billed.
- Give your meter a name (e.g., "Leads Enrichment Meter").
- Assign an event name (e.g., "leads_enrichment_meter").
- Click "Create meter" to save.
With the new meter selected, you'll return to the pricing section. Click "Next" to go back to the "Add a product" form.
Finally, click "Add product" to complete the product creation.

Now you have a product ready for billing. Next, add your client as a Stripe "Customer" and create a new subscription for them:

Go to the "Customers" section and click "Add customer". Fill in their details, such as name, email, and preferred billing information.
Navigate to the "Subscriptions" section and click "Create subscription".
- In the form, select the customer you just created.
- Select the product you configured earlier.
- Save the subscription.

That's it! You've established the necessary Stripe product and customer setup to bill for your agent's services. While Stripe offers many advanced features, this covers the essentials to get you started.

Step 2: Preparing Your Agent for Deployment

To deploy our agent as an API service, we'll use Itura's deployment platform. Itura simplifies deploying your agent as an API, monitoring its execution, and integrating with GitHub for automatic deployments upon pull request merges. Crucially, Itura also integrates with Stripe to send meter events when your agent is triggered.

Preparing your agent for Itura deployment is straightforward: wrap its core logic in a Flask endpoint. This endpoint will receive data from the customer's webhook and trigger the agent. Upon successful execution, the endpoint sends a callback to the customer's webhook with the results. Itura handles updating the Stripe meter with the consumed units.

For our Lead Enrichment Agent, the Flask endpoint will look like this:

from flask import Flask, request
from agent import process_and_callback, RequestData # Assuming RequestData is defined elsewhere

app = Flask(__name__)

@app.route("/run", methods=["POST"])
def run_agent():
    data = request.get_json()

    # Extract data using type-safe models.
    # Ensure RequestData.from_dict() is implemented to parse the JSON payload
    request_data = RequestData.from_dict(data)

    return process_and_callback(request_data)

That's the core change! Simply wrap your agent's logic in a Flask endpoint and define a RequestData model (or similar mechanism) to parse incoming data. Additionally, ensure you have a requirements.txt file specifying the agent's dependencies (easily generated by running pip freeze > requirements.txt in your project's virtual environment).

With the Flask endpoint and requirements.txt file in place, push your code to a GitHub repository. Now you're ready for deployment.

Step 3: Deploying the Agent as an API Service

To deploy your agent, create an account at Itura. During signup, you'll be prompted to connect your GitHub and Stripe accounts. Once connected, create a new agent deployment:

Click the "New Agent" button.
Select the GitHub repository containing your agent's code.

The deployment process will take a couple of minutes. Once complete, you'll receive the agent's URL, which will look something like this:

https://{agentSlug}.agent.itura.ai/run

This URL is used to trigger your agent. However, before you can do that, you need to link the Stripe customer ID and subscription with the agent deployment in Itura. This enables Itura to automatically send meter events to Stripe when the agent is triggered and to invalidate the agent's API key if the subscription is canceled.

In your Itura agent dashboard:

Click on "Customers".
Add the customer by looking up their account from your connected Stripe account.
Select the subscription you created earlier for this customer.

Once the customer is added, you'll likely need to:

Create a new API key for this customer: Navigate to the "API keys" view and select the customer you just added. Itura will generate a new API key for them.
Add necessary environment variables: For the Lead Enrichment Agent, you'll need to add CALLBACK_URL and GEMINI_API_KEY. These environment variables (especially CALLBACK_URL) should ideally be associated with a specific customer, allowing you to use the same agent deployment for multiple clients, each with their unique configurations.

Step 4: Triggering the Agent and Billing the Customer

With everything set up, your agent is ready to be triggered using its unique URL and the customer-specific API key. For example, the following cURL command will trigger the agent:

curl -X POST \
  -H "Authorization: Bearer {customer-specific-api-key}" \
  -H "Content-Type: application/json" \
  -d '{"callbackInfo": {"rowNumber": 1, "spreadsheetId": "your_spreadsheet_id"}, "rowData": {"Email": "john.doe@example.com"}}' \
  https://{agentSlug}.agent.itura.ai/run

Behind the scenes, Itura will inspect the provided API key, identify the associated customer, and trigger the agent, injecting the customer's environment variables into the serverless execution environment. If the agent executes successfully, Itura sends an event to Stripe, reporting 1 unit of subscription consumption. The customer will then be billed accordingly at the end of their billing cycle. If needed, you can also customize your agent's logic to return a specific number of units consumed by including a "billedUnits" field in its response.

Conclusion

Transforming a locally successful Python AI agent into a consistent revenue stream presents unique challenges, from articulating its value to managing deployments and ensuring ongoing reliability. However, by adopting an outcome-based pricing model and leveraging the right tools, these hurdles can be effectively overcome.

This post has guided you through a practical approach to productizing your AI agent. We've explored how:

Outcome-based pricing aligns your incentives with your customer's, simplifying sales by focusing on tangible results and ROI.
Stripe can be configured to support this model, enabling you to meter usage and bill customers based on the value your agent delivers.
Preparing your agent with a simple Flask API makes it ready for scalable, cloud-based deployment.
A platform like Itura can drastically simplify deployment, customer management, and billing integration, allowing you to deploy, monitor, and manage customer-specific configurations and API keys with ease.

By following these steps, you can shift your focus from operational complexities to what you do best: building intelligent AI agents that solve real-world problems. The path from a clever script to a profitable product is now clearer, empowering you to turn your AI creations into sustainable and scalable businesses.

Deploy CrewAI as an API service (in 10 min)⚡

Tarek Oraby — Wed, 16 Apr 2025 12:46:57 +0000

This guide walks you through the process of deploying a CrewAI agent to a cloud endpoint. We'll use Itura's free tier to host the agent in a serverless environment. The deployment will be connected to a GitHub repository, so any changes to the code will automatically be reflected in the deployment.

(This guide is also available in video format: https://youtu.be/HFWlnIfXc_o)

What you need

About 15 minutes
A GitHub account
A basic understanding of CrewAI
The uv package manager (the default package manager for CrewAI)
An OpenAI API key (or other LLM API key)

What you will deploy

You will deploy a simple CrewAI agent to a serverless cloud endpoint. The endpoint will accept POST requests at a URL similar to https://<your-agent-slug>.agent.itura.ai/run. When a POST request is sent to this URL, the agent will be initiated and will start executing. Environment variables can be added to the agent, which will be injected at runtime.

Our starting point is a simple CrewAI crew consisting of two agents that work together to create a research report on the current state of LLMs. The key entry point for the crew is the main.py file.

#!/usr/bin/env python
from crewai_deployment_example.crew import CrewaiDeploymentExample

def run():
    """
    Run the crew.
    """
    inputs = {
        'topic': 'AI LLMs',
        'current_year': '2025'
    }

    CrewaiDeploymentExample().crew().kickoff(inputs=inputs)

Optionally, if you want, run the agent locally by adding the necessary entries to the .env file. For example:

MODEL=gpt-4o-mini
OPENAI_API_KEY=<your-openai-api-key>

Then using the CrewAI CLI

crewai run

How to complete this guide

Download and unzip the source repository for this guide, or clone it using Git: git clone https://github.com/Itura-AI/crewai-deployment-example.git
cd into crewai-deployment-example

When you finish, you can check your work against the code in the crewai-deployment-example/complete branch.

Deploying a CrewAI agent

Step 1: Install Flask

To deploy the agent to the Itura cloud platform, we'll need to create a simple HTTP endpoint that will be used to initiate the agent process. We'll use Flask to create this endpoint. First, install Flask using the uv package manager.

uv add flask

Step 2: Create a /run endpoint

To initiate the agent process, Itura looks for a /run endpoint that accepts POST requests. Let's add this endpoint to our agent code. Adjust the main.py file to include the following:

#!/usr/bin/env python
from crewai_deployment_example.crew import CrewaiDeploymentExample

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route('/run', methods=['POST'])
def run():
    """
    Run the crew.
    """
    inputs = {
        'topic': 'AI LLMs',
        'current_year': '2025'
    }

    # You can get the inputs from the request body
    # inputs = request.json

    try:
        result = CrewaiDeploymentExample().crew().kickoff(inputs=inputs)
        return jsonify({"output": result.raw})
    except Exception as e:
        return jsonify({"error": str(e)})

## This is optional,
## but uncomment if you want to run the Flask server locally
# if __name__ == '__main__':
#   app.run(host='0.0.0.0', port=5000)

Step 3: Create a requirements.txt file

You need to create a requirements.txt file to specify the dependencies for our agent. Itura will use this file to install the dependencies for our agent when it is deployed.

uv pip compile pyproject.toml -o requirements.txt

Once you have the Flask run endpoint and the requirements.txt file, push your code to a GitHub repository.

Step 4: Deploy to Itura Cloud

Go to app.itura.ai and create a new agent project. You'll be prompted to connect your GitHub account and select the repository and branch you want to deploy. Select the branch holding your agent code (with the Flask run endpoint and requirements.txt file), and click Deploy.

Note that the deployment might take a couple of minutes to complete.

Once the deployment is complete, you'll be able to see an auto-generated API key (e.g., sk-agent-aa3f96a3-43e9-448f-ad94-84a38e64c229). Save this key in a secure location. You can generate a new key at any time from the project dashboard.

Step 5: Add environment variables

When the deployment is complete, you will also be able to add environment variables to the agent. These will be injected at runtime with each request to the agent endpoint. Add the OpenAI API key as an environment variable from the UI.

Step 6: Initiate the agent

Now that the agent is deployed, you can initiate one or more instances of it by sending a POST request to the agent endpoint. From the project dashboard, you can see the agent endpoint URL (e.g., https://<your-agent-slug>.agent.itura.ai/run). Using this URL and the API key, you can initiate the agent by sending a POST request to the endpoint.

curl --request POST \
  --url https://{agentSlug}.agent.itura.ai/run \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "input": "Your agent parameters here"
}'

If successful, you'll receive a 202 Accepted response. This means your request is queued.

{
  "run_id": "unique-run-id",
  "message": "Run request accepted and queued for execution",
  "status": "PENDING"
}

You can check the status of the run from Itura's dashboard. Or, by sending a GET request to the /status endpoint.

NOTE* The status endpoint is provided by Itura platform. So, you don't need to add it to your code.

curl --request GET \
  --url https://{agentSlug}.agent.itura.ai/status/{run_id} \
  --header 'Authorization: Bearer <token>'

Conclusion

That's it! You've just deployed a CrewAI agent to the cloud and initiated it using a simple HTTP request. From the Itura dashboard, you can see the agent's logs, metrics, and more. Though not covered in this guide, you can also parse the POST request body to the agent. This way, you can pass more complex data to the agent as input.

If you want to update the deployment code, you can do so by pushing a new commit to the GitHub repository. Itura will automatically detect the changes and update the deployment.

To learn more about itura, visit itura.ai.

Spring Boot and React in Harmony

Tarek Oraby — Sat, 30 Sep 2023 16:59:34 +0000

For many full-stack developers, the combination of Spring Boot and React has become a staple in building dynamic business applications. Yet, while powerful, this pairing has its set of challenges. From type-related errors to collaboration hurdles, developers often find themselves navigating a maze of everyday issues.

Enter Hilla, a framework that aims to simplify this landscape. If Hilla hasn't crossed your radar yet, this article will provide an overview of what it offers and how it can potentially streamline your development process when working with Spring Boot and React.

Spring Boot, React, and Hilla

For full-stack developers, the combination of Java on the backend and React (with TypeScript) on the frontend offers a compelling blend of reliability and dynamism. Java, renowned for its robust type system, ensures data behaves predictably, catching potential errors at compile-time. Meanwhile, TypeScript brings a similar layer of type safety to the JavaScript world, enhancing React's capabilities and ensuring components handle data as expected.

However, while both Java and TypeScript offer individual type-safe havens, there's often a missing link: ensuring that this type-safety is consistent from the backend all the way to the frontend. This is where the benefits of Hilla shine, enabling

End-to-End Type Safety
Direct Communication Between React and Spring Services
Consistent Data Validation and Type Safety

End-To-End Type Safety

Hilla takes type safety a step further by ensuring it spans the entire development spectrum. Developers spend less time perusing API documentation and more time coding. With automatically generated TypeScript services and data types, Hilla allows developers to explore APIs directly within their IDE. This seamless integration means that if any code is altered, whether on the frontend or backend, any inconsistencies will trigger a compile-time error, ensuring that issues are caught early and rectified.

Direct Communication Between React and Spring Services

With Hilla, the cumbersome process of managing endpoints or deciphering complex queries becomes a thing of the past. Developers can directly call Spring Boot services from their React client, receiving precisely what's needed. This is achieved by making a Spring @Service available to the browser using Hilla's @BrowserCallable annotation. This direct communication streamlines data exchange, ensuring that the frontend gets exactly what it expects without any unnecessary overhead.

Here's how it works. First, you add @BrowserCallable annotation to your Spring Service:

@BrowserCallable 
@Service
public class CustomerService {
    public List<Customer> getCustomers() {
        // Fetch customers from DB or API
    }
}

Based on this annotation, Hilla auto-generates TypeScript types and clients that enable calling the Java backend in a type-checkable way from the frontend (no need to declare any REST endpoints):

function CustomerList() {
    // Customer type is automatically generated by Hilla
    const [customers, setCustomers] = useState<Customer[]>([]);

    useEffect(() => {
      CustomerService.getCustomers().then(setCustomers);
    }, []);

    return (
      <ComboBox items={customers} ></ComboBox>
    )
}

Consistent Data Validation and Type Safety

One of the standout features of Hilla is its ability to maintain data validation consistency across the stack. By defining data validation rules once on the backend, Hilla auto-generates TypeScript validations for the frontend. This not only enhances developer productivity but also ensures that data remains consistent, regardless of where it's being processed.

For instance, if a field is marked as @NotBlank in Java, Hilla ensures that the same validation is applied when this data is processed in the React frontend.

public class Customer {

    @NotBlank(message = "Name is mandatory")
    private String name;

    @NotBlank(message = "Email is mandatory")
    @Email
    private String email;

    // Getters and setters

}

The Hilla useForm hook uses the generated TypeScript model to apply the validation rules to the form fields.

function CustomerForm() {

    const {model, field, read, submit} = useForm(CustomerModel, {
        onSubmit: CustomerService.saveCustomer
    });

    return (
    <div>
      <TextField label="Name" {...field(model.name)} />
      <EmailField label="Email" {...field(model.email)} />
      <Button onClick={submit}>Save</Button>
    </div>
    )
}

Batteries and Guardrails Included

Hilla streamlines full-stack development by offering pre-built tools, enhancing real-time capabilities, prioritizing security, and ensuring long-term adaptability.

The framework provides a set of pre-built UI components designed specifically for data-intensive applications. These components range from data grids and forms to various select components and editors. Moreover, for those looking to implement real-time features, Hilla simplifies the process with its support for reactive endpoints, removing the need for manual WebSocket management. Another notable aspect of Hilla is its security integration. By default, it's connected with Spring Security, offering robust access control mechanisms to safeguard data exchanges. The framework's stateless design ensures that as user demands increase, the application remains efficient.

Hilla's design approach not only streamlines the current development process but also future-proofs your app. It ensures that all components integrate seamlessly, making updates, especially transitioning from one version to another, straightforward and hassle-free.

In Conclusion

Navigating the complexities of full-stack development in Spring Boot and React can be complex. This article highlighted how Hilla can alleviate many of these challenges. From ensuring seamless type safety to simplifying real-time integrations and bolstering security, Hilla stands out as a comprehensive solution. Its forward-thinking design ensures that as the tech landscape evolves, your applications remain adaptable and updates remain straightforward.

For those immersed in the world of Spring Boot and React, considering Hilla might be a step in the right direction. It's more than just a framework; it's a pathway to streamlined and future-ready development.

How to style a Vaadin Application

Tarek Oraby — Fri, 11 Mar 2022 08:40:31 +0000

In this guide, we learn the basics of styling a Vaadin application using Cascading Style Sheets (CSS).

While it’s possible to style some parts of a Vaadin application using the Java API, this approach tends to be rather limited for mid- to large-sized applications. So, it is recommended to do the styling of a Vaadin application by adding CSS inside standard style sheets. This guide focuses only on the recommended style-sheets approach.

The following topics are discussed in this guide:

Create a custom theme
Where to place your CSS
How to override Lumo values
How to selectively style views and components
When to add CSS under the /components directory
How to selectively style the internals of a Vaadin component
How to selectively style a sub-component of a Vaadin component

Create a custom theme

To start, we need to inform Vaadin about the location in which we will place our CSS. The easiest way to do this is to add the @Theme annotation on a class that implements AppShellConfigurator.

For example, in a Spring Boot application, you could have an application configuration class such as the following:

@Theme("myapp")
public class Application extends SpringBootServletInitializer
                         implements AppShellConfigurator {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

}

By default, projects downloaded from start.vaadin.com come with the @Theme annotation added to the Application class.

Where to place your CSS

Custom CSS should be placed inside a specific folder located under the frontend/themes directory in your project. This directory has something like the following minimal structure:

frontend
└── themes              
    └── myapp        
        ├── components/ 
        └── styles.css

NOTE: the myapp sub-folder might be named differently in your case. This folder should have the same name as the one used for the custom theme using the @Theme annotation.

Custom CSS should be placed in one of the following locations:

Inside the styles.css file
As a separate CSS file, which is imported from within styles.css
Under the components/ directory in order to style the (local CSS) internals of Vaadin components (such as a Vaadin Button)

The first and simplest option is to place the custom CSS directly inside the styles.css file. For example, adding the following snippet to styles.css would change the font family and size of the whole Vaadin application.

* {
    font-family: "Lucida Console", "Courier New", monospace;
    font-size: 1rem;
}

However, as the number of custom stylings increases, it becomes convenient to organize the styles into separate .css files. In order to do this, simply create a new CSS file and import that from within styles.css.

For example, if we create a new CSS file called main-view.css at frontend/themes/myapp/views/main-view.css, then we would import this file by adding the following line to styles.css:

@import url('./views/main-view.css');

Further below, we will discuss the scenarios in which we should add our CSS to the components/ directory.

How to override Lumo values

The default look and feel of a Vaadin application is based on a built-in theme, called Lumo. This theme is essentially a bunch of CSS custom properties (such as colors, typography, and sizes) that each has a CSS variable assigned to it.

One easy way to customize the look and feel of our application requires us to override the default values of the Lumo variables.

For example, suppose our application has a bunch of TextField, ComboBox, DatePicker, and Button components. By default, they will look as follows:

Suppose that we want to increase the roundedness of their corners. By default, the Lumo theme provides these components with a small rounded corner whose value is defined in the --lumo-border-radius-m variable. In order to increase the roundedness of the corners, we can override the default value of this variable. Specifically, we can add the following inside the styles.css file in order to increase the default roundedness value:

html {
    --lumo-border-radius-m: 1em;
}

This style will increase the corner roundedness of many components at once, so that they will look similar to the following screenshot:

But what if one wants to override the defaults for only a subset of components? No problem; simply use the name of the components as the CSS selector.

For example, if one wants to override the rounded corner defaults for only the TextField and ComboBox components, then the following should be added inside the styles.css file:

vaadin-text-field, vaadin-combo-box {
    --lumo-border-radius-m: 1em;
}

This will change the defaults for the TextField and ComboBox only, leaving other components, such as the DatePicker and Button, with their default values.

How to selectively style views and components

So far, we have been discussing ways to style all components belonging to a certain kind in the same way. For example, if we want to set the width of all Button components within our application, we could add something like the following to the styles.css file:

vaadin-button {
    width: 140px;
}

However, we often want to style the same kind of component differently, depending on its function within our application. For example, we might want to style the Button component differently in our application, depending on whether it is a “Signup” button or a “Delete account!” button.

The easiest way to style views and components selectively is by providing them with a CSS class name from the Java API. All Vaadin components and views have an addClassNames() method that can be used for this purpose.

To illustrate, the following view makes use of the addClassNames() method to assign a CSS class name to the view itself, as well as to the components that it includes.

import com.vaadin.flow.component.button.Button;
import com.vaadin.flow.component.html.H1;
import com.vaadin.flow.component.html.Paragraph;
import com.vaadin.flow.component.orderedlayout.VerticalLayout;
import com.vaadin.flow.router.Route;

@Route("")
public class MyView extends VerticalLayout {

    public MyView() {
        addClassNames("my-view"); //CSS classname for the whole view

        H1 heading = new H1("My View Title");
        Paragraph text = new Paragraph("Thanks for shopping with us. Click the button to submit your order.");

        Button submitButton = new Button("Submit");
        submitButton.addClassNames("submit-button", "wide-button"); //CSS classnames for the submit button

        Button cancelButton = new Button("Cancel");
        cancelButton.addClassNames("cancel-button", "wide-button"); //CSS classnames for the cancel button

        add(heading, text, submitButton, cancelButton);

    }
}

These CSS class names can then be used to selectively style this view and its components. For example, adding the following to styles.css demonstrates how selective styling can be achieved.

.my-view  {
    padding: 1em;
}

.my-view h1 {
    /*Only applies to the h1 headers inside my-view*/
    margin-top: 0;
}

p {
    /*Applies to all p elements regardless of whether they are
     located inside my-view*/
    font-size: 1.2em;
}

.submit-button {
    color: green;
}

.cancel-button {
    color: red;
}

.wide-button {
    width: 120px;
}

Now after the selective styling is applied, MyView will look as follows:

When to add CSS under the `/components` directory

Customizing styling using the two aforementioned options (namely by placing styles inside styles.css, or inside a separate file imported from within styles.css) should work for views and layouts created within the project. However, these two options are not guaranteed to work if one aims to style the internals of a Vaadin component.

Custom styling of the internals of Vaadin components, such as the Grid or ComboBox components, needs to be placed under the frontend/themes/myapp/components/ directory. (Note again that the myapp folder name might be different in your case.)

For example, let’s assume we want to increase the size of the toggle icon that opens the dropdown menu of a ComboBox.

If we inspect this toggle in Chrome (right-click on the toggle and select the Inspect option), we will see that it has an attribute called part whose value is equal to toggle-button.

To style this part, we need to create a file called vaadin-combo-box.css and place it under the frontend/themes/myapp/components/ directory. In this file, we can do something like the following to increase the size of the ComboBox toggle.

[part="toggle-button"] {
    font-size: 2em;
}

Note that the files placed under the /components directory have to exactly match the Vaadin component names as they appear in the DOM. For example, the ComboBox component appears in the DOM as vaadin-combo-box. Accordingly, the styling file of the ComboBox has to be named exactly vaadin-combo-box.css.

How to selectively style the internals of a Vaadin component

We just saw how to style the internals of a Vaadin component by using the part property and placing the CSS inside a specific file placed under the frontend/themes/myapp/components/ directory. Specifically, we saw how to style the ComboBox toggle using the [part="toggle-button"] selector and placing the CSS in a file called vaadin-combo-box.css that resides under the frontend/themes/myapp/components/ directory.

However, this approach will style all toggle elements of all the ComboBox components in our application in the same way. What if we want our styling to apply to only some toggle elements but not to others?

Here we can follow the same approach as we did above to selectively style views and components. That is, we can provide the component with a CSS class name from the Java API using the addClassNames() method. Then we can use this CSS class name to selectively style the internal part of the component.

For example, suppose that our application has the following two ComboBox components:

ComboBox<String> firstCombo = new ComboBox<>();
comboBox.setItems("Earth", "Mars");

ComboBox<String> secondCombo = new ComboBox<>();
comboBox.setItems("Mercury", "Venus");

If we want to style the toggle part of these two components differently, we can do this by providing each ComboBox with a different classname as follows

firstCombo.addClassNames("first-combo");
secondCombo.addClassNames("second-combo");

Then, in frontend/themes/myapp/components/vaadin-combo-box.css, we can selectively style the toggle element of the two components differently by doing something like:

:host(.first-combo) [part="toggle-button"] {
    font-size: 2em;
}

:host(.second-combo) [part="toggle-button"] {
    font-size: 1em;
}

Notice that we have to use here the :host() shadow function because the toggle-button part is located inside the shadow DOM of the ComboBox.

How to selectively style a sub-component of a Vaadin component

There are cases in which we cannot use CSS classnames to do the selective styling. Specifically, we cannot rely on CSS classnames when we want to style the sub-components of a Vaadin component.

What do I mean by a sub-component? Some Vaadin components create sub-components that exist in the DOM independently from the Vaadin component itself. This is noticeably the case for the Vaadin components, such as the DatePicker, Dialog, and ComboBox, that open an overlay during user interaction. For example, a ComboBox adds a vaadin-combo-box-overlay element in the DOM when the user clicks on the toggle button to see the list of the available options. This vaadin-combo-box-overlay element exists in the DOM independently from the vaadin-combo-box element itself.

The fact that they exist in the DOM independently from their parent component means that a CSS classname that is given to the parent element is not propagated down to the sub-component. For example, if we give a ComboBox a classname from the Java API, only the vaadin-combo-box element will get this classname in the DOM, not the vaadin-combo-box-overlay element.

Instead of classnames, we can rely on the theme attribute as a selector in order to selectively style a sub-component of a Vaadin component. Unlike the classnames, the theme attribute has the benefit of propagating through to the sub-components.

We can provide a component with a theme attribute from the Java API using the addThemeName() method.

For example, like before suppose that our application has the following two ComboBox components:

ComboBox<String> firstCombo = new ComboBox<>();
firstCombo.setItems("Earth", "Mars");

ComboBox<String> secondCombo = new ComboBox<>();
secondCombo.setItems("Mercury", "Venus");

Suppose that we want to change the background color of the drop-down list of items that the user sees when they open the ComboBox. This background color is controlled by the background-color property of the overlay part of the vaadin-combo-box-overlay element. This overlay part in the DOM is highlighted in the following screenshot.

To selectively style the background of the overlay of the two ComboBox components, we can first give each one of them a theme name using the Java API as follows:

firstCombo.addThemeName("first-combo");
secondCombo.addThemeName("second-combo");

Then, we need to create a new file called vaadin-combo-box-overlay.css and place it under the frontend/themes/myapp/components directory.

In this file, we can selectively style the overlay part of the vaadin-combo-box-overlay element of the two ComboBoxes by doing something like:

:host([theme="first-combo"]) [part="overlay"] {
    background-color: rosybrown;
}

:host([theme="second-combo"]) [part="overlay"] {
    background-color: aquamarine;
}

Here again we are using the :host() shadow function because the overlay part is located inside the shadow DOM of the vaadin-combo-box-overlay element.

Conclusion

In this guide, we learned the basics of styling a Vaadin application using CSS. Be sure to check the official Vaadin documentation for more advanced use cases.

Forem: Tarek Oraby

End-to-end testing of Gen AI Apps

The Challenges of Testing Gen AI Apps

Introducing SigmaEval: A Statistical Approach to Gen AI Evaluation

How SigmaEval Works

Getting Started with SigmaEval

Core Concepts

Key Features

Practical Applications

Resources

From Code to Cash: Monetizing Python AI Agents ⚡

Why Embrace Outcome-Based Pricing?

Example: A Lead Enrichment Agent

Step 1: Setting Up Stripe

Step 2: Preparing Your Agent for Deployment

Step 3: Deploying the Agent as an API Service

Step 4: Triggering the Agent and Billing the Customer

Conclusion

Deploy CrewAI as an API service (in 10 min)⚡

What you need

What you will deploy

How to complete this guide

Deploying a CrewAI agent

Step 1: Install Flask

Step 2: Create a /run endpoint

Step 3: Create a requirements.txt file

Step 4: Deploy to Itura Cloud

Step 5: Add environment variables

Step 6: Initiate the agent

Conclusion

Spring Boot and React in Harmony

Spring Boot, React, and Hilla

End-To-End Type Safety

Direct Communication Between React and Spring Services

Consistent Data Validation and Type Safety

Batteries and Guardrails Included

In Conclusion

How to style a Vaadin Application

Create a custom theme

Where to place your CSS

How to override Lumo values

How to selectively style views and components

When to add CSS under the /components directory

How to selectively style the internals of a Vaadin component

How to selectively style a sub-component of a Vaadin component

Conclusion

When to add CSS under the `/components` directory