Forem: El Bruno

AI Agents Built My Shopping Cart (And I Just Watched) 🤖

El Bruno — Mon, 07 Jul 2025 13:05:00 +0000

Hi!

FYI: This was written with some AI help 😂

Okay, so picture this: I’m sitting here with my morning coffee, and I decide to add a shopping cart feature to the eShopLite demo scenarios. But instead of cracking my knuckles and diving into code like usual… I just watched AI agents do ALL the work. 🤯

I’m not kidding. In 8 minutes, I recorded the entire journey from “hey, we need a shopping cart” to a complete, production-ready e-commerce feature. And honestly? It blew my mind.

TL:DR: here is the video

The Setup: eShopLite Needs a Shopping Cart

So I was playing around with eShopLite – our sample e-commerce app that’s built with .NET Aspire, Blazor, and has this cool Semantic search feature using Azure AI Foundry models.

Pretty neat stuff, but it was missing something obvious: you couldn’t actually buy anything. 🛒

The old me would’ve fired up Visual Studio, and start coding. But 2025 me? I decided to see what happens when you let AI agents handle the entire thing.

Round 1: Claude Sonnet Gets the Requirements Right

First up was Claude Sonnet 4 in VS Code. I basically said “Hey, look at this codebase and write me a proper Product Requirements Document for a shopping cart feature.”

What came back was… honestly incredible. This wasn’t some half-baked bullet point list. We’re talking about a full 12-section PRD with:

📊 Executive summary with actual success metrics
⚡ Functional requirements that covered everything from cart management to checkout flows
🏗 Technical architecture with real code examples (not pseudo-code!)
📅 4-phase implementation plan broken down by weeks
🧪 Testing strategy covering unit tests, integration tests, the works
⚠ Risk assessment with actual mitigation strategies

Check out some of the entity models it generated:


public class CartItem
{
    public int ProductId { get; set; }
    public Product Product { get; set; }
    public int Quantity { get; set; }
    public decimal UnitPrice { get; set; }
    public decimal TotalPrice => Quantity * UnitPrice;
}

public class Cart
{
    public string SessionId { get; set; }
    public List<CartItem> Items { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime UpdatedAt { get; set; }
    public decimal TotalAmount => Items.Sum(i => i.TotalPrice);
    public int TotalItems => Items.Sum(i => i.Quantity);
}

Round 2: GitHub Copilot Becomes Project Manager

Next, I told GitHub Copilot to take all that PRD goodness and create a proper GitHub issue in the Azure-Samples/eShopLite repo.

And wow, it didn’t just slap together some basic “add shopping cart pls” issue. It created Issue #31 with:

🎯 Complete technical specifications
📋 Implementation phases with clear deliverables
🏷 Proper labels (enhancement, feature-request, shopping-cart, e-commerce)
✅ Detailed acceptance criteria
📚 Links to all the relevant docs

Round 3: GitHub Copilot Coding Agent Goes Full Beast Mode

Instead of coding it myself, I assigned the issue to GitHub Copilot Coding Agent.

I literally just typed:

Add the Issue #31 to @copilot”

Then I sat back and watched the magic happen. This AI agent:

🔍 Analyzed the entire codebase to understand how everything worked
🌿 Created a new branch (copilot/fix-31) like a good developer
🏗 Built multiple projects – CartEntities, service interfaces, Blazor components
💻 Implemented EVERYTHING with proper error handling and validation
📝 Created a pull request with detailed docs explaining all the changes

Plot Twist: I Became the AI Whisperer 🎭

Now, don’t get me wrong – I wasn’t just sitting there doing nothing. My role shifted to something way more interesting, because, in every iteration, once GH Copilot Coding Agent generated some code, I started to do this:

🎯 Requirements : Making sure the AI “got” what we actually needed.
👀 Code Reviewer : Spotting edge cases and suggesting improvements, and hey, marked down some errors!
🎨 UX Advocate : Ensuring the user experience didn’t suck
🛡 Quality Guardian : Making sure this was actually production-ready

The conversation with the Copilot agent was surprisingly natural. When I said “hey, the checkout validation could be better,” it immediately understood and enhanced the implementation. When I mentioned mobile responsiveness, boom – updated CSS on the spot.

It felt less like programming and more like directing a really smart intern who happens to code at superhuman speed. 🚀

What Actually Got Built

The final result? A complete shopping cart system that would make any e-commerce developer proud:

🛠 Tech Stack

.NET + Aspire : Microsoft’s cloud-native platform
Blazor Server : For those sweet real-time UI updates
Session Storage : Using ProtectedSessionStorage (secure guest checkout)
Bootstrap + Font Awesome : Because we’re not savages

✨ Features That Actually Work

🛒 Add to Cart : With quantity selection and visual feedback
📝 Cart Management : View, update, remove items, clear everything
💳 Checkout Process : Multi-step form with progress indicators

🏆 Code Quality

This wasn’t some weekend hackathon code. The AI generated:

Comprehensive error handling with proper logging
Client and server-side validation
WCAG 2.1 AA accessibility compliance
Performance optimizations for concurrent users
Security best practices throughout

The final Pull Request #32 was a thing of beauty:

📁 8 new project files with complete cart functionality
🎨 15+ Blazor components for the UI
⚙ Complete service layer with interfaces and implementations
💄 Responsive styling that actually looks good
📖 Documentation explaining everything

Done in almost an hour by AI agents.

What This Means for Us Developers

Okay, so this experiment kind of broke my brain a little. Here’s what I think is happening:

🎭 We’re Becoming AI Orchestrators

Instead of writing every line of code, we’re becoming conductors of AI-powered development orchestras. The skill isn’t knowing every API by heart – it’s knowing which AI agent to use and how to coordinate them.

📋 Requirements Are Everything Now

With AI handling the implementation, the quality of your requirements becomes the main bottleneck. Garbage in, garbage out – but good requirements in, amazing code out.

🎨 Creativity Goes to User Experience

While AI handles the technical heavy lifting, we get to focus on the fun stuff – user experience, business logic, creative problem-solving.

🔍 Code Review Gets Strategic

Instead of hunting for syntax errors, we’re validating architecture, ensuring business requirements are met, and checking that the AI understood the bigger picture.

Want to Try This Yourself?

If you’re fired up to experiment with AI-driven development (and you should be!), here’s how to get started:

🛠 Get the tools : VS Code + GitHub Copilot extension
🐣 Start small : Try simple features first, build up your AI-whispering skills
📝 Write clear requirements : Seriously, this is like 80% of the battle now
🔄 Embrace iteration : Use AI feedback loops to refine implementations
👀 Learn to review AI code : Develop your “AI code quality radar”

Final stuff

Look, I’ve been coding for years, and this felt like a glimpse into a completely different future. We’re not just getting AI assistance anymore – we’re orchestrating AI agents to build entire solutions while we focus on the strategic stuff.

The shopping cart that would normally take a long time to properly plan, implement, and test? AI agents knocked it out in minutes. And it wasn’t some demo-quality code – this was (almost) production-ready, well-architected, properly tested software.

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

More info in https://beacons.ai/elbruno

🔗 Useful Links

🧠✨ Testing GPT-4o’s Image Generation – From C# with ❤️ and Microsoft.Extensions.AI

El Bruno — Thu, 27 Mar 2025 15:41:03 +0000

Hi!

OpenAI just dropped some exciting news:

👉 GPT-4o can now generate images directly from prompts.

Announced here: Introducing GPT-4o Image Generation, this new feature lets you go from words to stunning visuals – including photorealistic scenes, illustrations, logos, and more.

The fun part? You can now generate images just by chatting with GPT-4o.

The challenging part? If you’re a developer trying this via the OpenAI API… it’s not quite ready yet 😅

🎯 My Test: “Make Me a Cute Sticker”

So, naturally, I tried this in ChatGPT — and it worked beautifully.

Just uploaded a photo of my cat, added this prompt:

“Make me a cute minimalist sticker based on the provided image. Use a thick white border and transparent background.”

Boom 🐾 — instant sticker! Minimalist lines, clear cat expression, perfect for printing or slapping on your laptop.

🔧 Now Let’s Try in C# (Spoiler: it’s not ready… yet)

Being a .NET fanboy and AI tinkerer, I fired up a quick console app using the awesome new Microsoft.Extensions.AI library — designed to unify and simplify AI model calls from .NET.

Here’s my code using the OpenAIClient with GPT-4o:

📉 The Actual Output

Here’s what I got back from GPT-4o via API:


Prompt: make me a cute minimalist sticker based on the provided image...

Response:
To create a cute minimalist sticker from the provided image, follow these steps:
1. Crop and Simplify...
2. Add a thick white border...
3. Use a transparent background...

You can use tools like Adobe Illustrator or Canva.

Basically — the model understood the task, but instead of returning a new image, it gave instructions 📄.

❗ Why the Disconnect?

The image generation capability is live in ChatGPT, but not yet exposed via the OpenAI API.

As of now:

GetResponseAsync() from Microsoft.Extensions.AI supports image inputs ✅
But image generation as an output is not yet supported ❌

So developers: sit tight, it’s coming.

💡 Takeaways

GPT-4o’s new image generation is 🔥 — in ChatGPT.
If you’re building with .NET and Microsoft.Extensions.AI, you’re already in a great spot to tap into these APIs as soon as image outputs are supported.
Until then, your code can analyze and interpret images with GPT-4o, but it can’t yet generate them.

👀 What’s Next?

I’m keeping this code snippet ready for when OpenAI opens up the feature.

And I’m already thinking about using this in:

Sticker generators 🐱
Avatar creators 🧙
Meme bots 🤖
Or even something weird, like “What if your plant could draw?”

Let me know if you want to explore this together — or if you’re building cool stuff with GPT-4o and .NET.

Until then, happy coding!

— Bruno 💬 🐾

Using Large Language Models with .NET: Generating Image Alt Text Automatically

El Bruno — Tue, 07 Jan 2025 14:00:00 +0000

Hi! The following post is generated using some super cool LLMs, and hey it’s also about LLMs!

Introduction

In this blog post, we’ll explore a practical use case for LLMs: generating alt text for images automatically, ensuring your applications are more inclusive and accessible.

By the end of this post, you’ll learn how to:

Leverage Llama 3.2 Vision for image analysis.
Implement the solution in a .NET project.
Test the application using a sample GitHub repository.

This is the main repo: https://github.com/elbruno/Image-AltText-Generator

Let’s dive in!

Main Demo Repo

What Does the Application Do?

The application generates descriptive alt text for images using gpt-40-mini (online) or Llama3.2-Vision (locally). Alt text is essential for accessibility, enabling visually impaired users to understand the content of an image. This solution uses AI to:

Upload and analyze an image.
Generate an accurate, human-readable description of the image content.
Provide the description as alt text that can be used in web applications or reports.

Technology Stack

.NET: main desktop app.
ollama: to run LLMs locally.
Llama 3.2 Vision: Visual language model for understanding and describing images.
OpenAI APIs: Integration with advanced LLMs to enhance text generation capabilities.

How to Run the Test Using the Sample Code

Prerequisites

Before running the application, ensure you have the following:

.NET SDK : Version 9.0 or higher.
Git : To clone the sample repository.
To analyze the image, 2 options:
- OpenAI API Key : For text generation.
- Ollama : To run the Llama 3.2 Vision model.

Step-by-Step Guide

1. Clone the Repository

Start by cloning the sample repository:


git clone https://github.com/elbruno/Image-AltText-Generator.git
cd Image-AltText-Generator

2. For local use, set Up the Llama 3.2 Vision Model

Installing Ollama: To install Ollama, follow the instructions provided on the Ollama website.
Download the Llama3.2-vision Model: After installing Ollama, download the Llama3.2-vision model by following the instructions on the Ollama website or using the Ollama CLI

4. Run the Application

Launch the .NET application:


dotnet run

5. Test the Application

Copy an image to the clipboard and run the application. The application will:

Analyze the image using a Vision model.
Generate alt text .

For example, uploading a picture of a cat might return:

“A fluffy orange cat sitting on a wooden bench surrounded by green plants.”

Conclusions

Integrating AI models like Llama 3.2 Vision into .NET applications unlocks powerful, real-world use cases. By automating tasks like generating alt text, developers can enhance user experiences and build more inclusive applications effortlessly.

This tutorial demonstrates how easy it is to use LLMs for image understanding and description in .NET. With tools like Docker, OpenAI APIs, and accessible GitHub repositories, you’re just a few steps away from embedding AI capabilities into your projects.

So, what are you waiting for? Clone the repo, test the application, and start building your next AI-powered solution!

Happy coding!

Resources:

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

More info in https://beacons.ai/elbruno

CPU vs GPU: Which Wins for Running LLMs Locally?

El Bruno — Thu, 02 Jan 2025 15:06:48 +0000

Introduction

Running large language models (LLMs) locally has become increasingly accessible, thanks to advancements in hardware and model optimization. For .NET programmers, understanding the performance differences between CPUs and GPUs is crucial to selecting the best setup for their use case.

In this blog post, we’ll explore these differences by benchmarking the Llama 3.2 Vision model using a locally hosted environment with ollama running in docker.

Watch the Video Tutorial

Before diving in, check out the video tutorial for a quick overview of the process and key concepts covered in this blog post.

Goal of the Comparison

The goal of this exercise is to evaluate:

Execution Time : How fast can the model process queries on CPU vs GPU?
Resource Utilization : How do hardware resources (memory, power) compare between the two setups?
Suitability : Which setup is better for different programming tasks?

By the end, you’ll have a clear understanding of the trade-offs and be equipped to choose the most appropriate setup for your projects.

How to Run the Test Using the Sample Code

1. Setup

Start by cloning the sample GitHub repository:


git clone https://github.com/elbruno/Ollama-llama3.2-vision-Benchmark
cd Ollama-llama3.2-vision-Benchmark

Ensure you have the necessary dependencies installed:

For .NET > Install .NET SDK

2. Model Download

Install Docker.
Run the Docker container for GPU and CPU:
- GPU running on port 11434 (default)

docker run -d –gpus=all -v ollama:/root/.ollama -p 11434:11434 –name ollama ollama/ollama
CPU running on port 11435
docker run -d -v ollamacpu:/root/.ollamacpu -p 11435:11434 –name ollamacpu ollama/ollama
1. On each container pull the llama3.2-vision image. Run the command ollama run llama3.2-vision
2. You will have docker running 2 instances of ollama, similar to this image:

3. Run Code

For benchmarking, we are using BenchmarkDotNet for .NET.

Open the OllamaBenchmark.sln and run the solution.

4. Results Analysis

The .NET solution will output detailed performance metrics, including execution time and resource usage. Compare these metrics to identify the strengths of each hardware setup.

Conclusions

Performance : GPUs consistently outperform CPUs in execution time for LLMs, especially for larger models like Llama 3.2 Vision.
Resource Efficiency : While GPUs are faster, they consume more power. CPUs, on the other hand, are more energy-efficient but slower.
Use Cases :
- CPU : Best for lightweight, cost-sensitive tasks or development environments.
- GPU : Ideal for production workloads requiring high throughput or real-time inference.

Running benchmarks is a straightforward way to determine the best hardware for your specific needs. By following the steps outlined here, you can confidently experiment with LLMs and optimize your local environment for maximum efficiency.

For a detailed walkthrough, check out the video tutorial.

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

More info in https://beacons.ai/elbruno

Code Sample: Integrating Azure OpenAI Search with #SemanticKernel in .NET

El Bruno — Fri, 14 Jun 2024 13:59:42 +0000

Hi!

Today I’ll try to expand a little the scenario described in this Semantic Kernel blog post: “Azure OpenAI On Your Data with Semantic Kernel“.

The code below uses an GPT-4o model to support the chat and also is connected to Azure AI Search using SK. While runing this demo, you will notice the mentions to [doc1], [doc2] and more. Extending the original SK blog post, this sample shows at the bottom the details of each one of the mentioned documents.

A similar question using Azure AI Studio, will also shows the references to source documents.

Semantic Kernel Blog Post

The SK team explored how to leverage Azure OpenAI Service in conjunction with the Semantic Kernel to enhance AI solutions. By combining these tools, you can harness the capabilities of large language models to work effectively with data, using Azure AI Search capabilities. The post covered the integration process, highlighted the benefits, and provided a high-level overview of the architecture.

The post showcases the importance of context-aware responses and how the Semantic Kernel can manage state and memory to deliver more accurate and relevant results. This integration between SK and Azure AI Search empowers developers to build applications that understand and respond to user queries in a more human-like manner.

The blog post provides a code sample showcasing the integration steps. To run the scenario, you’ll need:

Upload our data files to Azure Blob storage.
Vectorize and index data in Azure AI Search.
Connect Azure OpenAI service with Azure AI Search.

For more in-depth guidance, be sure to check out the full post here.

Code Sample

And now it’s time to who how we can access the details of the response from a SK call, when the response includes information from Azure AI Search.

Let’s take a look at the following program.cs to understand its structure and functionality.

The sample program is a showcase of how to utilize Azure OpenAI and Semantic Kernel to create a chat application capable of generating suggestions based on user queries.
The program starts by importing necessary namespaces, ensuring access to Azure OpenAI, configuration management, and Semantic Kernel functionalities.
Next, the program uses a configuration builder to securely load Azure OpenAI keys from user secrets.
The core of the program lies in setting up a chat completion service with Semantic Kernel. This service is configured to use Azure OpenAI for generating chat responses, utilizing the previously loaded API keys and endpoints.
To handle the conversation, the program creates a sample chat history. This history includes both system and user messages, forming the basis for the chat completion service to generate responses.
An Azure Search extension is configured to enrich the chat responses with relevant information. This extension uses an Azure Search index to pull in data, enhancing the chat service’s ability to provide informative and contextually relevant responses.
Finally, the program runs the chat prompt, using the chat history and the Azure Search extension configuration to generate a response.
This response is then printed to the console. Additionally, if the response includes citations from the Azure Search extension, these are also processed and printed, showcasing the integration’s ability to provide detailed and informative answers.

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

Sample Code using the new OpenAI library for .NET

El Bruno — Fri, 07 Jun 2024 10:30:04 +0000

Hi!

The new OpenAI SDK for NET was officially announced, so today, let’s review the annoucement and show some sample code on how to use it.

The current blog scenarios are

Sample Chat demo
Sample Audio to Text demo
Sample Image analisis demo

Here is a sample output for the chat demo! And yes, with a funny system message and questions about France!

Code Repository: GPT-4o Labs

OpenAI SDK for .NET

Microsoft Build 2024 has unveiled new AI investments for .NET developers, including the first beta release of the official OpenAI .NET library, version 2.0.0-beta.1. This library facilitates smooth integration with OpenAI and Azure OpenAI, complementing existing libraries for Python and TypeScript/JavaScript.

Developed on GitHub, the .NET library will stay current with OpenAI’s latest features, with ongoing work to refine it based on community feedback. The release acknowledges Roger Pincombe’s pioneering work on the initial OpenAI .NET package and encourages continued innovation from community library developers. Participation and collaboration within the community are highly encouraged as the project progresses.

_ Note: Part of the content of this post was generated by Microsoft Copilot, an AI assistant._

Scenario 1: Chat

Time to share the “Hello World” of using GPT models. The following sample is a console application that interacts with the OpenAI API to generate chat responses.

It retrieves an API key from user secrets and specifies the model to be used for the chat.
It initializes a ChatClient with the model and API key.
The system message and user question are defined and added to a list of chat messages.
The chat is then completed using the ChatClient and the response from the chat is retrieved.
Finally, the system prompt, user question, and chat response are displayed in the console.

This is the sample output for this program:

System Prompt: You are a useful assitant that replies using a funny style.

User Question: What is the capital of France?

Response: Well, let me put on my fancy beret and sip some imaginary café au lait while I tell ya – the capital of France is the one and only Paris! Yes, indeed, the city of love, croissants, and an Eiffel Tower that’s basically the world’s largest toothpick! So, if you’re planning to hunt for baguettes or have a romantic escapade, Paris is where the magic happens! ??????

Scenario 2: Audio

The SDK also allow us to work with Whisper and audio files.

The following code is a C# console application that uses the OpenAI API to transcribe audio files.

It sets up the configuration to retrieve the OpenAI API key and specifies the model to be used for transcription.
It creates an AudioClient instance with the model and API key.
The audio file to be transcribed is specified, and transcription options are set, including the response format and granularities for timestamps.
The audio file is then transcribed, and the transcription text is printed to the console.
Additionally, the start and end times for each word and segment in the transcription are also printed to the console.

This is a sample output for this program:


Transcription:
Estás escuchando No Tiene Nombre, un podcast sobre tecnología que es bastante probable que sea escrito por una inteligencia artificial, por ejemplo, con chat GPT. El host, autor, editor y encargado de los efectos visuales del podcast es Bruno Capuano. Puedes contactarlo en Twitter en arroba elbruno o en las redes sociales también buscando por elbruno. Mi nombre es Elena, de Italia YOS invito al episodio de hoy. Subtítulos por la comunidad de Amara.org

Words:
            Estás : 700 - 1420
       escuchando : 1420 - 2060
               No : 2060 - 2420
            Tiene : 2420 - 2580
           Nombre : 2580 - 3000
               un : 3340 - 3500
          podcast : 3500 - 3780
            sobre : 3780 - 4100
       tecnología : 4100 - 4720
              que : 4720 - 4960
               es : 4960 - 5120
         bastante : 5120 - 5560
         probable : 5560 - 6020
              que : 6020 - 6260
              sea : 6260 - 6480
          escrito : 6480 - 6820
              por : 6820 - 7100
              una : 7100 - 7460
     inteligencia : 7460 - 7820
       artificial : 7820 - 8380
              por : 8820 - 9020
          ejemplo : 9020 - 9300
              con : 9680 - 9880
             chat : 9880 - 10060
              GPT : 10060 - 11340
               El : 11360 - 11760
             host : 11760 - 12100
            autor : 12660 - 12660
           editor : 13140 - 13140
                y : 13140 - 13340
        encargado : 13340 - 13740
               de : 13740 - 13920
              los : 13920 - 14300
          efectos : 14300 - 14420
         visuales : 14420 - 14860
              del : 14860 - 15020
          podcast : 15020 - 15400
               es : 15400 - 15760
            Bruno : 15760 - 15920
          Capuano : 15920 - 16400
           Puedes : 17640 - 17700
      contactarlo : 17700 - 18320
               en : 18320 - 18500
          Twitter : 18500 - 18740
               en : 18740 - 18920
           arroba : 18920 - 19160
          elbruno : 19160 - 19540
                o : 19540 - 19960
               en : 19960 - 20080
              las : 20080 - 20360
            redes : 20360 - 20360
         sociales : 20360 - 20800
          también : 20800 - 21180
         buscando : 21180 - 21560
              por : 21560 - 21960
          elbruno : 21960 - 22340
               Mi : 23480 - 23560
           nombre : 23560 - 23840
               es : 23840 - 24280
            Elena : 24280 - 24440
               de : 24780 - 24940
           Italia : 24940 - 25360
              YOS : 25360 - 26180
           invito : 26180 - 26660
               al : 26660 - 26880
         episodio : 26880 - 27340
               de : 27340 - 27720
              hoy : 27720 - 27720
       Subtítulos : 30000 - 31540
              por : 31540 - 31540
               la : 31540 - 31540
        comunidad : 31540 - 31540
               de : 31540 - 31540
            Amara : 31540 - 31540
              org : 31540 - 31540

Segments:
      Estás escuchando No Tiene Nombre, un podcast sobre tecnología que es bastante probable : 700 - 6480
                 que sea escrito por una inteligencia artificial, por ejemplo, con chat GPT. : 6480 - 12660
    El host, autor, editor y encargado de los efectos visuales del podcast es Bruno Capuano. : 13140 - 18500
   Puedes contactarlo en Twitter en arroba elbruno o en las redes sociales también buscando por elbruno. : 18500 - 26180
                                Mi nombre es Elena, de Italia YOS invito al episodio de hoy. : 26180 - 27720
                                                    Subtítulos por la comunidad de Amara.org : 30000 - 31540

Scenario 3: Using Vision with GPT4o.

And the final scenario will be on how to use vision capabilities with this new SDK. The following program project is a C# console application that uses the OpenAI API to interact with an AI assistant.

It starts by reading the OpenAI API key from the user secrets and initializing the OpenAI client with this key.
It uploads an image file named foggyday.png from the imgs directory to the OpenAI server for the purpose of vision tasks.
After the image is uploaded, it creates an AI assistant with a specific instruction to reply in a funny style.
It starts a new thread with the assistant, sending an initial message asking the assistant to describe the uploaded image.
The program then enters a loop, listening for updates from the assistant.
When a new message is received from the assistant, it is printed to the console.
The program continues to listen for updates until it is manually stopped.

This is a sample output for this program, and below is the image used to test:


--- Run started! ---
Well, well, well! What do we have here? Is that paradise? A spa? Or just someone's backyard on a dreamy, misty morning? There's a pristine blue pool just begging for a cannonball, surrounded by neat paving stones that scream, "We mean business!" Looming over the scene is a terrific tree that looks like it's auditioning for the role of "Majestic Sentinel."

In the distance, the fog is putting up a great show, hiding whatever secrets lie beyond this yard-maybe a herd of unicorns or a neighborhood of lawn gnomes planning their next move. The whole scene feels like it's on the set of the next great mystery movie... or maybe just an ad for pool cleaning services. Either way, sign me up for a dip!
D:\sk\gpt4ol-sk-csharp\src\OAINETSDK_lab03\bin\Debug\net8.0\OAINETSDK_lab03.exe (process 3668) exited with code 0.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

Conclusion

The new OpenAI SDK for .NET is a great one. I’ll keep testing and sharing samples, for each new scenario supported by the SDK!

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

Powering Up #NET Apps with #Phi-3 and #SemanticKernel

El Bruno — Fri, 31 May 2024 13:49:48 +0000

Hi!

Introducing the Phi-3 Small Language Model

Phi-3 is an amazing Small Language Model. And hey, it’s also an easy one to use in C#. I already wrote how to use it with ollama, now it’s time to hit the ONNX version.

Introduction to Phi-3 Small Language Model

The Phi-3 Small Language Model (SLM) represents a significant leap forward in the field of artificial intelligence. Developed by Microsoft, the Phi-3 family of models is a collection of the most capable and cost-effective SLMs available today. These models have been meticulously crafted to outperform other models of similar or even larger sizes across various benchmarks, including language understanding, reasoning, coding, and mathematical tasks.

Phi-3 models are not only remarkable for their performance but also for their efficiency and adaptability. They are designed to operate across a wide range of hardware, from traditional computing devices to edge devices like mobile phones and IoT devices. This makes Phi-3 particularly suitable for developers looking to integrate advanced AI capabilities into applications that require strong reasoning, limited compute resources, and low latency.

C# Phi3-Labs GitHub Repository

The GitHub repository in question serves as a practical guide for developers looking to harness the power of the Phi-3 SLM within their applications. It provides a set of sample projects to demonstrate how to use Phi-3 and C#.

LabsPhi301. This is a sample project that uses a local phi3 model to ask a question. The project loads a local ONNX Phi-3 model using the Microsoft.ML.OnnxRuntime libraries.
LabsPhi302. This is a sample project that implements a Console chat using Semantic Kernel.
LabsPhi303 (Comming soon!). This is a sample project that uses Phi-3 Vision to analyze images.

The repository contains detailed instructions for setting up the development environment, cloning the necessary models from Hugging Face, and installing the Phi-3 model.

You can learn more about Phi-3 in Hugging Face.

A final running application will look like this.

()

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

#SemanticKernel and GPT-4o: Image analysis labs in C#

El Bruno — Tue, 14 May 2024 13:21:26 +0000

Hi!

Introduction to GPT-4o

GPT-4o, developed by OpenAI, represents a significant leap forward in AI technology. Dubbed “omni” for its all-encompassing capabilities, GPT-4o is a multimodal model that can process and generate text, audio, and images.

It’s designed to facilitate more natural human-computer interactions, responding to audio inputs in as little as 232 milliseconds. This model is not only faster but also 50% cheaper to use in the API, making it a cost-effective solution for developers and businesses alike.

Sample GitHub Repository

I decided to test some of the new features, and created this repository with some samples using Semantic Kernel and the new GPT-4o model.

https://github.com/elbruno/gpt4ol-sk-csharp/

Repository Content

The repo describes the basic steps to set up and utilize the GPT-4o model with .NET. The repository includes sample code, and links to further reading materials.

The initial demo shows how to use GPT-4o to analyze the following image:

With the following output:


The image appears to be a screenshot of a terminal window running on a Raspberry Pi device. The user has executed the `neofetch` command with `sudo`, and the terminal displayed system information. Additionally, the `ollama list` command was executed, showing a list of local models.

Here's the breakdown of the terminal output:

### System Information (Neofetch Output)
- **OS:** Debian GNU/Linux 12 (bookworm) aarch64
- **Host:** Raspberry Pi 5 Model B Rev 1.0
- **Kernel:** 6.6.20+rpt-rpi-2712
- **Uptime:** 3 mins
- **Packages:** 694 (dpkg)
- **Shell:** bash 5.2.15
- **CPU:** 4 cores @ 2.400GHz
- **Memory:** 640MiB / 8052MiB

### Models List (Ollama List Command Output)
- **llama3:latest**
- **ID:** 71a106a91016
- **Size:** 4.7 GB
- **Modified:** 4 days ago
- **phi3:latest**
- **ID:** a2c89ceaed85
- **Size:** 2.3 GB
- **Modified:** 3 days ago

I’ll keep updating the samples with more scenarios.

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

Demo – API Manifest Plugins for Semantic Kernel

El Bruno — Wed, 08 May 2024 13:00:00 +0000

Hi!

Today, it’s time to share some insights into the world of API Manifest Plugins for Semantic Kernel, these are a cool way to work with OpenAI’s Large Language Models (LLMs).

Introduction to API Manifest Plugins for Semantic Kernel

Semantic Kernel is at the forefront of AI development, allowing developers to import plugins from OpenAPI documents. However, importing entire OpenAPI documents for large APIs can be inefficient. To tackle this, Microsoft introduced API Manifest Plugins for Semantic Kernel

An API Manifest is a document that outlines an application’s API dependencies, including links to API descriptions, the requests made, and their authorization requirements. These plugins enable Semantic Kernel functions to be generated from API Manifest files, which can then be called by LLMs in various scenarios.

Key features include:

Slicing large API descriptions into task-specific parts.
Packaging multiple API dependencies into a single plugin.
Defining authorization requirements for API calls.

This approach not only streamlines the process but also enhances security and efficiency when integrating third-party APIs.

The Importance of Function Calling while working with OpenAI LLMs

Function calling is a game-changer for developers working with OpenAI LLMs. It allows the models to intelligently output JSON objects containing arguments to call one or many functions, based on the input provided^.

This feature is crucial for:

Creating assistants that answer questions by calling external APIs.
Converting natural language into API calls.
Extracting structured data from text.

Supported by the latest models, function calling ensures that developers can get structured data back from the model more reliably.

Reference to the GitHub Repository

My GitHub repository “sk-API-Manifest-Plugins” is a sample entry for developers looking to leverage API Manifest Plugins with Semantic Kernel.

Scenario

In this scenario, we will assume that there are public APIs with information about superheroes, and a pet database with information about pets and their owner information.

There is a fictitious Superhero Pet Day, so the owner of a Pet Store will trigger a plan to find which clients have pets with superhero names.

The plan will detect these pets and send an email congratulating them for the Superhero Pet Day with information about their pet name super hero.

Prerequisites

Once you cloned the repository, ensure you have:

.NET 8
Visual Studio 2022 or Visual Studio Code
Access to OpenAI APIs or Azure OpenAI Services

Components

The repository includes:

2 Sample API projects: PetStoreAPI and SuperHeroAPI.
1 Semantic Kernel test project with a plan to solve the scenario. The project uses API Manifest to embed the plugins from the 2 APIs.
Bonus: the SK test also use a native plugin that emulates the email sent.

Running the Demo

To run the demo:

Clone the repository from GitHub.
Install the necessary dependencies listed in the repository.
Follow the step-by-step guide to set up the environment and execute the demo.

Here is the start page of the console demo:

And the responses from the SK Planner:

The last step also show a table with all the details and steps that the planner performed to solve the scenario.

This hands-on experience will give you a practical understanding of how API Manifest Plugins work within Semantic Kernel.

For more detailed information and to explore the full capabilities of API Manifest Plugins and function calling with OpenAI LLMs, visit the official Semantic Kernel blog and the OpenAI documentation. Embrace the future of AI with these powerful tools at your disposal.

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

#SemanticKernel: Local LLMs Unleashed on #RaspberryPi 5

El Bruno — Wed, 01 May 2024 12:00:00 +0000

Hi!

Welcome to the exciting world of local Large Language Models (LLMs) where we’re pushing the boundaries of what’s possible with AI.

Today let’s talk about a cool topic: run models locally, especially on devices like the Raspberry Pi 5. Let’s dive into the future of AI, right in our own backyards.

Ollama and using Open Source LLMs

OLLAMA stands out as a platform that simplifies the process of running open-source LLMs locally on your machine. It bundles model weights, configuration, and data into a single package, making it accessible for developers and AI enthusiasts alike. The key benefits of using Ollama include:

Simplicity : Easy setup process without the need for deep machine learning knowledge.
Cost-Effectiveness : Eliminates cloud costs, making it wallet-friendly.
Privacy : Ensures data processing happens on your local machine, enhancing user privacy.
Versatility : Suitable for various applications beyond Python, including web development.

Using Local LLMs like Llama3 or Phi-3

Local LLMs like Llama3 and Phi-3 represent a significant shift towards more efficient and compact AI models. Llama3, with its Mixture-of-Experts (MoE) architecture, offers specialized neural networks for different tasks, providing high-quality outputs with a smaller parameter count.

Phi-3, developed by Microsoft, uses advanced training techniques like quantization to maximize efficiency, making it ideal for deployment on a wide range of devices.

The use of local LLMs offers several advantages:

Reduced Latency : Local models eliminate network latency associated with cloud-based solutions.
Enhanced Privacy : Data remains on your local device, offering a secure environment for sensitive information.
Customization : Local models allow for greater flexibility to tweak and optimize the models as per your needs.

How to Set Up a Local Ollama Inference Server on a Raspberry Pi 5

I already wrote a couple of times, my own version of the 1st time setup for a Raspberry Pi (link). Once the device is ready, setting up Ollama on a Raspberry Pi 5 (or older) is a straightforward process. Here’s a quick guide to get you started:

Installation : Use the official Ollama installation script to install it on your Raspberry Pi OS.

The main command is:


curl -fsSL https://ollama.com/install.sh | sh

Running Models : After installation, you can run various LLMs like tinyllama, phi, and llava, depending on your RAM capacity.

In example to install and run llama 3, we can use the following command:


ollama run llama3

Once ollama is installed and a model is downloaded, the console should look similar to this one:

For a detailed step-by-step guide, including setting up Docker and accessing the Ollama WebUI, check out the resources available on GitHub.

Tip: to check the realtime journal of the ollama service, we can run this command:


journalctl -u ollama -f

Important: by defaul ollama server is available only for local calls. In order to enable access from other machines, you need to follow these steps:

– Edit the systemd service by calling this command.


sudo systemctl edit ollama.service

– This will open an editor.

– Add a line Environment under section [Service]:


[Service]
Environment="OLLAMA_HOST=0.0.0.0"

– Save and exit.

– Reload systemd and restart Ollama:

More information in the ollama FAQ.

How to Use Semantic Kernel to Call a Chat Generation from a Remote Server

Let’s switch and write some code. This is a “Hello World” sample using Semantic Kernel and Azure OpenAI Services.

You can learn more about this ai samples in: https://aka.ms/dotnet-ai.

Now , to use a remote LLM, like Llama 3 in a Raspberry Pi, we can add a service to the Builder, that uses OpenAI API specification. In the next sample this change in the line 35:

This makes the trick! And with just a single change in a line.

And we also have the question about the performance, adding a StopWatch we can get a sense of the time elapsed for the call. For this simple call, the response is around 30-50 seconds.

Not bad at all for a small device!

Conclusion

The advent of local LLMs like Ollama is revolutionizing the way we approach AI, offering unprecedented opportunities for innovation and privacy. Whether you’re a seasoned developer or just starting out, the potential of local AI is immense and waiting for you to explore.

This blog post was generated using information from various online resources, including cheatsheet.md, anakin.ai, and techcommunity.microsoft.com, to provide a comprehensive guide on local LLMs and Ollama.

Happy coding!

Greetings

El Bruno

#RaspberryPi – 1st setup no monitor 📺: Wifi 📶 auto connect, SSH, rename, update, docker 🐳, rust and more! Update 2023-Jan-04

El Bruno — Tue, 30 Apr 2024 12:00:00 +0000

Latest version 2024-Apr-30 here

Content

Create SD card using Raspberry Pi Imager
Configure Wireless connection (if needed)
Enable SSH (if needed)
Find the IP address in your network
Access via SSH
Change Password (if needed)
Rename the device (if needed)
Expand FileSystem
Update the device
Install neofetch
Install Docker
Setup SSH password-less access to the Raspberry Pi
Setup password-less access to remote work with docker 🐳
Run Docker 🐳] commands without sudo
Install Rust

Hi!

Let’s start installing the latest Raspberry Pi OS image in an SD card. Next steps will be focus on how to access and control remotely your device you may want to follow this steps.

This tutorial and tips works for Raspberry Pi 3, 4 and Zero.

The version 1.6 of Raspberry Pi Imager includes a new feature that allows to

Define HostName
Enable SSH
Configure Wifi
Set locale

Directly from the tool pressing the [Config] button

_ Note: Open the setting using this button is new in the latest Raspverry Pi imager version._

We can define this settings on the tool screen.

I’ll leave the old-school standard methods below just as a reference.

Configure Wireless connection (if needed)

In the SD Card, you need to create a file named [wpa_supplicant.conf] in the root of the SD card with the following information:


country=ca
update_config=1
ctrl_interface=/var/run/wpa_supplicant

network={
 scan_ssid=1
 ssid=" Your WiFi SSID"
 psk="You amazing password"
}

The file content is very straight forward to understand. Main values to complete are [ssid] and [psk].

Once you put the SD card in the device and start the device, it will automatically connect to the configured WiFi.

Enable SSH (if needed)

If you also want to enable SSH, you need to create a blank file named [ssh] to the main partition.

Once you put the SD card in the device and start the device, it will automatically enable the SSH service.

So, you need to create and copy 2 files to the root of your SD card

wpa_supplicant.conf
ssh

Find the IP address in your network

And that’s it, your Raspberry Pi will be connected to the Wifi and with SSH enabled. At this moment we can use a tool like AngryIp (see references) to detect the new device in the network

My new device IP is: 192.168.1.246

I’m trying to avoid Java updates, and even install java, so lately I use a mobile app: Fing, and after a search the results are nicer.

Access via SSH

I used to like Putty to connect to my device, however during the past months I’ve been using Windows Terminal and Powershell. In order to access the device I need to execute the command


ssh user@deviceaddress

and my data is

user: pi
ip: 192.168.1.246
password: raspberry

You can now start working with your Raspberry Pi !

If you get a SSH error message similar to this one


@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
---
Host key verification failed.

You need to run this command to fix the host hey.


# ssh-keygen -R <host>
ssh-keygen -R 192.168.1.247

_ Important: the default password is raspberry, please follow next step!_

Change Password (if needed)

The default password for the device is “raspberry”, and as usual, it’s recommended to change it. In order to do this, in the ssh terminal, let’s access to the device configuration


sudo raspi-config

This will open the configuration for the device.

Option number 1 will allow us to change the password.

Rename the device (if needed)

In the same section we can change the Host Name.

And define the new name for the Raspberry Pi device.

Expand FileSystem

Another important option in the configuration is to expand the SD disk.In the same configuration screen, select

6. Advanced Options
Expand File System

Now we need to reboot and after the reboot the file system should have been expanded to include all available space on your micro-SD card. Reboot with the command


sudo reboot

Update the device

There are 2 ways to update the device, using commands and using raspi-config.

In the Raspi Config main menu, the option 8 will launch the update commands.
If you prefer to manually type an update command, this one works for me


sudo -- sh -c 'apt-get update; apt-get upgrade -y; apt-get dist-upgrade -y; apt-get autoremove -y; apt-get autoclean -y'

Install neofetch

Neofetch is a shell script that requires bash 3.2+ to function. By default, it displays an ASCII art of your Distro’s logo. Neofetch supports almost 150 different operating systems. From Linux to Windows.

Installation is super easy, just run the command:


sudo apt install neofetch

And then run the command


sudo neofetch

The output will be similar to this one:

Install Docker

The information is available from the Official Docker Documentation


curl -fsSL https://get.docker.com -o get-docker.sh

sudo sh get-docker.sh

And then, a simple check for the docker version with the command


docker version

Setup ssh password-less access to the Raspberry Pi

The main resource is an official one fro the Raspberry Pi org: Passwordless SSH access. Here is a quick resume for Windows 11 users. First let’s run this command


ls ~/.ssh

I already keys on my computer, so I found id_sa and id_rsa.pub.

Now, let’s copy them to the device. For this this command:


cat ~/.ssh/id_rsa.pub | ssh <USERNAME>@<IP-ADDRESS> 'mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys'

And that’s it! Now we can ssh the Raspberry Pi without password.

_ Disclaimer: you know that this is not a very secure practice?_

Setup SSH password-less access to remote work with docker 🐳

Once we finished ssh password-less access, we can easily configure a TCP-enabled Docker.

I followed the next steps. First, let’s edit docker.service with the command:


sudo nano /lib/systemd/system/docker.service

And add [-H tcp://0.0.0.0:2375] at [ExecStart] section:

And restart the device or just the needed services

run sudo systemctl daemon-reload
run sudo systemctl restart docker

Run Docker 🐳 commands without sudo

Now let’s upgrade our docker commands, so we can run them without sudo. Just run this commands


sudo groupadd docker
sudo gpasswd -a $USER docker

And also restart the service:


sudo service docker restart

Install Rust

To install Rust run the following in your terminal, then follow the on-screen instructions.


curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

These are the options, I’ll go with the default one

After a couple of minutes, we have rust installed !

I closed my terminal and opened again, and I can test the rust and cargo version with the commands


rustc --version
cargo --version

I can even create a new app using cargo, edit the main file and run it!

Rust is working fine !

Conclusion

And that’s it, we have our device updated and running with the latest software versions and we didn’t use a monitor! I’ll update this post frequently to make it relevant with my personal best practices.

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

My posts on Raspberry Pi ⚡🐲⚡

Dev posts for Raspberry Pi

Tools and Apps for Raspberry Pi

Setup the device

Hardware

🦙 Harnessing Local AI: Unleashing the Power of .NET Smart Components and Llama2

El Bruno — Tue, 09 Apr 2024 12:00:00 +0000

Hi!

.NET Smart Components are an amazing example of how to use AI to enhace the user experience in something as popular as a combobox.

.NET Smart Components also support the use of local LLMs, so in this post I’ll show how to configure these components to use a local Llama 2 inference server. The following image shows the Smart TextArea doing completions with a local server, in the right we can check the local server journal to check the local http requests to the server.

Introduction to .NET Smart Components

.NET Smart Components are a groundbreaking addition to the .NET ecosystem, offering AI-powered UI controls that seamlessly integrate into your applications. These components are designed to enhance user productivity by providing intelligent features such as Smart Paste , Smart TextArea , and Smart ComboBox¹.

Smart Paste simplifies data entry by automatically filling out forms using data from the user’s clipboard. Smart TextArea enhances the traditional textarea by providing autocomplete capabilities for sentences, URLs, and more. Lastly, Smart ComboBox improves the traditional combo box by offering suggestions based on semantic matching^.

These components are currently available for Blazor, MVC, and Razor Pages with .NET 6 and later, and they represent an experiment in integrating AI directly into user interfaces ¹.

The Importance of Local LLMs like Llama2

Local Large Language Models (LLMs) like Llama2 offer significant advantages, particularly in terms of data privacy and security. In example: running LLMs locally allows organizations to process sensitive data without exposing it to external servers, ensuring compliance with data protection regulation.

Llama2 is an open-source model that provides robust performance across various tasks, including common-sense reasoning, mathematical abilities, and general knowledge. It supports a context length of 4096 tokens, which is double that of its predecessor, Llama1. This makes Llama2 an ideal choice for organizations looking to leverage AI while maintaining control over their data and infrastructure.

How to run .NET Smart Components with a Local Ollama Inference Server

In previous posts, I shared how to run a local Ollama Inference Server in Ubuntu (blog). Lucky us, you can also do this in Windows now.

And once you clone the main Smart Component repository, you only need to add a small change to run the samples locally.

Open the file [RepoSharedConfig.json]

Add the following configuration to use the local ollama model


{
  "SmartComponents": {

   // local demo with ollama self-hosted
    "SelfHosted": true,
    "DeploymentName": "llama2",
    "Endpoint": "http://localhost:11434"
  }
}

And that’s it!, now you can run the either the Blazor or the MVC demos and they will use the local Ollama server to run the completions!

And hey, let’s keep an eye on the Smart Components, they are going to provide an amazing new user experience powered by AI!

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

More info in https://beacons.ai/elbruno