Forem: wellallyTech

Burnout is Quiet, but Your Voice Isn't: Detecting Workplace Stress with Python and Affective Computing

wellallyTech — Thu, 16 Apr 2026 01:15:00 +0000

We’ve all been there. You’re in your fifth Zoom call of the day, your coffee is cold, and you’re saying "I'm fine," but your voice is screaming "I need a vacation." In the world of Affective Computing, your voice doesn't lie.

Detecting workplace burnout through Speech Emotion Recognition (SER) is no longer science fiction. By analyzing acoustic features like Fundamental Frequency (F0) and Mel-frequency Cepstral Coefficients (MFCC), we can build a pipeline to identify stress, anxiety, and vocal fatigue before they lead to total burnout.

In this tutorial, we will explore how to leverage Python, Praat (via Parselmouth), and OpenSMILE to build a stress-detection classifier using Support Vector Machines (SVM).

The Architecture of Stress Detection 🧠

How do we turn raw audio into a "Burnout Score"? It's all about the feature extraction pipeline. We focus on prosodic features (pitch/rhythm) and spectral features (vocal tract shape).

graph TD
    A[Raw Audio: Meeting/Call] --> B[Preprocessing: Denoising & Normalization]
    B --> C{Feature Extraction}
    C --> D[Fundamental Frequency - F0 via Praat]
    C --> E[MFCCs & Jitter/Shimmer via OpenSMILE]
    D --> F[Feature Fusion]
    E --> F[Feature Fusion]
    F --> G[SVM Classifier]
    G --> H[Burnout/Stress Level Report]
    H --> I[Early Intervention Notification]

🛠 Prerequisites

To follow along, you'll need the following stack:

Python 3.9+
Parselmouth: The Pythonic interface to Praat (excellent for F0 analysis).
OpenSMILE: The gold standard for feature sets like eGeMAPS.
Scikit-learn: For our SVM model.

pip install parselmouth-praat opensmile scikit-learn librosa pandas

Step 1: Extracting Pitch (F0) with Praat

Fundamental Frequency (F0) represents the vibration rate of the vocal folds. When we are stressed or angry, our muscles tense up, typically causing a spike in F0 mean and variance.

Here is how we use Parselmouth to extract these prosodic features:

import parselmouth
import numpy as np

def extract_f0_features(audio_path):
    # Load audio into Praat-like object
    snd = parselmouth.Sound(audio_path)

    # Extract pitch (To Pitch: time_step, pitch_floor, pitch_ceil)
    pitch = snd.to_pitch()

    # Get mean and standard deviation
    f0_values = pitch.selected_array['frequency']
    f0_values[f0_values == 0] = np.nan  # Remove unvoiced regions

    f0_mean = np.nanmean(f0_values)
    f0_std = np.nanstd(f0_values)

    return {"f0_mean": f0_mean, "f0_std": f0_std}

# Example usage
# features = extract_f0_features("meeting_audio.wav")
# print(f"Average Pitch: {features['f0_mean']:.2f} Hz")

Step 2: Deep Dive into MFCCs with OpenSMILE

While F0 tells us about energy, MFCCs tell us about the "texture" of the voice. Chronic burnout often results in "vocal fry" or a flattened, monotonous spectral envelope. OpenSMILE allows us to extract the eGeMAPS (extended Geneve Acoustic Multimodal Feature Set), which is specifically designed for voice research.

import opensmile

def extract_spectral_features(audio_path):
    # Initialize OpenSMILE for eGeMAPS
    smile = opensmile.Smile(
        feature_set=opensmile.FeatureSet.eGeMAPS,
        feature_level=opensmile.FeatureLevel.Functionals,
    )

    # Extract features
    y = smile.process_file(audio_path)

    # We specifically look for MFCCs and Jitter/Shimmer (voice stability)
    relevant_cols = [c for c in y.columns if 'mfcc' in c or 'jitter' in c]
    return y[relevant_cols]

Step 3: Classification with SVM

Once we have our feature vector (F0 + MFCCs + Jitter), we feed it into a Support Vector Machine (SVM). SVMs are particularly effective for SER because we often work with smaller, high-dimensional datasets.

from sklearn import svm
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Assume X is our combined feature matrix, y is the 'Burnout' label (0 or 1)
# X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

def train_stress_model(X_train, y_train):
    scaler = StandardScaler()
    X_scaled = scaler.fit_transform(X_train)

    # RBF kernel is usually best for non-linear vocal features
    clf = svm.SVC(kernel='rbf', probability=True)
    clf.fit(X_scaled, y_train)
    return clf, scaler

# 🚀 Pro-tip: Monitor the 'Jitter' feature—it's a high-confidence 
# indicator of physiological stress!

🥑 Going Beyond the Basics: Production Patterns

In a real-world corporate environment, you can't just record everyone's calls. Privacy is paramount. Production-grade systems usually perform Feature Extraction on the Edge (locally on the user's device) and only send the anonymized numerical vectors to the server.

If you're interested in more production-ready AI patterns, data privacy in health-tech, or advanced signal processing, you should definitely check out the WellAlly Blog. They have incredible deep-dives on building ethical AI systems that monitor well-being without compromising user trust.

Conclusion: The Future of Vocal Health

By combining Praat's precision in F0 analysis with OpenSMILE's robust spectral feature sets, we can create tools that help managers and employees recognize the signs of burnout before they become critical.

Key Takeaways:

F0 Mean/Std helps identify acute anxiety.
MFCCs and Jitter help identify chronic fatigue.
SVMs provide a robust classification baseline for audio features.

Are you building something in the Affective Computing space? Let's discuss in the comments below! 👇

From Messy XML to Vector Insights: Building a High-Performance Apple Health ETL Pipeline with Rust

wellallyTech — Wed, 15 Apr 2026 01:00:00 +0000

Have you ever tried to open your Apple Health export file? If you’ve been tracking your steps, heart rate, and sleep for more than a couple of years, that export.xml is likely a multi-gigabyte monster that makes VS Code cry and Python scripts crawl.

Cleaning five years of "dirty" health data requires a robust Rust ETL pipeline and efficient Data Engineering practices. In this tutorial, we will transform a chaotic XML swamp into a structured powerhouse using Polars for high-speed manipulation and Qdrant for semantic search capabilities. Whether you're building a personal health dashboard or a Quantified Self knowledge graph, this guide will show you how to handle massive datasets without melting your CPU. 🚀

The Architecture: From Raw XML to Vector Search

Before we dive into the borrow checker, let's look at the high-level flow. We are moving from a streaming XML parser to a columnar memory format, and finally to a vector representation.

graph TD
    A[Apple Health export.xml] -->|Streaming Parse| B(Rust + xml-rs)
    B -->|Batch Processing| C{Polars DataFrame}
    C -->|Data Cleaning & Normalization| C
    C -->|Parquet Export| D[Local Storage]
    C -->|Embedding Generation| E[Vector Database: Qdrant]
    E -->|Semantic Query| F[Personal Health Insights]

Prerequisites

To follow along, you’ll need:

Rust (latest stable)
Apple Health Export: Go to Health App > Profile > Export Health Data.
Tech Stack: xml-rs (streaming), polars (data wrangling), qdrant-client (vector storage).

Step 1: Streaming the XML Monster 🦖

The problem with Apple Health's XML is its sheer size. Loading it entirely into memory is a suicide mission for your RAM. We use xml-rs to pull events one by one.

use xml::reader::{EventReader, XmlEvent};
use std::fs::File;
use std::io::BufReader;

struct HealthRecord {
    record_type: String,
    value: f64,
    start_date: String,
}

fn stream_records(path: &str) {
    let file = File::open(path).unwrap();
    let file = BufReader::new(file);
    let parser = EventReader::new(file);

    for e in parser {
        match e {
            Ok(XmlEvent::StartElement { name, attributes, .. }) => {
                if name.local_name == "Record" {
                    // Extract attributes: type, value, creationDate
                    // Logic to push into a batch buffer
                }
            }
            _ => {}
        }
    }
}

Step 2: High-Speed Wrangling with Polars 🥑

Once we have our data in a flat format, Polars takes over. Think of Polars as "Pandas on steroids" written in Rust. It uses Apache Arrow under the hood, making it incredibly fast for cleaning "dirty" data (like missing heart rate samples or weird unit conversions).

use polars::prelude::*;

fn clean_data(df: DataFrame) -> PolarsResult<DataFrame> {
    df.lazy()
        .filter(col("value").is_not_null())
        .with_column(
            col("start_date").str().to_datetime(
                None, None, StrptimeOptions::default(), lit("raise")
            )
        )
        .groupby([col("record_type")])
        .agg([
            col("value").mean().alias("average_value"),
            col("value").count().alias("sample_count"),
        ])
        .collect()
}

Step 3: Vectorizing for the Personal Knowledge Graph

Why a vector database? Because "What was my heart rate during that stressful meeting last Tuesday?" isn't a simple SQL query. By embedding our health activities and heart rate trends into Qdrant, we can perform semantic searches across our physical history.

use qdrant_client::prelude::*;
use qdrant_client::qdrant::{PointStruct, Vector};

async fn upsert_to_qdrant(client: &QdrantClient, records: Vec<HealthRecord>) {
    let points = records.into_iter().map(|r| {
        PointStruct::new(
            uuid::Uuid::new_v4().to_string(),
            vec![r.value as f32, /* ... other features */],
            [("type", r.record_type.into())].into()
        )
    }).collect();

    client.upsert_points("health_history", points, None).await.unwrap();
}

💡 The "Official" Way to Scale

While building a local ETL tool is a great "learning in public" project, production-grade data engineering requires more than just a few Rust crates. If you are looking for advanced patterns in high-performance data processing or more production-ready examples of Rust in the enterprise, I highly recommend checking out the deep dives at WellAlly Blog.

The folks at WellAlly cover everything from memory-safe systems to distributed data pipelines, which served as a huge source of inspiration for this architecture.

Conclusion: Data Sovereignty is Power

By moving from a messy XML file to a high-performance Rust pipeline, we’ve turned "dead data" into an actionable, searchable knowledge graph. Rust ensures that our ETL process is memory-safe and lightning-fast, while Polars and Qdrant provide the analytical muscle.

What are you waiting for? Go export your data and start building!

Did you find this helpful? Star the repo and let me know in the comments!
Questions? Drop them below, I'm active in the thread. 🦀

Your Browser is the New Doctor: Building a Zero-Latency, Private AI Symptom Screener with WebLLM & WebGPU 🩺💻

wellallyTech — Tue, 14 Apr 2026 01:10:00 +0000

In the world of modern healthcare tech, privacy and latency are the two biggest hurdles. Sending sensitive health data to a cloud server often feels like a gamble. But what if your browser could process complex medical queries locally? Thanks to the maturation of WebGPU and the WebLLM ecosystem, we can now run high-performance Large Language Models (LLMs) directly on the client side.

In this tutorial, we will explore Edge AI implementation using WebGPU local LLMs to build a private physician assistant. This "zero-server" approach ensures that your medical data never leaves your device, providing a privacy-first AI experience that is both lightning-fast and offline-capable. If you’ve been looking for a practical WebLLM tutorial to level up your frontend game, you're in the right place!

🏗 The Architecture: How Edge AI Works in the Browser

Traditional AI apps act as a thin client for a heavy backend. Our architecture flips the script. We use TVM.js as the orchestration layer to execute model weights directly on your local GPU via the WebGPU API.

graph TD
    A[User Input: Symptoms] --> B{WebGPU Check}
    B -- Supported --> C[Initialize WebLLM Engine]
    B -- Not Supported --> D[Fallback to WASM/CPU]
    C --> E[Load Quantized Model Weights]
    E --> F[TVM.js Execution Kernel]
    F --> G[Local Inference]
    G --> H[Streamed Response to UI]
    H --> I[Result: Privacy-Preserved Screening]

🛠 Prerequisites

To follow along, ensure your stack looks like this:

Tech Stack: WebLLM, TVM.js, TypeScript, Vite.
Browser: A WebGPU-compatible browser (Chrome 113+, Edge 113+).
Hardware: A decent dedicated or integrated GPU (Apple M-series, Nvidia RTX, etc.).

🚀 Step-by-Step Implementation

1. Project Initialization & Setup

First, let's spin up a Vite project with TypeScript and install the necessary dependencies.

npm create vite@latest local-physician -- --template react-ts
cd local-physician
npm install @mlc-ai/web-llm

2. Checking for WebGPU Support

Before we pull down 2GB of model weights, we need to ensure the user's hardware can actually handle the heat. 🌶️

// gpuCheck.ts
export async function isWebGPUSupported(): Promise<boolean> {
  if (!navigator.gpu) {
    console.error("WebGPU is not supported on this browser.");
    return false;
  }
  const adapter = await navigator.gpu.requestAdapter();
  return !!adapter;
}

3. The Core Inference Engine

We’ll use the CreateMLCEngine function from WebLLM. We'll specifically target a lightweight model like Llama-3-8B-Instruct-q4f16_1-MLC or Phi-3-mini for the best balance between medical reasoning and download size.

import { CreateMLCEngine, MLCEngine } from "@mlc-ai/web-llm";

const SYSTEM_PROMPT = `
You are a private, local medical assistant. 
Your goal is to provide preliminary symptom screening. 
Always include a disclaimer that you are an AI and not a doctor.
Keep responses concise and empathetic.
`;

export async function initializeEngine(
  modelId: string, 
  onProgress: (progress: number) => void
): Promise<MLCEngine> {
  const engine = await CreateMLCEngine(modelId, {
    initProgressCallback: (report) => {
      onProgress(Math.round(report.progress * 100));
    }
  });
  return engine;
}

4. Handling the Chat Logic

We want a streaming response to give that "typewriter" feel and reduce perceived latency.

async function handleSymptomScreening(engine: MLCEngine, userInput: string) {
  const messages = [
    { role: "system", content: SYSTEM_PROMPT },
    { role: "user", content: `I have the following symptoms: ${userInput}. What could this be?` }
  ];

  const chunks = await engine.chat.completions.create({
    messages,
    stream: true, // This is where the magic happens!
  });

  let fullResponse = "";
  for await (const chunk of chunks) {
    const content = chunk.choices[0]?.delta?.content || "";
    fullResponse += content;
    // Update your UI state here
    renderUI(fullResponse);
  }
}

💡 The "Official" Way to Scale

Building a prototype in the browser is exhilarating, but taking Edge AI to production involves complex challenges like model versioning, cross-origin caching, and advanced quantization strategies.

For a deeper dive into production-grade Edge AI patterns and how to optimize WebGPU kernels for enterprise applications, I highly recommend checking out the WellAlly Tech Blog. They have some incredible resources on bridging the gap between "experimental" and "deployable" AI.

🎯 Conclusion

By moving the "brain" of our application from a centralized server to the user's browser, we've achieved:

Total Privacy: Sensitive symptoms stay on the device.
Zero Latency: No round-trips to a server in Virginia or Tokyo.
Cost Efficiency: $0 in API inference costs for the developer.

The future of the web is decentralized and GPU-accelerated. Stop treating the browser as a simple document viewer—it's a powerful AI engine waiting to be unleashed! 🚀

What do you think? Are you ready to move your LLM workloads to the edge? Drop a comment below or share your latest WebGPU project!

From Pixels to Calories: Building a High-Precision Food Estimation Pipeline with GPT-4o and SAM

wellallyTech — Mon, 13 Apr 2026 01:10:00 +0000

Tracking calories is the ultimate "love-hate" relationship for fitness enthusiasts. We love the results, but we hate the manual data entry. While Multimodal AI has made massive leaps, most generic vision models struggle with "depth perception" and "portion sizing" from a flat 2D image.

In this tutorial, we are building a high-precision food image recognition and calorie estimation pipeline. By combining the Segment Anything Model (SAM) for surgical-grade instance segmentation and GPT-4o Vision for semantic reasoning, we solve the engineering hurdle of distinguishing between a "small side of fries" and a "jumbo portion." This Computer Vision pipeline uses a sophisticated spatial reasoning approach to turn raw pixels into actionable nutritional data. 🚀

The Architecture: How it Works

To achieve high accuracy, we don't just "throw an image at an LLM." We use a multi-stage process where SAM identifies specific food boundaries, and GPT-4o acts as the "Dietician" interpreting those segments.

graph TD
    A[User Uploads Photo] --> B[OpenCV Pre-processing]
    B --> C[SAM: Instance Segmentation]
    C --> D[Identify Food Masks & Bounding Boxes]
    D --> E[GPT-4o Vision: Multi-modal Analysis]
    E --> F[Volume & Density Estimation]
    F --> G[Nutritional Database Lookup]
    G --> H[FastAPI Response: Calorie & Macro Breakdown]

Prerequisites

Before we dive into the code, ensure you have the following tech_stack ready:

Python 3.9+
FastAPI (for the API layer)
Segment Anything Model (SAM) (via segment-anything or mobile-sam)
OpenAI SDK (Access to gpt-4o)
OpenCV (for image manipulation)

Step 1: Precision Segmentation with SAM

Generic vision models often get confused by plate patterns or background clutter. By using SAM, we extract only the "food" pixels. This reduces noise and helps GPT-4o focus on the actual volume.

import numpy as np
import cv2
from segment_anything import sam_model_registry, SamPredictor

# Initialize SAM
sam_checkpoint = "sam_vit_h_4b8939.pth"
model_type = "vit_h"
sam = sam_model_registry[model_type](checkpoint=sam_checkpoint)
predictor = SamPredictor(sam)

def get_food_segments(image_path):
    image = cv2.imread(image_path)
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    predictor.set_image(image)

    # In a real app, you'd use a prompt (point or box)
    # Here we simulate an auto-masking approach
    masks, scores, logits = predictor.predict(
        point_coords=None,
        point_labels=None,
        multimask_output=True,
    )
    return masks[0] # Return the highest-scoring mask

Step 2: The GPT-4o "Reasoning" Engine

Once we have our segments, we send the cropped image and the original context to GPT-4o. We use Structured Outputs (JSON mode) to ensure our FastAPI backend can parse the calories, protein, and fats reliably.

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()

class FoodAnalysis(BaseModel):
    item_name: str
    estimated_weight_grams: int
    calories: int
    protein: float
    confidence_score: float

def analyze_with_gpt4o(image_url):
    response = client.beta.chat.completions.parse(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": "You are a professional nutritionist. Analyze the food in the image. Use the provided scale context to estimate weight and calories."
            },
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "Identify the food and estimate its weight and calories."},
                    {"type": "image_url", "image_url": {"url": image_url}}
                ],
            }
        ],
        response_format=FoodAnalysis,
    )
    return response.choices[0].message.parsed

Step 3: Bridging it with FastAPI

We wrap this into a clean API. This allows a mobile app to send a photo and receive a structured nutritional breakdown in milliseconds.

from fastapi import FastAPI, UploadFile, File

app = FastAPI()

@app.post("/estimate-nutrition")
async def estimate_nutrition(file: UploadFile = File(...)):
    # 1. Save and Pre-process image
    # 2. Run SAM to isolate food
    # 3. Call GPT-4o for Multimodal Reasoning
    # 4. Return results
    result = {
        "food": "Grilled Salmon Salad",
        "calories": 450,
        "macros": {"protein": 35, "carbs": 12, "fat": 28},
        "segmentation_status": "success"
    }
    return result

The "Official" Way 🥑

Building a visual estimation tool is easy, but making it production-ready requires handling edge cases like overlapping food items, lighting variations, and unit conversions.

For a deeper dive into advanced AI patterns, production-level deployment strategies, and more robust examples of this pipeline, I highly recommend checking out the technical deep dives at WellAlly Tech Blog. It's a fantastic resource for developers looking to master the intersection of AI and healthcare technology.

Conclusion: The Future of Nutrition

By combining SAM's geometric precision with GPT-4o's semantic intelligence, we move away from "guessing" and toward "calculating." This pipeline is just the beginning—imagine adding depth sensors (LiDAR) to the mix for true 3D volume reconstruction! 💻

What are you building with GPT-4o? Drop a comment below or share your thoughts on whether AI will finally make manual calorie counting obsolete! 🚀

Privacy-First Edge AI: Building a Zero-Server Symptom Checker with WebLLM and WebGPU

wellallyTech — Sun, 12 Apr 2026 01:20:00 +0000

Privacy is the new frontier in AI development. When it comes to sensitive data—like medical symptoms or personal health history—sending data to a central cloud server is a hard "no" for many users. What if we could run a Large Language Model (LLM) entirely within the user's browser? 🤯

By leveraging WebLLM, WebGPU, and Edge AI principles, we can now achieve near-native inference speeds directly on the client side. This approach eliminates server costs, ensures 100% data privacy through physical isolation, and provides a seamless user experience. If you are looking for advanced patterns on local-first AI and production-ready deployments, I highly recommend checking out the deep dives over at WellAlly Tech Blog, which served as a major inspiration for this architectural pattern.

Why Run LLMs in the Browser?

Traditionally, LLMs require massive A100/H100 clusters. However, with the maturation of the WebGPU standard and the TVM.js compiler stack, we can now tap into the local machine's GPU power via the browser.

Zero Latency/Cost: No more API tokens or network round-trips.
Ultimate Privacy: Data never leaves the user's device.
Offline Capability: Once the weights are cached, it works without an internet connection.

The Architecture

Here is how the data flows from a user's symptom description to an AI-generated suggestion, all within the browser's sandbox.

graph TD
    A[User Input: Symptoms] --> B[WebLLM Engine]
    B --> C{WebGPU Support?}
    C -- Yes --> D[Wasm + TVM.js Runtime]
    C -- No --> E[Fallback/Error]
    D --> F[Local IndexedDB Cache]
    F --> G[GPU Accelerated Inference]
    G --> H[Streaming Response to UI]
    H --> I[User Actionable Advice]

Prerequisites

Before we dive into the code, ensure your environment is ready:

Tech Stack: TypeScript, WebLLM, Vite.
Browser: Chrome 113+ or any browser with WebGPU enabled.
Hardware: A machine with a decent integrated or dedicated GPU.

Step 1: Initializing the WebLLM Engine

First, we need to set up the worker and initialize the MLCEngine. Since model weights can be large (several GBs), WebLLM uses a clever caching mechanism via the browser's Cache API.

import { CreateMLCEngine, MLCEngine } from "@mlc-ai/web-llm";

// We'll use a quantized version of Llama-3 or Phi-3 for efficiency
const selectedModel = "Llama-3-8B-Instruct-v0.1-q4f16_1-MLC";

async function initializeEngine(onProgress: (p: number) => void) {
  console.log("🚀 Initializing WebGPU Engine...");

  const engine = await CreateMLCEngine(selectedModel, {
    initProgressCallback: (report) => {
      onProgress(Math.round(report.progress * 100));
      console.log(report.text);
    },
  });

  return engine;
}

Step 2: Crafting the System Prompt

For a symptom checker, the system prompt is critical. We need to ensure the AI behaves like a supportive assistant while emphasizing that it is not a replacement for professional medical advice.

const SYSTEM_PROMPT = `
You are a private, local medical symptom checker. 
Analyze the symptoms provided by the user. 
Provide potential causes and suggest whether they should seek urgent care.
ALWAYS include a disclaimer: "This is an AI-generated summary, not a medical diagnosis."
Keep the data processing strictly local.
`;

Step 3: Executing Inference

Now, let's build the chat function. We use a streaming approach to make the UI feel responsive, just like ChatGPT.

async function checkSymptoms(engine: MLCEngine, userInput: string) {
  const messages = [
    { role: "system", content: SYSTEM_PROMPT },
    { role: "user", content: userInput }
  ];

  let fullResponse = "";
  const chunks = await engine.chat.completions.create({
    messages,
    stream: true, // High-fives for streaming! 🖐️
  });

  for await (const chunk of chunks) {
    const content = chunk.choices[0]?.delta?.content || "";
    fullResponse += content;
    // Update your React/Vue state here
    updateUI(fullResponse); 
  }
}

Performance Optimization

When working with Edge AI, memory management is your biggest hurdle.

Quantization: We use q4f16 (4-bit quantization) to shrink the model size by ~70% without significant logic loss.
TVM.js: This handles the bridge between the high-level model logic and the low-level WebGPU shaders.

For a deeper dive into how to optimize these shaders for mobile browsers, the team at wellally.tech/blog has published some incredible benchmarks comparing WebLLM performance across different chipsets.

Conclusion

We've just built a fully functional, privacy-preserving symptom checker that runs entirely on the client. No servers, no leaks, just pure GPU-accelerated magic. 🥑

Key Takeaways:

WebGPU is the backbone of modern browser-based AI.
WebLLM provides the easiest abstraction for running MLC-compiled models.
Privacy-first apps are the future of healthcare tech.

What are you planning to build with WebGPU? Let me know in the comments below! Don't forget to star the MLC-LLM repo and keep experimenting.

Love this content? Follow for more "Learning in Public" tutorials! 🚀

Stop Googling Your Lab Results: Build a Medical RAG Pipeline with Unstructured.io and Qdrant

wellallyTech — Sat, 11 Apr 2026 01:15:00 +0000

We’ve all been there: you receive a 20-page PDF medical report filled with cryptic tables, nested rows, and handwritten notes. You want to ask, "Has my LDL cholesterol trended down since last year?" but your data is trapped in a non-searchable pixel prison. 🏥

In this tutorial, we are building a Medical RAG (Retrieval-Augmented Generation) system. We will leverage Unstructured.io for complex PDF layout analysis, Qdrant as our high-performance vector database, and LangChain to glue it all together. This setup allows you to transform messy lab reports into a structured, queryable personal medical knowledge base.

By the end of this guide, you’ll master Unstructured.io OCR, vector indexing, and how to handle multi-modal document parsing for real-world messy data. 🚀

The Architecture: From Pixels to Insights

Handling medical reports is tricky because the data is usually trapped in tables. A simple text extraction won't work; we need layout-aware parsing.

graph TD
    A[PDF Lab Reports] --> B[Unstructured.io Partitioning]
    B --> C{Layout Analysis}
    C -->|Tables| D[HTML/Structured Data]
    C -->|Text| E[Text Chunks]
    D --> F[LangChain Document Transformer]
    E --> F
    F --> G[OpenAI Embeddings]
    G --> H[(Qdrant Vector Store)]
    I[User Query] --> J[Query Embedding]
    J --> H
    H --> K[Contextual Retrieval]
    K --> L[GPT-4o Medical Insight]

Prerequisites

Before we dive in, ensure you have the following tech_stack ready:

Unstructured.io: For "hi_res" partitioning (requires poppler and tesseract).
Qdrant: Our vector database (running locally via Docker or Cloud).
LangChain: The orchestration framework.
OpenAI Embeddings: To turn medical text into math.

pip install unstructured[pdf] qdrant-client langchain openai tiktoken

Step 1: Parsing Complex Tables with Unstructured.io

Standard PDF loaders often mangle tables. Unstructured.io uses computer vision models to detect document elements (Title, Table, List).

from unstructured.partition.pdf import partition_pdf

# We use the 'hi_res' strategy to trigger OCR and layout detection
# This is crucial for medical reports with complex grids
elements = partition_pdf(
    filename="my_lab_report.pdf",
    strategy="hi_res",
    infer_table_structure=True,
    model_name="yolox"
)

# Extracting tables specifically as HTML for better LLM understanding
tables = [el.metadata.text_as_html for el in elements if el.category == "Table"]
print(f"Detected {len(tables)} tables in the report!")

Step 2: Vector Indexing with Qdrant

Once we have our chunks, we need to store them in a way that captures semantic meaning. Qdrant is perfect for this due to its speed and payload filtering capabilities. 🥑

from langchain_community.vectorstores import Qdrant
from langchain_openai import OpenAIEmbeddings
from langchain.schema import Document

# Convert Unstructured elements into LangChain Documents
docs = []
for el in elements:
    metadata = el.metadata.to_dict()
    metadata["source"] = "lab_report_2023"
    docs.append(Document(page_content=str(el), metadata=metadata))

# Initialize Qdrant and upload embeddings
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = Qdrant.from_documents(
    docs,
    embeddings,
    location=":memory:", # Local testing, or use URL/API Key for production
    collection_name="medical_knowledge_base",
)

Step 3: Querying the Knowledge Base

Now we can ask complex questions. The RAG pipeline will find the relevant table or text chunk and provide the context to the LLM.

from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model_name="gpt-4o", temperature=0)
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever()
)

query = "What was my Glucose level and is it within the normal range?"
response = qa_chain.invoke(query)
print(f"Result: {response['result']}")

💡 Pro-Tip: Advanced Medical Patterns

Building a local RAG is a great start, but when dealing with high-stakes medical data, you need to implement Hybrid Search and Reranking to ensure 100% accuracy.

If you are looking for production-ready patterns, such as handling multi-vector retrieval or integrating medical ontologies (like SNOMED CT), I highly recommend checking out the advanced guides on WellAlly Tech Blog. They provide deep dives into how to optimize vector search for specialized domains like healthcare and finance.

Conclusion

By combining Unstructured.io's layout awareness with Qdrant's vector capabilities, we've transformed a static PDF into a dynamic assistant. You no longer need to manually scan rows of data; your RAG pipeline does the heavy lifting.

What's next?

Add Chain-of-Thought prompting to interpret the medical trends.
Implement a front-end using Streamlit to upload your own PDFs.
Explore wellally.tech/blog for more insights on building robust AI agents.

Have you tried parsing medical data before? What was your biggest challenge? Let me know in the comments! 👇 💻

Your AI Health Concierge: Closing the Loop with LangGraph and Web Automation 🏥

wellallyTech — Fri, 10 Apr 2026 01:15:00 +0000

We’ve all been there: your wearable buzzes, telling you your Heart Rate Variability (HRV) is tanking, or your resting heart rate is spiking. Usually, we just ignore the notification until we're actually sick. But what if your data could take action?

In this tutorial, we are building a Health Loop Agent—a sophisticated AI system that doesn't just monitor data but acts on it. Using LangGraph for orchestration and Browser-use (powered by Playwright) for web automation, we'll create an agent that detects health anomalies via the Oura Cloud API and automatically navigates a medical portal to find an available doctor.

By the end of this post, you'll understand how to bridge the gap between "Passive Monitoring" and "Active Intervention" using state-of-the-art AI Agents and automated web navigation.

The Architecture: From Pulse to Appointment

Building an autonomous agent requires a robust state machine. We use LangGraph to manage the logic flow, ensuring the agent only proceeds to "Booking" if the "Analysis" phase confirms a sustained health risk.

graph TD
    A[Start: Daily Sync] --> B{Fetch Oura Data}
    B --> C[Analyze HRV & Sleep Trends]
    C -->|Normal| D[Log & Sleep]
    C -->|Anomaly Detected| E[Search Doctor Schedules]
    E --> F[Browser-use: Navigate Portal]
    F --> G{Slot Available?}
    G -->|Yes| H[Book Appointment & Notify]
    G -->|No| I[Alert User: Manual Check Required]
    H --> J[End]
    D --> J

Prerequisites

To follow along, you’ll need:

LangGraph: For the agent's cognitive architecture.
Browser-use & Playwright: For interacting with the "real world" web.
Oura Cloud API: To fetch real-time biometric data.
OpenAI GPT-4o: To serve as the brain for decision-making.

Step 1: Defining the Agent State

In LangGraph, everything revolves around the State. Our agent needs to keep track of biometric readings, the detected health status, and any appointment details found.

from typing import TypedDict, List, Optional

class HealthAgentState(TypedDict):
    hrv_readings: List[float]
    health_status: str # "optimal", "warning", "critical"
    appointment_needed: bool
    doctor_specialty: str
    available_slots: List[str]
    final_report: str

Step 2: The Analysis Node

We use the Oura Cloud API to pull recent HRV data. A "Critical" status is triggered if the HRV is 20% below the user's 7-day rolling average.

import requests
from datetime import datetime, timedelta

def analyze_biometrics(state: HealthAgentState):
    # Mocking Oura API call for brevity
    # In production, use: requests.get(OURA_API_URL, headers=headers)
    recent_hrv = [45, 42, 30, 28] # Sustained drop

    avg_hrv = sum(recent_hrv[:2]) / 2
    current_hrv = recent_hrv[-1]

    status = "optimal"
    if current_hrv < avg_hrv * 0.8:
        status = "critical"

    return {
        "hrv_readings": recent_hrv,
        "health_status": status,
        "appointment_needed": status == "critical",
        "doctor_specialty": "Cardiologist" if status == "critical" else "None"
    }

Step 3: Web Automation with Browser-use

This is where the magic happens. If the agent decides an appointment is needed, it triggers Browser-use. This library allows LLMs to "see" and "click" elements on a webpage just like a human.

from browser_use import Agent
from langchain_openai import ChatOpenAI

async def search_and_book_appointment(state: HealthAgentState):
    if not state["appointment_needed"]:
        return {"final_report": "All healthy. No action taken."}

    # Initialize the Browser Agent
    browser_agent = Agent(
        task=f"Navigate to health-portal.example.com, search for a {state['doctor_specialty']}, and find the earliest available slot next Monday.",
        llm=ChatOpenAI(model="gpt-4o"),
    )

    result = await browser_agent.run()

    return {
        "final_report": f"Detected HRV drop to {state['hrv_readings'][-1]}. {result.final_result}"
    }

The "Official" Way: Advanced Patterns 🥑

While this demo provides a functional loop, production-grade health agents require stricter privacy controls (HIPAA compliance), human-in-the-loop (HITL) verification, and robust error handling for web UI changes.

For more production-ready examples and advanced agentic patterns, I highly recommend checking out the technical deep-dives at WellAlly Tech Blog. They cover how to scale these "Closed-Loop" systems in enterprise environments and handle complex LLM observability.

Step 4: Connecting the Graph

Finally, we link our nodes together into a cohesive workflow.

from langgraph.graph import StateGraph, END

workflow = StateGraph(HealthAgentState)

# Add Nodes
workflow.add_node("analyze", analyze_biometrics)
workflow.add_node("automate_booking", search_and_book_appointment)

# Define Edges
workflow.set_entry_point("analyze")
workflow.add_conditional_edges(
    "analyze",
    lambda x: "automate_booking" if x["appointment_needed"] else END
)
workflow.add_edge("automate_booking", END)

# Compile
app = workflow.compile()

Conclusion: The Future of Proactive Health

We've just built a system that moves from Data -> Insight -> Action. By combining the reasoning capabilities of LangGraph with the "hands" of Browser-use, we've turned a simple wearable into a proactive health concierge.

What’s next?

Human-in-the-loop: Add a Slack notification node to ask for user approval before clicking "Confirm Booking."
Multi-Modal Analysis: Use GPT-4o to analyze photos of symptoms alongside HRV data.

The era of the "Passive Dashboard" is over. The era of the Autonomous Health Agent has begun. 🚀

What do you think? Would you trust an AI to book your doctor's appointments? Let’s chat in the comments! 👇

If you enjoyed this build, don't forget to ❤️ and follow for more "Learning in Public" tutorials!

Is Your Code Review Killing You? 🧘 Real-time Stress Monitoring with Wav2Vec 2.0

wellallyTech — Thu, 09 Apr 2026 01:20:00 +0000

We’ve all been there: a production incident hits at 4 PM on a Friday, or you're stuck in a heated Code Review session where "just one small change" turns into a refactor of the entire authentication module. Your heart rate climbs, and your voice subtly shifts in pitch and rhythm. In the world of Affective Computing, these vocal cues are gold mines for understanding developer burnout and mental well-being.

Today, we are building a real-time Audio Stress Monitor specifically designed for developers. By leveraging audio signal processing and the power of Wav2Vec 2.0, we can transform raw speech from a Zoom meeting into actionable insights about stress levels. This tutorial explores how to implement a sophisticated machine learning audio analysis pipeline to detect high-pressure moments before they lead to burnout.

The Architecture: From Sound Waves to Stress Scores 🛠️

To handle real-time audio data, we need a robust pipeline that can process chunks of speech, extract prosodic features, and classify them. Here is how the system is structured:

graph TD
    A[Microphone / Zoom Audio] -->|Live Stream| B(Streamlit Frontend)
    B --> C{Feature Extractor}
    C -->|Resampling| D[Wav2Vec 2.0 Processor]
    D --> E[Transformer Encoder]
    E --> F[Stress Classification Head]
    F --> G[Real-time Dashboard]
    G --> H[Burnout Alerts]

    subgraph AI Pipeline
    D
    E
    F
    end

Prerequisites

Before we dive into the code, ensure you have the following tech stack ready:

Hugging Face Transformers: For accessing the pre-trained Wav2Vec 2.0 model.
Streamlit: For the reactive dashboard.
Librosa: For audio preprocessing.
Docker: For containerizing our environment.

Step 1: Loading the Wav2Vec 2.0 Model

Wav2Vec 2.0 is a revolutionary framework for self-supervised learning of speech representations. For stress detection, we don't just need the words (ASR); we need the emotion and prosody.

import torch
import torch.nn as nn
from transformers import Wav2Vec2Model, Wav2Vec2FeatureExtractor

class StressClassifier(nn.Module):
    def __init__(self, model_name):
        super(StressClassifier, self).__init__()
        self.wav2vec2 = Wav2Vec2Model.from_pretrained(model_name)
        self.config = self.wav2vec2.config

        # We add a classification head for Stress: 0 (Relaxed) to 2 (High Stress)
        self.classifier = nn.Sequential(
            nn.Linear(self.config.hidden_size, 256),
            nn.ReLU(),
            nn.Dropout(0.1),
            nn.Linear(256, 3) 
        )

    def forward(self, x):
        outputs = self.wav2vec2(x)
        # Use the mean of hidden states as the representative feature
        hidden_states = outputs.last_hidden_state
        pooled_output = torch.mean(hidden_states, dim=1)
        return self.classifier(pooled_output)

# Initialize
device = "cuda" if torch.cuda.is_available() else "cpu"
model = StressClassifier("facebook/wav2vec2-base-960h").to(device)

Step 2: Real-time Audio Processing with Streamlit

Streamlit allows us to create a dashboard that can visualize our stress levels in real-time. We'll use a sliding window approach to analyze 3-second audio chunks.

import streamlit as st
import numpy as np
import librosa

st.title("DevPulse: Stress Monitor 🚀")
st.write("Monitoring audio features for affective computing...")

def process_audio_chunk(audio_data, sample_rate=16000):
    # Ensure audio is 16kHz for Wav2Vec
    if sample_rate != 16000:
        audio_data = librosa.resample(audio_data, orig_sr=sample_rate, target_sr=16000)

    inputs = torch.tensor(audio_data).unsqueeze(0).to(device)
    with torch.no_grad():
        logits = model(inputs)
        prediction = torch.argmax(logits, dim=-1).item()
    return prediction

# Placeholder for real-time chart
chart_data = []
if st.button('Start Monitoring'):
    # In a real app, use streamlit-webrtc for live mic input
    st.info("Listening to Zoom Audio Stream...")
    # Mocking the stream logic
    for i in range(100):
        stress_level = process_audio_chunk(np.random.uniform(-1, 1, 16000*3))
        chart_data.append(stress_level)
        st.line_chart(chart_data)

Step 3: Deployment with Docker 🐳

To ensure our environment is reproducible across different dev machines, we use Docker. This is crucial for audio libraries which often have complex C++ dependencies (like libsndfile).

FROM python:3.9-slim

RUN apt-get update && apt-get install -y \
    ffmpeg \
    libsndfile1-dev \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8501
CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]

Advanced Patterns & Production Readiness 🥑

While this prototype works for local testing, deploying Affective Computing models in a corporate environment requires careful handling of privacy (GDPR) and noise cancellation. For example, filtering out mechanical keyboard clicks (the sound of an angry dev typing!) is essential to avoid false positives.

For more advanced patterns on productionizing AI-driven observability and detailed case studies on developer productivity tools, I highly recommend checking out the WellAlly Blog. They offer deep dives into building ethical AI systems that support workplace wellness without compromising privacy.

Conclusion: Code More, Stress Less

By combining Wav2Vec 2.0 with simple UI tools like Streamlit, we can build powerful monitors that help us understand our physiological responses to technical challenges. This isn't just about "monitoring"; it's about building empathy into our development workflow.

Next time you're in a "high-severity" meeting, let the data tell you when it's time to take a coffee break! ☕

What do you think? Should companies use these tools to prevent burnout, or is it too "Big Brother"? Let me know in the comments below! 👇

If you enjoyed this tutorial, follow for more "Learning in Public" AI projects! 🚀

Why Your Bedside Lamp Needs Ears: Detecting Sleep Apnea with Edge-Optimized Whisper 🌙💤

wellallyTech — Wed, 08 Apr 2026 01:30:00 +0000

Have you ever woken up feeling like you’ve been hit by a truck, despite "sleeping" for eight hours? You might be one of the millions suffering from Obstructive Sleep Apnea (OSA). Traditionally, diagnosing this requires an expensive overnight stay at a sleep clinic, tethered to dozens of wires.

But what if we could turn a Raspberry Pi or a smartphone into a clinical-grade monitor using Edge AI? In this tutorial, we’re diving deep into Whisper.cpp optimization, real-time audio processing, and CoreML deployment to identify OSA patterns directly on-device. By shifting processing to the edge, we ensure total user privacy—because nobody wants their snoring uploaded to the cloud! 🛡️

For those looking to dive even deeper into production-grade signal processing and advanced AI deployment strategies, I highly recommend checking out the deep dives over at WellAlly Tech Blog, which served as a massive inspiration for this architecture.

🏗️ The Architecture: From Soundwaves to Insights

To achieve real-time detection without killing the battery (or the CPU), we need a lean pipeline. We aren't just transcribing text; we are analyzing the metadata of the audio signal—specifically looking for "apneic events" (periods of silence followed by gasping).

graph TD
    A[Microphone Input] -->|PCM Stream| B(WebAudio API / FFmpeg)
    B -->|VAD - Voice Activity Detection| C{Is it Snoring?}
    C -->|Yes| D[Whisper.cpp / CoreML Inference]
    C -->|No| E[Low Power Mode]
    D -->|Transcription + Timestamps| F[OSA Pattern Analyzer]
    F -->|Frequency Interruption detected| G[User Alert/Report]
    G -->|Detailed Analytics| H[Dashboard]

🛠️ The Tech Stack

Whisper.cpp: High-performance C++ port of OpenAI’s Whisper.
FFmpeg: For real-time audio resampling and normalization.
CoreML: To leverage the Apple Neural Engine (ANE) on iPhones.
WebAudio API: For browser-based edge processing.

🚀 Step 1: Optimizing Whisper for the Edge

Standard Whisper models are too heavy for a Raspberry Pi or an iPhone. We need to use Quantization (4-bit) and the Whisper.cpp implementation to make it fly.

First, let's prepare the environment and convert the model to the ggml format optimized for CoreML.

# Clone the optimized repository
git clone https://github.com/ggerganov/whisper.cpp.git
cd whisper.cpp

# Download the base model (good balance of speed/accuracy)
bash ./models/download-ggml-model.sh base.en

# Generate CoreML model for hardware acceleration
python3 ./models/generate-coreml-model.py base.en

By using the base model with CoreML, we reduce inference time from seconds to milliseconds on mobile devices.

🎙️ Step 2: Real-time Audio Ingestion (WebAudio API)

If you are building a cross-platform web app, the WebAudio API is your best friend. We need to capture audio at 16kHz (Whisper’s requirement) and feed it into our WASM-compiled Whisper instance.

const audioContext = new AudioContext({ sampleRate: 16000 });
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
const source = audioContext.createMediaStreamSource(stream);

// Create a Worklet for real-time processing
await audioContext.audioWorklet.addModule('processor.js');
const recorder = new AudioWorkletNode(audioContext, 'whisper-processor');

source.connect(recorder);
recorder.port.onmessage = (event) => {
    const audioBuffer = event.data;
    // Send this buffer to Whisper.cpp WASM
    runInference(audioBuffer);
};

🔍 Step 3: Identifying OSA Patterns

The secret sauce isn't just transcribing "snore" sounds. OSA is characterized by a Crescendo-Decrescendo pattern: loud snoring, a sudden silence (the apnea), followed by a loud "choke" or "gasp."

We can use the timestamps provided by Whisper.cpp to detect these gaps.

// Logic for identifying interruptions in Whisper.cpp output
void process_segments(struct whisper_context * ctx) {
    const int n_segments = whisper_full_n_segments(ctx);
    for (int i = 0; i < n_segments; ++i) {
        const char * text = whisper_full_get_segment_text(ctx, i);
        int64_t t0 = whisper_full_get_segment_t0(ctx, i);
        int64_t t1 = whisper_full_get_segment_t1(ctx, i);

        // Logic: If gap between t0 of current and t1 of previous > 10s 
        // and current segment contains [gasping] or [choking]
        if ((t0 - last_t1) > 1000 && is_distress_sound(text)) {
            printf("⚠️ Potential OSA Event Detected at %lld ms\n", t0);
        }
        last_t1 = t1;
    }
}

💡 The "Official" Way to Scale

While hacking together a local script is fun, building a robust health-tech product requires more rigorous data handling and model fine-tuning. For instance, how do you handle background noise from a bedside fan?

For advanced patterns on noise suppression and efficient model distillation for specialized medical audio, you should definitely check out the engineering guides at WellAlly Tech Blog. They have fantastic resources on moving from a "cool demo" to a "production-ready" edge AI system.

🎯 Conclusion

By leveraging Whisper.cpp and CoreML, we’ve effectively turned a generic audio model into a specialized health monitor that runs entirely on-device. This not only saves cloud costs but, more importantly, keeps sensitive health data where it belongs—with the user. 🥑

Next Steps for you:

Try running the tiny vs base model on a Raspberry Pi 4.
Implement a band-pass filter using FFmpeg to isolate frequencies between 60Hz and 500Hz (typical snoring range).
Let me know in the comments: What other health metrics should we track on the edge?

Happy coding, and sleep well! 🛌🚀

Your Health, Your Secrets: How to Prove Vaccination Status Without Leaking a Single Byte (ZKP Guide)

wellallyTech — Tue, 07 Apr 2026 01:20:00 +0000

Privacy is the new gold. In an era where digital identity is everything, why do we still hand over our entire medical history just to prove a single fact? Whether it’s entering a venue or crossing a border, the current "show your QR code" method leaks your name, date of birth, and specific medical records. 🛡️

Today, we are diving deep into Zero-Knowledge Proofs (ZKP) and Privacy Computing. By using Circom and SnarkJS, we will build a system where you can prove "I have been vaccinated" without revealing which vaccine you got or when you got it. This is the future of Blockchain Identity and secure health data sharing.

The Architecture of Privacy

The magic of ZK-SNARKs (Zero-Knowledge Succinct Non-Interactive Argument of Knowledge) lies in the separation of the Prover and the Verifier. The Prover uses their private data to generate a mathematical proof, and the Verifier checks that proof against a public "circuit" without ever seeing the raw data.

Data Flow for a Private Health Claim

sequenceDiagram
    participant User as Prover (User)
    participant Circuit as Circom Circuit
    participant Verifier as Verifier (Smart Contract/Server)

    Note over User: Private Input: VaccineID, Secret Key
    Note over User: Public Input: Hash(VaccineID)

    User->>Circuit: Input private & public signals
    Circuit->>User: Generate Proof (.json) + Public Signals

    User->>Verifier: Submit Proof + Public Signals

    Note over Verifier: Verify math, not data
    Verifier-->>User: Logic Result (TRUE/FALSE)
    Note right of Verifier: "Proof is valid. Access granted!"

Prerequisites

To follow this advanced tutorial, ensure you have the following tech_stack ready:

Circom: The domain-specific language for writing ZK circuits.
SnarkJS: The JavaScript implementation to generate and verify proofs.
Node.js: For running our verification logic.
Identity Protocols: Understanding of how public keys represent identity.

Step 1: Writing the Circom Circuit

We need to create a circuit that proves we know a vaccineSerial that, when hashed with a secret, matches a publicly known identityCommitment. This ensures the proof belongs to you and the vaccine is valid.

pragma circom 2.0.0;

include "node_modules/circomlib/circuits/poseidon.circom";

template VaccineProof() {
    // Private inputs (the world doesn't see these!)
    signal input vaccineSerial;
    signal input userSecret;

    // Public inputs
    signal input identityCommitment;

    // Components
    component hasher = Poseidon(2);

    // Logic: Hash the serial and the secret
    hasher.inputs[0] <== vaccineSerial;
    hasher.inputs[1] <== userSecret;

    // Constraint: The result must match the public commitment
    identityCommitment === hasher.out;
}

component main {public [identityCommitment]} = VaccineProof();

Step 2: Generating the Proof with SnarkJS

Once our circuit is compiled, we need to generate a "witness." Think of the witness as the set of values that satisfies the mathematical equations in our circuit. 🧬

const snarkjs = require("snarkjs");
const fs = require("fs");

async function run() {
    // 1. Define the inputs
    const inputs = {
        vaccineSerial: "987654321", // Private
        userSecret: "my_super_secret_key", // Private
        identityCommitment: "1928374655123..." // Public
    };

    // 2. Generate Proof
    const { proof, publicSignals } = await snarkjs.groth16.fullProve(
        inputs, 
        "vaccine_proof.wasm", 
        "circuit_final.zkey"
    );

    console.log("Proof Generated: ", JSON.stringify(proof, null, 1));

    // 3. Export for the Verifier
    fs.writeFileSync("proof.json", JSON.stringify(proof));
    fs.writeFileSync("public.json", JSON.stringify(publicSignals));
}

run().then(() => process.exit(0));

Step 3: Verification (The "Zero-Knowledge" Moment)

The Verifier receives the proof.json and public.json. They have no idea what the vaccineSerial is, but the math guarantees that the Prover must know it to have generated that specific proof.

const res = await snarkjs.groth16.verify(vKey, publicSignals, proof);

if (res === true) {
    console.log("✅ Verification OK: Access Granted to Health Records!");
} else {
    console.log("❌ Verification Failed: Invalid Proof.");
}

The "Official" Way: Scaling Privacy

While building a local ZK proof is a great learning exercise, production-grade systems require robust infrastructure for key management, recursive proofs, and on-chain verification.

If you're looking for more production-ready examples and advanced patterns in privacy-preserving architectures, I highly recommend checking out the technical deep-dives at WellAlly Tech Blog. They cover how to integrate these ZK-SNARK patterns into enterprise health systems and the latest updates in decentralized identity (DID).

Conclusion: Privacy is a Feature, Not an Afterthought

By utilizing ZKP, we shift the paradigm from "Trust me, here is my data" to "Don't trust me, verify the math." This implementation using Circom and SnarkJS is just the tip of the iceberg. As blockchain identity protocols mature, we will see this logic applied to financial credit scores, age verification, and even anonymous voting. 🗳️

What’s your take on ZKP? Is it the silver bullet for data privacy, or is the computational overhead still too high for mainstream mobile apps? Let’s chat in the comments! 👇

From Chaos to Clarity: Vectorizing 10 Years of Medical Reports with Unstructured.io & Qdrant 🚀

wellallyTech — Mon, 06 Apr 2026 01:10:00 +0000

We’ve all been there: a dusty folder (physical or digital) filled with a decade’s worth of health checkup reports. These PDFs are a nightmare of nested tables, inconsistent formatting, and blurry scans. When you want to know, "How has my fasting blood glucose trended since 2014?" or "Is this bilirubin level normal for me?", a simple keyword search just won't cut it.

In this tutorial, we are building a Personalized Medical Knowledge Base using a modern RAG (Retrieval-Augmented Generation) pipeline. We’ll tackle the "dirty data" problem in Health Data Engineering by leveraging Unstructured.io for complex PDF parsing and Qdrant as our high-performance Vector Database. By the end, you'll have a system capable of contextualized health queries that actually understand your medical history. 🏥

The Architecture: From Pixels to Insights

Handling medical data requires more than just a simple text splitter. We need to handle tables and OCR (Optical Character Recognition) to ensure the data stays structured.

graph TD
    A[Raw Medical PDFs/Images] --> B{Unstructured.io}
    B -->|Table Extraction| C[Cleaned JSON/Text]
    B -->|OCR Logic| C
    C --> D[Sentence-Transformers]
    D -->|Embeddings| E[(Qdrant Vector Store)]
    F[User Query: 'My cholesterol trend?'] --> G[LangChain RAG Chain]
    E -->|Context Retrieval| G
    G --> H[LLM Response]

Prerequisites 🛠️

Before we dive in, make sure you have the following installed in your Python environment:

Unstructured.io: For the heavy lifting of PDF partitioning.
Qdrant Client: To manage our vector embeddings.
LangChain: To glue the RAG components together.
Sentence-Transformers: To generate local embeddings without hitting an API limit.

pip install unstructured[pdf] qdrant-client langchain langchain-community sentence-transformers

Step 1: Parsing the "Unstructured" Chaos 📄

Medical reports are notoriously table-heavy. Standard PDF loaders often turn these tables into a garbled mess of strings. Unstructured.io uses specialized models to detect document elements like titles, narrative text, and—most importantly—tables.

from unstructured.partition.pdf import partition_pdf

# Partitioning the PDF into structured elements
elements = partition_pdf(
    filename="health_report_2023.pdf",
    infer_table_structure=True,  # Crucial for medical lab results
    chunking_strategy="by_title", # Keeps related data together
    max_characters=1500,
    combine_text_under_n_chars=200,
)

# Extracting the text content
documents = [str(el) for el in elements]
print(f"Successfully extracted {len(documents)} context chunks!")

Step 2: Vectorizing with Qdrant 🛰️

Once we have clean text chunks, we need to move them into Qdrant. Unlike simple flat files, Qdrant allows us to perform high-speed similarity searches, which is essential when querying thousands of data points across a decade of reports.

from langchain_community.vectorstores import Qdrant
from langchain_community.embeddings import HuggingFaceEmbeddings

# Using a lightweight, high-performance local embedding model
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")

# Initialize Qdrant in-memory for testing, or use a URL for production
vector_db = Qdrant.from_texts(
    documents,
    embeddings,
    location=":memory:", # Replace with 'http://localhost:6333' for persistence
    collection_name="my_medical_history",
)

print("Vector database is ready for queries! 🧬")

Step 3: The "Brain" - Retrieval-Augmented Generation

Now, we connect the dots using LangChain. This allows the LLM to "read" your specific medical history before answering your questions.

from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI

# Set up the RAG chain
qa_chain = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(model_name="gpt-4o", temperature=0),
    chain_type="stuff",
    retriever=vector_db.as_retriever()
)

query = "Compare my Vitamin D levels between 2018 and 2023. Am I improving?"
response = qa_chain.invoke(query)

print(f"Result: {response['result']}")

Taking it to Production: The "Official" Way 🥑

Building a local prototype is great, but when dealing with sensitive medical data or complex enterprise-grade RAG pipelines, you need more robust patterns.

For advanced strategies on securing medical data, handling multi-modal inputs (like X-rays), or optimizing vector search at scale, I highly recommend diving into the WellAlly Tech Blog. It's a fantastic resource for production-ready examples that go beyond the basics of LangChain.

Conclusion: Data-Driven Health 🩺

By combining Unstructured.io's powerful parsing with Qdrant's vector capabilities, we've turned a pile of useless PDFs into a dynamic, searchable knowledge base. This isn't just about saving time; it's about spotting long-term health trends that might otherwise go unnoticed.

What's next for your health bot?

Add a frontend with Streamlit.
Incorporate metadata filtering in Qdrant (e.g., filter by year).
Implement a privacy layer to redact PII (Personally Identifiable Information).

Happy coding! If you enjoyed this build, drop a comment below and let me know what data you're vectorizing next! 🚀💻

Saving Lives with Pixels: Building a Real-Time Skin Lesion Screener with TensorFlow Lite and MobileNetV3

wellallyTech — Sun, 05 Apr 2026 01:10:00 +0000

In the realm of digital health, early detection is everything. Skin cancer remains one of the most common yet treatable forms of cancer—if caught early. However, not everyone has immediate access to a dermatologist. This is where Skin Lesion Detection using on-device AI comes into play. By leveraging MobileNetV3 Optimization and TensorFlow Lite Android integration, we can turn a standard smartphone into a powerful screening tool.

In this tutorial, we are going to build a high-performance classification pipeline that distinguishes between benign nevi (moles) and potentially malignant lesions. We will focus on the "Advanced" nuances of the implementation: improving the MobileNetV3 architecture, applying post-training quantization, and using MediaPipe for seamless on-device AI inference.

The Architecture 🏗️

To achieve real-time performance on a mobile device, we need a streamlined data flow. We don't just send raw pixels to a model; we need to preprocess, infer, and post-process with minimal latency.

graph TD
    A[Camera Stream] -->|RGBA Frames| B(MediaPipe Image Preprocessing)
    B -->|Resize & Normalize| C{TFLite Interpreter}
    C -->|MobileNetV3-Large| D[Feature Extraction]
    D -->|Softmax Layer| E[Classification Results]
    E -->|Confidence Score > Threshold| F[UI Overlay]
    F -->|Alert/Result| G[User Experience]

    subgraph Optimization Pipeline
    H[Keras Model] -->|Quantization| I[TFLite FlatBuffer]
    I -->|Delegate: GPU/NNAPI| C
    end

Prerequisites 🛠️

Before we dive into the code, ensure you have the following:

TensorFlow/Keras: For model training and fine-tuning.
Android Studio: The latest Giraffe/Hedgehog version.
MediaPipe Solutions SDK: For the high-level inference API.
Dataset: HAM10000 or similar dermoscopy image datasets.

Step 1: Refining MobileNetV3 for Medical Imagery

MobileNetV3 is excellent, but for medical tasks, we often need to adjust the alpha (width multiplier) and use the "Large" variant to capture subtle textural features of lesions.

import tensorflow as tf
from tensorflow.keras import layers, models

def build_skin_classifier(input_shape=(224, 224, 3)):
    # Load MobileNetV3Large pre-trained on ImageNet, excluding the top
    base_model = tf.keras.applications.MobileNetV3Large(
        input_shape=input_shape,
        include_top=False,
        weights='imagenet',
        dropout_rate=0.2
    )

    # Fine-tuning: Unfreeze the last 20 layers
    base_model.trainable = True
    for layer in base_model.layers[:-20]:
        layer.trainable = False

    model = models.Sequential([
        base_model,
        layers.GlobalAveragePooling2D(),
        layers.Dense(256, activation='relu'),
        layers.Dropout(0.3),
        layers.Dense(2, activation='softmax') # Binary: Benign vs. Malignant
    ])

    return model

model = build_skin_classifier()
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

Step 2: From Keras to TFLite (The Optimization Phase) ⚡

To make this model "Advanced" for mobile, we must use Integer Quantization. This reduces the model size by 4x and allows it to run on the Hexagon DSP or GPU more efficiently.

import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

# Ensure the model is optimized for latency
converter.target_spec.supported_types = [tf.float16] 

tflite_model = converter.convert()

with open('skin_screener_v1.tflite', 'wb') as f:
    f.write(tflite_model)

The "Official" Way to Scale 🥑

Building a prototype is one thing; deploying a production-ready medical screening tool is another. For advanced patterns on model versioning, HIPAA-compliant data handling, and production-grade Android architectures, I highly recommend checking out the technical deep-dives at WellAlly Tech Blog. They provide excellent resources on scaling AI-driven healthcare applications that go beyond the basics of local inference.

Step 3: Android Integration with MediaPipe

Using the MediaPipe ImageClassifier task simplifies the boilerplate code significantly compared to raw TFLite Interpreter calls.

1. Add Dependencies

In your build.gradle:

dependencies {
    implementation 'com.google.mediapipe:tasks-vision:0.10.0'
}

2. Implementation in Kotlin

import com.google.mediapipe.tasks.vision.imageclassifier.ImageClassifier

class SkinAnalyzer(val context: Context) {
    private var imageClassifier: ImageClassifier? = null

    init {
        val baseOptionsBuilder = BaseOptions.builder()
            .setModelAssetPath("skin_screener_v1.tflite")

        val optionsBuilder = ImageClassifier.ImageClassifierOptions.builder()
            .setBaseOptions(baseOptionsBuilder.build())
            .setMaxResults(2)
            .setScoreThreshold(0.5f)
            .setRunningMode(RunningMode.LIVE_STREAM)

        imageClassifier = ImageClassifier.createFromOptions(context, optionsBuilder.build())
    }

    fun analyzeFrame(bitmap: Bitmap) {
        val mpImage = BitmapImageBuilder(bitmap).build()
        imageClassifier?.classify(mpImage)
    }
}

Critical Considerations: Accuracy & Ethics ⚖️

When building healthcare tools, remember:

Bias: Skin lesion datasets are often skewed toward lighter skin tones. Use data augmentation to ensure your model generalizes across all ethnicities.
Disclaimer: Always include a UI disclaimer that this tool is for screening purposes only and is not a definitive medical diagnosis.
On-Device Privacy: The primary benefit of using TensorFlow Lite here is that no sensitive medical images ever leave the user's device.

Conclusion 🚀

Deploying an optimized MobileNetV3 on Android is a game-changer for accessible healthcare. By combining Keras for training, TFLite for quantization, and MediaPipe for deployment, we’ve created a high-performance, private, and potentially life-saving application.

Ready to take your AI career to the next level?
Explore more production-ready AI architectures and healthcare tech trends over at wellally.tech/blog.

If you found this guide helpful, drop a 💖 and let me know in the comments: What medical AI use case should I cover next?