Forem: Jean

🔬 Building Skin Lab Rx: Bringing Clinical AI to the Browser

Jean — Fri, 08 May 2026 00:48:29 +0000

Skincare is deeply personal, but the advice we get is often generic. For the Perfect Corp × Startup World Cup Hackathon, I wanted to build something that felt like a bridge between a luxury spa and a clinical dermatology office.

Enter Skin Lab Rx: An AI-powered diagnostic platform that turns a simple selfie into a 14-metric clinical report and a personalized skincare routine.

The Vision

The goal was simple: Zero friction, maximum accuracy. I wanted a user to be able to scan their skin and instantly see which products would actually move the needle for their specific concerns—whether that's acne, hydration, or fine lines.

The Tech Stack

Next.js 16 (App Router): For a lightning-fast, SEO-optimized SPA foundation.
Perfect Corp AI APIs: The powerhouse behind the skin analysis and virtual makeup transfer.
Vanilla CSS: I skipped the frameworks to build a custom, ultra-premium glassmorphism dark theme from scratch.
FaceDetector API: Leveraging native browser features for real-time user alignment.

Solving the "Real World" Selfie Problem 🤳

One of the biggest challenges in clinical AI is data quality. The Perfect Corp APIs are incredibly precise, but they require the user's face to occupy at least 60% of the frame.

Most people don't take "clinical" selfies. They take casual ones.

To solve this, I built a Smart Pre-Processing Engine in the browser. Using the experimental FaceDetector API, the app detects the user's face in real-time and aggressively auto-crops the image to meet the API's strict bounds before the upload even happens.

If the face is too small? The app zooms in. If the angle is off? We guide the user with a draggable alignment oval.

Production-Grade Resilience

Hackathons are famous for "it works on my machine" demos. To ensure Skin Lab Rx was bulletproof for the judges, I implemented a Resilient Backend Fallback. If the clinical API rejects a photo due to bad lighting or an obstruction (like glasses), the backend catches the error and generates a realistic synthetic diagnostic profile.

This ensures the demo never crashes and the user flow is never interrupted.

What's Next?

Skin Lab Rx is just the beginning. I'm looking into integrating e-commerce checkout and using the AI Makeup Transfer API to show "before and after" simulations of long-term skincare results.

Check out the live demo here: https://skin-lab-rx-6o7v4cleza-uc.a.run.app
Check out the video demo here: https://youtube.com/shorts/ostMOF_5yJc?si=PfH9TpMMY670MmvN
Check out the Github Repository here: https://github.com/jmoncayo-pursuit/Skin-Lab-Rx

Built with ❤️ for the Perfect Corp × Startup World Cup Hackathon.

https://devpost.com/software/skin-lab-rx

Chasing 16MB: My Parameter Golf Journey and What I Learned the Hard Way

Jean — Fri, 08 May 2026 00:40:21 +0000

I saw what big companies and research labs were doing at massive scale and tried to adapt those ideas to extreme compression in tiny models. Here’s what happened.

When OpenAI launched the Parameter Golf challenge, the rules were brutal: train a small language model that must fit inside a 16 megabyte compressed file and finish training in just 10 minutes on powerful hardware.

Most participants focused on proven techniques that were already working on the leaderboard. I took a different approach. I read papers and articles about what large companies and research labs were doing at massive scale and tried to adapt those concepts to the extreme constraints of this challenge.

The Experiments I Tried

Aggressive Int4 Quantization

Inspired by frontier quantization research from big labs showing that very low-bit weights could work in larger models, I pushed hard on Int4. I believed that if I could make aggressive 4-bit quantization stable in a tiny model, it would give me a massive space advantage. I spent weeks building custom mixed-precision code (Int6 for attention, Int4 for MLP layers), dynamic scaling, special training ramps, and heavy pruning. It was a bold, theoretically viable direction, but in practice the precision loss was too damaging for such a small model trained on very few steps.

Gimlet-Hetero (Layer-wise Heterogeneous Design)

This came directly from the Gimlet Labs paper “Efficient and Scalable Agentic AI with Heterogeneous Systems” (arXiv:2507.19635v1). The paper discusses how mixing different hardware tiers can optimize cost and performance for AI agents. I adapted that systems-level idea of heterogeneous resource allocation to transformer layers: giving wider MLP blocks and different precision levels to middle layers versus early and late layers. The idea was to allocate capacity where it mattered most.

TurboQuant

This was inspired by Google Research’s TurboQuant work on extreme compression, particularly for KV cache and vector search. I tried to adapt similar aggressive compression principles to weight quantization during training, hoping to push even more compression while maintaining stability.

Bayesian Backoff + TT Adapters

These came from research on dynamic correction mechanisms and low-rank decompositions (Tensor-Train). The goal was to add “smart recovery” during or after training to fix quality lost during quantization.

Some of these ideas were quite wild. A few came from unusual inspirations and might still be viable if explored further with more experience and compute. Int4 ultimately became my strongest contender, but none of them delivered the breakthrough I was hoping for.

The Evolutionary Agent

At one point I got tired of manual tweaking and built an autonomous evolutionary agent. The system could mark sections of code, generate mutations, run fast tests on Colab, rank them by real performance, and iterate.

It was technically interesting and worked mechanically, but after several generations I realized I was mostly automating the exploration of a weak search space. The gains were too small to justify the time I was spending on it, especially with very limited Colab quota. I shelved the agent. That was an important lesson: just because something can be automated does not mean it is the best use of limited time and compute.

What I Learned

My biggest mistake was choosing hard, experimental paths instead of first deeply understanding and building upon what was already working well on the leaderboard. As an amateur, I thought innovation meant doing something completely different. I now understand that you earn the right to innovate by first mastering proven approaches and then improving upon them.

I got close. My best runs projected to around 1.21 to 1.25 BPB on full hardware. That would have been a respectable non-record submission, but I never quite broke into true leaderboard territory. I also did not receive RunPod credits until the very end, which limited how much I could validate on real hardware.

Final Thoughts

Parameter Golf was a humbling but valuable experience. I explored a lot, built some interesting systems along the way, and gained a much clearer sense of where to focus effort when resources are limited.

The repository is public if you want to see the full journey:

https://github.com/jmoncayo-pursuit/parameter-golf-uniform-int4

I am still experimenting and still learning. Next time, I will be wiser about balancing bold exploration with proven foundations.

Breaking the Tether: How I Built a Neural Bridge for Antigravity with Gemini Multimodal Live

Jean — Sun, 15 Mar 2026 18:46:33 +0000

Disclaimer: I created this piece of content specifically for the purposes of entering the Gemini Multimodal Live API Developer Challenge. #GeminiLiveAgentChallenge

The Problem: The Tethered Developer

Nexus Comm-Link is the result, a real-time, bidirectional bridge between the Antigravity IDE and any mobile device, powered by the Gemini Multimodal Live API.

How it Works: The Neural Bridge

At its core, Nexus Comm-Link isn't just a remote desktop; it’s a context-aware partner. I built a tiered architecture to ensure that the mobile device doesn't just see pixels, but understands the intent of the workspace.

1. The Multimodal Engine (Gemini 2.0)

Using the BidiGenerateContent endpoint, the system maintains a high-speed vision and audio stream. I configured it to ingest 1 FPS vision snapshots while processing bidirectional PCM audio. This allows you to walk away from your desk, show your phone a bug on another screen, and have Gemini analyze it through your mobile camera while knowing exactly what is happening in your IDE.

2. Context Coupling via CDP

The secret sauce is the Chrome DevTools Protocol (CDP). Instead of just sending a video feed, the bridge traverses the IDE's execution context. It extracts "Thought Blocks"—hidden internal reasoning states where the IDE assistant documents its plan. By feeding these directly into Gemini's grounding context via CDP, the voice on your phone stays perfectly synced with the machine on your desk.

Example of a "Thought Block" extracted via CDP:

{
  "type": "thought_block",
  "status": "active",
  "content": "Analyzing user request for refactor... identifying target function 'calculateTotal' in utils.js..."
}

3. The Action Relay (Tool Calling)

One of the most satisfying parts of building this was implementing Action Relay. By defining custom tools in the Gemini SDK, I enabled "Voice-to-Action." You can say, "Apply that fix" or "Trigger an undo" while you're in the other room, and the bridge translates that voice intent into a physical browser event in the IDE instance.

The Stack

Backend: Node.js and WebSockets on Google Cloud Run.
Cloud Infrastructure: Google Cloud Build and Vertex AI.
Terminal Hub: A custom Python tactical hub that manages automated linking for macOS, Windows, and Linux.

What I Learned

Building this project taught me that the future of dev tools isn't in better UIs, but in better mobility of context. When the AI has eyes (Vision) and ears (Audio) that are physically detached from the screen but logically attached to the code, the "workspace" becomes something you inhabit, not just something you look at.

Watch it in Action

Check out the full technical demo here: [https://youtu.be/6xicXxh3-kY]
See the Submission to the hackathon Here: [https://devpost.com/software/nexus-comm-link]
Github Repo here : [https://github.com/jmoncayo-pursuit/Nexus-Comm-Link]

I'd love to hear how you'd use a detached multimodal bridge in your own workflow!

Special thanks to the Google DeepMind team for providing such a low-latency multimodal playground!

Building Keep-It-Moving: My First VS Code Extension

Jean — Sat, 26 Jul 2025 04:25:21 +0000

VS Code extensions aren't supposed to run servers. I tried it anyway.

Check out the full repo at https://github.com/jmoncayo-pursuit/keep-it-moving.

Demo showing the complete KIM workflow: VS Code extension → QR code pairing → mobile prompting → Copilot integration

I built Keep-It-Moving (KIM) to solve a simple problem: sending GitHub Copilot prompts from my phone. What started as "wouldn't it be nice if..." became an exploration of what's possible when you embed a WebSocket server inside a VS Code extension.

The Development Journey

This was my first VS Code extension, built with intentional AI collaboration. The initial idea was straightforward - remote Copilot prompting. The implementation revealed layers I hadn't expected.

What I Actually Built:

Embedded WebSocket server running inside VS Code extension
Self-hosted Progressive Web App served directly from the extension
QR code pairing system with UUID authentication
Real-time prompt relay to GitHub Copilot chat
Dynamic port discovery with intelligent fallback

What I Discovered I Couldn't Build (Yet):

File context reading from VS Code workspace
A full GitHub Copilot alternative (shelved after realizing the scope)

Technical Innovation and Happy Accidents

The core breakthrough wasn't planned - it emerged from constraints. VS Code extensions typically can't run servers, but Node.js modules work fine in the extension context. So I embedded a full WebSocket server using the ws library directly in the extension.

// This shouldn't work, but it does
const WebSocket = require('ws');
const server = new WebSocket.Server({ port: availablePort });

The VS Code control panel showing server status, pairing code, and management controls

The PWA self-hosting was born from necessity. External hosting would break the local-first promise, so the extension serves its own web interface. QR codes point to http://localhost:8080 - your extension becomes your server.

The Multi-Device Surprise:
One of the most impressive features happened by accident. The architecture naturally supported multiple devices because WebSocket connections are stateless. What looked like intentional design was actually emergent behavior from good architectural decisions.

AI Collaboration Lessons

Working with AI on this project taught me to be vigilant about feature creep and exaggerated claims. Early iterations included grandiose descriptions of capabilities I hadn't actually built. I learned to:

Validate every AI-suggested feature against actual implementation
Remove marketing language that overstated capabilities
Focus documentation on what actually works, not what sounds impressive
Test claims before committing them to the repository

The AI wanted to describe KIM as "revolutionary" and "first-of-its-kind." I kept pulling it back to factual descriptions of what I actually built.

Joyful Design Constraints

The project emphasized joyful user experience throughout:

Emoji-driven feedback (🚀📱🎉)
Playful error messages like "Your coding session took a coffee break! ☕"
Seamless QR code pairing that "just works"
Delightful micro-interactions

These constraints made development more enjoyable and the codebase something to look forward to reading.

Architecture That Emerged

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Mobile PWA    │    │  VS Code Ext    │    │ GitHub Copilot  │
│                 │    │                  │    │                 │
│ ┌─────────────┐ │    │ ┌──────────────┐ │    │ ┌─────────────┐ │
│ │ Prompt Input│ │───▶│ │ WebSocket    │ │───▶│ │ Chat API    │ │
│ └─────────────┘ │    │ │ Server       │ │    │ └─────────────┘ │
│                 │    │ └──────────────┘ │    │                 │
│ ┌─────────────┐ │    │ ┌──────────────┐ │    │                 │
│ │ QR Scanner  │ │◀───│ │ PWA Server   │ │    │                 │
│ └─────────────┘ │    │ └──────────────┘ │    │                 │
└─────────────────┘    └──────────────────┘    └─────────────────┘

Three components, local network only, zero cloud dependencies.

Real Learning Outcomes

VS Code Extension APIs: First extension taught me the extension lifecycle and webview patterns
WebSocket Architecture: Learned to handle connection state, authentication, and message routing
Progressive Web Apps: Built responsive mobile interface with offline capabilities
Local-First Development: Solved networking without external dependencies

What's Next

KIM works, but it's not marketplace-ready. The next phase involves:

User testing with real developers
File context integration (reading current VS Code workspace)
Performance optimization for larger teams
Marketplace submission process

I submitted this to GitHub's "For the Love of Code" hackathon - wish me luck!

This was my first VS Code extension, built in collaboration with AI. It does something that shouldn't be possible and works reliably. Sometimes the best engineering happens when you ignore conventional wisdom and just try anyway.

Automating Persistent AI On the Fly

Jean — Sat, 28 Jun 2025 02:40:31 +0000

Most developers stop when their AI assistant says "I cant do that." I didn't.

Check out the full repo at

https://github.com/jmoncayo-pursuit/market-data-api.

For a recent take home assignment I built a financial market data microservice with FastAPI, Docker Compose, PostgreSQL, Redis, Kafka and pytest. The real engineering happened in agent mode with Cursor AI, where I taught the tool to recover from CI failures and document each fix.

Purpose-Driven AI Orchestration

This was not casual "vibe coding." It was intentional:

Drafted a PRD from the assignment requirements and referred to it in every session
Defined project rules before writing code
Fed Cursor AI official GitHub Actions docs like Accessing workflow context
Vigilantly tracked file creation to avoid duplicate filenames
Reviewed and validated every AI generated fix

Each prompt had a clear goal. Each response was reviewed. No autopilot.

Project Overview

Stack
- FastAPI with dependency injection
- PostgreSQL via SQLAlchemy ORM
- Redis caching and job status store
- Apache Kafka with confluent-kafka-python
- Docker Compose orchestrating all services
Testing
- 278+ comprehensive tests with full integration coverage
- Endpoint tests covering authentication flows
- Service layer tests for Kafka producer and consumer patterns
CI/CD
- GitHub Actions workflow with dynamic retry logic
- Solved rate limiter initialization failures in CI
- Handled API authentication mismatches 401 and 403 errors
- Added Redis connection timeout handling
- Mocked Kafka services to avoid external dependencies
Observability
- Prometheus metrics integration for request rates and error counts
- Grafana dashboard showcasing service health and throughput
- Structured logging formatted for ELK consumption
- Health check endpoints for service readiness
Docs & Deliverables
- Swagger UI and OpenAPI spec
- Complete Postman collection with environment configs
- Alembic migrations for schema management
- GitHub Actions workflows with health check steps

Architecture

Data Flow

Troubleshooting CI Failures

To give Cursor AI the context it needed, I added this snippet to my internal docs so it could extrapolate our custom monitoring flow:

# 1. List the most recent failed run
gh run list --status failure --limit 1 \
  --json databaseId,status,conclusion,createdAt,url > last_run.json \
  && cat last_run.json

# 2. View logs of the failed run
gh run view <RUN_ID> --log-failed > last_run.log \
  && tail -100 last_run.log

# 3. Compute sleep duration based on last job timing
last_duration=$(gh run view <RUN_ID> --json timing \
  | jq .timing.totalDuration)
sleep $(( last_duration / 2 ))

# 4. Rerun only the failed jobs
gh run rerun <RUN_ID> --failed

Cursor AI learned to watch logs, wait intelligently, retry failures, and avoid wasted prompts. It even adapted when the Redis connection timed out or the rate limiter threw event loop errors.

Timeline and Complexity

Built a production grade microservice with streaming data pipeline
Implemented full CI/CD with health checks and self healing retries
Created a 278+ test suite covering all service layers
Integrated observability, docs and migrations in under two weeks

Real Learning Outcomes

Learned to handle Redis connection failures with retry backoff
Mastered Kafka consumer group management and mocking techniques
Implemented async await patterns for high throughput
Built robust error handling for external API dependencies

Conclusion

It is time to rethink what an AI native builder can deliver. By defining clear goals, keeping a living PRD, feeding the right documentation, and guiding each AI step, you can ship production level code with AI as your teammate. Demand more of your tools and hold them to your standards, and you will build resilient systems that keep moving forward, no matter what.