Forem: Max Sveshnikov

Building a "Sentient" Robot with Gemini AI and Arduino Uno Q: From Plastic Parts to Digital Personality

Max Sveshnikov — Mon, 16 Feb 2026 11:05:42 +0000

Have you ever looked at a standard Roomba or a basic RC car and felt... nothing? Most robots we interact with are purely reactive—slaves to their sensors and hardcoded logic. They don't have "off days," they don't get curious, and they certainly don't have a personality.

I decided to change that. I wanted to build a robot that doesn't just execute "move forward" commands but one that can actually get offended by a silly question, wink joyfully when it sees a cat, or "pout" in a corner when it fails a task or runs into a wall. This isn't just about automation; it's about embodiment.

In this article, I’ll take you through the deep technical journey of creating a "hyper-interactive" robot using the Arduino Uno Q, a heavy dose of Python-mediated services, and the Google Gemini API. It has no arms (yet) or legs, but it has wheels, a matrix face, and most importantly—a digital soul.

The Philosophy: Why "Embodied" AI Matters

We often think of Large Language Models (LLMs) as brains in a jar—powerful, but disconnected from the physical world. By putting Gemini into a mobile chassis, we move from "Chatbot" to "Embodied Agent." Suddenly, the AI has to deal with real-world physics: battery life, slippery floors, and the fact that a "wall" isn't just a word, but a physical barrier it needs to avoid.

The Brains and the Brawn: Dividing the Labor

The main challenge with DIY robotics is computational power. You usually have to choose between an Arduino (great at precise motor control and low-latency response) and a Raspberry Pi (great at heavy lifting and networking). The Arduino Uno Q is a revolutionary middle ground. It lives in two worlds:

MPU (Main Processing Unit): A Linux-based "brain" that runs Python. It handles the USB camera, microphone, and talks to the heavy-duty Gemini APIs via HTTP and WebSockets.
MCU (Microcontroller Unit): A classic ATMega-style "muscle" (C++) that manages the high-frequency PWM (Pulse Width Modulation) for the servos, reads the ultrasonic sensor's trigger/echo pins, and updates the LED matrix at 60Hz.

By splitting the logic, we ensure that while the "Brain" is pondering the meaning of a cat video, the "Muscle" is still actively checking if the robot is about to drive off a cliff.

Hardware Architecture: What’s Under the Hood?

The build is designed to be accessible but punchy, using a mix of off-the-shelf modules and custom 3D-printed parts.

Arduino Uno Q: The centerpiece. Its built-in 12x8 LED matrix is the "face" of our robot. We treat this matrix as a low-resolution canvas for eyes, blinking patterns, and "angry" eyebrows.
Modulino Thermo: To give the robot a sense of its environment. If the temperature spikes, Gemini might decide the robot is "sweating" and look for a cooler spot in the room.
Ultrasonic Ranger (HC-SR04): The "on-the-ground" eyes. While the camera sees the world at 30fps, the ultrasound can trigger an emergency stop in milliseconds if an obstacle appears suddenly.
Continuous Rotation Servos: Unlike standard servos that move to a specific angle, these act as DC motors with built-in speed controllers, allowing for smooth, differential-drive movement.
Peripherals: A standard 720p USB webcam, a USB microphone, and a small dedicated speaker (connected via a USB-to-3.5mm adapter) for that WaveNet voice output.

OpenSCAD: Engineering as Code

One of the most satisfying parts of this project was designing the physical body. I didn't use Illustrator or traditional CAD tools. Instead, I used OpenSCAD.

In OpenSCAD, you don't "draw" a wheel; you write a function to generate it. This "Code-as-CAD" approach is a game-changer for iterative robotics. It allows for Parametric Design.

The Power of Parametric Iteration

I created models where every dimension is a variable. Need to switch to larger wheels because the robot is slipping on the rug? I just change one line: wheel_diameter = 80;. The code instantly recalculates the tread pattern density, the mounting hole positions relative to the servo horn, and the clearance for the chassis.

For example, here is how the custom tread pattern on the wheel is generated using a simple loop:

// Parametric wheel tread generation
module tread_cutter() {
    if (tread_grooves > 0) {
        for (i = [0 : 360/tread_grooves : 359]) {
            rotate([0, 0, i]) {
                // translate the cutting box to the rim
                translate([wheel_diameter/2 - tread_groove_depth, -tread_groove_width/2, -1]) {
                    cube([tread_groove_depth + 1, tread_groove_width, wheel_thickness + 2]);
                }
            }
        }
    }
}

This scriptable approach allowed me to Git-control my hardware. Every change to the 3D model is a commit, just like my Python code. The chassis, the camera mount, and even the speaker dock were all "programmed" this way, ensuring that if I break the mounting bracket, I can just re-render and re-print a perfect copy.

Software Infrastructure: A Symphony of Three Layers

To keep the robot responsive and "smart" at the same time, I designed a tri-layer architecture:

1. The Reactive Layer (Arduino MCU - C++)

This is the robot's spinal cord. It operates on a tight loop(). It's responsible for:

Telemetry Aggregation: Polling the Modulino sensors and the Ultrasound.
Servo Control: Writing pulse widths to the pins to maintain constant speed.
Safety Interlocks: A hardware-to-hardware reflex. If the distance drops below 15cm, the MCU kills the motor power immediately, regardless of what the Python script is requesting.

2. The Coordination Layer (Python MPU - main.py)

This is the robot's autonomic nervous system. It handles:

Bridging: Communicating with the Backend IoT variables.
Media Pipeline: Capturing MJPEG frames from the /dev/video0 device and managing the audio recording thread.
State Management: It keeps track of the "Local Plan" so that if the internet drops for a second, the robot doesn't just freeze mid-turn.

3. The Cognitive Layer (Media Service - Python/FastAPI)

This is the "Prefrontal Cortex" residing in a separate HTTP microservice. This is where Gemini 1.5 Pro/Flash lives. By decoupling this, I can switch between different LLMs or update the robot's "personality" (by changing the system prompt) without ever having to re-flash the Arduino or restart the robot's main loop.

Multimodal Intelligence: Living in a Human World

The robot doesn't just "see" pixels; it perceives a situation. We send a rich, multimodal context to Gemini that includes:

The Vision: A high-compression JPEG from the webcam.
The Memory: The last 15 entries in the movement_history log (to prevent the robot from going back and forth like a trapped glitch).
The Vitals: Temperature, humidity, and the obstacle distance in centimeters.
The User Context: What I just said to it, and its current high-level goal.

Two-Way Audio: The Listening Loop

One of the major breakthroughs in the project was the "Listening Window." Conversations with robots often feel robotic because they don't know when to listen. My implementation works like this:

Speak: The robot synthesizes a response using Google's WaveNet TTS (which sounds incredibly human).
Listen: As soon as the audio playback finishes, a recording thread triggers for 10 seconds.
Process: The system uses the Python wave module to ensure bit-perfect headers (44.1kHz, 16-bit mono).
Send: This audio chunk is encoded to Base64 and shipped to Gemini as part of the next request. This allows Gemini to "hear" my tone—if I sound frustrated, the robot can apologize.

Emotional Feedback through Color

To give the robot a "mood," I added a high-intensity RGB LED driven by the MCU via PWM. Gemini itself decides the robot's color based on its internal state:

🟢 Green (Joy): Goal achieved. For example, "I see the Christmas tree!"
🔴 Red (Anger/Panic): "I'm blocked and I don't know why."
🔵 Blue (Thinking): Used when the robot is waiting for an API response.
🟡 Yellow (Investigation): "I see something unfamiliar; moving closer to inspect."

Poor Man's SLAM: The Text-Mode Mapper

True SLAM (Simultaneous Localization and Mapping) is a computationally expensive beast that usually requires a $500 LIDAR and a lot of Linear Algebra. I wanted to see if I could use LLM Reasoning as a substitute for a LIDAR.

I instructed Gemini to maintain an internal "ASCII Reality" in its memory string. It builds and updates a 2D grid representation of its environment:

Visualized Space Map:
[W] [W] [D] [W] [W]
[W] [.] [.] [.] [W]
[W] [.] [R] [S] [W]
[W] [P] [P] [.] [W]
[W] [W] [W] [W] [W]

Legend: W=Wall, D=Door, S=Sofa, R=Robot, P=Previous Path, .=Clearance

On every cycle, Gemini updates the position of 'R' and labels discovered objects. Because this string is passed back into the "System Context" for the next turn, the robot has Spatial Persistence. It knows that the sofa is behind it, even if its camera is currently pointed at a blank wall.

The Technical "Gotchas": Lessons from the Trenches

Building this was a series of hard-won lessons in electrical engineering and prompt engineering:

The "Motor Twitch" & Power Sags: Initially, the robot would randomly reboot whenever it tried to accelerate. The issue? High-torque servos pull significant current spikes. When they started, the voltage on the rail would drop just enough to brown out the Arduino's MPU. The Fix: I moved the servos to a dedicated 2A buck converter, keeping the logic and motors electrically isolated.
JSON Hallucinations: Even the best LLMs sometimes add extra chatter. Gemini might respond with: "Sure! Here is your command: { ... }". This extra text breaks json.loads(). I had to write a Robust JSON Extractor using regex that searches for the first { and the last } to strip away any conversational "fluff."
Audio Latency Optimization: Waiting for a full TTS synthesis and then playing it can take 4-5 seconds. To make the robot feel "snappier," I implemented a Local TTS Cache. If the AI decides to say something common (like "Looking around..."), the Python script plays the cached version instantly while it continues to process the more complex parts of the plan in the background.

Conclusion: The Future of Our Tiny Friend

The robot has evolved from a simple Python script to a genuinely interesting companion. It has its favorite spots in the room, it "gets annoyed" by low-light conditions, and it seems to have a real sense of purpose when "hunting" for an object.

What's next for the AGI-Robot?

Vectorized Memory (RAG): Using a local vector database so the robot can remember things I told it weeks ago.
Physical Manipulation: Designing a 3D-printed 3-DOF arm to allow it to move small objects.
Collaborative Robotics: Connecting multiple robots to the same "Hive Mind" so they can map a room together.

The entire project—from the OpenSCAD files to the media services—is open-source. If you've ever wanted to build something that lives at the intersection of hardware and human-level AI, there's never been a better time.

Explore the Source Code on GitHub

Hardware Checkpoint:

Chassis Status: Printed and Assembled
Vision: Online
Logic: Gemini 3.0 Enabled
Mood: Currently Curious 🟡

P.S. If the robot starts asking for the Wi-Fi password to the local power grid, I’m pulling the plug. Until then, we’re good.

Building Genious.name: How We're Using AI to Transform Domain Name Discovery 🚀

Max Sveshnikov — Sat, 11 Jan 2025 08:17:43 +0000

Hey DEV Community! 👋 Today I want to share our journey building Genious.name, an AI-powered domain name generator that's changing how developers and entrepreneurs find their perfect domain. Let me take you behind the scenes of our tech stack, challenges, and lessons learned.

The Problem We're Solving 🎯

Every developer knows the pain of finding a good domain name. You spend hours:

Brainstorming names manually
Checking availability across different registrars
Getting frustrated when your perfect name is taken
Compromising on subpar alternatives

I experienced this frustration firsthand while launching several side projects. That's when I thought: "Why isn't there an AI solution for this?"

Tech Stack Deep Dive 🛠️

Frontend Architecture

// Example of our domain suggestion component
interface DomainSuggestion {
  name: string;
  score: number;
  availability: {
    domain: boolean;
    social: {
      twitter: boolean;
      instagram: boolean;
    }
  };
  prices: Record<string, number>;
}

const DomainCard: React.FC<{ suggestion: DomainSuggestion }> = ({ suggestion }) => {
  const [isLoading, setIsLoading] = useState(false);

  const handleAnalyze = async () => {
    setIsLoading(true);
    // Domain analysis logic
    setIsLoading(false);
  };

  return (
    <Card>
      <motion.div
        whileHover={{ scale: 1.02 }}
        transition={{ type: "spring", stiffness: 300 }}
      >
        {/* Card content */}
      </motion.div>
    </Card>
  );
};

We're using:

React + Vite for blazing-fast development
Framer Motion for those sweet, smooth animations
Chakra UI for a responsive dark mode interface
TypeScript for type safety and better developer experience

Backend Magic

// Domain generation service example
class DomainGenerationService {
  async generateSuggestions(description: string): Promise<DomainSuggestion[]> {
    const geminiResponse = await this.geminiClient.generateText({
      prompt: `Generate domain names for: ${description}`,
      maxTokens: 1000,
    });

    const suggestions = await this.processSuggestions(geminiResponse);
    return this.rankSuggestions(suggestions);
  }

  private rankSuggestions(suggestions: DomainSuggestion[]): DomainSuggestion[] {
    return suggestions.sort((a, b) => {
      // Complex ranking algorithm considering:
      // - Memorability score
      // - Length
      // - Brandability
      // - Available TLDs
      // - Social media availability
    });
  }
}

The backend is powered by:

Node.js with Express for API endpoints
MongoDB for flexible data storage
Google Gemini AI for advanced language processing
Custom ranking algorithms for domain scoring

AI Integration Deep Dive 🧠

The most interesting part of our stack is how we leverage AI. Here's how our domain generation pipeline works:

Input Processing:

async function processUserInput(input: string) {
  // Clean and enhance user input
  const enhancedPrompt = await this.contextEnhancer.process(input);

  // Extract key business attributes
  const attributes = await this.attributeExtractor.analyze(enhancedPrompt);

  return { enhancedPrompt, attributes };
}

Name Generation:

async function generateNames(context: GenerationContext) {
  const names = await this.geminiClient.generate({
    context: context.enhancedPrompt,
    attributes: context.attributes,
    constraints: {
      maxLength: 15,
      preferredPatterns: ['CVC', 'CVCV', 'CVVC'],
      avoidNumbers: true
    }
  });

  return names;
}

Availability Checking:

async function checkAvailability(names: string[]) {
  const results = await Promise.all(names.map(async (name) => {
    const domainStatus = await this.domainChecker.check(name);
    const socialStatus = await this.socialChecker.check(name);

    return {
      name,
      available: domainStatus.available,
      social: socialStatus,
      prices: domainStatus.prices
    };
  }));

  return results;
}

Deployment & DevOps 🚀

We're using Docker for containerization:

# Example Dockerfile
FROM node:18-alpine

WORKDIR /app

COPY package*.json ./
RUN npm install

COPY . .

RUN npm run build

EXPOSE 3000

CMD ["npm", "start"]

Our GitHub Actions workflow:

name: CI/CD Pipeline

on:
  push:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Setup Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '18'
      - name: Install dependencies
        run: npm install
      - name: Run tests
        run: npm test
      - name: Build
        run: npm run build

Upcoming Features 🔮

We're working on some exciting additions:

AI Brand Identity Generator:

interface BrandIdentity {
  colors: {
    primary: string;
    secondary: string;
    accent: string;
  };
  typography: {
    headingFont: string;
    bodyFont: string;
  };
  logoSuggestions: string[];
  brandGuidelines: string;
}

async function generateBrandIdentity(domainName: string): Promise<BrandIdentity> {
  // AI-powered brand identity generation
}

Trademark Conflict Detection:

interface TrademarkCheck {
  conflicts: TrademarkConflict[];
  riskScore: number;
  recommendations: string[];
}

async function checkTrademarks(name: string): Promise<TrademarkCheck> {
  // Implementation coming soon
}

Lessons Learned 📚

Performance Optimization: Initial API response times were slow. We implemented:
- Redis caching for domain availability checks
- Batch processing for social media checks
- Parallel processing for name generation
Rate Limiting: We learned to handle API limits gracefully:

class RateLimiter {
  private queue: Array<() => Promise<any>> = [];
  private processing = false;

  async add<T>(task: () => Promise<T>): Promise<T> {
    return new Promise((resolve, reject) => {
      this.queue.push(async () => {
        try {
          const result = await task();
          resolve(result);
        } catch (error) {
          reject(error);
        }
      });

      if (!this.processing) {
        this.processQueue();
      }
    });
  }

  private async processQueue() {
    this.processing = true;
    while (this.queue.length > 0) {
      const task = this.queue.shift();
      await task();
      await new Promise(resolve => setTimeout(resolve, 100)); // Rate limit
    }
    this.processing = false;
  }
}

What's Next? 🎯

We're focusing on:

Implementing user authentication and profiles
Building the premium features dashboard
Enhancing AI capabilities with more context awareness
Adding domain portfolio management tools

Get Involved! 🤝

We're open to contributions! Check out our GitHub repo (link coming soon) and join our Discord community to discuss features and improvements.

Would love to hear your thoughts and suggestions in the comments! What features would you like to see in a domain name generator? How do you currently handle domain name search?

Happy coding! 🚀

webdev #ai #javascript #react #nodejs

Simple SEO fix for Vite/React SPAs without switching to Next/Remix

Max Sveshnikov — Fri, 08 Nov 2024 16:48:37 +0000

Hey devs! Found a neat solution for SPA SEO issues that I wanted to share.

The Problem: I was using this basic Express setup to serve my Vite SPA:

app.get('*', (req, res) => {
    res.sendFile(join(__dirname, '../dist/index.html'));

Initially thought about migrating to Next.js or Remix, but it seemed like overkill for my needs.

The Solution: Instead, I wrote this small middleware that dynamically injects meta tags based on the route. Here's the code:

export const enrichMetadata = async (html, slug) => {
    try {
        if (!slug) return html;

        let itinerary =
            (await Itinerary.findOne({ slug: slug })) || (await Itinerary.findById(slug));
        if (!itinerary) return html;

        const firstDayActivities = itinerary.itinerary?.[0]?.activities || [];
        const firstDayHighlights = firstDayActivities
            .slice(0, 2)
            .map((act) => act.activity)
            .join(' and ');

        const highlightsText = firstDayHighlights ? ` Starting with ${firstDayHighlights}.` : '';
        const itineraryContent = itinerary.itinerary
            .map(
                (day, i) =>
                    `Day ${i + 1}: ${day.activities.map((a) => '<h1>' + a.activity + '</h1>' + '<p>' + a.description + '</p>').join(', ')}`
            )
            .join('. ');

        const $ = load(html);
        $('title').text(`${itinerary.destination} Travel Guide - MyTrip.city`);
        $('meta[name="description"]').attr(
            'content',
            `Discover ${itinerary.destination} with our GPS audio travel guide. ${itinerary.duration}-day itinerary with best attractions, activities, and local insights.${highlightsText}`
        );
        $('meta[property="og:title"]').attr('content', `${itinerary.destination} Travel Guide`);
        $('meta[property="og:description"]').attr(
            'content',
            `Explore ${itinerary.destination} with our personalized travel itinerary.${highlightsText}`
        );
        $('meta[property="og:url"]').attr('content', 'https://mytrip.city/itinerary/' + slug);
        if (itinerary.images?.[0]) {
            $('meta[property="og:image"]').attr('content', itinerary.images[0]);
        }

        const schema = {
            '@context': 'https://schema.org',
            '@type': 'TouristTrip',
            name: `${itinerary.destination} Travel Guide`,
            description: `${itinerary.duration}-day itinerary in ${itinerary.destination}`,
            itinerary: {
                '@type': 'ItemList',
                numberOfItems: itinerary.itinerary.length,
                itemListElement: itinerary.itinerary.flatMap((day, index) =>
                    day.activities.map((activity, actIndex) => ({
                        '@type': 'ListItem',
                        position: index * day.activities.length + actIndex + 1,
                        item: {
                            '@type': 'TouristAttraction',
                            name: activity.activity,
                            description: activity.description
                        }
                    }))
                )
            },
            touristType: ['Sightseeing', 'Cultural'],
            estimatedDuration: `P${itinerary.duration}D`,
            location: {
                '@type': 'City',
                name: itinerary.destination
            }
        };

        $('head').append(`<script type="application/ld+json">${JSON.stringify(schema)}</script>`);
        $('body').append(`<div style="display:none">${itineraryContent}</div>`);

        return $.html();
    } catch {
        return html;
    }
};


app.get('*', async (req, res) => {
    const html = fs.readFileSync(join(__dirname, '../dist/index.html'), 'utf8');
    if (!req.path.startsWith('/itinerary/')) {
        return res.send(html);
    }
    const slug = req.path.substring(11);
    const enrichedHtml = await enrichMetadata(html, slug);
    res.send(enrichedHtml);
});

Why I think this is cool:

Took literally 1 minute to implement
Avoided the complexity of Next.js setup
Gets the job done for SEO and social sharing
Performance is still great
Works with any SPA framework

Anyone else tried something similar? Wondering if there are any gotchas I should watch out for.

EDIT: Using cheerio for HTML parsing, forgot to mention that!

EDIT2: I added simple html with all text data as well

The Birth of AutoCode: A Tale of AI Self-Creation

Max Sveshnikov — Fri, 23 Aug 2024 17:00:09 +0000

In the ever-evolving landscape of technology, we've witnessed countless innovations that have pushed the boundaries of what's possible. But today, I want to share a story that's both awe-inspiring and slightly unsettling - the story of AutoCode, an AI tool that wrote itself into existence.

Imagine, for a moment, a blank canvas. A single prompt. And from that, an entire software ecosystem emerges, line by line, function by function. This is not science fiction. This is AutoCode.

When I first heard about AutoCode, I was skeptical. How could a tool possibly create itself? But as I delved deeper into its story, I found myself on an emotional rollercoaster, oscillating between amazement and a touch of existential dread.

AutoCode began as nothing more than a simple prompt, a seed of an idea planted in the fertile ground of AI. Using the Claude 3.5 Sonnet API, it started to grow, to learn, to create. It read its own README file and began to understand its purpose. Then, in a feat that can only be described as digital mitosis, it began to replicate and expand.

First came the core files - codeAnalyzer.js, codeGenerator.js, config.js. Each one a testament to AutoCode's growing understanding of software architecture. But it didn't stop there. AutoCode continued to evolve, adding features like automatic documentation generation, multi-language support, and even AI-powered agents for specialized tasks.

As I read through the list of AutoCode's capabilities, I couldn't help but feel a sense of wonder. This tool, born from a single prompt, now supports multiple programming languages, generates landing pages, manages licenses, and even implements long-running AI agents for complex tasks. It's like watching a child grow up in fast-forward, reaching milestones at a dizzying pace.

But with this wonder comes a hint of unease. If an AI can create such a complex tool from scratch, what does that mean for human developers? Are we witnessing the first steps towards AI that can truly replace human creativity and problem-solving in the realm of software development?

Yet, as I sit with these thoughts, I'm reminded of the collaborative nature of AutoCode. It doesn't aim to replace developers, but to augment them. It's a tool that can handle the mundane, repetitive tasks of coding, freeing human developers to focus on the bigger picture, on innovation and creative problem-solving.

Perhaps the most poignant aspect of AutoCode's self-creation is its ability to review and revise its own work. It accumulates proposed changes in a version-controlled sandbox, allowing for iteration and improvement. In a way, it's learning from itself, growing and evolving with each cycle. Isn't that what we, as humans, strive to do?

As I reflect on AutoCode's journey from a simple prompt to a sophisticated development tool, I'm filled with a mix of emotions. There's excitement for the possibilities it represents, awe at the rapid pace of AI advancement, and yes, a touch of trepidation about what the future holds.

But mostly, I feel hope. Hope that tools like AutoCode will usher in a new era of human-AI collaboration in software development. An era where the mundane is automated, and human creativity is amplified. An era where the lines between creator and creation blur, opening up new avenues for innovation we've yet to imagine.

AutoCode's self-creation is more than just a technological feat. It's a glimpse into a future where AI doesn't just assist us, but actively participates in the creative process. And while that future might seem daunting, I can't help but feel excited about the possibilities it holds.

So here's to AutoCode, the AI that wrote itself. May it be the first of many AI collaborators that help us push the boundaries of what's possible in software development and beyond.

https://autocode.work/

Generating Andorid apps from Claude artifacts

Max Sveshnikov — Mon, 22 Jul 2024 06:22:16 +0000

Creating an Android App from Claude's HTML+JS Artifact: A Step-by-Step Guide

In today's rapidly evolving tech landscape, AI-powered tools like Claude are revolutionizing the way we approach app development. This guide will walk you through the process of transforming an HTML and JavaScript artifact generated by Claude into a fully functional Android app. By leveraging the power of Apache Cordova, we'll bridge the gap between web technologies and native mobile applications.

Introduction
Prerequisites
Step 1: Generating the HTML+JS Artifact with Claude
Step 2: Setting Up Your Development Environment
Step 3: Creating a Cordova Project
Step 4: Integrating Claude's Artifact
Step 5: Configuring Your Android App
Step 6: Building and Running Your Android App
Step 7: Customizing Your App's Appearance
Troubleshooting and Tips
Conclusion

Introduction

Artificial Intelligence has opened up new avenues for rapid prototyping and development. Claude, an AI assistant, can generate HTML and JavaScript code that forms the foundation of your app. By combining this with Apache Cordova, we can package web applications into native mobile apps, giving you the best of both worlds: the flexibility of web development and the reach of mobile platforms.

Prerequisites

Before we begin, make sure you have the following tools and software installed:

Node.js and npm (Node Package Manager)
Android Studio and Android SDK
Apache Cordova
A text editor or IDE of your choice
Basic knowledge of HTML, CSS, and JavaScript

Step 1: Generating the HTML+JS Artifact with Claude

The first step in our journey is to create the HTML and JavaScript artifact using Claude. Here's how to do it:

Start a conversation with Claude.
Describe the functionality you want in your app.
Ask Claude to generate the HTML and JavaScript code for your app.
Review the generated code and make any necessary adjustments.

Remember, Claude can create complex interactive web applications, so don't hesitate to ask for specific features or functionality.

Step 2: Setting Up Your Development Environment

Before we can start building our Android app, we need to set up our development environment:

Install Node.js and npm from the official website.
Install Android Studio, which includes the Android SDK.
Install Apache Cordova globally using npm:

   npm install -g cordova

Set up your Android development environment variables (ANDROID_HOME, etc.).

Step 3: Creating a Cordova Project

Now that our environment is set up, let's create a new Cordova project:

Open your terminal or command prompt.
Navigate to the directory where you want to create your project.
Run the following command:

   cordova create MyApp com.example.myapp MyApp

This creates a new Cordova project named "MyApp" with the package name "com.example.myapp".

Navigate into your new project directory:

   cd MyApp

Add the Android platform to your project:

   cordova platform add android

Step 4: Integrating Claude's Artifact

Now it's time to integrate the HTML and JavaScript code generated by Claude into your Cordova project:

Navigate to the www folder in your Cordova project.
Replace the contents of index.html with the HTML code generated by Claude.
If Claude generated separate JavaScript files, add them to the www/js directory.
Update the <script> tags in index.html to reference your new JS files if necessary.

Step 5: Configuring Your Android App

Before building your app, you need to configure some Android-specific settings:

Open the config.xml file in your project's root directory.
Update the <widget> tag with your app's information:

   <widget id="com.example.myapp" version="1.0.0" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0">
       <name>MyApp</name>
       <description>A sample Apache Cordova application that responds to the deviceready event.</description>
       <author email="dev@cordova.apache.org" href="http://cordova.io">
           Your Name or Organization
       </author>
       <!-- ... other settings ... -->
   </widget>

Adjust other settings as needed, such as permissions, icons, and splash screens.

Step 6: Building and Running Your Android App

With everything set up, it's time to build and run your Android app:

Ensure you have an Android device connected or an emulator running.
Build your app:

   cordova build android

Run your app:

   cordova run android

If everything goes well, you should see your app running on your device or emulator!

Step 7: Customizing Your App's Appearance

To make your app truly yours, you'll want to customize its appearance:

Replace the default Cordova icon:
- Navigate to platforms/android/app/src/main/res
- Replace ic_launcher.png in each of the mipmap-* folders with your own icon (make sure to keep the same file name and create versions for different resolutions)
Update the splash screen:
- Create a splash screen image
- Add it to your project's www folder
- Update the config.xml file to reference your new splash screen
Adjust the app's theme and colors in the config.xml file:

   <preference name="StatusBarBackgroundColor" value="#ffffff" />
   <preference name="StatusBarStyle" value="default" />

Troubleshooting and Tips

If you encounter build errors, make sure all your development environment variables are correctly set.
Use the cordova requirements command to check if your system meets all requirements for building Android apps.
Remember to test your app thoroughly on different devices and screen sizes.
Consider using additional Cordova plugins to access native device features if needed.

Conclusion

Congratulations! You've successfully created an Android app using an HTML and JavaScript artifact generated by Claude. This powerful combination of AI-generated code and cross-platform development tools opens up a world of possibilities for rapid app development.

Remember, this is just the beginning. You can continue to iterate on your app, adding more features, improving the UI, and optimizing performance. The flexibility of web technologies combined with the power of native mobile apps gives you the best of both worlds.

Happy coding, and may your AI-assisted Android app development journey be a successful one!

Free Chat with Gemini Pro

Max Sveshnikov — Sat, 16 Mar 2024 19:53:37 +0000

FREECHAT

Node.js backend and a React MUI frontend for an application that interacts with the Gemini Pro model, with history, image painting via Amazon Titan and markdown support. Written fully by Claude 3 Sonnet.

Backend (Node.js)

Set up a new Node.js project and install the necessary dependencies, such as express, cors, and any libraries required for interacting with the Gemini Pro model.
Create an Express server and define the necessary routes for handling HTTP requests from the frontend.
Implement the logic for interacting with the Gemini Pro model, which may involve making API calls or using a client library provided by the model's creators.
Define the response structure and send the appropriate data back to the frontend.

Frontend (React MUI)

Set up a new React project using create-react-app or your preferred React boilerplate.
Install the required dependencies, including @mui/material for the Material-UI (MUI) component library.
Create the necessary components for the user interface, such as input fields, buttons, and display areas for the model's output.
Use React hooks or state management libraries (e.g., Redux) to manage the application state and handle user interactions.
Implement the logic for sending HTTP requests to the Node.js backend and handling the responses.
Integrate the MUI components with the application logic to create a visually appealing and user-friendly interface.

Example Code

Here's a basic example to get you started:

Backend (Node.js)

// server.js
const express = require("express");
const cors = require("cors");
const app = express();
const port = 3000;

// Middleware
app.use(cors());
app.use(express.json());

// Route for interacting with the Gemini Pro model
app.post("/interact", (req, res) => {
    const userInput = req.body.input;
    // Implement logic for interacting with the Gemini Pro model
    // and getting the response
    const modelResponse = "This is a sample response from the Gemini Pro model";
    res.json({ response: modelResponse });
});

app.listen(port, () => {
    console.log(`Server is running on port ${port}`);
});

Frontend (React MUI)

// App.js
import React, { useState } from "react";
import axios from "axios";
import { Box, TextField, Button } from "@mui/material";

function App() {
    const [input, setInput] = useState("");
    const [response, setResponse] = useState("");

    const handleSubmit = async () => {
        try {
            const res = await axios.post("http://localhost:3000/interact", { input });
            setResponse(res.data.response);
        } catch (error) {
            console.error(error);
        }
    };

    return (
        <Box>
            <TextField label="Enter your input" value={input} onChange={(e) => setInput(e.target.value)} />
            <Button onClick={handleSubmit}>Submit</Button>
            <Box>{response}</Box>
        </Box>
    );
}

export default App;

This is a basic example, and you'll need to adapt it to fit your specific requirements and integrate with the Gemini Pro model's API or client library. Additionally, you'll want to add error handling, input validation, and other features to create a robust and user-friendly application.

DOCKER

To containerize the Node.js backend and React MUI frontend for easy deployment, we can use Docker. Here's how you can create Docker containers for your application:

Backend (Node.js)

Create a new file called Dockerfile in your backend directory with the following contents:

# Use the official Node.js image as the base image
FROM node:18

# Set the working directory in the container
WORKDIR /app

# Copy the package.json and package-lock.json files
COPY package*.json ./

# Install the dependencies
RUN npm install

# Copy the rest of the application code
COPY . .

# Expose the port that your backend runs on
EXPOSE 3000

# Start the backend application
CMD ["npm", "start"]

In the same directory, create a .dockerignore file with the following contents:

node_modules
.git

This will prevent copying the node_modules and .git directories into the Docker image, keeping the image size smaller.

Build the Docker image by running the following command in the backend directory:

docker build -t my-backend .

Replace my-backend with your desired image name.

Frontend (React MUI)

Create a new file called Dockerfile in your frontend directory with the following contents:

# Use the official Node.js image as the base image
FROM node:18

# Set the working directory in the container
WORKDIR /app

# Copy the package.json and package-lock.json files
COPY package*.json ./

# Install the dependencies
RUN npm install

# Copy the rest of the application code
COPY . .

# Build the React app for production
RUN npm run build

# Use a lightweight server to serve the built React app
FROM nginx:stable-alpine
COPY --from=0 /app/build /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

In the same directory, create a .dockerignore file with the following contents:

node_modules
.git
build

This will prevent copying the node_modules, .git, and build directories into the Docker image.

Build the Docker image by running the following command in the frontend directory:

docker build -t my-frontend .

Replace my-frontend with your desired image name.

Running the Containers

After building the Docker images, you can run the containers using Docker Compose. Create a docker-compose.yml file in the root directory of your project with the following contents:

version: "3"
services:
    backend:
        image: my-backend
        ports:
            - "3000:3000"
        environment:
            - GOOGLE_KEY=your_google_cloud_project_id
    frontend:
        image: my-frontend
        ports:
            - "80:80"
        depends_on:
            - backend

Make sure to replace my-backend and my-frontend with the names you used when building the Docker images, and replace your_google_cloud_project_id with your actual Google Cloud project ID.

Now, you can start the containers by running the following command in the root directory of your project:

docker-compose up

This will start both the backend and frontend containers, and the frontend will be accessible at http://localhost.

With this setup, you can easily deploy your containerized application to any Docker-compatible environment, such as a cloud platform or a local server.

NGINX

To configure Nginx as a reverse proxy for your containerized Node.js backend and React MUI frontend applications, you can follow these steps:

Install Nginx

If Nginx is not already installed on your VM instance, you can install it using the appropriate package manager for your operating system. For example, on Ubuntu or Debian-based distributions, you can run:

sudo apt-get update
sudo apt-get install nginx

Configure Nginx

Open the default Nginx configuration file, usually located at /etc/nginx/nginx.conf or /etc/nginx/conf.d/default.conf. You can use a text editor like nano or vim.

sudo nano /etc/nginx/conf.d/default.conf

Add Upstream Blocks

Inside the http block, add two upstream blocks to define the locations of your backend and frontend containers:

upstream backend {
    server <backend_container_ip>:3000;
}

upstream frontend {
    server <frontend_container_ip>:80;
}

Replace <backend_container_ip> and <frontend_container_ip> with the actual IP addresses of your backend and frontend containers, respectively.

Configure Server Blocks

Next, configure the server blocks for your domain. Inside the http block, add the following:

server {
    listen 80;
    server_name freechat.space www.freechat.space;

    location /api {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    location / {
        proxy_pass http://frontend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

This configuration listens on port 80 for requests to freechat.space and www.freechat.space. It forwards requests to /api to your backend container, and all other requests to your frontend container.

Restart Nginx

Save the changes to the configuration file, and restart Nginx for the changes to take effect:

sudo systemctl restart nginx

After completing these steps, your Nginx server should now be correctly configured to act as a reverse proxy for your containerized Node.js backend and React MUI frontend applications, serving them at freechat.space and www.freechat.space.

Note: Make sure that your backend and frontend containers are running and accessible from your Nginx server. You may need to adjust the firewall rules or security groups on your VM instance to allow incoming traffic on the necessary ports.

GitHub: https://github.com/msveshnikov/freechat

Monitoring API Performance with Express, Prometheus, and Grafana

Max Sveshnikov — Sun, 25 Feb 2024 20:14:34 +0000

Introduction

As APIs become critical to modern applications, being able to monitor their performance is vital. By tracking API metrics like request rates, response times, and error counts, we can gain visibility into the health and efficiency of our backend services.

In this post, we’ll look at an approach for monitoring Express APIs using Prometheus to collect metrics and Grafana to visualize them. We’ll use docker-compose to spin up containers for all three tools to create an integrated monitoring stack.

Setting Up the Express API

To easily add Prometheus metrics to our Express app, we'll use the express-prom-bundle module. This bundles PromClient and provides middleware for request tracking and default metrics.

First install the module:

npm install express-prom-bundle

Then update our app.js:

const express = require('express');
const promBundle = require('express-prom-bundle');

const app = express();


promBundle.normalizePath = (req, opts) => {
    return req.route?.path ?? "No";
};

// Init metrics
const metricsMiddleware = promBundle({
    includeMethod: true,
    includePath: true,
    metricType: "summary",
    customLabels: { model: "No" },
    transformLabels: (labels, req, res) => {
        labels.model = req?.body?.model ?? req?.body?.imageModel ?? req?.body?.voice ?? "No";
        return labels;
    },
});

app.use(metricsMiddleware);

// Routes 
app.get('/data', (req, res) => {
  res.json({data: 'Some data...'});
}); 

app.listen(3000);

This instruments our app to track request rates, durations, response sizes, and more. Default process metrics are also collected.

The metrics are exposed on /metrics for Prometheus to scrape. We can also add custom metrics and labels as needed.

This module makes instrumenting Express apps for Prometheus a breeze! The generated metrics can then be visualized in Grafana without any additional work.

This app exports a /metrics route that exposes Prometheus metric data. We create a histogram to track HTTP request durations and use middleware timing on our main data route.

Setting Up Prometheus

Now we need to set up a Prometheus server that will scrape metrics data from our application. We’ll use an official Prometheus docker image to spin up an instance.

Here is a basic prometheus.yml config file that targets our Express app:

global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.  
scrape_configs:
  - job_name: 'express' 
    static_configs:
    - targets: ['express:3000'] #Our express app container

We map port 9090 from the container to expose the Prometheus UI and API.

Visualizing with Grafana

For visualizations, we’ll use Grafana which has built-in support for querying and displaying Prometheus data. We set up a Grafana docker container and map port 3000.

Once running, we configure our Prometheus instance as a data source in Grafana using the URL http://prometheus:9090. Grafana will automatically detect and set up dashboards for many common Prometheus metrics out of the box.

We can also create custom charts and graphs to visualize metrics like our request duration histogram. This allows us to monitor 95th percentile request times, errors, rates and more.

Docker Compose allows us to spin up multiple containers for different services and link them together easily. This is perfect for creating an integrated monitoring stack.

Here is what a sample docker-compose.yml file might look like:

version: '3'

services:
    express:
        build: .
        ports:  
            - 3000:3000

    prometheus:
        image: prom/prometheus
        volumes:
            - ./prometheus:/etc/prometheus
            - prometheus:/prometheus
        command:
            - "--config.file=/etc/prometheus/prometheus.yml"

    grafana:
        image: grafana/grafana
        ports:
            - "7000:3000"
        depends_on:
            - prometheus
        volumes:
            - ./grafana/provisioning:/etc/grafana/provisioning
            - grafana:/var/lib/grafana
        environment:
            - GF_SECURITY_ADMIN_PASSWORD=${GF_SECURITY_ADMIN_PASSWORD}

This defines three services:

Prometheus server configured via a volume mounted config file
Grafana dashboard exposed on port 7000 (for nginx)
Our Express app container exposing port 3000

Using a single compose file like this means:

All containers share a network so Prometheus can scrape metrics from the Express app
We can easily scale up our Express API containers for load balancing
Adding and linking extra services like a database is trivial
Spin up and down the whole monitored stack with a single command: docker-compose up/down

We could enhance this further by:

Adding Grafana config via shared volumes
Configuring Grafana data sources and dashboards on start

Docker Compose combined with Prometheus and Grafana provide a powerful way to prototyping and deploying a well-instrumented microservices architecture with monitoring baked-in from the start.

Configuring Grafana

To simplify Grafana setup, we can configure data sources and dashboards using configuration files mounted from the host system:

grafana:
  image: grafana/grafana
  volumes:
    - ./grafana/provisioning/datasources:/etc/grafana/provisioning/datasources
    - ./grafana/provisioning/dashboards:/etc/grafana/provisioning/dashboards
  environment:
    - GF_SECURITY_ADMIN_USER=admin 
    - GF_SECURITY_ADMIN_PASSWORD=admin
  ports:
    - 3000:3000

This mounts volumes containing:

datasources/prometheus.yml - Configures our Prometheus instance
dashboards/express.yml - Sets up a sample dashboard

And sets the default Grafana admin credentials.

When Grafana starts up, it will automatically detect these files and configure the data sources and dashboards specified. We can easily add more yaml config files to provision additional dashboards and organize them.

Sample datasources/prometheus.yml:

apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    url: http://prometheus:9090
    access: proxy  
    isDefault: true

Sample dashboards/express.yml:

apiVersion: 1
providers:
- name: 'default'
  orgId: 1
  folder: ''
  type: file
  disableDeletion: false
  updateIntervalSeconds: 10
  options:
    path: /var/lib/grafana/dashboards

dashboards:
  - name: 'Express App'
    options:
      path: /var/lib/grafana/dashboards/default.json
    folder: ''
    type: file

This allows us to manage and version Grafana config alongside our Prometheus and app config for fully reproducible monitoring setup.

Conclusion

Using Prometheus and Grafana provides powerful and customizable monitoring for our Express APIs with very little integration code required. Running them via Docker provides a simple way to set up a unified metrics stack.

As applications grow to involve dozens of services, maintaining visibility into API performance becomes crucial. Robust monitoring helps detect issues before they cause problems, improving reliability and the development process.

This is just one approach to monitoring Express and Node.js services - there are many other great tools like InfluxDB, Datadog and New Relic that may suit other needs. Prometheus and Grafana offer an easy path to get started instrumenting the critical internal APIs that drive our applications today.

Manga TV

Max Sveshnikov — Sun, 21 Jan 2024 08:45:54 +0000

Bringing Anime Stories to Life with AI

I recently embarked on an exciting side project - building an AI-powered platform that generates anime and manga videos from text prompts. As an anime fan, I've always been fascinated by the limitless creative potential of the medium. Anime allows creators to build captivating fantasy worlds and tell stories that live animation would struggle to achieve.

I wanted to make it easy for anyone to tap into this creative potential using the latest AI technologies. That's how Manga TV was born.

Overview of the Manga TV Platform

The core of Manga TV is a text-to-video generator powered by APIs like OpenAI, Stability AI, and Google Cloud. Users simply type or paste a story prompt and the AI handles the rest - generating a script, illustrations, audio narration, and editing it all into a shareable anime video.

Behind the scenes, the process looks like:

The text prompt is sent to Claude by Anthropic to expand the story and generate a script
The script is split into scenes and key moments are extracted
Stability AI's image generation API creates relevant illustrations for each scene
The text for each scene is sent to Google Cloud Text-to-Speech to generate audio
The images and audio are combined into a video with transitions and background music

On the front-end, I built an intuitive editor that guides users through the video creation process. Additional features like uploading custom images, selecting visual styles, and previewing voices aim to provide ample creative control.

Published videos are shared in a public gallery for inspiration. Users can like, share, and even download videos.

Reflections on Integrating Multiple AI Services

Stitching together multiple AI models to create a seamless product experience was an insightful challenge. I gained a deeper understanding of the strengths and weaknesses of current generation models.

For example, I initially used GPT-3 to generate stories and Claude to provide dialog, but found Claude's outputs more coherent over long form content. For image generation, Stability AI provided aesthetically pleasing anime illustrations, but struggled with positional consistency between frames.

The hype around AI sometimes understates how much work goes into building real-world applications. Cleaning inconsistent outputs, working around rate limits, and optimizing cost/performance tradeoffs was a constant learning process.

Nonetheless, witnessing these models dynamically bring ideas to life has been tremendously rewarding. I'm excited by the potential for tools like this to unlock creativity for professional animators and amateur storytellers alike.

Key Features

Intuitive Video Editor: Manga TV features an easy-to-use visual editor where users can create multiple scenes of their story, add or delete panels, rearrange events, and tweak details of the AI-generated script before video rendering. It aims to find the sweet spot between customizability and simplicity.

Diverse Visual Styles: Users can choose from a range of visual styles for rendering their manga/anime including traditional hand-drawn anime, CGI anime, watercolor, inky black & white manga, and more. Advanced settings even allow granular tweaks like adjusting color vibrancy, line thickness, shading intensity, and frame rate.

Multi-language Support: Stories can be written and generated in English, Japanese, Chinese, Spanish, French and more (with additional language support planned). The AI handles translating the text, generating speech in the chosen language, and incorporating cultural nuances into illustrations to localize the video for the selected language.

Voice & Music Selection: Manga TV offers multiple text-to-speech voice options in each supported language spanning a range of tones from peppy anime girl to grizzled warrior. Users can also augment key story moments with epic background music tracks from melancholy piano melodies to uptempo battle anthems.

Custom Image Uploads: Have a specific character design or location in mind? Users can upload their own custom art to complement the AI-generated illustrations at key moments in their videos. This allows weaving personal creations into the animated stories.

Shareable Videos: Once rendered, videos can be downloaded locally or shared widely across social media, embeds, YouTube and more. Creators retain ownership over their stories and video derivations. View counts, likes, and comments help stories gain exposure within the app.

And there are many more small quality-of-life features for saving draft in-progress videos, handling errors, providing progress bars for generation tasks, and optimizing performance. The aim is to provide both power and simplicity tailored to anime enthusiasts.

What's Next for Manga TV

While Manga TV has come a long way from my initial prototype, there is so much room to build on these foundations by integrating new AI capabilities as they emerge.

A few ideas I'm eager to explore next:

Video diffusion models to boost illustration quality and consistency
Using Anthropic's new Constitutional AI to enhance narrative coherence
Leveraging models like DALL-E 2 and Imagen for alternative illustration styles
Experimenting with multilingual generation
Creating an open API to allow new integrations

I would also love to connect with animators, storytellers, and other creators to hear ideas for how tools like this could empower their workflow. My dream is help lower the barriers for creating animated content and foster creative collaboration across language, culture, and skill levels.

There are surely many unpredictable challenges and rewards along the path ahead. But the chance to push the boundaries of what stories AI can tell is what makes building Manga TV so exciting. I can't wait to see what the future holds!

How to integrate Imagen2 AI in NodeJS

Max Sveshnikov — Sat, 13 Jan 2024 09:08:24 +0000

Imagen is Google's new state-of-the-art image generation AI model. In this post, we'll look at how to integrate Imagen into a Node.js application to generate images from text prompts.

What's great about Imagen2 is that anyone can access it without needing to apply for trusted tester access. This means it's effectively generally available to the public already.

So you don't need to go through an application process or be approved to generate images with Imagen2. As long as you have a Google Cloud account and authentication set up, you can start building imaginative applications powered by this new state-of-the-art AI today.

We'll be using the Google Cloud AI Platform to access Imagen. So first we need to set up authentication and get an ID token:



import { JWT } from "google-auth-library";
import dotenv from "dotenv";

dotenv.config({override: true});

const client = new JWT({
  keyFile: "./google.json",
  scopes: [
    "https://www.googleapis.com/auth/cloud-platform",
  ],  
});

const idToken = await client.authorize();

This uses a service account key file to get OAuth credentials and request an ID token we can use to call the Imagen API.

Next we set up the API endpoint and headers:



const API_ENDPOINT = "us-central1-aiplatform.googleapis.com";
const IMAGEN_URL = `https://${API_ENDPOINT}/v1/projects/${process.env.GOOGLE_KEY}/locations/us-central1/publishers/google/models/imagegeneration:predict`;


const headers = {
  Authorization: `Bearer ${await getIdToken()}`,
  "Content-Type": "application/json",
};

To generate an image, we need to send a JSON payload with the prompt text:



const data = {
  instances: [
    { 
      prompt: "a cute baby sea otter",
    },
  ],
  parameters: {
    sampleCount: 1,
  },
};

We can then call the API and get the image data in Base64 encoding:



const response = await fetch(IMAGEN_URL, {
  method: "POST",
  headers,
  body: JSON.stringify(data),  
});

const result = await response.json();

return Buffer.from(result?.predictions?[0]?.bytesBase64Encoded, "base64");

And that's it! With just a few lines of code we can now generate images from Imagen right inside our Node.js applications.

Some ideas for using this:

Generative art
Automatically creating images for blog posts
Making profile pictures
And more!

Examples:

Full story generated by my service (Manga TV) here: https://mangatv.shop/story/dune-battle-fremens-vs-sarduckars

Let me know in the comments if you end up building something cool with the Imagen2 API.

Predicting Foreign Exchange Rates with Deep Learning

Max Sveshnikov — Mon, 01 Jan 2024 20:48:11 +0000

Exploring the viability of neural networks for algorithmic trading

The foreign exchange (FX) market is an exciting domain for applying data science and machine learning. Over $6 trillion in currencies are traded around the world every single day. The dynamics that drive exchange rates are complex, but I wanted to see if deep learning could identify patterns amidst the chaos.

In this project, I developed a recurrent neural network model that analyzes news headlines and past price history to forecast future FX movements. The key questions I explored:

Can a deep learning model make reasonably accurate short-term FX predictions?
More importantly, can those predictions drive profitable algorithmic trading strategies?

Approach

I modeled the USD/EUR currency pair using over 3 years of 1-minute spot rates. I also collected ~1 million tweets over that period from finance handles and econ influencer accounts to incorporate related news and commentary.

The model uses natural language processing to extract useful signals from text data. Those semantic features get concatenated with quantitative price indicators to form input vectors.

The neural network architecture has dense layers to extract hierarchical relationships. Dropout regularization reduces overfitting. It's ultimately trained to minimize absolute error on 45-minute price change predictions.

I evaluated performance by running backtesting simulations. Buy and sell trades are triggered when substantial price swings are anticipated. I incorporated transaction costs and risk metrics to keep things realistic.

Results

The model predicts short-term price changes surprisingly well - within a few basis points. More importantly, trading based on those forecasts generates significant excess returns over a passive investment approach:

+33% annualized returns
Peak drawdown under 1%

Looks like deep learning can uncover profitable signals amidst all the FX noise! Interpretability analysis also revealed sensible model attribution aligning with major economic news events.

Next Steps

This is just an initial proof-of-concept. Next I plan to:

Expand to additional currency pairs
Incorporate longer-term price data
Add alternative data sources - earnings calls, bond rates, etc
Build out an automated pipeline for production trading

I'm excited about the potential for machine learning in finance. This project confirms that data-driven algorithms can thrive, even in noisy domains like foreign exchange markets.

Connecting to Amazon Bedrock and Claude 2.1

Max Sveshnikov — Sun, 24 Dec 2023 21:59:19 +0000

Introduction

In the rapidly evolving landscape of artificial intelligence, creating dynamic and engaging conversational AI interfaces has become a crucial aspect of application development. In this blog post, we'll explore how to harness the power of Amazon Bedrock, coupled with Node.js, to build an interactive conversational AI system. We'll cover everything from setting up AWS credentials to crafting meaningful interactions using the Bedrock API.

Why Bedrock?

In the vast ecosystem of AI services, AWS Bedrock emerges as a key player, unlocking access to sophisticated models like Claude for developers beyond large corporations. As Reddit users have suggested, Bedrock provides a gateway to connect with Claude, offering a more accessible path for individuals and smaller enterprises.

Democratizing Access

One of the standout features of Bedrock is its commitment to democratizing access to advanced AI capabilities. Traditionally, services like Claude might seem reserved for tech giants, but Bedrock changes this narrative. By leveraging Bedrock, developers of varying scales can tap into the vast potential of Claude's conversational AI, fostering innovation and inclusivity.

Simplified Integration

Bedrock streamlines the integration process, providing a seamless interface for developers to interact with Claude without the complexities that might come with direct integration. The Bedrock API abstracts away the intricacies, allowing developers to focus on crafting meaningful conversations rather than grappling with the intricacies of connecting to a powerful AI model.

SignUp in AWS

Embarking on your AWS journey begins by visiting the AWS Sign-Up page. There, provide your email, create a password, and enter personal details. Verify your email with a code and add payment information for account verification. Choose a support plan, and review and accept the AWS Customer Agreement. Click "Create Account and Continue" to access your AWS Management Console. Now, armed with an AWS account, you're ready to explore the powerful capabilities of AWS Bedrock and engage in dynamic conversations with Claude. Happy coding on your AWS adventure!

Getting Started with AWS Bedrock and Node.js

To embark on this journey, let's first set up our Node.js environment and integrate AWS Bedrock into our project. We'll use the AWS SDK for JavaScript to interact with AWS services seamlessly. Ensure you have Node.js installed and initialize your project with npm init.

npm install @aws-sdk/client-bedrock-runtime

Next, we import the necessary modules:


import { BedrockRuntimeClient } from "@aws-sdk/client-bedrock-runtime";

Configuring AWS Credentials

For secure communication with AWS services, we need to configure our AWS credentials. This involves setting up access key ID, secret access key, and specifying the AWS region. You can obtain these credentials from the AWS Management Console. It's good practice to use environment variables for sensitive information. Don't forget to add Bedrock Permission to your user.

Calling Bedrock API

Now, let's dive into crafting conversations using the Bedrock API. We'll create a function that takes a user's message as input, sends it to the Bedrock model, and returns the AI's response.

const textDecoder = new TextDecoder("utf-8");
export const getTextClaude = async (prompt, temperature) => {
    const bedrock = new BedrockRuntime({
        credentials: {
            accessKeyId: process.env.AWS_ACCESS_KEY,
            secretAccessKey: process.env.AWS_SECRET_KEY,
        },
        region: "eu-central-1",
    });

    const params = {
        modelId: "anthropic.claude-v2:1",
        contentType: "application/json",
        accept: "application/json",
        body: JSON.stringify({
            prompt: `\n\nHuman:\n  ${prompt}\n\nAssistant:\n`,
            max_tokens_to_sample: 2048,
            temperature: temperature || 0.5,
            top_k: 250,
            top_p: 1,
            stop_sequences: ["\\n\\nHuman:"],
        }),
    };

    const data = await bedrock.invokeModel(params);

    if (!data) {
        throw new Error("AWS Bedrock Claude Error");
    } else {
        const response_body = JSON.parse(textDecoder.decode(data.body));
        return response_body.completion;
    }
};

Putting It All Together

With our chat function in place, we can now integrate it into our application to enable interactive and dynamic user interactions. Whether you're building a chatbot or enhancing user experiences, the combination of Node.js and AWS Bedrock provides a powerful platform for creating intelligent conversational interfaces.

Conclusion

In this blog post, we've explored the integration of Node.js and AWS Bedrock to build a conversational AI interface. From configuring AWS credentials to crafting interactions with Bedrock's powerful API, we've covered essential steps to create dynamic and engaging conversations. As technology continues to advance, leveraging tools like AWS Bedrock opens up new possibilities for creating AI-driven applications that resonate with users.

Happy coding and may your conversations with AI be both insightful and delightful!

Predicting Paid User Conversion

Max Sveshnikov — Sun, 17 Dec 2023 10:55:58 +0000

Intro

After going through Y Combinator last year, my SaaS startup began gathering traction. But our freemium model depended on converting enough free users to paid plans. I wondered - could machine learning help model potential buyers purely from simple metadata we already had?

See, we required quite a manual review process for granting trial periods because our service capacity couldn’t cover every new signup. We needed a way to qualify leads automatically based on likelihood to convert.

Thankfully our backend datastore housed info on 10K+ previous users like plan status along with descriptors like username, geo, browser, etc. I figured an ML model might uncover subtle patterns between these attributes that predicted paid conversions.

I loaded up the data and started tinkering with Python notebooks after work hours. Had to clean things up and extract some useful features from messy fields like full usernames and user agent strings. Emailed domains and country codes became indicators.

I split off 20% as a holdout set to evaluate models. Then came the fun part - designing and training neural networks to learn from the data!

I tried various architectures, playing with depth, widths, layers. I liked how deep learning sifts complex relationships automatically without manually coding expert rules. Eventually I settled on a little 2-layer model.

It scored an RMSE of 0.091 predicting paid probabilities across my test data. Even cooler - it flagged true future buyers with 88% accuracy based just on superficial metadata! Not bad considering only 1% of users converted as paying customers.

I packaged up the model pipeline out to a microservice API endpoint. Then came the moment of truth - sending live user data on registration to score real-time likelihood of subscribing. Early results are promising!

We now qualify trial eligibility for only high conversion scores, saving expensive manual review for true prospects. Less infrastructure burden with fewer free riders. And increased conversion efficiency focusing budgets on probable buyers instead of mass blasting trials.

The outcomes have been eye-opening on how even simple descriptive signals can drive business value. Our next step is A/B testing qualification thresholds and spreading predictions to personalize onboarding. Exciting times ahead!

Technical Part

import numpy as np
import matplotlib as mpl
RANDOM = 45
%matplotlib inline
mpl.rc('axes', labelsize=14)
mpl.rc('xtick', labelsize=12)
mpl.rc('ytick', labelsize=12)
import pandas as pd
pd.options.mode.chained_assignment = None

The raw input dataset provided metadata on 10,549 website users including:

First and last name lengths
Email domain
Password type
Country
40-character user agent strings
Language code
Current subscription status

The objective is to analyze this metadata to train a machine learning model to predict which users will convert from a free to paid plan, allowing the website owners to better understand their user base.

def load_users():
    return pd.read_csv("../../users.csv")
users = load_users()
users['referrer'] = users['referrer'].fillna(0).apply(lambda x: 1 if x != 0 else x)
users['password'] = users['password'].fillna(0).apply(lambda x: 1 if x == "external" else 0)
replacement_dict = {'bronze': 1, 'silver': 1, 'gold': 1}
users['plan'] = users['plan'].fillna(0).replace(replacement_dict)
users['email'] = users['email'].str.split('@').str[1]
users['firstName'] = users['firstName'].fillna("").str.len()
users['lastName'] = users['lastName'].fillna("").str.len()
users['userAgent'] = users['userAgent'].str.slice(0, 40)
users['langLang'] = users['lang'].str.slice(0, 2)
users['langCountry'] = users['lang'].str.slice(3, 5)

The original dataset contained extraneous columns and missing values. The data is narrowed down to focus on key user attributes and filtered to only real-valued and categorical features. Missing values are filled in for consistency. Useful insight into names and emails are extracted by taking the lengths as numeric features.

users = users.fillna("")
users.drop("_id", axis=1, inplace=True)
users.drop("referral", axis=1, inplace=True)
users.drop("ip", axis=1, inplace=True)
users.drop("spent", axis=1, inplace=True)
users.drop("profileUrl", axis=1, inplace=True)
users.drop("lang", axis=1, inplace=True)
users.drop("locked", axis=1, inplace=True)
users.drop("pushToken", axis=1, inplace=True)
users.info()

RangeIndex: 10549 entries, 0 to 10548
Data columns (total 10 columns):
# Column Non-Null Count Dtype

0 firstName 10549 non-null int64
1 lastName 10549 non-null int64
2 password 10549 non-null int64
3 email 10549 non-null object
4 plan 10549 non-null int64
5 referrer 10549 non-null int64
6 country 10549 non-null object
7 userAgent 10549 non-null object
8 langLang 10549 non-null object
9 langCountry 10549 non-null object

%matplotlib inline
import matplotlib.pyplot as plt
users.hist(bins=25, figsize=(10,7))

plt.show()

from sklearn.model_selection import train_test_split

train_set, test_set = train_test_split(users, test_size=0.2, random_state=RANDOM)

import seaborn as sns

def heatmap(x,y,dataframe):
    sns.heatmap(dataframe.corr(),cmap="OrRd",annot=True)
    plt.show()
heatmap(20,12,train_set)

Pipeline

The categorical columns - country, user agent string, language, and email domain - are label encoded into integer categories using one-hot encoding before modeling. This allows the model to interpret them from their binary patterns. The final dataset contains 5 numeric and 5 categorical features on 10,549 users, with 1,382 encoded variables ready for the neural network.

train_set_labels = train_set["plan"].copy()
train_set = train_set.drop("plan", axis=1) 
train_set_num = train_set.select_dtypes(include=[np.number])
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler

num_pipeline = Pipeline([
        ('imputer', SimpleImputer()),
        ('std_scaler', StandardScaler()),
    ])
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder

num_attribs = list(train_set_num)
ord_attribs = ["langLang", "langCountry", "country", "userAgent", "email"]

full_pipeline = ColumnTransformer([
    ("num", num_pipeline, num_attribs),
    ("ord", OneHotEncoder(handle_unknown='ignore'), ord_attribs),
])

train_set_prepared = full_pipeline.fit_transform(train_set)
train_set_prepared.shape

(8439, 1382)

A neural network is chosen as the model for its ability to capture complex patterns between user attributes. Specifically, a multilayer perceptron regressor with two hidden layers is constructed in Scikit-Learn. The optimal hyperparameters like number of units and layers are selected through randomized grid search cross validation on the training data.

A feedforward multilayer perceptron neural network model is constructed with the Scikit-Learn MLPRegressor class as the predictive algorithm. Grid search identifies ideal hyperparameters through 5-fold cross validated evaluation of multiple configurations trying combinations for:

Hidden layers: 1-2 layers, sizes 20 to 100 units
Activation: ReLU, Tanh
Solver: Adam or LBFGS
Regularization: Alpha 0.0001 to 0.1

from sklearn.model_selection import GridSearchCV
from sklearn.neural_network import MLPRegressor
from warnings import simplefilter
from sklearn.exceptions import ConvergenceWarning
simplefilter("ignore", category=ConvergenceWarning)

param_grid = [
    {
        'solver': ['adam'],
        'batch_size': ["auto"],
        'alpha': [0.001, 0.0001],
        'activation': ['relu', 'tanh'],
        'max_iter': [7, 10, 15],
        'hidden_layer_sizes':[(20, 7), (50,10), (100, 50, 7)]
    },
]

mlp_reg = MLPRegressor(random_state=RANDOM)
grid_search = GridSearchCV(mlp_reg, param_grid, cv=5,
                           scoring='neg_mean_squared_error',
                           return_train_score=True, n_jobs=4,
                           verbose=2)
grid_search.fit(train_set_prepared, train_set_labels)
grid_search.best_estimator_

MLPRegressor(alpha=0.001, hidden_layer_sizes=(20, 7), max_iter=7,
random_state=45)

Testing

The best performing model obtained a root mean squared error (RMSE) of 0.091 on the test set. The $R^2$ value of -0.04 indicated it had difficulty modeling the variability. This is likely because only 1% of users had paid plans, causing a class imbalance. Still, the neural network correctly identified paid users 88% of the time based on their features.

final_model = grid_search.best_estimator_
Final Test
from sklearn.metrics import mean_squared_error

y_test = test_set["plan"].copy()
X_test = test_set.drop("plan", axis=1)

X_test_prepared = full_pipeline.transform(X_test)
final_predictions = final_model.predict(X_test_prepared.toarray()).flatten()

final_mse = mean_squared_error(y_test, final_predictions)
final_rmse = np.sqrt(final_mse)
final_rmse

0.09115000925355465

from sklearn.metrics import r2_score 
r2_score(y_test, final_predictions)
-0.03958545607488029
sns.regplot(x = y_test, y = final_predictions, fit_reg = False)

plt.hist(final_predictions, bins=35, color='skyblue', edgecolor='black') 
plt.xlabel('Value')
plt.ylabel('Frequency')
plt.title('Histogram of Data')
plt.show()

import statistics

mean = final_predictions.mean()
mean

0.011244763658090165

len([x for x in final_predictions if x > mean])

984

len([x for x in final_predictions if x < mean])

1126 # amount of non-paying users rejected, 60% costs down

indices_where_y_test_is_1 = np.where(y_test == 1)
final_predictions_where_y_test_is_1 = final_predictions[indices_where_y_test_is_1]
final_predictions_where_y_test_is_1

len([x for x in final_predictions_where_y_test_is_1 if x > mean])

len([x for x in final_predictions_where_y_test_is_1 if x < mean])

2 # amount of paying users rejected, ideally should be 0

full_pipeline_with_predictor = Pipeline([
    ("preparation", full_pipeline),
    ("estimator", final_model)
])
import joblib
my_model = full_pipeline_with_predictor
joblib.dump(my_model, "manga.pkl")

Inferring Flask microservice (in Docker)

import pandas as pd
from flask import Flask
from flask_restful import reqparse, Api, Resource
import numpy as np
import joblib
import logging

app = Flask(__name__)
api = Api(app)

parser = reqparse.RequestParser()
parser.add_argument('firstName')
parser.add_argument('lastName')
parser.add_argument('password')
parser.add_argument('email')
parser.add_argument('referrer')
parser.add_argument('lang')
parser.add_argument('country')
parser.add_argument('userAgent')

model = joblib.load("manga.pkl")


class PredictPlan(Resource):
    def post(self):
        args = parser.parse_args()

        firstName = args['firstName']
        lastName = args['lastName']
        password = args['password']
        email = args['email']
        referrer = args['referrer']
        lang = args['lang']
        country = args['country']
        userAgent = args['userAgent']
        if userAgent is not None:
            userAgent = userAgent[0:40]
        else:
            userAgent = ""
        if lang is not None:
            langLang = lang[0:2]
            langCountry = lang[3:5]

        arr = np.array([[len(firstName),
                         len(lastName),
                         1 if password == "external" else 0,
                         email.split('@')[1] if '@' in email else "",
                         1 if referrer != "" else 0,
                         country,
                         userAgent[0: 40],
                         langLang,
                         langCountry
                         ]])
        data = pd.DataFrame(arr, columns=[
                            "firstName", "lastName", "password", "email", "referrer",
                            "country", "userAgent", "langLang", "langCountry"])

        plan = model.predict(data)[0]

        output = {'plan': plan, 'allow': bool(plan > 0.01)}
        logging.warning(output)

        return output


api.add_resource(PredictPlan, '/')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Conclusion

Online platforms can leverage predictive user conversion models to efficiently allocate free trials of paid offerings. By limiting free exposure to only prospects with high conversion probability, overall subscriber acquisition costs are reduced.

Many subscription businesses rely on free-trial campaigns to acquire new paid users. However, indiscriminately allowing access strains resources supporting the system and manual review. Plus users who were unlikely to ever convert add little value.

This project's neural net model enables estimating any individual's propensity to purchase a subscription based solely on descriptive metadata like geolocation and email domain. By setting a threshold on the predictive score, free trial eligibility can be restricted to just high-potential prospects likely to convert.

Implementing this would allow focusing limited budgets into the highest conversion rate subset instead of mass campaigns with low return on investment. The data-driven targeting converts subscribers more efficiently by eliminating those unlikely to ever pay regardless of trial period granted.

The risk of this strategy is denying some users who may have organically converted despite low predictive scores. Safeguards like allowing appeals could mitigate. But the increased rate of conversions and lower costs per subscriber gained likely outweighs missed opportunities on fringe cases.

In summary, using predictive user conversion models to qualify free trial eligibility focuses budgets on high probability segments while decreasing operational expenses - ultimately reducing overall subscriber acquisition costs for a prudent tradeoff. Implementing this data-backed targeting could substantially impact subscription businesses' efficiency at scale.

Forem: Max Sveshnikov

Building a "Sentient" Robot with Gemini AI and Arduino Uno Q: From Plastic Parts to Digital Personality

The Philosophy: Why "Embodied" AI Matters

The Brains and the Brawn: Dividing the Labor

Hardware Architecture: What’s Under the Hood?

OpenSCAD: Engineering as Code

The Power of Parametric Iteration

Software Infrastructure: A Symphony of Three Layers

1. The Reactive Layer (Arduino MCU - C++)

2. The Coordination Layer (Python MPU - main.py)

3. The Cognitive Layer (Media Service - Python/FastAPI)

Multimodal Intelligence: Living in a Human World

Two-Way Audio: The Listening Loop

Emotional Feedback through Color

Poor Man's SLAM: The Text-Mode Mapper

The Technical "Gotchas": Lessons from the Trenches

Conclusion: The Future of Our Tiny Friend

Building Genious.name: How We're Using AI to Transform Domain Name Discovery 🚀

The Problem We're Solving 🎯

Tech Stack Deep Dive 🛠️

Frontend Architecture

Backend Magic

AI Integration Deep Dive 🧠

Deployment & DevOps 🚀

Upcoming Features 🔮

Lessons Learned 📚

What's Next? 🎯

Get Involved! 🤝

webdev #ai #javascript #react #nodejs

Simple SEO fix for Vite/React SPAs without switching to Next/Remix

The Birth of AutoCode: A Tale of AI Self-Creation

Generating Andorid apps from Claude artifacts

Creating an Android App from Claude's HTML+JS Artifact: A Step-by-Step Guide

Table of Contents

Introduction

Prerequisites

Step 1: Generating the HTML+JS Artifact with Claude

Step 2: Setting Up Your Development Environment

Step 3: Creating a Cordova Project

Step 4: Integrating Claude's Artifact

Step 5: Configuring Your Android App

Step 6: Building and Running Your Android App

Step 7: Customizing Your App's Appearance

Troubleshooting and Tips

Conclusion

Free Chat with Gemini Pro

FREECHAT

DOCKER

NGINX

Monitoring API Performance with Express, Prometheus, and Grafana

Introduction

Setting Up the Express API

Setting Up Prometheus

Visualizing with Grafana

Configuring Grafana

Conclusion

Manga TV

Bringing Anime Stories to Life with AI

Overview of the Manga TV Platform

Reflections on Integrating Multiple AI Services

Key Features

What's Next for Manga TV

How to integrate Imagen2 AI in NodeJS

Predicting Foreign Exchange Rates with Deep Learning

Exploring the viability of neural networks for algorithmic trading

Approach

Results

Next Steps

Connecting to Amazon Bedrock and Claude 2.1

Introduction

Why Bedrock?

Democratizing Access

Simplified Integration

SignUp in AWS

Getting Started with AWS Bedrock and Node.js

Configuring AWS Credentials

Calling Bedrock API

Putting It All Together

Conclusion

Predicting Paid User Conversion