Forem: Mehdi Amrane

BrightMentions: Building a 24/7 Brand Monitoring AI Agent with n8n and Bright Data

Mehdi Amrane — Mon, 01 Sep 2025 06:55:41 +0000

This is a submission for the AI Agents Challenge powered by n8n and Bright Data

What I Built

I built an intelligent Brand and Keyword Mention Monitoring Agent that runs every 2 hours, searching the web for brand mentions and delivering curated reports directly to your inbox. It's essentially a tireless assistant that can process hundreds of posts while you sleep, filtering out the noise and highlighting what actually matters.

The core problem this solves: manual brand monitoring is both overwhelming and incomplete. You're either drowning in irrelevant mentions or missing important conversations entirely. This agent finds the sweet spot between comprehensive coverage and actionable insights.

Demo

The system processes hundreds of potential mentions autonomously, delivering only the most relevant insights through formatted email reports.

n8n Workflow

Complete workflow available here: N8N BrightMentions Workflow JSON

Setup Guide

Import & Clone
- Import the workflow JSON into your n8n instance
- Clone this Airtable database to your account
Connect Your Services
- Airtable: Create a token for your BrightMentions database
- Bright Data: Connect your account and create two zones: serp_api1 and web_unlocker1
- OpenRouter: Add your API credentials (recommended for rate limit flexibility)
- SMTP: Configure email credentials for notifications
Configure Your Settings
- Follow the yellow notes in the workflow
- Set your keywords, exclusion URLs, and email recipients in Airtable

Activate the workflow and monitoring begins automatically.

Technical Implementation

AI Agent Configuration

System Instructions: The heart of this system is a prompt that transforms the AI into a specialized mention analysis expert. Rather than generic sentiment analysis, the agent understands the context, urgency levels, and response strategies. It evaluates mentions across multiple dimensions: sentiment intensity, potential impact, author influence, and recommended actions.

Model Choice: I specifically chose OpenAI's GPT-4.1 mini and nano models via OpenRouter for two key reasons:

Massive context windows: These models can handle extremely long extracted content without hitting token limits, which is crucial when analyzing detailed social media posts or lengthy articles
No rate limiting on OpenRouter: Unlike direct API access, this ensures the workflow never stalls due to request limits during high-volume processing periods

Memory: This agent operates statelessly by design. Each mention analysis is independent, with all necessary context provided in the prompt. This approach ensures consistency and eliminates potential memory-related errors in automated workflows.

Tools: The agent uses n8n's Structured Output Parser to ensure the AI returns properly formatted JSON responses. This tool validates the output format and prevents parsing errors that could break the downstream workflow. All mention analysis happens within a single, comprehensive system instruction that guides the AI through structured evaluation processes.

The prompt instructs the agent to return structured JSON output covering sentiment, urgency classification, impact assessment, and specific response recommendations, essentially turning raw mentions into actionable data.

Bright Data Verified Node

Bright Data makes the "unstoppable" part of this workflow actually work. Instead of building and maintaining scrapers for dozens of platforms, I leverage their pre-built solutions that handle the complexity for me.

Pre-built Scrapers: The system uses 18 different Bright Data verified scrapers covering major platforms:

Social networks: X, Instagram, Facebook, LinkedIn, TikTok, BlueSky
Content platforms: YouTube, Pinterest, Reddit
Professional networks: LinkedIn

SERP API Integration: Clean, structured Google search results without dealing with CAPTCHAs or anti-bot measures. The serp_api zone handles search queries with time filters and exclusions seamlessly.

Web Unlocker: For any URL that doesn't fit pre-built scrapers, the Web Unlocker extracts content from virtually any website, bypassing anti-bot protections automatically.

Batch Processing: The workflow uses snapshot IDs for efficient parallel processing. Submit multiple URLs, receive a snapshot ID, then poll for completion - this approach scales naturally as monitoring requirements grow.

The reliability here is what sets this apart from DIY scraping solutions. When Instagram changes their layout, Bright Data's scrapers adapt automatically. When a site implements new bot detection, the Web Unlocker handles it transparently.

Journey

Key Challenges

The Keyword Ambiguity Problem
Initial versions captured every mention of a keyword regardless of context. Monitoring "n8n" would return results about random abbreviations, postal codes, basically anything containing those three letters in sequence.

Solution: Context-aware filtering. The system scrapes context URLs to understand what each keyword actually represents, then uses AI to filter search results before expensive content extraction begins.

URL Routing Complexity
Initially built routing with switch cases to direct URLs to appropriate scrapers. Problem: some URLs failed with Web Unlocker but worked perfectly with pre-built scrapers, creating unpredictable extraction failures.

Solution: Migrated to regex-based routing that intelligently categorizes URLs and automatically selects the optimal extraction method. Much cleaner and handles edge cases seamlessly.

Reliability at Scale
Web scraping fails frequently, even with robust infrastructure. Early versions would crash when individual requests failed, losing entire batches of data.

Solution: Comprehensive error handling with email notifications. Failed extractions are logged and reported, while successful ones continue processing. This provides visibility into problematic domains while maintaining system resilience.

Technical Insights

Context is everything: The difference between useful monitoring and spam is understanding what you're actually looking for. Automated context extraction from URLs transformed accuracy dramatically.

Smart routing matters: Categorizing URLs and directing them to appropriate extraction methods (pre-built scrapers vs. Web Unlocker) optimizes both speed and reliability.

Simple architectures scale better: Using stateless AI analysis rather than complex memory systems keeps the workflow maintainable and predictable.

Results

The system delivers measurable improvements over manual monitoring:

Comprehensive coverage: 24/7 monitoring across all major platforms
High accuracy: Context-aware filtering eliminates irrelevant mentions
Cost efficiency: Pay-per-use pricing scales with actual monitoring needs
Zero maintenance: Bright Data handles platform changes automatically
Actionable insights: AI analysis provides response recommendations, not just raw data

Redis Place: Building r/place with 9 Redis Data Structures

Mehdi Amrane — Mon, 11 Aug 2025 06:59:20 +0000

This is a submission for the Redis AI Challenge: Beyond the Cache.

What I Built

Redis Place is a collaborative pixel art canvas inspired by Reddit's r/place, where users can paint on a shared 1000x1000 canvas in real-time. But here's the twist: every single feature is powered by Redis as the primary database - no traditional database in sight!

What makes this project special:

🎨 Real-time collaboration with instant pixel updates across all users
📊 Advanced analytics dashboard with live user leaderboards and activity feeds
🔄 Time-travel replay system to watch the canvas evolve from the first pixel
🗺️ Activity heatmaps showing hot zones across different time ranges
🏆 Achievement system with automated badge awarding
⚡ Ultra-optimized performance with Redis pipelines and intelligent caching

The entire application demonstrates Redis capabilities far beyond caching - it's a comprehensive data platform handling everything from pixel storage to user authentication, analytics, and real-time messaging.

Tech Stack

Frontend: Vite + React
Backend: Node.js + Socket.io + Redis 8

Demo

🚀 Live Demo: https://redisplace.mehdi.co/
🐙 Source Code: https://github.com/mehdiamrane/redis-place

ASMR Demo Video

Key Features Showcase:

Main Canvas:
Real-time collaborative pixel art

Time-Travel Replay:
Watch the canvas evolve over time with chronological event reconstruction

Activity Heatmap:
Visual overlay showing activity hot zones with time-based filtering

Analytics Dashboard:
User leaderboards, statistics, and activity feed

User Profile:
User profile with statistics and badges

How I Used 9 Redis Data Structures in Redis 8

This project pushes Redis beyond caching into primary database territory, leveraging 9 different Redis data structures to create a feature-rich collaborative application:

1. 🔢 Bitfields: Ultra-Efficient Pixel Storage

// Store 1M pixels in just 625KB using 5-bit encoding per pixel
const bitOffset = (y * 1000 + x) * 5;
await redis.bitfield("canvas:pixels", "SET", "u5", bitOffset, colorId);

Challenge: Store 1M pixels efficiently
Solution: Each pixel uses 5 bits (32 color IDs), reducing memory from ~4MB to 625KB
Impact: 85% memory reduction with instant pixel access

2. 📡 Pub/Sub: Real-Time Collaboration

// Instant pixel updates across all connected users
await redis.publish("canvas:updates", JSON.stringify({ x, y, color, userId }));

Real-time broadcasting of pixel changes to all connected clients
Decoupled architecture separating storage from notifications
Instant synchronization across unlimited concurrent users

3. 🏆 Sorted Sets: Dynamic Leaderboards

// Increment user's pixel count
await redis.zincrby("leaderboard:users", 1, userId);
// Get top 10 users with most pixels
const topUsers = await redis.zrevrange("leaderboard:users", 0, 9, "WITHSCORES");

Automatic ranking without complex sorting algorithms
Efficient range queries for top N users
Real-time score updates with each pixel placement

4. 📊 HyperLogLog: Probabilistic Analytics

// Add user ID to daily visitor set
await redis.pfadd(`visitors:daily:${date}`, userId);
// Count unique daily visitors with constant memory
const uniqueVisitors = await redis.pfcount(`visitors:daily:${date}`);

Memory-efficient counting of unique values
Daily visitor tracking with minimal overhead

5. 🌊 Streams: Event Sourcing & Time Travel

// Complete event history for replay feature
await redis.xadd("stream:activity", "*", "userId", userId, "x", x, "y", y, "color", color);

// Time-based filtering for canvas replay
const events = await redis.xrange("stream:activity", startTime, endTime);

Complete event history for every pixel placement
Time-based filtering using native Redis timestamps
Canvas reconstruction by replaying events chronologically

6. 🗃️ JSON Documents: Rich User Profiles and Rate Limiting

// Structured user data with atomic updates
await redis.call(
  "JSON.SET",
  `userprofile:${userId}`,
  "$",
  JSON.stringify({
    pixelsPlaced: 150,
    favoriteColor: calculateFavorite(colorUsage),
    colorUsage: { "1": 45, "5": 30, "12": 15 },
    badges: ["first_pixel", "color_master"],
  })
);

// Rate limiting using JSON profiles
const profileData = await redis.call("JSON.GET", profileKey, "$.lastPixelTime");
if (timeSinceLastPixel < backendCooldownMs) {
  socket.emit("rate-limited", { remainingSeconds });
}

Atomic profile updates in a single operation
Complex data structures with nested objects and arrays
Automatic favorite color calculation from usage patterns
Badge system integration with real-time achievements
Rate limiting using JSON profiles with lastPixelTime field

7. ⏱️ Time Series: Activity Heatmaps

// Track activity zones with time-based aggregation
await redis.call("TS.CREATE", `heatmap:${zoneX}:${zoneY}`, "RETENTION", 604800000);
await redis.call("TS.ADD", `heatmap:${zoneX}:${zoneY}`, Date.now(), 1);

// Query with time ranges for dynamic heatmaps
const activity = await redis.call("TS.RANGE", `heatmap:${x}:${y}`, fromTime, "+");

400-zone grid tracking (400 areas of 50x50 pixels zones across 1000x1000 pixels canvas)
7-day retention with automatic expiration
Pipeline optimization reducing 30s queries to 180ms
5-minute caching further improving performance to 90ms when cache hits

8. 🗂️ Hashes: Authentication & Statistics

// Storing user credentials
await redis.hset(`user:${username}`, "password", bcryptHash, "createdAt", timestamp);
// Incrementing global color usage statistics
await redis.hincrby("stats:colors", colorId, 1);

Secure credential storage with bcrypt hashing
Global color statistics with atomic increments
Efficient field-based operations for related data

9. 🎯 Strings: Intelligent Caching and Session Management

// Event-driven cache invalidation
await redis.set("canvas:snapshot:cache", JSON.stringify(snapshot));
// Clear on pixel changes
await redis.del("canvas:snapshot:cache");

// 30-day sliding sessions with automatic renewal
await redis.setex(`session:${token}`, 2592000, username);
// Validate session against logged-in username
const username = await redis.get(`session:${sessionToken}`);

Smart cache invalidation only when canvas changes
Race condition handling with polling mechanisms
Performance boost: 4s generation → instant cached loads
Sliding expiration for sessions, refreshed on activity

Performance Optimizations That Matter

Canvas Loading: 500x Performance Improvement

Before: 20+ seconds for full canvas load
After: 4 seconds (first load), instant (cached)
Technique: 50K-operation pipelines (50k operations max. to prevent stack overflow) + intelligent Redis caching

Heatmap Queries: 165x Faster

Before: 30 seconds for 400 zone queries
After: 180ms (pipelines) → 90ms (with cache)
Technique: Pipeline batching + 5-minute Redis cache layer

Canvas Storage Efficiency: 85% Reduction

Traditional approach: ~4MB for 1M pixels
Bitfield approach: 625KB for same data
Bonus: Room for 32 color states because of 5-bit encoding

Conclusion - Redis is not just a cache

This project proves Redis isn't just a cache - it's a complete data platform:

✅ Primary Database: No other database needed

✅ Real-Time Engine: Pub/Sub for instant collaboration

✅ Analytics Platform: Multiple data structures for complex queries

✅ Event Store: Complete history with Streams

✅ Session Store: Secure authentication with sliding expiration