The latest articles on Forem by Timosha (@timoshaciv). https://forem.com/timoshaciv https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3896721%2Fbd74e98a-0ed3-48b8-a549-485e225b3d04.jpeg https://forem.com/timoshaciv en Timosha Fri, 24 Apr 2026 21:29:03 +0000 https://forem.com/timoshaciv/i-built-an-ai-system-that-collects-1800-business-signals-per-day-3k1k https://forem.com/timoshaciv/i-built-an-ai-system-that-collects-1800-business-signals-per-day-3k1k <p>I built a signal intelligence system that scrapes Reddit, Hacker News, Google Trends, patent filings, job boards, and more -- collecting 1,800+ signals per day about business opportunities.</p> <h2> What It Does </h2> <p>The system runs 32 automated cron jobs across 14+ sources. Each signal gets scored on:</p> <ul> <li> <strong>Revenue potential</strong> (dollar signals mentioned in posts)</li> <li> <strong>Pain level</strong> (complaints, frustrations, switching behavior)</li> <li> <strong>Competition</strong> (how many existing solutions)</li> <li> <strong>Demand evidence</strong> ("I'd pay for...", "why hasn't someone built...")</li> </ul> <h2> How It Works </h2> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>Reddit (40 subreddits) ─┐ Hacker News ─┤ Google Trends ─┤ Federal Register ─┤──→ Signal DB ──→ Scoring ──→ Daily Digest Patent Filings ─┤ Job Boards ─┤ Google News ─┤ GitHub Trending ─┘ </code></pre> </div> <p>Every morning, I get a digest with the top signals ranked by score. The system detects:</p> <ul> <li> <strong>Breakout topics</strong> (sudden interest spikes)</li> <li> <strong>Cross-source convergence</strong> (same topic appearing in multiple places)</li> <li> <strong>Explicit demand</strong> (people literally saying they'd pay for something)</li> </ul> <h2> What I've Found So Far </h2> <p>After collecting 9,500+ signals:</p> <ol> <li> <strong>Raptive dropped their entry threshold from 100K to 25K pageviews</strong> -- most bloggers still think it's 100K</li> <li> <strong>Building permit data is the most underrated real estate indicator</strong> -- permits show development months before ground breaks</li> <li> <strong>The "boring business" SaaS market is massively underserved</strong> -- HVAC, plumbing, and cleaning companies still use spreadsheets</li> </ol> <h2> The Stack </h2> <ul> <li>Python collectors with PRAW, requests, pytrends</li> <li>SQLite for signal storage</li> <li>Cron for scheduling (32 jobs)</li> <li>Raspberry Pi for residential IP scraping (avoids rate limits)</li> <li>Custom scoring algorithm with enrichment pipeline</li> </ul> <h2> What I'm Building With This </h2> <p>I've launched 10 micro-experiments based on signals from this system -- each one is a landing page testing demand for a specific tool:</p> <ul> <li> <strong>PermitPulse</strong> -- Building permit intelligence</li> <li> <strong>RegRadar</strong> -- Regulatory change monitoring</li> <li> <strong>FilingDiff</strong> -- SEC filing change detection</li> <li> <strong>GovConRadar</strong> -- Government contract alerts</li> <li>And 6 more...</li> </ul> <p>Each experiment collects early access signups. After 7 days, I kill the losers and build MVPs for the winners.</p> <h2> Key Insight </h2> <p>Most people build first, then look for customers. This system flips it: <strong>find the demand first, then build exactly what people are asking for.</strong></p> <p>The signal-to-noise ratio is about 5% (105 actionable signals out of 1,853 collected yesterday). But those 105 signals are gold -- real people expressing real pain with real willingness to pay.</p> <p><em>What signals do you use to validate business ideas? I'd love to hear other approaches.</em></p> ai startup datascience business