We built scrapers for 11 major platforms — here's what we learned

Emery Patricks — Fri, 13 Mar 2026 03:35:50 +0000

Why we built ScraperNode

We kept running into the same problem: we needed structured data from platforms like LinkedIn, Instagram, and TikTok, and every time we'd write a custom scraper, it would break within weeks. Layout changes, rate limits, auth flows — maintaining scrapers for major platforms is a full-time job.

So we turned it into an actual product. ScraperNode is a scraping API with maintained scrapers for 11 platforms. You send a request, you get structured data back.

What we cover

22 scrapers across 11 platforms:

Platform	What you can extract
LinkedIn	Profiles, companies, posts, job listings, people search
Instagram	Profiles, posts, reels, hashtag search
TikTok	Profiles, posts, hashtag search
YouTube	Channels, videos, comments
Twitter/X	Profiles, tweets
Facebook	Profiles, pages, posts, groups, marketplace
Indeed	Job listings, company info
Glassdoor	Reviews, salaries, job listings
Yelp	Business listings, reviews
GitHub	Repository data
Crunchbase	Company data

The hard parts

The reason we built this as a service instead of an open-source library: these scrapers need constant maintenance. LinkedIn alone changes their markup every few weeks. Instagram's auth flow has changed three times in the last year. TikTok's anti-bot detection gets more aggressive every month.

We handle the infrastructure — proxy rotation, browser fingerprinting, session management, rate limiting — so you don't have to.

How it works

REST API

You send a request with the data you want, you get JSON back.

curl https://api.scrapernode.com/v1/linkedin/profile \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{"url": "https://linkedin.com/in/someone"}'

You get back structured data — name, headline, experience, education, skills — not raw HTML you have to parse.

n8n community node

If you use n8n for workflow automation, we have a community node:

Go to Settings > Community Nodes
Install n8n-nodes-scrapernode
Add your API key in credentials

The node has three operations:

Create — start a scraping job
Get — check job status
Get Results — retrieve the structured data

A typical workflow looks like: Trigger → ScraperNode Create → Wait → Get Results → Process data

The node is also AI agent compatible, so you can use it with n8n's AI workflow nodes to build intelligent scraping pipelines.

Pricing

Credits-based, pay per scrape. No monthly minimum. Most scrapers cost 1-2 credits per result.

What people use it for

Lead generation — pull company and people data from LinkedIn for outreach
Market research — track competitors on social platforms
Job market analysis — aggregate listings from Indeed and Glassdoor
Content monitoring — track brand mentions across platforms
AI training data — feed structured social data into models and workflows
n8n automations — plug into automated workflows without writing scraper code

Other things we've built

We also put together awesome-n8n-templates — 8,697 n8n workflow templates organized by category, integration, and use case. We use n8n heavily for our own data pipelines and this repo makes it easier to find relevant templates.

We organized 8,697 n8n workflow templates into a browsable GitHub repo

Emery Patricks — Fri, 13 Mar 2026 02:58:01 +0000

The problem

n8n has thousands of community workflow templates, but finding the right one is harder than it should be. The official template library is keyword-search-only, and when you're exploring — trying to see what's possible with a specific tool or use case — there's no good way to browse.

We're ScraperNode — we build web scraping tools and use n8n daily. This was a problem we kept running into, so we decided to fix it.

What we built

We scraped all 8,697 templates from n8n.io and organized them into a GitHub repo with three browsing dimensions:

25 categories
AI agents, content creation, lead generation, CRM, DevOps, web scraping, invoice processing, HR, crypto trading, and more. Each category page lists every template with view counts and descriptions.

236 integrations
Every n8n node has its own page. Want to see every workflow that uses Google Sheets? OpenAI? Telegram? They're all grouped and sorted by popularity.

11 use cases
Cross-category collections for real-world needs: marketing automation, web scraping, data pipelines, freelancer toolkit, e-commerce, customer success, and others.

How each template is organized

Every template gets its own directory:

templates/lead-generation/2076-scrape-crunchbase/
├── README.md       # Full description, nodes used, import instructions
└── workflow.json   # The actual n8n workflow — import directly

The README has the complete description from the original template, which nodes it uses, and instructions for importing.

Numbers

8,697 templates
25 categories
236 integrations (any node used by 5+ templates gets its own page)
11 use case collections

Link

ScraperNode / awesome-n8n-templates

🔥 The largest open-source collection of n8n workflow templates — 8,697+ ready-to-import workflows across 25 categories, 236 integrations, and 11 use cases.

🔥 Awesome n8n Templates

🚀 The largest open-source collection of n8n workflow templates on GitHub.

8,697+ ready-to-import automation workflows across 25 categories — from AI agents and email automation to web scraping, CRM integrations, and beyond.

Whether you're automating your first workflow or building complex multi-step integrations, this collection has you covered. Every template is a .json file you can import directly into n8n in seconds.

⚡ Quick Start

Getting started takes less than a minute:

🔍 Browse the categories below to find what you need
📥 Download the .json template file
🚀 Import into n8n: Workflow menu → Import from File

That's it — your automation is ready to configure and run! 🎉

📚 New to n8n? Check out our step-by-step importing guide and credential setup walkthrough.

⭐ Featured Templates

Hand-picked templates showcasing the most powerful automation patterns — these are the workflows the community loves most.

Template

…

View on GitHub

MIT licensed. If there are categories or integrations you'd want to see, open an issue or PR.

Forem: Emery Patricks