I want to research and scrape the web
Servers for internet search, web crawling, and DOM extraction. Your agent aggregates multiple sites, scrapes raw fields, and links source URLs.
MCP servers for this
- MCP server
- Official
- Featured

Firecrawl MCP Server
Firecrawl
Firecrawl, the context API for AI agents: search, scrape, interact with the web, plus paper search.
7.7kActive- Runs locally
- Remote
- API key
- OAuth
Setup documented for VS Code (GitHub Copilot), Cursor, Claude Desktop, Windsurf- MCP server
- Official
- Featured

exa
Exa
Connect AI agents to Exa for web search, content fetching, and multi-step research.
5.2kActive- Remote
- API key
- OAuth
- Freemium
Setup documented for Claude Code, VS Code (GitHub Copilot), Codex, Cursor and 8 more- MCP server
- Official

browser-use
Browser Use
Browser agents and stealth cloud browsers for AI: run web tasks, fill forms, extract data.
119kActive- Runs locally
- Remote
- API key
Setup documented for Claude Code, Cursor- MCP server
- Official

Apify MCP server
Apify
Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡
14kActive- Remote
- API key
- OAuth
Setup documented for VS Code (GitHub Copilot), Cursor, ChatGPT, Claude Desktop- MCP server

Scrapling MCP Server
Web scraping with stealth HTTP, real browsers, and Cloudflare bypass. CSS selectors supported.
90kActive- Runs locally
- No auth
- MCP server

AIHawk
AI browser agent: browses, clicks, types, and reads real web pages from plain-English instructions.
32kActive- Runs locally
- API key
- MCP server

Skyvern
Skyvern
AI-powered browser automation — navigate, click, fill forms, and extract data from any website.
23kActive- Remote
- API key
Setup documented for VS Code (GitHub Copilot)- MCP server

SeleniumBase MCP
Stealthy browser automation, testing, and web-scraping via CDP Mode.
13kActive- Runs locally
- No auth
- Free
Setup documented for VS Code (GitHub Copilot)- MCP server

Robin: AI-Powered Dark Web OSINT
Dark web OSINT over Tor: search onion engines, scrape pages, report with your own model.
7.7kActive- Runs locally
- API key
Setup documented for Claude Code, Codex, Claude Desktop- MCP server
- Official

CRW Web Scraper
fastCRW
Open-source web scraper for AI agents with scrape, crawl, and map tools
1.2kActive- Runs locally
- API key
- Freemium
Setup documented for Claude Code, Codex, Cursor, Gemini CLI and 2 more- MCP server
- Official

Brightdata MCP server
Bright Data
Bright Data's Web MCP server enabling AI agents to search, extract & navigate the web
2.7kActive- Runs locally
- API key
Setup documented for Claude Code, VS Code (GitHub Copilot), Cursor, Claude Desktop and 4 more- MCP server

ui-tars-desktop
MCP server for web search operations
39kActive- Runs locally
- MCP server

wigolo
Local-first web intelligence MCP server for AI coding agents
5.5kActive- Runs locally
- API key
- Free
Setup documented for Claude Code, VS Code (GitHub Copilot), Codex, Cursor and 5 more- MCP server
- Official

brave-search-mcp-server
Brave
Brave Search MCP Server: web results, images, videos, rich results, AI summaries, and more.
1.5kActive- Runs locally
- API key
Setup documented for VS Code (GitHub Copilot), Claude Desktop- MCP server
- Official

Anime Garden
Anime Garden
Anime Garden MCP for anime torrent discovery.
1.4kActive- Remote
- No auth
Setup documented for Claude Code, Codex- MCP server

SearXNG Search
SearXNG
MCP server for SearXNG — privacy-respecting web search with pagination, URL reading
1.3kActive- Runs locally
- API key
- OAuth
- Free
Setup documented for Cursor, Claude Desktop- MCP server

tavily-mcp
Tavily
MCP server for advanced web search using Tavily
2.5kActive- Runs locally
- API key
- OAuth
Setup documented for Claude Code, Cursor- MCP server
- Official

CRW Web Scraper
fastCRW
Open-source web scraper for AI agents with scrape, crawl, and map tools
1.2kActive- Runs locally
- API key
- Freemium
Setup documented for Claude Code, Codex, Cursor, Gemini CLI and 2 more- MCP server

invisible_playwright_mcp
AI browser agent: browses, clicks, types, and reads real web pages from plain-English instructions.
887Active- Runs locally
Setup documented for Claude Code, Gemini CLI- MCP server

Unbrowse
Websites as APIs: read pages, run site tasks via learned APIs, cloud browser with saved logins.
810Active- Runs locally
- Remote
- API key
Setup documented for Claude Code- MCP server
- Official

Fetch
A Model Context Protocol server that provides web content fetching capabilities. This server enables LLMs to retrieve and process content from web pages, converting HTML to markdown for easier consumption.
92kActive- Runs locally
- Remote
- No auth
Setup documented for VS Code (GitHub Copilot)- MCP server

idea-reality-mcp
Pre-build reality check. Scans GitHub, HN, npm, PyPI, Product Hunt — returns 0-100 signal.
827Active- Runs locally
- API key
Setup documented for Claude Code, Cursor, Claude Desktop- MCP server

browser
Lightpanda
Requesting 933 real web pages over the network on a AWS EC2 m5.large instance. See benchmark details.
37kActive- Runs locally
- No auth
- MCP server

TrueSource
First-hand market signals: filings, procurement wins, R&D expansion, macro data
575Active- Runs locally
- API key
Setup documented for Claude Desktop
Skills for this
- Skill

Agent-Reach
Agent Reach 让你的 Agent 真正连上互联网,用上它自己够不着的平台和能力。
105kActive- Bundles scripts
- Needs network
Setup documented for Claude Code, Codex- Skill

reddit-fetch
Reddit
Fetch content from Reddit via its JSON API using a browser session (DuckDuckGo-hop unlock). Use when accessing Reddit URLs, researching topics on Reddit, or when Reddit returns 403/blocked errors.
10kActive- No scripts
- Needs network
- Skill

browser-act
BrowserAct
Browser automation CLI for AI agents. NEVER run browser-act commands directly via Bash — always invoke this skill first. Use browser-act when a user mentions it by name, includes or asks to run a browser-act CLI command (e.g., browser-act browser list), or to: fetch, view, or extract rendered…
6.3kActive- No scripts
- Needs network
Setup documented for Claude Code, Codex, Cursor, Windsurf- Skill
- Official

kanecli-skill
TestMu AI
Use kane-cli for any task that requires a real browser: navigating websites, clicking elements, filling forms, searching, testing web UI, taking screenshots, or verifying deployments. Use --agent for run, testmd run, and generate.
381Active- Bundles scripts
- Skill
- Official

puppeteer-skill
TestMu AI
Generates Puppeteer scripts for browser automation, scraping, and PDF generation. Triggers on: "Puppeteer", "headless Chrome", "page.goto", "scrape", "PDF generation".
381Active- No scripts
- Skill
- Official

venice-augment
Venice.ai
Venice augmentation endpoints for agent pipelines. Covers POST /augment/text-parser (extract text from PDF/EPUB/DOCX/PPTX/XLSX/XLS, plain text and source code; multipart, up to 25MB; JSON or plain-text response), POST /augment/scrape (fetch a URL and return markdown; blocks X/Reddit and…
160Active- No scripts
- Skill

social-media-research-skills
ScrapeCreators
Practical AI agent skills for social media research, powered by ScrapeCreators.
5kActive- Bundles scripts
- Needs network
Setup documented for Claude Code, VS Code (GitHub Copilot), GitHub Copilot CLI, Codex and 3 more- Skill

boss-zhipin-scraper
Boss直聘
Scrape BOSS直聘 (job listing site) via Chrome CDP. Searches jobs by keyword/city/filters, fetches JD details, outputs structured JSON/CSV with plaintext salary, and can summarize scraped results into a job-market prompt. Optional --company locally filters results by company name (boss_name…
1.7kActive- Bundles scripts
- Needs network
- Skill

youtube-skills
TranscriptAPI
YouTube Skills gives your AI agent instant access to YouTube transcripts, video search, channel data, and playlist extraction. No yt-dlp (YouTube blocks all major cloud IPs), no headless browsers, no binaries — just a fast API call that works everywhere.
1.1kActive- No scripts
- Needs network
Setup documented for Claude Code, Codex, Cursor, Google Antigravity- Skill

unbrowse (skill)
Search and call websites through Unbrowse's hosted API or remote MCP, reuse indexed site tools, read pages, and learn missing routes in its cloud browser. Use for structured website tasks, and authenticated site access with Unbrowse.
810Active- No scripts
Setup documented for Claude Code- Skill

youtube-fetcher
Retrieve transcripts from YouTube, Instagram, TikTok, X, Vimeo and other video sites, summarize or analyze what was said (and shown on screen), or save an Obsidian-ready Markdown knowledge-base note with captions or a Whisper transcript, creator metadata, frames, language, and source provenance…
512Active- Bundles scripts
- Skill

deep-research
Execute autonomous multi-step research using Google Gemini Deep Research Agent. Use for: market analysis, competitive landscaping, literature reviews, technical research, due diligence. Takes 2-10 minutes but produces detailed, cited reports. Costs $2-5 per task.
447Active- Bundles scripts