Best Web Scraping MCP Servers

DeepWiki

Official Remote

Instantly turn any Deepwiki article into clean, structured Markdown you can use anywhere. Deepwiki MCP Server safely crawls deepwiki.com pages, removes clutter like ads and navigation, rewrites links for Markdown, and offers fast performance with customizable output formats. Choose a single document or organize content by page, and easily extract documentation or guides for any supported library. It’s designed for secure, high-speed conversion and clear, easy-to-read results—making documentation and learning seamless.

Integrates with Jina AI's web services to enable web content extraction, search, and fact-checking through natural language interactions.

W

Web Fetcher

Remote

Fetches and extracts web content using Playwright's headless browser capabilities, delivering clean, readable content from JavaScript-heavy websites in HTML or Markdown format for research and information gathering.

DuckDuckGo Search

Remote

Integrates with DuckDuckGo to provide web search capabilities, content fetching, and parsing, with results formatted for large language model consumption.

C

Chinese Trends Hub

Remote

Provides real-time access to trending topics and content from major Chinese platforms including Weibo, Zhihu, Douyin, Bilibili, Douban, Toutiao, and 36kr through separate tools with temporary caching for improved performance.

FetchSERP

Official

Integrates with FetchSERP API to provide SEO analysis, SERP data retrieval, web scraping, keyword research, backlink analysis, and domain intelligence across Google, Bing, Yahoo, and DuckDuckGo search engines.

Provides specialized documentation scraping and retrieval from GitHub, NPM, PyPI, and web pages, enabling accurate reference to up-to-date library documentation without disrupting workflow.

Enables browser automation through Selenium WebDriver with support for Chrome, Firefox, and Edge browsers, providing navigation, element interaction, form handling, screenshot capture, JavaScript execution, and advanced actions for automated testing and web scraping tasks.

A

Airbnb

Remote

Integrates with Airbnb to enable vacation rental search and detailed property information retrieval without requiring API keys

DuckDuckGo Search

Remote

Provides web search capabilities through DuckDuckGo, enabling content retrieval, URL processing, and metadata extraction with customizable filtering options

Enables comprehensive web research by leveraging Tavily's Search and Crawl APIs to aggregate information from multiple sources, extract detailed content, and structure data specifically for generating technical documentation and research reports.

B

Baidu Search

Remote

Provides web search capabilities through Baidu's search engine, enabling retrieval of search results and webpage content with robust error handling and content parsing.

Integrates with the Serper API to enable web searches and webpage content extraction, supporting research, content aggregation, and data mining tasks.

Enables AI to perform Google searches via the Serper API with support for location, language, and time period filters.

Fetches and analyzes YouTube video transcripts by accepting URLs or video IDs and returning formatted transcript data with timestamps for video content analysis without watching.

G

Google News & Trends

Remote

Integrates with Google News RSS feeds and Google Trends to provide news article search, trending topic retrieval, and optional content summarization for news monitoring and trend analysis workflows.

Playwright

Automate web browsers for testing, scraping, and visual analysis.

Apify Actor

Official Remote

Use 4,000+ pre-built cloud tools, known as Actors, to extract data from websites, e-commerce, social media, search engines, maps, and more.

Transforms webpage content into a fully inlined, script-free HTML document with base64-encoded resources, enabling comprehensive web page analysis and extraction.

Provides a unified search and web scraping platform that integrates multiple search providers like SearxNG and Tavily, along with Firecrawl for advanced web content extraction, enabling flexible web data retrieval and structured information gathering.

Enables LLM-powered browser automation for web tasks including navigation, interaction, and content extraction through Playwright's comprehensive browser control capabilities.