Web Scout MCP Server

An MCP server for web search using DuckDuckGo and content extraction, with support for multiple URLs and memory optimizations.

✨ Features

- 🔍 DuckDuckGo Search: Fast and privacy-focused web search capability
- 📄 Content Extraction: Clean, readable text extraction from web pages
- 🚀 Parallel Processing: Support for extracting content from multiple URLs simultaneously
- 💾 Memory Optimization: Smart memory management to prevent application crashes
- ⏱️ Rate Limiting: Intelligent request throttling to avoid API blocks
- 🛡️ Error Handling: Robust error handling for reliable operation

📦 Installation

$3

To install Web Scout for Claude Desktop automatically via Smithery:

``bash
npx -y @smithery/cli install @pinkpixel-dev/web-scout-mcp --client claude
`

$3

`bash
npm install -g @pinkpixel/web-scout-mcp
`

$3

`bash
npm install @pinkpixel/web-scout-mcp
`

🚀 Usage

$3

After installing globally, run:

`bash
web-scout-mcp
`

$3

Add this to your MCP client's config.json (Claude Desktop, Cursor, etc.):

`json
{
"mcpServers": {
"web-scout": {
"command": "npx",
"args": [
"-y",
"@pinkpixel/web-scout-mcp@latest"
]
}
}
}
`

$3

Set the WEB_SCOUT_DISABLE_AUTOSTART=1 environment variable when embedding the package and calling createServer() yourself. By default running the published entrypoint (for example node dist/index.js or npx @pinkpixel/web-scout-mcp) automatically bootstraps the stdio transport.

🧰 Tools

The server provides the following MCP tools:

$3

Initiates a web search query using the DuckDuckGo search engine and returns a well-structured list of findings.

Input:
- query (string): The search query string
- maxResults (number, optional): Maximum number of results to return (default: 10)

Example:
`json
{
"query": "latest advancements in AI",
"maxResults": 5
}
`

Output:
A formatted list of search results with titles, URLs, and snippets.

$3

Fetches and extracts clean, readable content from web pages by removing unnecessary elements like scripts, styles, and navigation.

Input:
- url: Either a single URL string or an array of URL strings

Example (single URL):
`json
{
"url": "https://example.com/article"
}
`

Example (multiple URLs):
`json
{
"url": [
"https://example.com/article1",
"https://example.com/article2"
]
}
`

Output:
Extracted text content from the specified URL(s).

🛠️ Development

`bash


Clone the repository

git clone https://github.com/pinkpixel-dev/web-scout-mcp.git
cd web-scout-mcp

Install dependencies

npm install

Build

npm run build

Run

npm start
``

📚 Documentation

For more detailed information about the project, check out these resources:

- OVERVIEW.md - Technical overview and architecture
- CONTRIBUTING.md - Guidelines for contributors
- CHANGELOG.md - Version history and changes

📋 Requirements

- Node.js >= 18.0.0
- npm or yarn

📄 License

This project is licensed under the MIT License.

<sub>Made with ❤️ by Pink Pixel


✨ Dream it, Pixel it ✨</sub>