Production-ready MCP server for Hyperbrowser integration with AI-powered API discovery, browser automation, and web scraping
npm install deepcrawler-mcpProduction-ready Model Context Protocol (MCP) server for Hyperbrowser integration with AI-powered API discovery, advanced web scraping, and browser automation.
npx without prior installation.env files in multiple locations``bash`
npm install -g deepcrawler-mcp
`bash`
npx deepcrawler-mcp
`bash`
pip install deepcrawler-mcp
- OpenRouter API Key: Get one at https://openrouter.ai (for DeepCrawler AI agents)
- Hyperbrowser API Key: Get one at https://hyperbrowser.ai (for browser automation)
Create a .env file in your project root:
`env`
OPENROUTER_API_KEY=sk_live_your_key_here
HYPERBROWSER_API_KEY=your_hyperbrowser_key_here
LOG_LEVEL=info
Or set environment variables:
`bash`
export OPENROUTER_API_KEY=sk_live_your_key_here
export HYPERBROWSER_API_KEY=your_hyperbrowser_key_here
`bashUsing npx
npx deepcrawler-mcp
$3
`bash
npx deepcrawler-mcp --list-tools
`Configuration
$3
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
|
HYPERBROWSER_API_KEY | Yes | - | Your Hyperbrowser API key |
| HYPERBROWSER_BASE_URL | No | https://api.hyperbrowser.ai | API base URL |
| HYPERBROWSER_TIMEOUT | No | 30000 | Request timeout in ms |
| HYPERBROWSER_RETRY_ATTEMPTS | No | 3 | Number of retry attempts |
| HYPERBROWSER_RETRY_DELAY | No | 1000 | Initial retry delay in ms |
| LOG_LEVEL | No | info | Log level: debug, info, warn, error |$3
The server checks for
.env files in this order:1.
./.env (project root)
2. ~/.mcp/.env (user home)
3. ~/.env (user home)
4. /etc/mcp/.env (system-wide)AI Assistant Configuration
This MCP server works with 20+ AI coding assistants. See SETUP_GUIDE.md for detailed configuration instructions for:
- Augment Code -
config-examples/augment-config.json
- Claude Desktop - config-examples/claude_desktop_config.json
- Cursor - config-examples/cursor-config.json
- Cline - config-examples/cline-config.json
- GitHub Copilot - config-examples/github-copilot-config.json
- Tabnine - config-examples/tabnine-config.json
- Cody - config-examples/cody-config.json
- And 13+ more...$3
For most assistants, add this to your MCP configuration:
`json
{
"mcp_server": {
"command": "npx",
"args": ["deepcrawler-mcp"],
"env": {
"OPENROUTER_API_KEY": "sk_live_your_key_here",
"HYPERBROWSER_API_KEY": "your_hyperbrowser_key_here"
}
}
}
`Then restart your AI assistant and verify tools are available.
Tools
$3
#### discover_apis
Discover hidden APIs on a website using AI agents.
Parameters:
-
url (string, required): Target website URL
- depth (number, optional): Crawl depth (1-5, default: 2)
- mode (string, optional): 'direct' or 'crew' (default: 'direct')
- include_websockets (boolean, optional): Include WebSocket analysis (default: true)
- include_static_analysis (boolean, optional): Include JavaScript analysis (default: true)
- timeout (number, optional): Timeout in ms (default: 300000)Example:
`json
{
"url": "https://example.com",
"depth": 2,
"mode": "direct"
}
`#### analyze_network_traffic
Analyze network traffic for API endpoints.
Parameters:
-
url (string, required): Target website URL
- duration (number, required): Analysis duration in seconds (1-300)
- filter_by_type (string, optional): Filter by request type#### analyze_javascript_code
Extract APIs from JavaScript code.
Parameters:
-
url (string, required): Target website URL
- include_comments (boolean, optional): Include code comments
- include_strings (boolean, optional): Include string literals#### solve_captcha
Detect and solve CAPTCHAs.
Parameters:
-
url (string, required): Target website URL
- captcha_type (string, required): Type of CAPTCHA (recaptcha_v2, recaptcha_v3, hcaptcha, image, audio)#### generate_openapi_schema
Generate OpenAPI specifications from discovered APIs.
Parameters:
-
endpoints (array, required): List of API endpoints
- base_url (string, required): API base URL
- title (string, optional): API title
- version (string, optional): API version#### analyze_websockets
Analyze WebSocket connections.
Parameters:
-
url (string, required): Target website URL
- duration (number, required): Analysis duration in seconds (1-300)
- filter_by_type (string, optional): Filter by message type$3
$3
Extract all hyperlinks from a webpage.
Parameters:
-
url (string, required): Target webpage URL
- include_markdown (boolean, optional): Also return page content in markdown format
- include_tags (array, optional): CSS selectors to include
- exclude_tags (array, optional): CSS selectors to exclude
- only_main_content (boolean, optional): Extract only main content linksExample:
`json
{
"url": "https://example.com",
"include_markdown": true,
"only_main_content": true
}
`Response:
`json
{
"links": [
{
"url": "https://example.com/page1",
"text": "Page 1",
"context": "Navigation link"
}
],
"markdown": "# Example\n\nContent here...",
"metadata": {
"total_links": 42,
"unique_links": 38,
"scraped_at": "2025-01-15T10:30:00Z"
}
}
`$3
Crawl multiple linked pages from a starting URL.
Parameters:
-
url (string, required): Starting webpage URL to crawl
- followLinks (boolean, optional): Whether to follow links to other pages (default: false)
- maxPages (number, optional): Maximum number of pages to crawl, 1-100 (default: 10)
- outputFormat (array, optional): Desired output formats: markdown, html, links, screenshotExample:
`json
{
"url": "https://example.com",
"followLinks": true,
"maxPages": 5,
"outputFormat": ["markdown", "links"]
}
`Response:
`json
{
"pages": [
{
"url": "https://example.com",
"title": "Example",
"content": "Page content...",
"links": [
{
"url": "https://example.com/page1",
"text": "Page 1"
}
]
}
],
"metadata": {
"total_pages": 5,
"crawled_at": "2025-01-15T10:30:00Z",
"duration_ms": 2500
}
}
`$3
Extract structured data from webpages using JSON schemas.
Parameters:
-
urls (array, required): List of URLs to extract data from
- schema (object, required): JSON schema defining the structure of data to extract
- prompt (string, optional): Custom prompt for extraction guidanceExample:
`json
{
"urls": ["https://example.com/product1", "https://example.com/product2"],
"schema": {
"title": { "type": "string" },
"price": { "type": "string" },
"description": { "type": "string" }
},
"prompt": "Extract product information"
}
`Response:
`json
{
"results": [
{
"url": "https://example.com/product1",
"data": {
"title": "Product 1",
"price": "$99.99",
"description": "Great product"
},
"success": true
}
],
"metadata": {
"total_urls": 2,
"successful": 2,
"failed": 0,
"extracted_at": "2025-01-15T10:30:00Z"
}
}
`$3
Execute advanced browser automation tasks with step-by-step execution.
Parameters:
-
task (string, required): Description of the browser task to execute
- url (string, optional): Starting URL for the task
- maxSteps (number, optional): Maximum number of steps to execute, 1-100 (default: 10)
- returnStepInfo (boolean, optional): Whether to return detailed step information (default: false)Example:
`json
{
"task": "Click the submit button and wait for confirmation",
"url": "https://example.com/form",
"maxSteps": 5,
"returnStepInfo": true
}
`Response:
`json
{
"result": "Task completed successfully",
"steps": [
{
"action": "navigate",
"result": "Navigated to page",
"timestamp": "2025-01-15T10:30:00Z"
},
{
"action": "click",
"result": "Clicked submit button",
"timestamp": "2025-01-15T10:30:01Z"
}
],
"metadata": {
"total_steps": 2,
"completed_at": "2025-01-15T10:30:02Z",
"success": true
}
}
`$3
Perform web searches using Bing search engine.
Parameters:
-
query (string, required): Search query string
- numResults (number, optional): Number of results to return, 1-50 (default: 10)Example:
`json
{
"query": "TypeScript best practices",
"numResults": 5
}
`Response:
`json
{
"results": [
{
"title": "TypeScript Best Practices",
"url": "https://example.com/typescript-best-practices",
"snippet": "Learn the best practices for writing TypeScript code..."
}
],
"metadata": {
"query": "TypeScript best practices",
"total_results": 5,
"searched_at": "2025-01-15T10:30:00Z"
}
}
`AI Assistant Configuration
$3
`json
{
"mcpServers": {
"hyperbrowser": {
"command": "npx",
"args": ["deepcrawler-mcp"],
"env": {
"HYPERBROWSER_API_KEY": "${HYPERBROWSER_API_KEY}"
}
}
}
}
`$3
`json
{
"mcpServers": {
"hyperbrowser": {
"command": "npx",
"args": ["deepcrawler-mcp"],
"env": {
"HYPERBROWSER_API_KEY": "${HYPERBROWSER_API_KEY}"
}
}
}
}
`$3
`json
{
"mcpServers": {
"hyperbrowser": {
"command": "npx",
"args": ["deepcrawler-mcp"],
"env": {
"HYPERBROWSER_API_KEY": "${HYPERBROWSER_API_KEY}"
}
}
}
}
`Development
$3
`bash
npm run build
`$3
`bash
npm test
npm run test:coverage
`$3
`bash
npm run lint
npm run format
`Troubleshooting
$3
Ensure
HYPERBROWSER_API_KEY is set correctly:`bash
echo $HYPERBROWSER_API_KEY
`$3
The server automatically retries with exponential backoff. Adjust retry settings:
`env
HYPERBROWSER_RETRY_ATTEMPTS=5
HYPERBROWSER_RETRY_DELAY=2000
`$3
Increase the timeout value:
`env
HYPERBROWSER_TIMEOUT=60000
`$3
Enable debug logging:
`env
LOG_LEVEL=debug
``See Hyperbrowser API Documentation
MIT - See LICENSE file for details
- NPM Package: https://www.npmjs.com/package/deepcrawler-mcp
- GitHub: TBD - To be configured
- Issues: TBD - To be configured
- Hyperbrowser Docs: https://docs.hyperbrowser.ai