n8n node for WebCrawler API integration - a website to markdown API for LLM and RAG
npm install n8n-nodes-webcrawlerapiWebcrawlerAPI is a website to markdown API for LLM and RAG. You can get content of the website or a single page with a
simple API call. Check more at WebCrawlerAPI. Our docs are available at WebCrawlerAPI Docs.
This package contains n8n community nodes for integrating with WebCrawler API.
The WebCrawlerAPI node allows you to scrape web content using the WebCrawlerAPI service.
#### Features
- Scrape content from any URL using the /v2/scrape endpoint
- Choose output format: Markdown, Cleaned, or HTML
- Remove unwanted elements using CSS selectors
- Use AI-powered prompts to extract specific information from the page
#### Credentials
To use this node, you need a WebCrawlerAPI credential:
- API Key: Obtain from WebCrawlerAPI Dashboard
#### Parameters
- URL to scrape (required): The URL of the page to scrape
- Output format (optional): Markdown (default), Cleaned, or HTML
- CSS selectors to remove (optional): Comma-separated list of selectors (e.g., .ads, .popup, #sidebar)
- Prompt (optional): AI prompt to extract specific information
#### Response Fields
- success: Boolean indicating if the request was successful
- status: Status of the scraping operation
- markdown: Content formatted as Markdown
- cleaned_content: Cleaned text content
- raw_content: Raw HTML content
- page_title: Title of the scraped page
#### Example Usage
1. Add the WebCrawlerAPI node to your workflow
!Add community node
2. Configure your WebCrawlerAPI credentials. Get your API key here.
!Configure credentials
3. Set the URL you want to scrape
4. Optionally, choose output format, add CSS selectors, or a prompt
5. Execute the workflow
#### Installation
``bash`
npm install n8n-nodes-webcrawlerapi
#### Development
`bashInstall dependencies
npm install
MIT