!logo


Web scraping and HTML-reprocessing. The easy way.

ineed allows you collect useful data from web pages using simple and nice API. Let's collect images, hyperlinks, scripts and stylesheets from http://google.com:

``js

var ineed = require('ineed');

ineed.collect.images.hyperlinks.scripts.stylesheets.from('http://google.com',
function (err, response, result) {
console.log(result);
});

`

Also, it can be used to build HTML-reprocessing pipelines (like jch/html-pipeline but for Node) with elegance. E.g. we have the following html:
`html







This is an H1
=============
* Red
* Green
* Blue


`

Let's render it's

This is an H1

• Red


• Green


• Blue

`


ineed doesn't build and traverse DOM-tree, it operates on sequence of HTML tokens instead. Whole processing is done in one-pass, therefore, it's blazing fast! The token stream is produced by parse5 which parses HTML exactly the same way modern browsers do.


ineed provides built-in collectors and reprocessors that covers a wide variety of common use cases. However, if you feel that something is missing, then you can easily extend ineed with custom plugins.

Install


`
$ npm install ineed
`

Usage

The general form:

#### `ineed.[....].`




$3


##### .fromHtml(html)
Accepts html string as an argument and synchronously returns result of the action.

Example:
`js
var result = ineed.collect.texts.fromHtml('

');
`

##### .from(options, callback)
Asynchronously loads specified page and invokes callback(err, response, result). It passes response object to callback in case if you need response headers or status codes. The first argument can be either a url or an options object.
The set of options is the same as in @mikeal's request, so you can use POST method, set request headers or do any other advanced setup for the page request.

Examples:
`js
ineed.collect.jsCode.from('https://github.com', function (err, response, result) {
console.log(response.statusCode);
console.log(response.headers);
});
`
`js
ineed.collect.title.from({
url: 'https://test.domain/',
method: 'POST',
form: 'input=test'
}, function (err, response, result) {
console.log(result);
});

`




$3


Collects information specified by plugin set. The result of the action is an object that contains individual plugin outputs as properties.

Example:
`js
var result = ineed.collect.texts.images.scripts.fromHtml(html);
`
will produce result:
`js
{
"texts" : ,
"images" : ,
"scripts" :
}
`
Built-in plugins:

| Plugin|Description| Output |
--- | --- | ---
.comments| Collects HTML comments| Array of comment string
.cssCode| Collects CSS code enclosed in