@e2fyi/streams

@e2fyi/streams
======================

This NodeJS library provides custom NodeJS streams
for specific use cases.

Currently, the following streams are available:

- DocumentTagger: A Transform stream to tag an auto-increment field to each object in the stream. Can also mutate the stream objects through a function or default Object.

- MongooseStream: A Transform stream which will bulk write the objects in the stream to mongodb via a mongoose model.

ChangeLog

- v1.0.0: new streams: DocumentTagger, MongooseStream.

Quick start

Using CommonJS module
``js
// importing DocumentTagger and MongooseStream
const {DocumentTagger, MongooseStream} = require('@e2fyi/streams');
`

API reference

The API documentation is also available at https://e2fyi.github.io/streams.

@e2fyi/streams

* @e2fyi/streams
* .DocumentTagger ⇐ stream.Transform
* new DocumentTagger(opts)
* .MongooseStream ⇐ stream.Transform
* new MongooseStream(opts)
* .DocumentTaggerSettings : Object
* .MongooseStreamSettings : Object

$3

Transform Object stream (objectMode=true) to tag with an autoIncrement id.
An object or function can be optionally provided to mutate each object in
the stream.

Kind: static class of @e2fyi/streams
Extends: stream.Transform
Emits: filtered

#### new DocumentTagger(opts)
Create a new DocumentTagger stream.

| Param | Type | Description |
| --- | --- | --- |
| opts | DocumentTaggerSettings | Settings for the stream. |

Example
`js
const docTagger = new DocumentTagger({autoIncrement: 'id', mutate: { project: 'test' }});
someReadableStreamFromArray([{text: 'abc'}, {text: 'efg'}])
.pipe(docTagger)
.pipe(process.stdout);
// stdout >
// {"text": "abc", "id": 0, "project": "test"}
// {"text": "efg", "id": 1, "project": "test"}
`


$3

A custom NodeJS Transform stream to mongo via mongoose.

Kind: static class of @e2fyi/streams
Extends: stream.Transform
Emits: mongoose-bulk-write

#### new MongooseStream(opts)
Create a Transform stream which bulkWrite to mongo based on the itemWaterMark.
model (mongoose Model) is a required field.


| Param | Type | Description |
| --- | --- | --- |
| opts | MongooseStreamSettings | Configuration for MongoStream. Default value for itemWaterMark is 50. |

Example
`js
var stream2mongo = new MongooseStream({mode: SomeMongooseModel});
someReadableStreamFromArray([{text: 'abc'}, {text: 'efg'}])
.pipe(stream2mongo) // writes to mongo (while stream are also passthrough)
.pipe(response); // stream same results back to some request
`


$3

Settings for DocumentTagger.

Kind: static typedef of @e2fyi/streams
Properties

| Name | Type | Description |
| --- | --- | --- |
| autoIncrement | String | The auto-increment field to tag onto the stream object. |
| ignoreNonObject | Boolean | If true, no errors will be emitted when the chunk in the stream cannot be parsed into Object. |
| readableObjectMode | Boolean | If true, Object will be emitted from the stream, otherwise a String or Buffer representation or will emitted instead. |
| objectMode | Boolean | If true, Object will be emitted from the stream, otherwise a String or Buffer representation or will emitted instead. Overwrites writableObjectMode. |
| filter | function | Function to filter the objects in the stream. Return true to keep the object. |
| mutate | function \| Object | Function or Object to mutate the stream. If an Object is provided, each stream object will be mutated with the Object.assign(streamObj, mutateObj). |

$3

Settings for MongooseStream.

Kind: static typedef of @e2fyi/streams
Properties

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| itemWaterMark | Number | 50 | The number of item collected before writing to mongodb. |
| passThrough | Boolean | | If false nothing will be emitted from the stream. |
| model | mongoose.Model | | mongoose Model. |




Development

Unit testing
`
npm test
`

Build documentation
`
npm run build
``