Intent tools

Tools for processing intents from RASA JSONs to Facebook fast-text learning sets.

Because there is good GUI: Rasa NLU Trainer

CLI Usage

- Convert RASA json to fast-text learning set

``bash
$ intools jsonToText ./testData.json ./testData.txt
`

- Convert RASA json to fast-text learning set and multiply by entities

`bash
$ intools jsonToText -m ./testData.json ./testData.txt
`

- Make word vectors learning set from wiki XML export

`bash
$ intools wikiToText ./testData.xml ./testData.txt
`
-----------------

API

Classes

MultiplicatorStream

MultiplicatorStream

Pipeline

Functions

jsonToText(input, output, [pipeline], [mapFn]) ⇒ Promise

Create fast-text learning set from Rasa intents json

wikiToText(input, output, [mapFn]) ⇒ Promise

Create a pretrained word vectors learning set from Wikipedia XML dump

normalize(str)

Preserves only letters (with or withour diacritics) and makes everything lowercased

MultiplicatorStream

Kind: global class

* MultiplicatorStream
* new MultiplicatorStream()
* new MultiplicatorStream(getVariants)

$3

Multiplicates a learning set data with available entities information

Example
`javascript
const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');

const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');

const entities = new main.EntitiesFromJson(from);

const pipeline = [
new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];

entities.loadEntities()
.then(() => main.jsonToText(from, to, pipeline))
.catch(e => console.error(e));
`


$3

| Param | Type |
| --- | --- |
| getVariants | function |

MultiplicatorStream

Kind: global class

* MultiplicatorStream
* new MultiplicatorStream()
* new MultiplicatorStream(getVariants)

$3

Multiplicates a learning set data with available entities information

Example
`javascript
const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');

const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');

const entities = new main.EntitiesFromJson(from);

const pipeline = [
new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];

entities.loadEntities()
.then(() => main.jsonToText(from, to, pipeline))
.catch(e => console.error(e));
`


$3

| Param | Type |
| --- | --- |
| getVariants | function |

Pipeline

Kind: global class

* Pipeline
* new Pipeline()
* .add(pipe) ⇒ this
* .promise() ⇒ promise

$3

Simple tool, which creates a Promise from pipeline of streams

$3

Append a stream

Kind: instance method of Pipeline

| Param | Type | Description |
| --- | --- | --- |
| pipe | ReadableStream \| Writable | the transform stream |

$3

Get a promise

Kind: instance method of Pipeline

jsonToText(input, output, [pipeline], [mapFn]) ⇒ Promise

Create fast-text learning set from Rasa intents json

Kind: global function

| Param | Type | Default | Description |
| --- | --- | --- | --- |
| input | string \| ReadableStream | | path of Rasa intent set or stream |
| output | string \| Writable | | path or stream to write fast-text learning set |
| [pipeline] | Array | | array of transform streams to modify the learning set |
| [mapFn] | function | | text normalizer function |

Example
`javascript
const path = require('path');
const { jsonToText } = require('intent-tools');

const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');

main.jsonToText(from, to)
.catch(e => console.error(e));
``

wikiToText(input, output, [mapFn]) ⇒ Promise

Create a pretrained word vectors learning set from Wikipedia XML dump

Kind: global function

| Param | Type | Default | Description |
| --- | --- | --- | --- |
| input | string \| ReadableStream | | path of Rasa intent set or stream |
| output | string \| Writable | | path or stream to write fast-text learning set |
| [mapFn] | function | | text normalizer function |

normalize(str)

Preserves only letters (with or withour diacritics) and makes everything lowercased

Kind: global function
Returs: string

| Param | Type | Description |
| --- | --- | --- |
| str | string | input string |