Tools for processing Rasa.ai JSONs to text learning sets for FB FastText
npm install intent-toolsTools for processing intents from RASA JSONs to Facebook fast-text learning sets.
Because there is good GUI: Rasa NLU Trainer
- Convert RASA json to fast-text learning set
``bash`
$ intools jsonToText ./testData.json ./testData.txt
- Convert RASA json to fast-text learning set and multiply by entities
`bash`
$ intools jsonToText -m ./testData.json ./testData.txt
- Make word vectors learning set from wiki XML export
`bash`
$ intools wikiToText ./testData.xml ./testData.txt
-----------------
PromiseCreate fast-text learning set from Rasa intents json
PromiseCreate a pretrained word vectors learning set from Wikipedia XML dump
Preserves only letters (with or withour diacritics) and makes everything lowercased
* MultiplicatorStream
* new MultiplicatorStream()
* new MultiplicatorStream(getVariants)
Example
`javascript
const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');
const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');
const entities = new main.EntitiesFromJson(from);
const pipeline = [
new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];
entities.loadEntities()
.then(() => main.jsonToText(from, to, pipeline))
.catch(e => console.error(e));
`
| Param | Type |
| --- | --- |
| getVariants | function |
* MultiplicatorStream
* new MultiplicatorStream()
* new MultiplicatorStream(getVariants)
Example
`javascript
const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');
const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');
const entities = new main.EntitiesFromJson(from);
const pipeline = [
new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];
entities.loadEntities()
.then(() => main.jsonToText(from, to, pipeline))
.catch(e => console.error(e));
`
| Param | Type |
| --- | --- |
| getVariants | function |
* Pipeline
* new Pipeline()
* .add(pipe) ⇒ this
* .promise() ⇒ promise
Kind: instance method of Pipeline
| Param | Type | Description |
| --- | --- | --- |
| pipe | ReadableStream \| Writable | the transform stream |
Kind: instance method of Pipeline
PromiseKind: global function
| Param | Type | Default | Description |
| --- | --- | --- | --- |
| input | string \| ReadableStream | | path of Rasa intent set or stream |
| output | string \| Writable | | path or stream to write fast-text learning set |
| [pipeline] | Array | | array of transform streams to modify the learning set |
| [mapFn] | function | | text normalizer function |
Example
`javascript
const path = require('path');
const { jsonToText } = require('intent-tools');
const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');
main.jsonToText(from, to)
.catch(e => console.error(e));
``
PromiseKind: global function
| Param | Type | Default | Description |
| --- | --- | --- | --- |
| input | string \| ReadableStream | | path of Rasa intent set or stream |
| output | string \| Writable | | path or stream to write fast-text learning set |
| [mapFn] | function | | text normalizer function |
Kind: global function
Returs: string
| Param | Type | Description |
| --- | --- | --- |
| str | string | input string |