#CSV2JSON
All you need nodejs csv to json converter. Support big json data, CLI, web server, powerful nested JSON, customised parser, stream, pipe, and more!

#IMPORTANT!!
Since version 0.3, the core class of csvtojson has been inheriting from stream.Transform class. Therefore, it will behave like a normal Stream object and CSV features will not be available any more. Now the usage is like:
``js
//Converter Class
var Converter=require("csvtojson").core.Converter;
var fs=require("fs");

var csvFileName="./myCSVFile";
var fileStream=fs.createReadStream(csvFileName);
//new converter instance
var csvConverter=new Converter({constructResult:true});

//end_parsed will be emitted once parsing finished
csvConverter.on("end_parsed",function(jsonObj){
console.log(jsonObj); //here is your result json object
});

//read from file
fileStream.pipe(csvConverter);
`

To convert from a string, previously the code was:
`js
csvConverter.from(csvString);
`

Now it is:
`js
csvConverter.fromString(csvString,callback);
`

The callback function above is optional. see Parse String.


##Menu
* Installation
* Example
* CLI Usage
* CLI Tool
* Web Service
* Demo Product
* Quick Start
* Parameters
* Customised Parser
* Webserver
* Events
* Built-in Parsers
* Example
* Big CSV File Streaming
* Process Big CSV File in CLI
* Column Array
* Parse String
* Empowered JSON Parser
* Change Log

GitHub: https://github.com/Keyang/node-csvtojson

##Installation
>npm install -g csvtojson


##Features

* Powerful library for you nodejs applications processing csv data.
* Extremly straight forward
* Multiple input support: CSV File, Readable Stream, CSV String etc.
* Highly extendible with your own rules and parsers for outputs.
* Multiple interfaces (webservice, command line)


##Usage

###Command Line Tools

>csvtojson

Example

>csvtojson ./myCSVFile

Or use pipe:

>cat myCSVFile | csvtojson

To start a webserver

>csvtojson startserver [options]

Advanced usage with parameters support, check help:

>csvtojson --help

$3

`js
var server=require("csvtojson").interfaces.web;

server.startWebServer({
"port":process.env.VCAP_APP_PORT || 8801
});
`
It uses HTTP Request as readable stream and HTTP Response as writable stream.

Quick Start

`js
//Converter Class
var Converter=require("csvtojson").core.Converter;
var fs=require("fs");

var csvFileName="./myCSVFile";
var fileStream=fs.createReadStream(csvFileName);
//new converter instance
var param={};
var csvConverter=new Converter(param);

//end_parsed will be emitted once parsing finished
csvConverter.on("end_parsed",function(jsonObj){
console.log(jsonObj); //here is your result json object
});

//read from file
fileStream.pipe(csvConverter);
`


Params

`js
//Parser Manager
var parserMgr=require("csvtojson").core.parserMgr;

parserMgr.addParser("myParserName",/^\parserRegExp\/,function (params){
var columnTitle=params.head; //params.head be like: parserRegExpColumnName;
var fieldName=columnTitle.replace(this.regExp, ""); //this.regExp is the regular expression above.
params.resultRow[fieldName]="Hello my parser"+params.item;
});
`

parserMgr's addParser function take three parameters:

1. parser name: the name of your parser. It should be unique.

2. Regular Expression: It is used to test if a column of CSV data is using this parser. In the example above any column's first row starting with parserRegExp will be using it.

3. Parse function call back: It is where the parse happens. The converter works row by row and therefore the function will be called each time needs to parse a cell in CSV data.

The parameter of Parse function is a JSON object. It contains following fields:

head: The column's first row's data. It generally contains field information. e.g. arrayitems

item: The data inside current cell. e.g. item1

itemIndex: the index of current cell of a row. e.g. 0

rawRow: the reference of current row in array format. e.g. ["item1", 23 ,"hello"]

resultRow: the reference of result row in JSON format. e.g. {"name":"Joe"}

rowIndex: the index of current row in CSV data. start from 1 since 0 is the head. e.g. 1

resultObject: the reference of result object in JSON format. It always has a field called csvRows which is in Array format. It changes as parsing going on. e.g.

`json
{
"csvRows":[
{
"itemName":"item1",
"number":10
},
{
"itemName":"item2",
"number":4
}
]
}
`

WebServer

`js
var webServer=require("csvtojson").interfaces.web;

var server=webServer.startWebServer({
"port":"8801",
"urlpath":"/parseCSV"
});
`

~~It will return an expressjs Application. You can add your own web app content there.~~ It will return http.Server object.

Events

Following events are used for Converter class:

* end_parsed: It is emitted when parsing finished. the callback function will contain the JSON object if constructResult is set to true.
* record_parsed: it is emitted each time a row has been parsed. The callback function has following parameters: result row JSON object reference, Original row array object reference, row index of current row in csv (header row does not count, first row content will start from 0)

To subscribe the event:

`js
//Converter Class
var Converter=require("csvtojson").core.Converter;

//end_parsed will be emitted once parsing finished
csvConverter.on("end_parsed",function(jsonObj){
console.log(jsonObj); //here is your result json object
});

//record_parsed will be emitted each time a row has been parsed.
csvConverter.on("record_parsed",function(resultRow,rawRow,rowIndex){
console.log(resultRow); //here is your result json object
});
`

Default Parsers

`json
{
"csvRows": [
{
"date": "2012-02-12",
"employee": {
"name": "Eric",
"age": "31",
"number": "51234",
"key": [
"key1",
"key2"
]
},
"address": [
"Dunno Street",
"Kilkeny Road"
]
},
{
"date": "2012-03-06",
"employee": {
"name": "Ted",
"age": "28",
"number": "51289",
"key": [
"key3",
"key4"
]
},
"address": [
"Cambridge Road",
"Tormore"
]
}
]
}
`

Big CSV File

`js
var Converter=require("csvtojson").core.Converter;
var csvConverter=new Converter({constructResult:false}); // The parameter false will turn off final result construction. It can avoid huge memory consumption while parsing. The trade off is final result will not be populated to end_parsed event.

var readStream=require("fs").createReadStream("inputData.csv");

var writeStream=require("fs").createWriteStream("outpuData.json");

readStream.pipe(csvConverter).pipe(writeStream);
`

The constructResult:false will tell the constructor not to combine the final result which would drain the memory as progressing. The output is piped directly to writeStream.

Convert Big CSV File with Command line tool

`
csvtojson [path to bigcsvdata] > converted.json
`

Or if you prefer streaming data in from another application:

`
cat [path to bigcsvdata] | csvtojson > converted.json
`

They will do the same job.

Column Array

`js
var columArrData=__dirname+"/data/columnArray";
var rs=fs.createReadStream(columArrData);
var result = {}
var csvConverter=new CSVAdv();
//end_parsed will be emitted once parsing finished
csvConverter.on("end_parsed", function(jsonObj) {
console.log(result);
console.log("Finished parsing");
done();
});

//record_parsed will be emitted each time a row has been parsed.
csvConverter.on("record_parsed", function(resultRow, rawRow, rowIndex) {

for (var key in resultRow) {
if (!result[key] || !result[key] instanceof Array) {
result[key] = [];
}
result[key][rowIndex] = resultRow[key];
}

});
rs.pipe(csvConverter);
`

Here is an example:

TIMESTAMP,UPDATE,UID,BYTES SENT,BYTES RCVED
1395426422,n,10028,1213,5461
1395426422,n,10013,9954,13560
1395426422,n,10109,221391500,141836
1395426422,n,10007,53448,308549
1395426422,n,10022,15506,72125

It will be converted to:

`json
{
"TIMESTAMP": ["1395426422", "1395426422", "1395426422", "1395426422", "1395426422"],
"UPDATE": ["n", "n", "n", "n", "n"],
"UID": ["10028", "10013", "10109", "10007", "10022"],
"BYTES SENT": ["1213", "9954", "221391500", "53448", "15506"],
"BYTES RCVED": ["5461", "13560", "141836", "308549", "72125"]
}
`

Parse String

`js
var testData=__dirname+"/data/testData";
var data=fs.readFileSync(testData).toString();
var csvConverter=new CSVConverter();

//end_parsed will be emitted once parsing finished
csvConverter.on("end_parsed", function(jsonObj) {
//final result poped here as normal.
});
csvConverter.fromString(data,function(err,jsonObj){
if (err){
//err handle
}
console.log(jsonObj);
});

`

#Empowered JSON Parser
Since version 0.3.8, csvtojson now can replicate any complex JSON structure.
As we know, JSON object represents a graph while CSV is only 2-dimension data structure (table).
To make JSON and CSV containing same amount information, we need "flatten" some information in JSON.

Here is an example. Original CSV:

`
fieldA.title, fieldA.children[0].name, fieldA.children[0].id,fieldA.children[1].name, fieldA.children[1].employee[].name,fieldA.children[1].employee[].name, fieldA.address[],fieldA.address[], description
Food Factory, Oscar, 0023, Tikka, Tim, Joe, 3 Lame Road, Grantstown, A fresh new food factory
Kindom Garden, Ceil, 54, Pillow, Amst, Tom, 24 Shaker Street, HelloTown, Awesome castle

`
The data above contains nested JSON including nested array of JSON objects and plain texts.

Using csvtojson to convert, the result would be like:

`json
[{
"fieldA": {
"title": "Food Factory",
"children": [{
"name": "Oscar",
"id": "0023"
}, {
"name": "Tikka",
"employee": [{
"name": "Tim"
}, {
"name": "Joe"
}]
}],
"address": ["3 Lame Road", "Grantstown"]
},
"description": "A fresh new food factory"
}, {
"fieldA": {
"title": "Kindom Garden",
"children": [{
"name": "Ceil",
"id": "54"
}, {
"name": "Pillow",
"employee": [{
"name": "Amst"
}, {
"name": "Tom"
}]
}],
"address": ["24 Shaker Street", "HelloTown"]
},
"description": "Awesome castle"
}]
``

Here is the rule for CSV data headers:

* Use dot(.) to represent nested JSON. e.g. field1.field2.field3 will be converted to {field1:{field2:{field3:< value >}}}
* Use square brackets([]) to represent an Array. e.g. field1.field2[< index >] will be converted to {field1:{field2:[< values >]}}. Different column with same header name will be added to same array.
* Array could contain nested JSON object. e.g. field1.field2[< index >].name will be converted to {field1:{field2:[{name:< value >}]}}
* The index could be omitted in some situation. However it causes information lost. Therefore Index should NOT be omitted if array contains JSON objects with more than 1 field (See example above fieldA.children[1].employee field, it is still ok if child JSON contains only 1 field).

Since 0.3.8, JSON parser is the default parser. It does not need to add "\json\" to column titles. Theoretically, the JSON parser now should have functionality of "Array" parser, "JSONArray" parser, and old "JSON" parser.

This mainly purposes on the next few versions where csvtojson could convert a JSON object back to CSV format without losing information.
It can be used to process JSON data exported from no-sql database like MongoDB.

#Change Log

##0.3.8
* Empowered built-in JSON parser.
* Change: Use JSON parser as default parser.
* Added parameter trim in constructor. default: true. trim will trim content spaces.

##0.3.5
* Added fromString method to support direct string input

##0.3.4
* Added more parameters to command line tool.

##0.3.2
* Added quote in parameter to support quoted column content containing delimiters
* Changed row index starting from 0 instead of 1 when populated from record_parsed event

##0.3
* Removed all dependencies
* Deprecated applyWebServer
* Added construct parameter for Converter Class
* Converter Class now works as a proper stream object