Stream based parser for files with character separated values
npm install xsvxsv is a lightweight, dependency-free and pretty fast stream parser for data with character separated values like csv, tsv and similar.
It's deliberately written in plain old JavaScript and can be used with any node version capable of Stream.Transform.
It can deal with some of the stupid people do to csv/tsv files, like empty lines, linebreaks in quoted fields and even comments.
``javascript
var fs = require("fs");
var xsv = require("xsv");
var opts = {
sep: ",", // as string
quote: 0x22, // or charcode integer
escape: Buffer.from("\\"), // or as Buffer
header: true,
};
fs.createReadStream("file.csv").pipe(xsv(opts).on("data", function(data){
console.log(data);
}).on("end", function(){
console.log("done.");
}));
`
* sep - Separation character: string, Buffer or int of byte code. First character/byte only. Default: ,
* - Quote character: string, Buffer or int of byte code; null for to disable. First character/byte only. Default: " for csv or null for tsvescape
* - Escape character: string, Buffer or int of byte code; null for to disable. First character/byte only. Default: \ for csv or null for tsvheader
* - First line is the header: bool||object; true emits data as object, false as array. Give an object to set a custom header. Default: true
* - Number of lines to skip: int; Ignores this number of lines before starting to parse. Handy to override the header given you know the field order. Default: 0
* - Comment character: string, Buffer or int of byte code; null for to disable. First character/byte only; Ignores all lines starting with this character. Default: \ for csv or null for tsvunescape
* - Transfer escape sequences (\t, \r, \n) in quoted fields to actual control characters: bool. Default: false
— Strip BOM from stream (if present, this will* mess up your data otherise): bool. Default: true
* — remove whitespace before and after values: bool. Default: false`
I was inspired to write this code because @substack blocked me after i asked nicely if they could fix the insecure minimist-dependency in optimist, which the also abandonned sv relies on. The nodejs ecosystem is presented to you by people being ignorant and hostile about maintaining their widely depended-upon code and abandonning their responsibility instead of transferring their orphaned code into careful long term maintainance.