Query (ND)JSON files on S3
npm install s3-query-json> Query JSON and NDJSON files on Amazon S3
The selectObjectContent API allows to easily query JSON and NDJSON data from S3. This node module provides a wrapper for this method, exposing the data as an aggregated result as a Promise.
See the official AWS documentation for more information.
``
$ npm install s3-query-json
`js
const {query} = require('s3-query-json');
query('foobarbaz', 'users.ndjson', 'SELECT s.name FROM S3Object s')
.then(data => console.log(data) // [{"name": "unicorn"}, {"name": "rainbow"}])
query('foobarbaz', 'users.ndjson', 'SELECT s.name FROM S3Object s', {documentType: 'NDJSON', compressionType: 'GZIP', stream: false, delimiter: '\n', scanRange: {start: 1, end: 100}})
.then(data => console.log(data) // [{"name": "unicorn"}, {"name": "rainbow"}])
`
#### bucket
Type: String
The name of the bucket containing your object.
#### key
Type: String
The key of the S3 object, either JSON or NDJSON format.
#### expression
Type: String
SQL query to be executed. Read the documentation on which SQL operations are allowed.
#### options
Type: Object
Configuration object.
##### [options.documentType]
Type: String
Default:
Accepts either JSON or NDJSON as input.
##### [options.compressionType]
Type: String
Default:
Compression used in the S3 object. Either NONE, GZIP or BZIP2
##### [options.stream]
Type: Boolean
Default:
Flag to indicate if the function returns a Promise with a original readstream. This allows more advanced custom processing.
##### [options.delimiter]
Type: string
Default:
Delimiter with a maximum length of 1 to separate the records that are being returned. To avoid difficult parsing, use a delimiter with no collision with the records returned.
##### [options.scanRange]
Type: Object
Default:
This property can only be used with non-compressed (ND)JSON files. Allows to scan a specified range of bytes. See the documentation for more information. When scanRange is provided, atleast one of keys start or end must be provided.
##### [options.scanRange.start]
Type: Number
Inclusive start of the range. Starts from 0
##### [options.scanRange.end]
Type: Number`
Inclusive end of the range. This is allowed to be out of bound of the object size.
MIT © Simon Jang