Generate rich random data based on a JSON template file.
npm install mgeneratejs_mgeneratejs_ generates structured, semi-random JSON data according to a
template object. It offers both a command line script and a JavaScript API.
``
npm install -g mgeneratejs
`
mgeneratejs '{"name": "$name", "age": "$age", "emails": {"$array": {"of": "$email", "number": 3}}}' -n 5
Results in:
`
{"name":"Glenn Simmons","age":32,"emails":["ivuge@afovopid.tt","gied@orsin.zw","wuhowbi@con.uk"]}
{"name":"Jane Santiago","age":57,"emails":["oliclon@ohaoni.la","hetoufi@em.ug","ecwawce@sewwato.kn"]}
{"name":"Winifred Martinez","age":59,"emails":["veag@gi.fm","liwfecor@vifbevof.gr","siwluz@habif.gf"]}
{"name":"Helena Chandler","age":65,"emails":["ga@latcon.tr","wur@helmawak.im","ovpifuva@gabruzup.vc"]}
{"name":"Gary Allison","age":30,"emails":["wiko@unuwudu.za","fog@zokje.sh","juppojer@jadi.tl"]}
You can also specify a JSON file instead of a JSON string:
`
mgeneratejs template.json -n 5
The input string or file must be valid JSON, with one exception: As a convenience, from version 0.3.0 onwards, it's ok to omit quotes around keys, so both these templates are equivalent:
`
{"name": "$name"}
{name: "$name"}
The output has the same shape as the input template (including nested keys), with
one exception: If a key is assigned the special value $missing, then the$missing
key is not present in the output (see below for an example).
All values are taken literally, except for special $-prefixed values. These
values are called "operators". A list of operators can be found below.
Operators are used either in string or object format. The string format is
a shortcut to call the operator with default options.
String format:
`
{"key": "$operator"}
Object format:
`
{"key": {"$operator": {
Most operators have sensible default values that are used for their string format.
Example: $year
`
mgeneratejs '{"born_in": "$year"}' -n 5
`
{"born_in":"2035"}
{"born_in":"2086"}
{"born_in":"2088"}
{"born_in":"2022"}
{"born_in":"2082"}
The object format allows to pass in additional options to the operator,
here, a minimum and maximum for the value:
`
mgeneratejs '{"born_in": {"$year": {"min": 1930, "max": 1970}}}'
`
{"born_in":"1936"}
{"born_in":"1953"}
{"born_in":"1964"}
{"born_in":"1932"}
{"born_in":"1943"}
See the definition of the operator for its default values.
Operators can be combined, where the result of one operator is passed in as
an option to another operator.
Example: Here we pass in a random number between 0 and 5 to the number option$array
of the operator to generate variable-length arrays.
`
mgeneratejs '{"ip_addresses": {"$array": {"of": "$ip", "number": {"$integer": {"min": 0, "max": 5}}}}}'
`
{"ip_addresses":["166.182.72.83","127.94.56.191","236.79.131.157","94.66.121.242"]}
{"ip_addresses":["48.227.145.186","160.173.45.84","24.86.124.235"]}
{"ip_addresses":[]}
{"ip_addresses":["21.45.212.198"]}
{"ip_addresses":["199.209.162.241"]}
#### General
- $array: Creates an array of values.
- $choose: Chooses one element from an array of possible choices.
- $inc: Generates natural numbers in increasing order.
- $join: Joins elements of an array to a string.
- $maxcard: Limits the cardinality of a random value.
- $pick: Returns an element from an array.
- $pickset: Returns a subset of an array.
#### Geospatial
- $coordinates: Returns a pair of longitude/latitude coordinates.
- $point: Returns a GeoJSON Point.
- $linestring: Returns a GeoJSON LineString.
- $polygon: Returns a GeoJSON Polygon.
- $geometries: Returns a GeoJSON GeometryCollection.
#### Native and MongoDB-specific Types
- $binary: Returns a MongoDB Binary type.
- $date: Returns a random date, optionally in a given range.
- $now: Returns the current date.
- $maxkey: Returns a MongoDB MaxKey object.
- $minkey: Returns a MongoDB MinKey object.
- $numberDecimal: Returns a MongoDB Decimal128 number.
- $numberLong: Returns a MongoDB Long (Int64) number.
- $numberInt: Returns a Int32 number.
- $objectid: Returns MongoDB ObjectID.
- $regex: Returns a Regular Expression object.
- $timestamp: Returns a MongoDB Timestamp.
Creates an array of values. Each new element is evaluated separately.
_Options_
- of (required) Defines an element of the array. Operators are evaluated separately for each element.number
- (optional) Number of elements. Default 0.
> Example
>
> `
> {"countries": {"$array": {"of": {"$country": {"full": true}}, "number": 3}}}
>
>
> Creates an array of 3 countries, e.g.
Returns a random MongoDB Binary value, optionally with a length and subtype.
_Options_
- length (optional) Length in bytes of binary value. Default 10.subtype
- (optional) Specific binary subtype (see [BSON spec][bson-spec]). Default 0.
> Example
>
> `
> {"blob": "$binary"}
>
>
> Returns a Binary object (stringified to extended JSON on stdout).
> e.g. .
Chooses one element from an array of possible choices with uniform probability.
Optionally chooses with probability proportional to a provided weights array.
_Options_
- from (required) Array of values or operators to choose from.weights
- (optional) Number of elements. Default 0.
> Example
>
> `
> {"status": {"$choose": {"from": ["read", "unread", "deleted"], "weights": [2, 1, 1]}}}
>
>
> Returns with probability 1/2, and {"status": "unread"} and{"status": "deleted"}
> each with probability 1/4.
Returns a 2-element array of longitude/latitude coordinates, optionally within
long_lim and/or lat_lim bounds.
_Aliases_
- $coord
-
_Options_
- long_lim (optional) Array of longitude bounds. Default [-180, 180].lat_lim
- (optional) Array of latitude bounds. Default [-90, 90].
> Example
>
> `
> {"position": {"$coordinates": {"long_lim": [-20, -19]}}}
>
>
> Returns a pair of coordinates with the longitude bounds between -20 and -19,
> e.g. .
Returns a random date object, optionally between specified min and max values.min
If and/or max are provided, they need to be in a format that [Date.parse()][date-parse]
can read, e.g. ISO-8601.
_Aliases_
- $datetime
_Options_
- min (optional) Minimum date, as parseable string.max
- (optional) Maximum date, as parsable string.
> Example
>
> `
> {"last_login": {"$date": {"min": "2015-01-01", "max": "2016-12-31T23:59:59.999Z"}}}
>
>
> Returns a random date and time between 2015 and 2016 (incl.), e.g.
> .
Returns a GeoJSON formatted GeometryCollection
with number geometries. By default, the geometries are chosen from Point,LineString, and Polygon. A subset of types can be specified with the types
option.
Additional options are passed onto each geometry, e.g. corners is passedlocs
to polygons, is passed to line strings.
_Options_
- number (optional) Number of geometries in the collection. Default 3.types
- (optional) Types of geometries to choose from. Default ["Point", "LineString", "Polygon"].locs
- (optional) Number of locations in a line string. Default 2.corners
- (optional) Number of corners in a polygon. Default 3. The last point in the coordinates array closes the polygon and does not count towards the number of corners.long_lim
- (optional) Array of longitude bounds. Default [-180, 180].lat_lim
- (optional) Array of latitude bounds. Default [-90, 90].
> Example
>
> `
> {"triangles": {"$geometries": {"types": ["Polygon"], "corners": 3, "number": 4}}}
>
>
> Returns a GeoJSON GeometryCollection with 4 triangles.
>
>
> {
> "triangles": {
> "type": "GeometryCollection",
> "geometries": [
> {
> "coordinates": [[[39.3259,-16.71813],[172.02089,-14.75681],[61.97122,-1.4036],[39.3259,-16.71813]]],
> "type": "Polygon"
> },
> {
> "coordinates": [[[57.66865,-18.3085],[-48.81722,-40.64912],[-145.11102,32.8189],[57.66865,-18.3085]]],
> "type": "Polygon"
> },
> {
> "coordinates": [[[110.68379,28.31158],[-73.67573,-19.54736],[-73.29514,52.07583],[110.68379,28.31158]]],
> "type": "Polygon"
> },
> {
> "coordinates": [[[-29.36382,79.19853],[138.84298,7.43148],[176.28313,36.83292],[-29.36382,79.19853]]],
> "type": "Polygon"
> }
> ]
> }
> }
>
Generate natural numbers in increasing order.
_Options_
- start (optional) starts counting at this value. Default 0.step
- (optional) increases by this amount each time. Default 1. Can also take negative value.
> Example
>
> `
> {"even_numbers": {"$inc": {"start": 0, "step": 2}}}
>
>
> Assigns the numbers 0, 2, 4, 6, ... to subsequent objects.
Takes an array array and a separator string sep and joins the elementssep
of the array (each cast to string) separated by . The default separator
is the empty string ''.
_Options_
- array (required) Array of values to be joined (cast to string).sep
- (optional) Separator string. Default '' (empty string).
> Example
>
> `
> {"code": {"$join": {"array": ["foo", "bar", "baz"], "sep": "-"}}}
>
>
> Returns .
Returns a GeoJSON formatted LineString
with optionally locs locations and within long_lim and/or lat_lim bounds.
_Options_
- locs (optional) Number of locations in the line string. Default 2.long_lim
- (optional) Array of longitude bounds. Default [-180, 180].lat_lim
- (optional) Array of latitude bounds. Default [-90, 90].
> Example
>
> `
> {"line": "$linestring"}
>
>
> Returns a GeoJSON line string with 2 locations,
> e.g. .
Limits the number of different values of a random generator to the specified maximum.
_Options_
- value The expression from which to draw the samples.max
- The upper limit of different values.
> Example
>
> `
> mgeneratejs '{"top_5_animals": {"$maxcard": {"value": "$animal", "max": 5}}}' -n 100
>
>
> Even though we are generating 100 documents, there will be at most 5 different animals
> in the data.
Returns the MongoDB MaxKey value.
> Example
>
> `
> {"upper_bound": "$maxkey"}
>
>
> Returns .
Returns the MongoDB MinKey value.
> Example
>
> `
> {"lower_bound": "$minkey"}
>
>
> Returns .
Returns the current date at creation time. Ideal for time-stamping documents.
_Options_
_none_
> Example
>
> `
> {"created": "$now"}
>
>
> Returns the extended JSON date and time at creation.
> .
Returns a MongoDB Decimal128 number.
_Aliases_
- $decimal
_Options_
- min (optional) minimum value. Default 0.max
- (optional) maximum value. Default 1000.fixed
- (optional) number of digits after the decimal. Default 2.
> Example
>
> `
> {"price": {"$numberDecimal": {"fixed": 3}}}
>
>
> Returns .
Returns a MongoDB Long (Int64) number.
_Aliases_
- $long
_Options_
- min (optional) minimum value. Default -2^53.max
- (optional) maximum value. Default 2^53.
> Example
>
> `
> {"price": {"$numberLong": {"min": 100000}}}
>
>
> Returns .
Returns ag 32-bit integer number.
_Aliases_
- $number
-
_Options_
- min (optional) minimum value. Default -2^31.max
- (optional) maximum value. Default 2^31.
> Example
>
> `
> {"price": {"$numberLong": {"min": 100000}}}
>
>
> Returns .
Returns a new MongoDB ObjectId.
_Aliases_
- $oid
> Example
>
> `
> {"_id": "$objectid"}
>
>
> Returns .
Takes an array and a number element and returns the element-th value of$missing
the array. If the number is larger than the length of the array, return instead, which will remove the key from the resulting document.element is zero-based (0 returns the first element).
_Options_
- array (required) Array of values or operators to choose from.element
- (optional) Index of the array element to pick. Default 0.
> Example
>
> `
> {"color": {"$pick": {"array": ["green", "red", "blue"], "element": 1}}}
>
>
> Returns .
Takes an array and a number quantity and returns a new n-element array$missing
containing unique values from the input array. If the number is larger than the
length of the array, return instead, which will remove the key from
the resulting document.
_Options_
- array (required) Array of values or operators to choose from.quantity
- (optional) The size of the output array. Default 1.
> Example
>
> `
> {"color": {"$pickset": {"array": ["green", "red", "blue"], "quantity": 2}}}
>
>
> Returns
Like $coordinates, but returns a GeoJSON formattedlong_lim
Point, optionally within and/or lat_lim bounds.
_Options_
- long_lim (optional) Array of longitude bounds. Default [-180, 180].lat_lim
- (optional) Array of latitude bounds. Default [-90, 90].
> Example
>
> `
> {"position": {"$point": {"long_lim": [-20, -19]}}}
>
>
> Returns a GeoJSON Point with the longitude bounds between -20 and -19,
> e.g. .
linestring: require('./linestring'),
polygon: require('./polygon'),
geometries: require('./geometries'),
Returns a GeoJSON formatted Polygon
(without holes) with corners corners, optionally within long_lim and/orlat_lim bounds. The last point in the coordinates array closes the polygon
and does not count towards the number of corners.
_Options_
- corners (optional) Number of corners in the polygon. Default 3.long_lim
- (optional) Array of longitude bounds. Default [-180, 180].lat_lim
- (optional) Array of latitude bounds. Default [-90, 90].
> Example
>
> `
> {"area": {"$polygon": {"corners": 5}}}
>
>
> Returns a GeoJSON polygon with 5 corners,
> e.g. .
Returns a [RegExp][regexp] object.
_Options_
- string (optional) The regular expression string. Default '.*'.flags
- (optional) Flags for the RegExp object. Default ''.
> Example
>
> `
> {"expr": {"$regex": {"string": "^ab+c$", "flags": "i"}}}
>
>
> Returns .
Returns a MongoDB Timestamp object.
_Options_
- t (optional) Set the low value to the specified value. Default random.i
- (optional) Set the high value to the specified value. Default random.
> Example
>
> `
> {"ts": {"$timestamp": {"t": 10, "i": 20}}}
>
>
> Returns .
All other $-prefixed strings that don't match any of the built-in operators aboveChance.js
are passed on to the [][chance-js] library. Use the string format for
default options, or pass in custom options with the object format.
Some Examples:
`
{"ip_address": "$ip"}
{"percent": {"$floating": {"min": 0, "max": 100, "fixed": 8}}}
{"birthday": {"$birthday": {"type": "child"}}}
{"phone_no": "$phone"}
{"full_name": {"$name": {"gender": "female"}}}
TBD.
In short, you can use handlebar template strings to build even more complex
values, e.g.
`
mgeneratejs '{"recipient": "{{chance.name()}} <{{chance.email()}}>"}' -n 3
`
{"recipient":"Lora Jimenez
{"recipient":"Elnora Brewer
{"recipient":"Howard Bryan
This is a JavaScript port from the [mgenerate][mgenerate-mtools] script in the
[mtools][mtools] library (of which I am also the author). It is mostly backwards
compatible except for the following breaking changes:
1. The "array" operator format is no longer supported, as it was confusing
which arguments need to be provided in which order. Instead, use the "object"
format with named options. See [array shortcut syntax][array-syntax].
2. The "$concat" operator has been renamed to "$join", as this operation is
called "join" in many languages, e.g. Python and JavaScript. "\$concat" is
reserved for a future operator to concatenate arrays.
3. mgeneratejs does not insert documents directly into MongoDB, it only outputs
to stdout. It doesn't make sense to re-implement all the authentication options
separately, when the resulting objects can simply be piped to mongoimport.
In addition, many more operators are supported through the inclusion of
the Chance.js` library, and the extended template syntax with handlebar templates.
Apache 2.0
[mgenerate-mtools]: https://github.com/rueckstiess/mtools/wiki/mgenerate
[array-syntax]: https://github.com/rueckstiess/mtools/wiki/mgenerate#parsing-the-json-document
[mtools]: https://github.com/rueckstiess/mtools/
[chance-js]: http://chancejs.com/
[regexp]: https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/RegExp
[bson-spec]: http://bsonspec.org/spec.html
[date-parse]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse
[travis_img]: https://img.shields.io/travis/rueckstiess/mgeneratejs.svg
[travis_url]: https://travis-ci.org/rueckstiess/mgeneratejs
[npm_img]: https://img.shields.io/npm/v/mgeneratejs.svg
[npm_url]: https://npmjs.org/package/mgeneratejs