NJSON - next-json

Next JavaScript Object Notation

[![Build Status][travis-badge]][travis-url]
[![NPM version][npm-badge]][npm-url]
[![Types][types-badge]][npm-url]
[![Dependents][deps-badge]][deps-url]

[![Code Climate][code-badge]][code-url]
[![Test Coverage][cover-badge]][code-url]
[![NPM downloads][npm-downloads-badge]][npm-url]

[![Stars][stars-badge]][stars-url]
[![Donate][donate-badge-ada]][donate-url-ada]
[![Donate][donate-badge-btc]][donate-url-btc]
[![Donate][donate-badge-eth]][donate-url-eth]

[code-badge]: https://codeclimate.com/github/iccicci/next-json/badges/gpa.svg
[code-url]: https://codeclimate.com/github/iccicci/next-json
[cover-badge]: https://codeclimate.com/github/iccicci/next-json/badges/coverage.svg
[deps-badge]: https://img.shields.io/librariesio/dependents/npm/next-json?logo=npm
[deps-url]: https://www.npmjs.com/package/next-json?activeTab=dependents
[donate-badge-ada]: https://img.shields.io/static/v1?label=donate&message=cardano&color=blue&logo=cardano
[donate-badge-btc]: https://img.shields.io/static/v1?label=donate&message=bitcoin&color=blue&logo=bitcoin
[donate-badge-eth]: https://img.shields.io/static/v1?label=donate&message=ethereum&color=blue&logo=ethereum
[donate-url-ada]: https://cardanoscan.io/address/DdzFFzCqrhsxfKAujiyG5cv3Bz7wt5uahr9k3hEa8M6LSYQMu9bqc25mG72CBZS3vJEWtWj9PKDUVtfBFcq5hAtDYsZxfG55sCcBeHM9
[donate-url-btc]: https://www.blockchain.com/explorer/addresses/btc/3BqXRqgCU2CWEoZUgrjU3b6VTR26Hee5gq
[donate-url-eth]: https://www.blockchain.com/explorer/addresses/eth/0x8039fD67b895fAA1F3e0cF539b8F0290EDe1C042
[github-url]: https://github.com/iccicci/next-json
[npm-downloads-badge]: https://img.shields.io/npm/dw/next-json?logo=npm
[npm-badge]: https://img.shields.io/npm/v/next-json?color=green&logo=npm
[npm-url]: https://www.npmjs.com/package/next-json
[stars-badge]: https://img.shields.io/github/stars/iccicci/next-json?logo=github&style=flat&color=green
[stars-url]: https://github.com/iccicci/next-json/stargazers
[travis-badge]: https://img.shields.io/travis/com/iccicci/next-json?logo=travis
[travis-url]: https://app.travis-ci.com/github/iccicci/next-json
[types-badge]: https://img.shields.io/static/v1?label=types&message=included&color=green&logo=typescript

Why this package?

Because JSON is awesome, but...

JSON is awesome mainly for two reasons:

1. it offers an easy way to serialize and deserialize complex data;
2. a valid JSON encoded string can be pasted in a JavaScript source file, a really awesome feature while developing /
debugging.

... but it has some limitations:

- ❌ loses undefined, NaN, Infinity, -0
- ❌ throws TypeError serializing circular references
- ❌ cannot handle BigInt, Date, Error, Map, RegExp, Set, URL
- ❌ doesn't support many other features...

This package is intended to offer something as great as JSON... trying to add something more.

> The main reason why NJSON was born is to provide a way to serialize data much more complex than what JSON can
> serialize, while maintaining the ability to copy and paste the serialized string into any JavaScript environment
> without the need of any additional library - something I strongly want while debugging / developing.

NJSON Features

- ✔ extends JSON
- ✔ safe parser: doesn't use eval
- ✔ JavaScript compatible: same result from parse and eval
- ✔ includes TypeScript types
- ✔ supports C style comments
- ✔ supports escaped new line in strings
- ✔ supports trailing commas
- ✔ supports circular and repeated references
- ✔ supports sparse arrays
- ✔ supports undefined
- ✔ supports -0, NaN and Infinity
- ✔ supports BigInt
- ✔ supports Date
- ✔ supports Error (with exception)
- ✔ supports Map
- ✔ supports RegExp
- ✔ supports Set
- ✔ supports TypedArrays (but Float16Array)
- ✔ supports URL

Example

``javascript
const set = new Set();
const arr = [, , NaN, , , set, , ,];

arr.push(arr);
set.add(arr).add(set);

const serialized = NJSON.stringify(arr);
const parsed = NJSON.parse(serialized);

console.log(parsed === parsed[8]); // true
console.log(Object.hasOwn(parsed, 7)); // false
console.log(isNaN(parsed[2])); // true
console.log(parsed[5] instanceof Set); // true
console.log(parsed === [...parsed[5]][0]); // true
console.log(serialized);
// ((A,B)=>Object.assign(B,{"5":A.add(B).add(A),"8":B}))(new Set(),[,,NaN])
`

Table of content

- Documentation
- NJSON extends JSON
- NJSON parser
- Usage
- JavaScript
- TypeScript
- Polyfills
- Server side
- Client side
- Not supported by design
- ArrayBuffer
- Function
- Symbol
- TypedArray
- The Error exception
- replacer / reviver
- Array
- Map
- Set
- TypedArray
- circular / repeated references
- Versions
- v0.5.0
- v0.5.1
- Installation
- MIME type
- API reference
- NJSON.empty
- [NJSON.parse(text[, reviver])](#njsonparsetext-reviver)
- [NJSON.parse(text[, options])](#njsonparsetext-options)
- [NJSON.stringify(value[, replacer[, space]])](#njsonstringifyvalue-replacer-space)
- [NJSON.stringify(value[, options])](#njsonstringifyvalue-options)
- interface NjsonParseOptions
- NjsonParseOptions.numberKey
- NjsonParseOptions.reviver
- interface NjsonStringifyOptions
- NjsonStringifyOptions.date
- NjsonStringifyOptions.numberKey
- NjsonStringifyOptions.omitStack
- NjsonStringifyOptions.replacer
- NjsonStringifyOptions.sortKeys
- NjsonStringifyOptions.space
- NjsonStringifyOptions.stringLength
- NjsonStringifyOptions.undef
- [expressNJSON([options])](#expressnjsonoptions)
- interface ExpressNjsonOptions
- ExpressNjsonOptions.parse
- ExpressNjsonOptions.stringify
- [Express.Response.njson(body[, options])](#expressresponsenjsonbody-options)
- [fetchNJSON([options])](#fetchnjsonoptions)
- [Response.njson([options])](#responsenjsonoptions)
- Compatibility
- TypeScript
- License
- Bugs
- Donating
- See also

Documentation

NJSON extends JSON

This doesn't mean it's 100% compliant: due its higher number of supported features the result string of the
serialization through NJSON.stringify may differs from the result of the serialization through JSON.stringify.

On the other hand, the result of the deserialization of a _valid JSON encoded string_ through NJSON.parse will
produce a value _deep equal_ to the value produced by JSON.parse and the reviver function will be called the same
amount of times, with the same parameters and in the same order.

Note: the reviver function still not implements the newly added context argument.

Taken the result of a JSON.parse call (i.e. a value which contains only _valid JSON values_), if serialized through
JSON.stringify or NJSON.stringify produces two equal strings and the replacer function will be called the same
amount of times, with the same parameters and in the same order.

NJSON parser

NJSON offers its own parser which means it doesn't use eval with its related security hole.

Even if the NJSON serialized string is _JavaScript compliant_, NJSON.parse is not able to parse any JavaScript
code, but only the subset produced by NJSON.stringify (otherwise it would have been another eval implementation).

Usage

$3

`javascript
import { NJSON } from "next-json";

const serialized = NJSON.stringify({ some: "value" });
const deserialized = NJSON.parse(serialized);
`

$3

`typescript
import { NJSON, NjsonParseOptions, NjsonStringifyOptions } from "next-json";

const serialized = NJSON.stringify({ some: "value" });
const deserialized = NJSON.parse<{ some: string }>(serialized);
`

Polyfills

$3

`javascript
import express from "express";
import { expressNJSON } from "next-json";

const app = express();

app.use(expressNJSON()); // install the polyfill
app.post("/mirror", (req, res) => res.njson(req.body)); // there is an 'n' more than usual
app.listen(3000);
`

$3

`javascript
import { NJSON, fetchNJSON } from "next-json";

fetchNJSON(); // install the polyfill

const payload = { infinity: Infinity };
payload.circular = payload;

const response = await fetch("http://localhost:3000/mirror", {
body: NJSON.stringify(payload), // there is an 'N' more than usual
headers: { "Content-Type": "application/njson" }, // there is an 'n' more than usual
method: "POST"
});
const body = await response.njson(); // there is an 'n' more than usual
`

Here payload deep equals payload.circular, which deep equals body, which deep equals body.circular, which deep
equals req.body in server side, which deep equals req.body.circular in server side! 🎉

Not supported by design

NJSON do not supports some Objects by design; when one of them is encountered during the serialization process
NJSON tries to act as JSON does. Nonetheless they take part in the _repeated references algorithm_ anyway. Follow
the details.

$3

ArrayBuffers can't be manipulated by JavaScript design: they are serialized as empty objects as JSON does.

$3

NJSON is designed to _serialize_ / _deserialize_ complex data to be shared between different systems, possibly
written with other languages than JavaScript (once implementations in other languages will be written). Even if
JavaScript can see a function as a piece of data, it is actually code, not data. More than this, for other languages,
may be a complex problem to execute JavaScript functions.

Last but not least, allowing the deserialization of a function would open once again the security hole implied by the
use of eval, and one of the reasons why NJSON was born, is exactly to avoid that security hole.

$3

A Symbol is something strictly bound to the JavaScript execution environment which instantiates it: sharing it
between distinct systems is something almost meaningless.

$3

Note: except for Int8Array, Uint8Array and Uint8ClampedArray, TypedArrays are platform dependant: they are
supported (but Float16Array as it is not supported by Node.js), but trying to transfer one of them between
different architectures may be source of unexpected problems.

The Error exception

Error's are special objects. By specifications the properties cause, message, name and stack are not
enumerable, NJSON serializes them as any other property. This, plus the nature of the stack property, originates
the Error exception to the rule that an _NJSON encoded string_ produces exactly the same value if parsed or
evaluated.

- cause:
- through NJSON.parse the result is a not enumerable property;
- through eval the result may be an enumerable or a not enumerable property depending on the running
JavaScript engine;

- stack:
- if absent:
- through NJSON.parse the result is a not enumerable property with value a _pseudo-stack_;
- through eval the result is the standard stack property for the running JavaScript engine;

- if present:
- through NJSON.parse the result is a not enumerable property;
- through eval the result may be an enumerable or a not enumerable property depending on the running
JavaScript engine;

The only option in my mind to avoid this exception is the use of Object.defineProperties, but it would increase both
the complexity of the parser and the size of the produced serialized string. Maybe in the future... configurable
through an option... if this can't be really tolerated.

replacer / reviver

Even if Date, RegExp, TypedArrays and URL are Objects, they are treated as native values i.e. replacer and
reviver will be never called with one of them as this context.

$3

For Arrays the key argument is a positive integer, but in a String form for JSON compatibility. This can be
altered (i.e. in a Number form) through the numberKey option.

Sparse arrays: When JSON.stringify processes a sparse array, it calls the replacer function with undefined as
the value for missing elements. NJSON tries to maintain compatibility as much as possible by behaving the same way.
However, if undefined (or any other value) were used in the same way, the serialized array could no longer be a
sparse array. For this reason, NJSON.empty was introduced: a _pseudo-value_ used by NJSON to indicate that an array
element is empty.

$3

Map's keys can be Functions and Symbols; for Maps the key argument is a positive integer in a Number form
and the value argument is the entry in the form [mapKey, mapValue]. This gives a way to _replace_/_revive_ keys
which can't be serialized. If replacer or reviver do not return a two elements array, the value is omitted.

$3

For Sets the key argument is a positive integer and it is passed in a Number form.

$3

Unlike JSON, NJSON does not call replacer and reviver for each element.

circular / repeated references

Regardless of whether they are omitted, serialized as native values or not, every Objects (but null), Functions
and Symbols take part in the _repeated references algorithm_; long Strings can take part as well (refer to
NjsonStringifyOptions.stringLength for details).

When a repeated reference is encountered, replacer and reviver are called against the reference, but it is not
called recursively against its properties. If a property of a repeated reference is changed, the same change has effect
in all other occurrences of the same reference.

Circular references are simply special cases of repeated references.

Versions

Important changes introduced by versions.

$3

v0.5.0 changes the stringification of Arrays from the use of push to the use Object.assign with some
advantages:

- enables the support of _sparse arrays_;

- enables the use of _body-less functions_ rather than _functions_ which makes the parser thinner;

and one disadvantage:

- breaks the compatibility with previous versions stringification.

To make the change as smooth as possible, v0.5.0 still supports the deserialization of previous versions
stringification (it maintains backward compatibility): support for parsing push will be removed in some later version
TBD.

> If NJSON is used to produce volatile values through stringify that are immediately parsed and discarded, no
> actions are required.
>
> If some values stringified using NJSON.version < v0.5.0 are stored (in some DBs or in some files), it is
> recommended to convert them serialization to the newer format; to do that, just parse and re-stringify them using
> v0.5.0 <= NJSON.version < TBD).

$3

v0.5.1 introduces sparse arrays support. If sparse arrays need to be handled, it is mandatory to use this version.

It is backward compatible.

Installation

With npm:

`sh
npm install --save next-json
`

MIME type

The MIME type for NJSON format is: application/njson .

API reference

NJSON.empty

A symbol used to indicate an empty value in an Array context. When a replacer or reviver function returns this
symbol:

- In an Array context: the element is removed from the array (creating or maintaining a sparse array)
- Outside an Array context: a TypeError is thrown

NJSON.parse(text[, reviver])

Just for compatibility with JSON.parse. Alias for:

`typescript
NJSON.parse(text, { reviver });
`

NJSON.parse(text[, options])

Converts a Next JavaScript Object Notation (NJSON) string into an object.

- text: <string> The text
to deserialize.
- options: <NjsonParseOptions> Deserialization options.
- Returns: <unknown> The _value_ result of
the deserialization of the NJSON encoded text.

NJSON.stringify(value[, replacer[, space]])

Just for compatibility with JSON.stringify. Alias for:

`typescript
NJSON.stringify(value, { replacer, space });
`

NJSON.stringify(value[, options])

Converts a JavaScript value to a Next JavaScript Object Notation (NJSON) string.

- value: <unknown> The value to
serialize.
- options: <NjsonStringifyOptions> Serialization options.
- Returns: <string> The
NJSON encoded serialized form of value.

interface NjsonParseOptions

- numberKey:
<boolean>
Alters the type of the key argument for reviver. Default: false.
- reviver:
<Function> Alters the
behavior of the deserialization process. Default: null.

$3

If true, the reviver function, for Array elements, will be called with the key argument in a Number form.

$3

As the
reviver
parameter of JSON.parse. See also replacer / reviver for NJSON specific details.

Note: the reviver function still not implements the newly added context argument.

interface NjsonStringifyOptions

- date:
<string>
Specifies Dates conversion method. Default: "time".
- numberKey:
<boolean>
Alters the type of the key argument for replacer. Default: false.
- omitStack:
<boolean>
Specifies if to stringify stack for Errors. Default: false.
- replacer:
<Function> |
<Array> | null Alters
the behavior of the serialization process. Default: null.
- sortKeys:
<boolean>
Specifies whether to sort Object keys. Default: false.
- space:
<number> |
<string> | null Specifies
the indentation. Default: null.
- stringLength:
<number> | null Makes
Strings to be treated as references. Default: null.
- undef:
<boolean>
Specifies the undefined behavior. Default: true.

$3

Specifies the method of Date objects used to serialize them. Follows the list of the allowed values and the relative
method used.

- "iso": Date.toISOString()
- "string": Date.toString()
- "time": Date.getTime() - the default
- "utc": Date.toUTCString()

$3

If true, the replacer function, for Array elements, will be called with the key argument in a Number form.

$3

For default NJSON.stringify serializes the stack property for Errors. If set to true, the property is omitted
from the serialized representation.

$3

As the
replacer
parameter of JSON.serialize. See also replacer / reviver for NJSON specific details.

$3

For default NJSON stringifies (and replaces as well) Object keys in the order they appear in the Object itself.
If set to true, Object keys are sorted alphabetically before both the processes. This can be useful to compare two
references: using this option, the stringified representation of two deep equal references are two equal strings.

$3

As the
space
parameter of JSON.serialize.

$3

If specified, Strings which length is greater or equal to the specified value take part in the _repeated references
algorithm_.

$3

For default NJSON.stringify serializes undefined values as well. If set to false, undefined values are
treated as JSON.stringify does.

expressNJSON([options])

An Express middleware which works as NJSON body parser and installs the
Express.Response.njson method.

- options: <NjsonStringifyOptions> NJSON Express middleware options.
- Returns: Express middleware The NJSON Express middleware.

interface ExpressNjsonOptions

- parse: <NjsonParseOptions> The options passed to
NJSON.parse by the middleware to parse the request body.
- stringify: <NjsonStringifyOptions> The
default options passed to NJSON.stringify by
Express.Response.njson to serialize the response body.

$3

The options passed to NJSON.parse by the middleware to parse the request body.

$3

The default options passed to NJSON.stringify by
Express.Response.njson to serialize the response.

Express.Response.njson(body[, options])

Encodes the body in NJSON format and sends it; sets the proper Content-Type header as well. Installed by
expressNJSON.

- body: <unknown> The body to be sent
serialized.
- options: <NjsonStringifyOptions> The options passed to
NJSON.stringify to serialize the response body; overrides the default
options.stringify passed to expressNJSON.

fetchNJSON([options])

Installs the Response.njson method.

- options: <NjsonParseOptions> The default options
passed to NJSON.parse by Response.njson to parse the response
body.

Response.njson([options])

Parses the response body with NJSON.parse. Installed by
fetchNJSON.

- options: <NjsonParseOptions> The options passed to
NJSON.parse to parse the response body; overrides the default options passed to
fetchNJSON.
- Returns: <unknown> The body parsed with
NJSON.parse.

Compatibility

Requires Node.js v14.

Exception: fetchNJSON` requires Node.js v18.

The package is tested under all Node.js versions
currently supported accordingly to Node.js Release.

TypeScript

TypeScript types are distributed with the package itself.

License

MIT License

Bugs

Do not hesitate to report any bug or inconsistency @github.

Donating

If you find useful this package, please consider the opportunity to donate on one of following cryptos:

ADA: DdzFFzCqrhsxfKAujiyG5cv3Bz7wt5uahr9k3hEa8M6LSYQMu9bqc25mG72CBZS3vJEWtWj9PKDUVtfBFcq5hAtDYsZxfG55sCcBeHM9

BTC: 3BqXRqgCU2CWEoZUgrjU3b6VTR26Hee5gq

ETH: 0x8039fD67b895fAA1F3e0cF539b8F0290EDe1C042

See also

Other projects which aim to solve similar problems:

- arson by Ben Newman
- devalue by Rich Harris
- jsesc by Mathias Bynens
- lave by Jed Schmidt
- oson by Steffen Trog
- serialize-javascript by Eric Ferraiuolo
- superjson by Blitz
- tosource by Marcello Bastéa-Forte