Manage binary data with JavaScript strictly typed Object-oriented programming
npm install @3m1/binary-object!Coverage
npm:shell
npm install binary-object
`With
:
`shell
yarn add binary-object
`Usage
$3
This library uses and TextDecoder to transform text to and from binary data. These are JavaScript native functions, but Node lacks them. You need to polyfill them first:
`javascript
if(!('TextEncoder' in global)) {
import('util').then(nodeUtil => {;
global.TextEncoder = nodeUtil.TextEncoder;
global.TextDecoder = nodeUtil.TextDecoder;
});
}
`For this to work on Node >= 14, you need to install
first:
`shell
npm install --save util
`$3
This library encourages the use of class member decorators, available in Typescript, but at a stage 2 proposal. To add it into your configuration, you will need something like:`json
"plugins": [
"@babel/plugin-transform-runtime",
[ "@babel/plugin-proposal-decorators", { "legacy": true } ],
[ "@babel/plugin-proposal-class-properties", { "loose": true } ]
]
`If you don't want to use decorators, you will need to use it like (not tested):
javascript
class MyBinaryClass extends Binary { / Functions/logic/statics / }
Object.defineProperty(MyBinaryClass.prototype, "someMember", binary(Types.Float32));
`$3
See autogenerated API documentation at this repo's GitHub Page.$3
`javascript
import {Binary, binary, Types} from 'binary-object';// Declare object shape
class BinaryTest extends Binary {
@binary(Types.Uint32)
id = 0;
@binary(Types.Float64)
testFloat = 0;
// Array of 10 low precision floating point numbers
@binary(Types.Array(Types.Float32, 10))
testFloatArray;
showId = () => console.log(
My id is ${this.id});
}// Allocate memory
const binTest = new ArrayBuffer( BinaryTest.binarySize );
// Instantiate memory parser
const proxyObject = new BinaryTest(binTest);
// Use it
proxyObject.id = 12345;
proxyObject.testFloat = 7.8;
expect(proxyObject.id).toBe(12345);
expect(proxyObject.testFloat).toBe(7.8);
`Accessing and modifying arrays and array elements also work:
javascript
const testArray = proxyObject.testFloatArray;// Iteratibility
expect([...testArray]).toEqual(new Array(10).fill(0));
// Elements access and modification
expect(testArray[0]).toBe(0);
testArray[0] = 1;
expect(testArray[0]).toBe(1);
expect(testArray[1]).toBe(0);
testArray[1] = 2;
expect(testArray[0]).toBe(1);
expect(testArray[1]).toBe(2);
// Modify full array
const newArr = [1,0,0,0,0,0,0,0,0,1];
proxyObject.testFloatArray = newArr;
expect([...testArray]).toEqual(newArr);
`You can define padded arrays for better performance and, maybe, enforced by API:
javascript
const type = Types.Uint32;
const length = 5;
class BinaryPadTest extends Binary {
@binary(Types.Uint8)
someMemberAtTheBegining; // Here,
true reffers to padding
@binary(Types.Array(type, length, true))
testArray; @binary(Types.Uint8)
someMemberAtTheEnd;
}
const binTest2 = new ArrayBuffer(BinaryPadTest.binarySize);
const testObj = new BinaryPadTest(binTest2);
// The first byte of someMemberAtTheBegining forces to consume
// a full testArray element before it (for padding)
const expectedSize = ((length + 1) * type.bytes) + 1;
expect(BinaryPadTest.binarySize).toBe(expectedSize);
expect(testObj.testArray).toBeInstanceOf(type.extractor); // Uint32Array
`Object composition is also allowed:
javascript
class BinaryArrayOfNestedTest extends Binary {
// Array of 3 BinaryTest objects
@binary( Types.Array( Types.Struct(BinaryTest), 3))
testNested; get id() { return this.testNested[0].id }
set id(value) {
this.testNested[0].id = value;
return true
}
@binary(Types.Uint32)
someNumber = 0;
showId = this.testNested[0].showId;
}
const binTest2 = new ArrayBuffer( BinaryArrayOfNestedTest.binarySize );
const proxyNested = new BinaryArrayOfNestedTest( binTest2 );
`Object composition from different memory sources is also allowed:
javascript
class Position extends Binary {
@binary(Types.Uint32)
x;
@binary(Types.Uint32)
y; static testCollision(pos1, pos2) {
...
}
collision = (pos2) => this.constructor.testCollision(this, pos2);
}
class Player extends Binary {
@binary( Types.Struct(Position) )
position;
@binary(Types.Float64)
life;
// Non managed binary data (like pointers)
bullets = [];
}
class Bullet extends Binary {
@binary( Types.Struct(Position) )
position;
@binary(Types.Float64)
direction;
}
class Game {
constructor() {
// malloc for players
this.binPlayers = new ArrayBuffer( Player.binarySize * 2 );
this.player1 = new Player( this.binPlayers );
this.player2 = new Player( this.binPlayers, Player.binarySize );
// malloc for bullets
this.maxBullets = 100;
this.binBullets = new ArrayBuffer( Bullet.binarySize * this.maxBullets );
// Optimize bullets by using a unique DataView for all of them
this.dvBullets = new DataView(this.binBullets);
// Half of the bullets for each player
Bullets.arrayFactory(
this.dvBullets,
maxBullets / 2,
0,
player1.bullets);
Bullets.arrayFactory(
this.dvBullets,
maxBullets / 2,
maxBullets / 2,
player2.bullets);
}
// You can move Bullets using a parallel worker or a WASM code block
// Sometimes, check if a bullet touched a victim
testTouched(attacker, victim) {
const {position: {collision}} = victim;
const touched = attacker.bullets.some( ({position}) => collision(position) );
}
}
`Transform your binary data into a JSON string:
javascript
console.log( JSON.stringify(proxyNested) );
`Assign data from JS objects:
javascript
proxyNested.testNested = {testFloatArray: [0,0,0,0,0,0,0,0,0,1]};
expect( [...proxyNested.testNested.testFloatArray] ).toEqual([0,0,0,0,0,0,0,0,0,1]);
`Allocate and parse a big memory chunk as array of objects:
javascript
const iterations = 5e5;
const binTest3 = new ArrayBuffer( BinaryTest.binarySize * iterations );
const dv3 = new DataView(binTest3);
const bObjList = new Array(iterations);
for(let i = 0; i < iterations; i++) {
bObjList.push(new BinaryTest(dv3, BinaryTest.binarySize * i));
}
bObjList.forEach( (obj, index) => obj.id = i${index});
const ids = bObjList.map( ({id}) => id);
`Memory owner
This library does not aims to own the memory, allowing to use it in different ways. The memory can be allocated from a parallel worker, from the WebAssembly code or from another library or API which can
be consumed using or SharedBuffer. It's recommended to wrap this library with the one managing the memory.Note: Opting in into managing your own memory (as in C) requires you to understand what you are doing at a low level. Things like dynamic array sizes or text manipulation can be a pain if you don't
understand the basics (not to say about endianness or memory padding). There are several standards and known ways to manage the memroy. This library aims to make it much easier to parse it (read and write),
but you'll still need to know how it works. Specifically, this libs does not aims to own the memory pieces: you'll need to create/allocate/move/copy/reallocate/free them around instantiating the JS binary objects.
Use cases
The environments where direct memory management would be desirable are minimal in JS. Still, there are some edge cases where it could be benefficial or mandatory to do so.$3
WebAssembly allows coding in some low level language (like C, C++ or Rust) and compiling to some binary code which can be executed by a JS interpreter (with WebAssembly API, of course).The code compiled into WebAssembly can use dedicated libraries to communicate with the JS code. It allows accessing the DOM (thus, the
window object), but also sharing pieces of memory
using WebAssembly.Memory. Using that, you could write complex but very fast code to do heavy calculations in your compiled code, and trigger some kind of JS re-render using the currently
calculated values at some time or frame-rate.$3
As app states is mostly managed by the ArrayBuffer (except some JS internal caches), you can minimize GC works, which will run your game (app) much smoothly without unexpected background GC tasks.$3
Similar to the WebAssembly use case, some one could create a SharedBuffer and use it from different Workers, allowing your app to use more than one CPU. For example in a game: you could have
the main worker which updates the DOM (or a canvas or a OpenGL) using the data from the buffer, while having several workers calculating 3D collisions.$3
As all the data is in a single memory piece, you can easily save it to somewhere and load it later to restore the state. This could be benefficial for games, as well as AIs, 3D renderers, scientific
calculations, password crackers, etc.$3
Dynamic binary reading, processing and writing (to server or directly to user), like images, audio, video, medical data (DICOM), etc.$3
You could take full advantage of browsers USB and/or Bluetooth APIs, but also you could easily communicate against binary API servers or IoT. Though, as those are usually read-only or
stream based, you would preffer using DataStreams.js` instead.