KvStorage

- Persistent key-value storage
- Node.js only, 16+ (no browser)
- API inspired by levelDB
- All keys are kept in memory
- Zero dependencies
- Small bundle size (10KB minified)
- Typescript + Javascript
- Concurrency is not supported (i.e. no multiple processes on the same file)
- Throttled writings to file

This class was designed to store small/medium data sets of JSON-able data types plus Date and Buffer,
where you can have the luxury of keeping keys in memory.
Small values are kept in memory as well (configurable threshold), while big ones are loaded on-demand.
You can fine-tune this to match your memory-usage vs performance on reading.

Installation

npm i @rejetto/kvstorage

API

By example

Write a value and read it later
``javascript
const { KvStorage } = require('@rejetto/kvstorage')
let db = new KvStorage()
// ...
await db.open('filename')
db.put('frank', { age: 33 })
setTimeout(async () => {
console.log(await db.get('frank'))
db.del('frank')
})
`

Same as above, but since we know data is not large, we can go sync
`javascript
db.put('frank', { age: 33 })
setTimeout(() => {
console.log(db.getSync('frank'))
db.del('frank')
})
`

Same as above, but different API
`javascript
const age = db.singleSync('age', 0)
aget.set(33)
setTimeout(() => {
console.log(age.get())
})
`

Methods

The any below actually means a value that can be JSON-encoded (plus Date and Buffer).

- constructor(options?)
- options:
- memoryThreshold: Above this number of bytes, a record is offloaded from memory (default: 1000).
- bucketThreshold: Above this number of bytes, a record will be stored in a separate but common file, called bucket (simple Buffers are saved as binaries) (default: 10000).
- fileThreshold: Above this number of bytes, a record will be stored in a dedicated file (simple Buffers are saved as binaries) (default: 100000).
- rewriteThreshold: Above this percentage (over the file size), a rewrite will be triggered at load time, to remove wasted space (default: 0.3).
- rewriteOnOpen: Enable rewriteThreshold on open(). (default: true)
- rewriteLater: Enable rewriteThreshold after open. (default: false)
- defaultPutDelay: default value for put(k,v,{delay}).
- maxPutDelay: default value for put(k,v,{maxDelay}).
- maxPutDelayCreate: default value for put(k,v,{maxDelayCreate}).
- reviver: A function passed to JSON.parse for custom deserialization (optional).
- keyToFileName: A function to customize the name of the files created by fileThreshold (optional). Including path-separator(s) in the name you can divide files in subfolders.
- open(path: string): Promise
- Opens the key-value store at the specified path. If clear is true, existing data will be deleted.
- isOpen(): boolean
- Current status.
- isOpening(): undefined | Promise
- Set while open() is ongoing.
- ready(): Promise
- You can await this promise to be sure it is open.
- get(key: string): Promise
- Get the value associated with the key, or undefined if not present.
- getSync(key: string): any
- Like get, but works only with values that are not offloaded, specifically those smaller than the memoryThreshold, bucketThreshold, and fileThreshold.
- put(key: string, value: any, { delay, maxDelay, maxDelayCreate }?): Promise
- Add/overwrite a value. Await-ing this is optional, unless you need to know that the file has been written.
- Adding a delay may save some writings for fast-changing values. Since the delay is renewed at each put, we can put a
limit to that, and can have a generic limit with maxDelay or one specific for keys that weren't written to disk yet,
as you may care more about the existence of a (persisted) record than its updates.
- del(key: string): Promise
- Remove a value. Await-ing this is optional, unless you need to know that the file has been written.
- has(key: string): boolean
- Check for existence
- size(): number
- Returns the number of key-value pairs stored.
- *iterator(options?): AsyncGenerator<[key, value]>
- Iterates over key-value pairs with optional filtering.
- *keys(options?): Generator
- Iterates over keys with optional filtering.
- firstKey(): string
- Return first entry of keys.
- sublevel(prefix: string): KvStorage-like
- Creates a sublevel key-value store with a specific prefix.
- close(): Promise
- Closes the key-value store, releasing resources.
- flush(): Promise
- Flushes any pending writes to disk.
- unlink(): Promise
- Deletes the entire key-value store at its location.
- clear(): Promise
- Equivalent to unlink + open.
- asObject(): Promise