A lightweight key-value storage library.
npm install medeaA persistent key-value storage library.
* Can be embedded in Node.js applications.
* Support for get, put, and remove operations.
* Influenced by Basho's Bitcask key-value store.
* Values can be any string or Buffer.
* Supports atomic batch writes.
* Allows snapshots for consistent views of data.
 
- Example
- Install
- Usage
- [db.open([directory], callback)](#dbopendirectory-callback)
- [db.get(key, [snapshot], callback)](#dbgetkey-snapshot-callback)
- db.put(key, value, callback)
- db.remove(key, callback)
- db.createSnapshot()
- db.createBatch()
- db.write(batch, callback)
- db.sync(callback)
- db.close(callback)
- db.listKeys(callback)
- db.mapReduce(options, callback)
- db.compact(callback)
- db.destroy(callback) / medea.destroy(director, callback)
- How It Works
- Keydir
- Data Files
- Hint Files
- Lock Files
- Limitations
- License
``javascript
var medea = require('medea');
var db = medea();
db.open(function(err) {
db.put('hello', 'world!', function(err) {
db.get('hello', function(err, val) {
console.log(val.toString());
db.close();
});
});
});
`
`bash`
$ npm install medea
Opens a Medea key-value store.
directory: Optional. Defaults to medea in the current directory.
callback: Takes one error parameter.
Returns the value associated with the given key.
key: identifier to retrieve the value
snapshot: a snapshot on which to query. optional.
callback: has the signature function (err, value) where value is the returned value.
Stores a value in Medea.
key: identifier
value: value associated with the key
callback: function that takes an error parameter
Removes an entry from Medea.
key: identifier for the item to remove
callback: Takes one error parameter.
Creates a transient snapshot of the data. Works with [db.get(key, [snapshot], callback)](#dbgetkey-snapshot-callback).
Returns a Snapshot object.
Example:
`js`
db.put('hello', 'world', function(err) {
var snapshot = db.createSnapshot();
db.put('hello', 'void', function(err) {
db.get('hello', snapshot, function(err, val) {
assert.equal(val.toString, 'world');
});
});
});
Creates a batch object for atomic writes. Works with db.write(batch, callback).
Returns a WriteBatch object.
Commits put and remove operations atomically.
batch: a WriteBatch object created with db.createBatch().
callback: Takes one error parameter.
Example:
`js
var batch = db.createBatch();
batch.put('key1', 'value1');
batch.remove('key2');
db.write(batch, function(err) {
console.log('batch written atomically');
});
`
Performs an fsync operation on the active data and hint files.
Closes Medea.
Returns an array of keys from the key-value store.
callback: A function that takes two parameters: an error and the array of keys.
Experimental.
Ad-hoc map-reduce queries over the key-value pairs. The query results are not indexed. A more robust map-reduce implementation will be provided in the near future.
See examples/map_reduce.js for an example.
Runs a compaction process on the database files. Reduces size of data on disk. Deletes old log entries.
callback: A function that takes an error parameter.
Destroys the database. If the folder only includes medea-files, the folder is removed also otherwise only the medea-files are removed and the folder is kept.
All Medea key-value stores reside on the file system in a directory. The directory is made up of data files, hint files, and lock files. All keys are held in an in-memory keydir for fast lookup. More explanation is below.
The keydir resides in memory. It contains a key, an associated data file ID, a value size, and a file offset. When a get is requested, a lookup happens in the keydir. The keydir entry allows one disk seek for each get operation. When a put or remove occurs, the value is updated in the keydir after the update occurs on-disk. The key set must fit in memory.
Format: {seq}.medea.data where {seq} is an incrementing number.
A new data file is created each time a Medea store is opened or after an active data file reaches a configured size limit. Data files are opened in an append-only mode and act as a log. For every put and remove a new entry is inserted into the currently active data file. Each entry contains a timestamp, the key size, the value size, the key, and the value itself. In addition, a CRC32 checksum is calculated, which can be used to test the integrity of the entry.
When existing keys are updated with new values, new entries are appended to the end of the data file. Note: When values are removed, the entry to the data file looks just like a put, but with a special tombstone value.
Format: {seq}.medea.hint where {seq} is an incrementing number.
Each data file has an associated hint file. Entries in a hint file contain a timestamp, a key size, a value size, an offset, and the key itself. The offset points to the latest value entry for a particular key. Hint files are used to build a new keydir when a Medea store is opened. This is much quicker than reading through the data files.
Format: medea.write.lock
Contains a process ID and the active file path.
Currently, multiple processes cannot access the same data directory. Run one process per directory. A workaround has been developed for servers using Node's cluster` module. Check out medea-clusterify to see how it works!
Repeated use leads to fragmentation and empty files. The compaction process needs to be run for cleanup. See: db.compact(callback).
Medea's design places all keys in memory. Therefore, the entire key set must fit in memory. All values are stored on disk.
MIT
Copyright 2013 Apigee Corporation and Contributors