Node binding for LMDB, the Lightning Memory-Mapped Database
npm install gs-node-lmdbAbout gs-node-lmdb
==================
This is an experimental, friendly fork of the original node-lmdb package. All of its changes are gonna be contributed back to the original module once they stabilizes.
node-lmdb
=========
This is a node.js binding for LMDB, an extremely fast and lightweight transactional key-value store database.
About
-----
Buffer API.
node-lmdb code is licensed to you under the terms of the MIT license. LMDB itself is licensed under its own OpenLDAP public license (which is similarly permissive).
require() the module.
javascript
var lmdb = require('node-lmdb');
`
#### Step 1: create an environment
Env represents a database environment. You can create one with the new operator and after that, you must open it before you can use it.
open() accepts an object literal in which you can specify the configuration options for the environment.
`javascript
var env = new lmdb.Env();
env.open({
path: __dirname + "/mydata",
mapSize: 210241024*1024, // maximum database size
maxDbs: 3
});
`
Close the environment when you no longer need it.
`javascript
env.close();
`
#### Step 2: open one or more databases
An environment (Env) can contain one or more databases. Open a database with env.openDbi() which takes an object literal with which you can configure your database.
`javascript
var dbi = env.openDbi({
name: "myPrettyDatabase",
create: true // will create if database did not exist
})
`
Close the database when you no longer need it.
`javascript
dbi.close();
`
#### Step 3: use transactions
The basic unit of work in LMDB is a transaction, which is called Txn for short. Here is how you operate with your data.
Every piece of data in LMDB is referred to by a key.
You can use the methods getString(), getBinary(), getNumber() and getBoolean() to retrieve something,
putString(), putBinary(), putNumber() and putBoolean() to store something and del() to delete something.
Currently only string, binary, number and boolean values are supported, use JSON.stringify and JSON.parse for complex data structures.
Because of the nature of LMDB, the data returned by txn.getStringUnsafe() and txn.getBinaryUnsafe() is only valid until the next put operation or the end of the transaction. Safer methods txn.getBinary() and txn.getString() will be garbage collected when there are no references to it.
IMPORTANT: always close your transactions with abort() or commit() when you are done with them.
`javascript
var txn = env.beginTxn();
var value = txn.getString(dbi, 1);
console.log(value);
if (value === null) {
txn.putString(dbi, 1, "Hello world!");
}
else {
txn.del(dbi, 1);
}
txn.putString(dbi, 2, "Yes, it's this simple!");
txn.commit();
`
$3
LMDB has four different entities:
* Env represents a full database environment. The same environment can be used by multiple processes, but a particular Env object must be used by one process only. You can operate with the same environment from multiple threads.
* Dbi represents a sub-database which belongs to a database environment. The same environment can contain either multiple named databases (if you specify a string name) or an unnamed database (if you specify null instead of a name).
Txn represents a transaction. Multiple threads can open transactions for the same Env, but a particular Txn object must only be accessed by one thread, and only one Txn object can be used on a thread at a time. (NOTE: The noTls option in the environment will change this behaviour for read-only transactions, so that a thread can then create any number of read-only transactions and any number of threads can access the same read-only transaction.) Note that only one write* transaction can be open in an environment in any given time. env.beginTxn() will simply block until the previous one is either commit()ted or abort()ed.
* Cursor objects can be used to iterate through multiple keys in the same database.
Here is how you use LMDB in a typical scenario:
* You create an Env and open() it with the desired configuration options.
* You open a Dbi by calling env.openDbi() and passing the database configuration options.
* Now you can create Txns with env.beginTxn() and operate on the database through a transaction by calling txn.getString(), txn.putString() etc.
* When you are done, you should either abort() or commit() your transactions and close() your databases and environment.
$3
You can find some in the source tree. There are some basic examples and I intend to create some advanced ones too.
The basic examples we currently have:
* examples/1-env.js - shows basic usage of Env, Dbi and Txn operating on string values
* examples/2-datatypes.js - shows how to use various data types for your data
* examples/3-multiple-transactions.js - shows how LMDB will behave if you operate with multiple transactions
* examples/4-cursors.js - shows how to work with cursors on a basic database
* examples/5-dupsort.js - shows how to use a dupSort database with cursors
* examples/6-asyncio.js - shows how to use the fastest (but also most dangerous) way for async IO
* examples/7-largedb.js - shows how to work with an insanely large database
* examples/8-multiple-cursors-single-transactions.js - shows how to use multiple cursors with a single transaction
* examples/9-unnamed-db.js - shows how to use an unnamed database
Advanced examples:
* examples/advanced1-indexing.js - this is a module pattern example which demonstrates the implementation of a search engine prototype
More will come later, so don't forget to check back!*
$3
* Only string, binary, number and boolean values are supported. If you want to store complex data structures, use JSON.stringify before putting it into the database and JSON.parse when you retrieve the data.
* Only string and unsigned integer keys are supported. Default is string, specify keyIsUint32: true to openDbi for unsigned integer. It would make the API too complicated to support more data types for keys.
Because of the nature of LMDB, the data returned by txn.getStringUnsafe() and txn.getBinaryUnsafe() is only valid until the next put operation or the end of the transaction. If you need to use the data later*, you can use the txn.getBinary() and txn.getString() methods.
* Fixed address map (called MDB_FIXEDMAP in C) features are not exposed by this binding because they are highly experimental
* Not all functions are wrapped by the binding yet. If there's one that you would like to see, drop me a line.
Contributing
------------
If you find problems with this module, open an issue on GitHub.
Also feel free to send me pull requests. Contributions are more than welcome! :)
$3
LMDB is bundled in node-lmdb so you can simply build this module using node-gyp.
`bash
Install node-gyp globally (needs admin permissions)
npm -g install node-gyp
Clone node-lmdb
git clone git@github.com:Venemo/node-lmdb.git
Build
cd node-lmdb
node-gyp configure
node-gyp build
`
$3
`bash
Adding upstream LMDB as remote
git remote add lmdb https://github.com/LMDB/lmdb.git
Fetch new remote
git fetch lmdb
Adding the subtree (when it's not there yet)
git subtree add --prefix=dependencies/lmdb lmdb HEAD --squash
Updating the subtree (when already added)
git subtree pull --prefix=dependencies/lmdb lmdb HEAD --squash
``