perstore

Perstore is a cross-platform JavaScript object store interface for mapping persistent
objects to various different storage mediums using W3C's IndexedDB object store API. Perstore
includes JavaScript object-relational mapping for SQL databases, JSON file storage,
and hopefully support for many other object/document style storage systems that
provide more direct object storage. Perstore provides model classes that wrap data
stores, and supports JSON Schema integrity enforcement, link management, and
prototype construction. Perstore also provides faceted access to models for an
object-capability based security model.

Setup
=====

It is recommended that you install Perstore such that it is available in require statements
under the "perstore" path. This can easily be done with a package mapping compliant module
loader like Nodules by using a mapping in your
package.json:

"mappings": {
"perstore": "http://github.com/kriszyp/perstore/zipball/master"
}

And you need a local.json file in your current working directory for your application that
defines any database settings such as connection information. There is a template
for local.json.

Model
=====

Typical usage of Perstore looks like:

// first setup the object store, here we use SQL/ORM store
var store = require("perstore/store/sql").SQLStore({
type: "mysql",
table: "my_table",
idColumn: "id"
});

// now we can setup a model that wraps the data store
var MyModel = require("perstore/model").Model("Example", store, {
properties: {
// we can define optionally define type constraints on properties
foo: String
},
prototype: {
// we can define functions on the prototype of the model objects as well
getFoo: function(){
return this.foo;
}
}
});
// now we can interact with the store and it's objects
var someObject = MyModel.get(someId); // retrieve a persisted object
someObject.getFoo(); // returns the current value of foo
someObject.foo = "bar"; // make a change
someObject.save(); // and save it

MyModel.delete(someOtherId); // delete an object

var MyFacet = require("facet").Restrictive(MyModel, {
});

MyFacet.delete(someId) -> will fail, as the facet has not allowed access to delete().

A model is defined with the Model constructor in the "MyModel" module. A Model definition
may follow the JSON schema definition for contractual constraints (usually defining property
type constraints in the "properties" property and relations with the "links" property).
property. It may also contain a prototype property which defines the prototype object
for all instances of the model. Methods can be defined on the prototype object, as well
as directly on the model. REST methods such as get, put, and delete are implemented
directly on the model, and can be overriden for specific functionality. Perstore roughly
follows the class definition structure used by Persevere 1.0

Perstore provides easy to use object persistence mechanism. Persisted model object
instances have two default methods and a property:

- save() - Saves any changes that have been made to an object to the data store.
- load() - If the object has not been fully loaded (sometime queries may return partial
object), the object will be fully loaded from the data store.
- schema - This is a reference to the schema for this object. Schema objects are augmented
(if it does not previously exist) with a getId method that can be used to retrieve the identity
of an object:

object.schema.getId(object) -> identity of object

In the initial example, object persistence is demonstrated with the "someObject"
variable. The object is loaded (via the get call to the model), modified, and saved
(with the save() call).

Facets provide secure, controlled access to models. The facet module comes provides
two facet constructors: Permissive and Restrictive. A Permissive facet allows all actions
on the model by default. Methods can be defined/overriden in the Permissive definition
to control or disable access to certain functionality. A Restrictive facet only allows read
access methods by default (get and query). One can define/override methods to allow
explicit access to other methods such as put or create. An example facet that only
allows read access and creation of new objects:

var facet = require("facet").Restrictive(model, {
create: function(object){ // allow create
return model.create(object);
}
});

Models wrap data stores, which provide the low level interaction with the database or
storage system. Perstore comes with several data stores including (in the store directory):

- mongodb - This is object store that uses a MongoDB database for storage.
- redis - This is object store that uses a Redis database for storage.
- sql - An SQL-based object store. This stores and retrieves objects as rows in
databases. Currently this only fully implemented in Rhino, but the sql data store can easily
wrap an SQL database provider that simple provides an W3C SQL database style
executeSql(sql) function.
- memory - An in-memory data store. None of the data in this store will be persisted
- js-file - Reads and stores all data in the store from a JSON (with JS extensions for
dates and other non-standard JSON types) file.
- remote - This can connect to a remote HTTP/REST based JSON server to store and
retrieve data.

Perstore also includes several store wrappers that can be used to compose more
sophisticate stores by adding functionality (also in the store directory):

- cache - Adds in-memory caching support to a provided store
- aggregate - Combines record data from multiple stores into a single object store
- replicated - Provides data replication across multiple stores
- full-text - Adds full text indexing (currently only available in Rhino through Lucene)
- inherited - Provides a super-sub type relationship between data stores

The following is store API for Perstore. The same API is used for data stores, store
models, and facets. All of the functions are optional. If they do not exist, it indicates
that the store or model does not support or allow the said functionality. All of the
functions may return a promise instead of
the actual return value if they require asynchronous processing to complete the
operation. They are roughly listed in order of importance
(get(id) is the most important function):

get(id, directives) - Finds the persisted record with the given identifier from the store and returns
an object representation (should always be a new object).

put(object, directives) - Stores the given object in storage. The record may or may not
already exist. The optional second parameter
defines the primary identifier for storing the object. If the second parameter is omitted, the
key may be specified the primary identifier property. If that is not specified, the key may be
auto-generated. The primary identifer for the object should be returned

delete(id, directives) - Deletes the record with the given identifier from the store.

query(queryString, directives) - This executes a query against the data store. The
queryString parameter defines the actual query, and the options parameter should be
an object that provides extra information. The following properties on the options
object may be included:

- start - The offset index to start at in the result set
- end - The offset index to end at in the result set
- parameters - An array of values for parameterized queries

The function should generally return an array representing the result set of the query
(unless the query creates a single aggregate object or value). Perstore is designed to leverage http://github.com/kriszyp/rql
for querying, and included stores use RQL, although stores can utilize alternate query languages.

add(object, directives) - Stores a new record. This acts similar to put, but should only be called
when the record does not already exist. Stores do not need to implement this
method, but may implement for ease of differentiating between creation of new
records and updates. This should return the identifier of the newly create record.

construct(object, directives) - This constructs a new persistable object. This does not
actually store the object, but returns an object with a save() method that
can be called to store the object when it is ready. This method does not apply to stores,
only models and facets.

subscribe(resource, callback) - Subscribes to changes in the given resource or set of
resources. The callback is called whenever data is changed in the monitored resource(s).

transaction() - Starts a new transaction for the store. This should return
a transaction object with the following functions. Each of these functions are optional
and only called if they exist:

- commit() - This is called when a transaction is committed.
- requestCommit() - This is called on all the databases/stores prior to committing the
transaction. If this succeeds (doesn't throw an error), the store should guarantee the
success of a subsequent commit() operation. This provides two phase commit
semantics.
- abort() - This is called when a transaction is aborted.
- suspend() - This is called when a transaction is suspended. This happens when an
event is finished, but a promise for the continuance of the action is still in progress.
After being suspended, this transaction is no longer the active transaction.
- resume() - This is called when a transaction is resumed. This happens when a promise
resumes the execution of an action.

(See Transactions section below for more information)

Perstore is designed to allow easy construction of new data stores. A data store
in Perstore is a JavaScript object with any or all of the functions defined above.

Querying
========

Perstore provides a query parsing and execution through http://github.com/kriszyp/rql
(RQL). RQL can be thought as basically a set of
nestable named operators which each have a set of arguments. RQL is designed to
have an extremely simple, but extensible grammar that can be written in a URL friendly query string. A simple RQL
query with a single operator that indicates a search for any resources with a property of
"foo" that has value of 3 could be written:

eq(foo,3)

RQL is a compatible superset of standard HTML form URL encoding. The following query
is identical to the query (it is sugar for the query above):

foo=3

We can use this query format to query stores and models. For example:

MyModel.query("foo=3").forEach(function(object){
// for each object with a property of foo equal to 3
});

We can also construct queries using chained operator calls in JavaScript. We could
write this query:

MyModel.query().eq("foo",3).forEach(...);

The RQL grammar is based around standard URI delimiters. The standard rules for
encoding strings with URL encoding (%xx) are observed. RQL also supersets FIQL.
Therefore we can write a query that finds resources with a "price" property below
10 with a "lt" operator using FIQL syntax:

price=lt=10

Which is identical (and sugar for call operator syntax known as the normalized form):

lt(price,10)

One can combine conditions with multiple operators with "&":

foo=3&price=lt=10

Is the same as:

eq(foo,3)<(price,10)

Which is also the same as:

and(eq(foo,3),lt(price,10))

And thus can be used to query a store:

MyModel.query("foo=3&price=lt=10")...

Or using chained JS calls to perform the same query:

MyModel.query().eq("foo",3).lt("price",10)...

The | operator can be used to indicate an "or" operation. We can also use paranthesis
to group expressions. For example:

(foo=3|foo=bar)&price=lt=10

Which is the same as:

and(or(eq(foo,3),eq(foo,bar)),lt(price,10))

And to query a model/store:

MyModel.query("(foo=3|foo=bar)&price=lt=10")...

And using chained JS calls:

var query = MyModel.query();
query.or(query.eq("foo",3),query.eq("foo","bar")).lt("price",10)...

Sometimes it makes sense to use the with statement (despite the fact that some
think it should never be used). This actually makes the syntax look very similar
to the query string format. For example:

with(MyModel.query()){
or(eq("foo",3),eq("foo","bar")).lt("price",10)...
}

For a more a complete reference guide to the RQL and the available query operators,
see [[http://github.com/kriszyp/rql]]. This also provides information on
the parsed query data structure which is important if you want to implement your
own custom stores.

Transactions
==========

Transactions provide a means for committing multiple changes to a database
atomically. The store API includes transaction semantics for communicating transactions
to the underlying databases. Perstore provides transactional management for delegating
transaction operations to the appropriate stores and databases. To start a transaction,
call the transaction function on the stores module with a callback that will perform any
of the actions of the transaction:

require("perstore/transaction").transaction(function(){
Model.put(...);
Model.delete(...);
});

The callback function may return a promise if the transaction will involve actions that
extend beyond the duration of the function call. When the promise is resolved the
transaction will be committed (or if the promise errors out, the transaction will be
aborted).

Perstore includes a JSGI middleware component for wrapping requests in transactions.
This will make the life of the request be one transaction, committed when the response
is ready to send (or aborted for an error).

transactionApp = require("perstore/jsgi/transactional").Transactional(nextApp);

Implementing Transactions
------------------------

If you are writing your store that needs to be transaction aware, there are two
different options for implementing transaction handling. The simplest approach is to
implement the implement the transaction method on your store and then use the
AutoTransaction store wrapper provided by the "stores" module:

var AutoTransaction = require("perstore/transaction").AutoTransaction;
myTransactionalStore = AutoTransaction({
transaction: function(){
// prepare the transaction
return {
commit: function{
// commit the transaction
},
// implement the rest of the handlers
abort:...
}
}
});

The AutoTransaction wrappers provides two important functions. First, if any of your
store methods are called outside of a global transaction, a transaction will automatically
be started before calling the method and committed afterwards. Second, if a global
transaction is in process, the transaction method will be called on the first access of
this store and be committed when the global transaction is committed.

The other approach to transaction handling is to provide a "database" object. This can
be useful for situations where transaction management needs to exist outside of
individual stores (and may cross stores). One can implement a "database" object that
provides the transaction method with the same API as the store's transaction method.
The database object can be registered with:

require("perstore/transaction").registerDatabase(transaction: function(){
// prepare the transaction
return {...}
});

This transaction method will be called whenever a global transaction is started.

Licensing
--------

Perstore is part of the Persevere project, and therefore is licensed under the
AFL or BSD license. The Persevere project is administered under the Dojo foundation,
and all contributions require a Dojo CLA.

Project Links
------------

See the main Persevere project for more information:

$3

$3

$3

$3

perstore

$3

$3

$3

$3