Peer-to-peer OpenStreetMap database over a kappa architecture.
npm install kappa-osm> Peer-to-peer OpenStreetMap database over kappa-core
A simple and easy-to-use geographic/spatial database that works offline, and
can synchronize with other instances of the database using a variety of
methods, internet and non (local wifi, USB keys, bluetooth, and more).
MOSTLY STABLE
Several upstream modules are using this now, and are being integrated into "real" apps. Expect minimal breaking changes going forward.
If you're interested in this project, leave an issue and share a bit about what
you're building & how we might collaborate!
``js
var kappa = require('kappa-core')
var ram = require('random-access-memory')
var memdb = require('memdb')
var Osm = require('kappa-osm')
var osm = Osm({
core: kappa(ram, { valueEncoding: 'json' }),
index: memdb(),
storage: function (name, cb) { cb(null, ram()) }
})
var node = {
type: 'node',
lat: '-12.7',
lon: '1.3',
tags: { feature: 'water fountain' },
changeset: 'abcdef'
}
osm.create(node, function (err, node) {
if (err) return console.error(err)
console.log('created node with id', node.id)
osm.query([1,-13,2,-11], function (err, nodes) {
if (err) console.error(err)
else console.log(nodes)
})
})
`
outputs
`
created node with id 58261217205dc19b
[
{
id: '58261217205dc19b',
type: 'node',
lat: '-12.7',
lon: '1.3',
tags: { feature: 'water fountain' },
changeset: 'abcdef' },
version: '366212350b5996f944df9df25e679a98545bdac98f507a06f493d167ff9d5f14@0',
links: [],
deviceId: '366212350b5996f944df9df25e679a98545bdac98f507a06f493d167ff9d5f14',
authorId: '366212350b5996f944df9df25e679a98545bdac98f507a06f493d167ff9d5f14'
}
]
`js`
var Osm = require('kappa-osm')
Expected opts include:
- core: a kappa-core instanceindex
- : a levelup instancestorage
- : a function (name, cb) {} that should provide acb
random-access-storage instance to
its callback
Create the new OSM element element and add it to the database. The resultingid
element, populated with the , version, and authorId, and deviceId fields, is returnedcb
by the callback .
authorId is the original author's deviceId, while deviceId represents the device
that most recently edited the element.
Fetch all of the newest OSM elements with the ID id. cb is called with an
array of OSM elements.
The reason an array is returned is because of the distributed nature of the
database: in the case that multiple peers modify an element prior to sync'ing
their databases with each other, there would be multiple latest elements
("heads") for that ID.
If no elements with id exist, an empty array is returned.
Fetch a specific OSM element by its version string. Returns null if not found,
otherwise the single element.
Update an existing element with ID id to be the OSM element element. The newtype
element should have all fields that the OSM element would have. The of
the element cannot be changed.
Updates work by replacing old heads (latest versions) with a new version. This
works by "linking" back to the version names of all previous heads you want to
replace. This happens automatically, but if an array of versions are passed
into opts.links, those elements will be replaced with this newer version
instead of the default current heads.
cb is called with the new element, including id, version, and deviceId
properties.
Marks the element id as deleted. Since all data is append-only in the{ deleted: true }
database, this does not actually delete data, but instead writes a brand new
version of the document with set on it.
Deleted ways, nodes, and relations are all still returned by the query API.
The nodes of a deleted way are not included in the results.
Create and update many elements atomically. ops is an array of operations
(objects) describing the elements to be added or updated or deleted.
`js`
{
type: 'put|del',
id: 'id',
value: { / element / }
}
If no id field is set, the element is created, otherwise it is updated withvalue
the element .
An operation type of 'put' inserts a new element or modifies an existing one,'del'
while a type of will mark the element as deleted.
Currently, doing a batch insert skips many validation checks in order to be as
fast as possible.
TODO: means to enable validation + error reporting / atomic write
Retrieves all nodes, ways, and relations within the bounding box bbox.
bbox is expected to be an array of the form [WEST, SOUTH, EAST, NORTH].(-85, 85)
Latitude (north/south) runs between , and longitude (west/east)(-180, 180)
between .
A callback parameter cb is optional. If given, it will be called as cb(err,
elements). If not provided, a Readable stream will be returned that can be
read from as elements are emitted. The distinction between the two is that the
callback will buffer all results before they are returned, but the stream will
emit results as they arrive, resulting in much less buffering. This can make a
large impact on memory use for queries on large datasets.
Elements are returned as governed by the query algorithm outlined by the OSM v0.6 API:
1. All nodes that are inside a given bounding box and any relations that
reference them.
2. All ways that reference at least one node that is inside a given bounding
box, any relations that reference them (the ways), and any nodes outside the
bounding box that the ways may reference.
3. All relations that reference one of the nodes, ways or relations included due
to the above rules. (This does not apply recursively; meaning that elements
referenced by a relation are not returned by virtue of being in that
relation.)
Accepted opts include:
- opts.observations (boolean): whether to include type === 'observation' objects as well as regular OSM types.
Fetch a list of all OSM elements that refer to the element with ID id. This
captures
1. elements with a changeset fieldnodes
2. all nodes referenced by a way's fieldmembers
3. all nodes, ways, and relations referenced by a relation's field
TODO: this could be made clearer -- maybe an example?
A callback parameter cb is optional. If given, it will be called as cb(err,
results). If not provided or set to null, a Readable stream will be returned
that can be read from as results are ready.
Objects of the following form are returned:
`js`
{
id: '...',
version: '...'
}
Return a readable stream r of all the documents in the db sorted bytimestamp or created_at (for observations). By default, returns least recent
documents first.
The following options are accepted via the opts parameter:
* opts.type(boolean) - additionally filter results by type as a stringopts.id
* (boolean) - only show results for a single string idopts.reverse
* (boolean) - when true, provide results from most to least recentopts.lt
* , opts.lte, opts.gt, opts.gte (string) - lexicographic sorting options
TODO: clarify how lt/lte/gt/gte work
There is a separate index for filtering by type and ID each, so queries should
be fast. Filtering by ID or type are exclusive options.
The lexicographic sorting options operate on timestamp/created_at keys which.toISOString()
are in ISO 8601 format, as you could get from :
`
> new Date().toISOString()
'2018-09-14T23:07:53.862Z'
Returns a readable stream t of all documents in the database of type type. Only the latest documents are returned, not historic data.
Create a duplex replication stream, that you can pipe into another kappa-osm's
instance's replication stream. The stream ends once all map data is exchanged
between the two peers.
Ensure that isInitiator to true to one side, and false on the other.
opts are passed in to multifeed's API
of the same name.
Event emitted when an error within kappa-osm has occurred. This is very
important to listen on, lest things suddenly seem to break and it's not
immediately clear why.
Documents (OSM elements, observations, etc) have a common format:
`js`
{
id: String,
type: String,
lat: String,
lon: String,
tags: Object,
changeset: String,
links: Array
version: String,
deviceId: String,
authorId: String
}
*Note: authorId is the original author's deviceId, while deviceId represents the device
that most recently edited the element.*
TODO: talk about forking data & forking architecture*
With npm installed, run
`
$ npm install kappa-osm
ISC