MFdb

MFdb is a portable P2P database backed by the IPFS Mutable File System.

Create large searchable datasets and easily share them with others. Works in Node and the browser.

Create a database locally and store it in IPFS. Then share the hash with others. They load it and can instantly search it after downloading. Since each commit is written to MFS you can also load a database at each intermediate state if you have the CID. Broadcast these changes and mirrored copies can stay up to date in real time. All merging happens at the Mutable File System level.

- A database contain multiple tables.
- Each table has a schema.
- Index multiple fields.
- Create unique and non-unique indexes. A b-tree is created for each index.
- Broadcast updates to peers via IPNS. (coming soon)
- Backup/load full database from hash.
- Lightweight. Less than 300kb total.

Query data by primary and non-primary indexes. Each table has a schema and when records are inserted we build btree indexes. B-trees are loaded into memory when fields are searched.

The entire database is stored in an IPFS Mutable File System folder and can be exchanged with a single hash. Broadcast the hash via IPNS and any peers can sync up to the latest version. The sync speed depends on many different factors but should be faster the more peers you have.

Install

``console
npm install ipfs mfdb
`

Usage

First initialize MFdb with IPFS.

`typescript
import assert from 'assert'
import { MFdb } from "mfdb"

const ipfs = await IPFS.create()

const db = new MFdb(ipfs)
`

Create a database definition and use it to create the database. In this example we're saving baseball players.

We'll add a single table with 4 different indexed fields. Indexes can be unique. A table must have exactly 1 primary key field defined.

`typescript

let definition: DatabaseDefinition = {
name: "test",
tables: [
{
name: "player",
columns: [
{ name: "id", unique: true, primary: true },
{ name: "name", unique: false },
{ name: "currentTeam", unique: false },
{ name: "battingHand", unique: false },
{ name: "throwingHand", unique: false }
]
}
]
}

await db.createDatabase(definition)
`

Use the database
`typescript
await db.useDatabase("my-database")
`


Add a record and retreive it by the primary key. Note: The primary key must be a string.
`typescript

//Start transaction
db.startTransaction()

//Save it
db.insert("player", {
id: '101',
name: "Andrew McCutchen",
currentTeam: "PIT",
battingHand: "R",
throwingHand: "R"
})

//Commit changes
await db.commit()


//Retrieve it
let player = await store.get("player", '101')
`

Now we're going to add a few more players

`typescript

//Start transaction
db.startTransaction()

store.insert("player", {
id: '102',
name: "Pedro Alvarez",
currentTeam: "BAL",
battingHand: "R",
throwingHand: "R"
})

store.insert("player", {
id: '103',
name: "Jordy Mercer",
currentTeam: "PIT",
battingHand: "L",
throwingHand: "R"
})

store.insert("player", {
id: '104',
name: "Doug Drabek",
currentTeam: "BAL",
battingHand: "L",
throwingHand: "R"
})

//Commit changes
await db.commit()
`


Now retreive the values by the secondary indexes.
`typescript
//Get players who play for PIT
let teamPIT = await store.searchIndex({
table: "players",
index: "currentTeam",
value: "PIT",
sortDirection: "desc",
offset: 0,
limit: 100
})


//Get players who who bat right handed.
let battingR = await store.searchIndex({
table: "players",
index: "battingHand",
value: "R",
sortDirection: "desc",
offset: 0,
limit: 100
})
`

Update a player with new info
`typescript
db.startTransaction()

db.update("test-table", {
id: '101',
name: "Andrew McCutchen",
currentTeam: "PIT",
battingHand: "L", //was R
throwingHand: "R"
})

await db.commit()
`

Delete a row
`typescript

db.startTransaction()
db.delete("test-table", '103')

await db.commit()
`

Get the CID of the database.
`typescript
let cid = await db.getDatabaseCid('players')
`

Load database from CID.
`typescript
await db.updateFromCid("test", "QmSRtaj7Vu8Q4YhcU32xaiHvmKR4KD5h7WR27azzeECCki")
`


Examples can be found in the tests.

API

MFdb exposes the following functions:

$3

* Discard any uncommitted changes and begin a new transaction. All puts and deletes must have an active transaction.

$3

* Flushes uncommitted changes to disk.

$3

* Creates a database with the passed in name.

$3

* Checks whether a database exists or not.

$3

* Drops a database by name.

$3

* Switch the active database to the passed in one. All table related functions will use this value.

$3

* Stores a record in a table. Throws exception if a record already exists with that primary key.

$3

* Updates a record in a table. Throws exception if the record does NOT exist already.

$3

* Calls insert if the record doesn't already exists. Calls update if it does exist. Slower than calling insert or update directly.

$3

* Query a table by primary key. Returns a single record.

$3

* Gets the CID of the value stored in a specific table with key.

$3

* Get rows in a table that match the passed value for the supplied index. If no index is supplied it will search the primary index.

Search request is an object with the following fields:

`typescript
interface IndexSearch {
table: string,
index:string,
value?:any,
sortDirection:SortDirection
offset: number
limit: number
}

//Example:

//Get players who who bat right handed.
let battingR = await store.searchIndex({
table: "players",
index: "battingHand",
value: "R",
sortDirection: "desc",
offset: 0,
limit: 100
})
`

$3

* Delete a row in a table by key.

$3

* Count the records in a table.

$3

* Flushes the database to MFS for backup.

$3

* Get the IPFS CID for the mutable file system folder where the database is stored.

Data Definition Interface

`typescript
interface DatabaseDefinition {
name:string
tables:TableDefinition[]
}

interface TableDefinition {
name: string
columns: ColumnDefinition[]
}

interface ColumnDefinition {
name: string
primary?: boolean
unique: boolean
}
``