Weavestore

A TypeScript-based database abstraction layer that provides traditional database operations (CRUD) on top of the IPFS/IPNS decentralized storage network.

Features

- 🗄️ Database-like operations on IPFS/IPNS
- 🏗️ Hierarchical structure: Database → Schema → Table organization
- 🔒 Type-safe TypeScript API
- 📦 Support for Lighthouse (IPFS and IPNS)
- 🆔 Configurable ID generation (Snowflake or UUID)
- 💾 Built-in caching with LRU eviction
- 🔄 Automatic retry and failover
- 🚀 Built with Bun for optimal performance
- 🔐 Better Auth adapter for decentralized authentication
- 🔙 Backward compatible with legacy flat table structure

Installation

``bash
bun add weavestore
`

Or with npm:

`bash
npm install weavestore
`

Quick Start

$3

`typescript
import { WeaveStore } from 'weavestore';

// Initialize the SDK
const sdk = new WeaveStore({
lighthouse: {
apiKey: 'your-lighthouse-api-key',
},
ipns: {
apiKey: 'your-lighthouse-ipns-api-key',
},
});

await sdk.initialize();

// Create a table (uses default database and schema)
const usersTable = await sdk.createTable('users');

// Insert a record
const userId = await usersTable.insert({
name: 'John Doe',
email: 'john@example.com',
age: 30,
});

// Query records
const users = await usersTable.find({ age: { $gte: 25 } });

// Update a record by ID
await usersTable.updateById(userId, { age: 31 });

// Update multiple records with filter
const updatedCount = await usersTable.update(
{ age: { $lt: 18 } },
{ status: 'minor' }
);

// Soft delete a record (default)
await usersTable.deleteById(userId);

// Hard delete a record (removes from Lighthouse storage)
await usersTable.deleteById(userId, true);

// Delete multiple records with filter
const deletedCount = await usersTable.delete({ status: 'inactive' });
`

$3

Weavestore now supports a hierarchical structure: Database → Schema → Table

`typescript
import { WeaveStore } from 'weavestore';

const sdk = new WeaveStore({
lighthouse: { apiKey: 'your-lighthouse-api-key' },
ipns: { apiKey: 'your-lighthouse-ipns-api-key' },
});

await sdk.initialize();

// Create a database
const myAppDb = await sdk.createDatabase('my_app');

// Create schemas within the database
const publicSchema = await myAppDb.createSchema('public');
const authSchema = await myAppDb.createSchema('auth');

// Create tables within schemas
const usersTable = await publicSchema.createTable('users');
const sessionsTable = await authSchema.createTable('sessions');

// Insert data
const userId = await usersTable.insert({
name: 'John Doe',
email: 'john@example.com',
});

// Access tables using path notation
const table = await sdk.getTableByPath('my_app.public.users');

// List all databases
const databases = await sdk.listDatabases();

// List schemas in a database
const schemas = await myAppDb.listSchemas();

// List tables in a schema
const tables = await publicSchema.listTables();
`

Hierarchical Structure

Weavestore organizes data in a three-tier hierarchy:

`
Database → Schema → Table → Records
`

$3

- Multi-tenancy: Separate data for different tenants/applications
- Logical Grouping: Organize related tables into schemas (e.g., 'public', 'auth', 'analytics')
- Namespace Management: Avoid table name conflicts across different contexts
- Better Organization: Scale to hundreds of tables without chaos

$3

`typescript
// Create a database
const database = await sdk.createDatabase('my_app', {
cache: { enabled: true, ttl: 600000 },
isolationLevel: 'read-committed',
});

// Get an existing database
const db = await sdk.getDatabase('my_app');

// List all databases
const databases = await sdk.listDatabases();

// Delete a database
await sdk.deleteDatabase('old_app'); // Soft delete
await sdk.deleteDatabase('old_app', true); // Hard delete
`

$3

`typescript
// Create a schema
const schema = await database.createSchema('public', {
validation: { enabled: true, strict: true },
defaultTableSettings: { idStrategy: 'uuid' },
namingConvention: { case: 'snake_case', prefix: 'tbl_' },
});

// Get an existing schema
const publicSchema = await database.getSchema('public');

// List all schemas
const schemas = await database.listSchemas();

// Batch load schemas
const [publicSchema, authSchema] = await database.batchGetSchemas(['public', 'auth']);

// Prefetch schemas for faster access
await database.prefetchSchemas(['public', 'auth']);

// Delete a schema
await database.deleteSchema('old_schema'); // Soft delete
await database.deleteSchema('old_schema', true); // Hard delete
`

$3

`typescript
// Create a table in a schema
const usersTable = await schema.createTable('users', {
name: { type: 'string', required: true },
email: { type: 'string', required: true },
age: { type: 'number' },
});

// Get a table from a schema
const table = await schema.getTable('users');

// List all tables in a schema
const tables = await schema.listTables();

// Batch load tables
const [usersTable, postsTable] = await schema.batchGetTables(['users', 'posts']);

// Prefetch tables
await schema.prefetchTables(['users', 'posts']);

// Delete a table
await schema.deleteTable('old_table'); // Soft delete
await schema.deleteTable('old_table', true); // Hard delete
`

$3

Access entities using intuitive path notation:

`typescript
// Get table by full path
const usersTable = await sdk.getTableByPath('my_app.public.users');

// Also supports slash notation
const table = await sdk.getTableByPath('my_app/public/users');

// Get schema by path
const schema = await sdk.getSchemaByPath('my_app.public');

// Get database by path
const database = await sdk.getDatabaseByPath('my_app');
`

$3

The legacy flat table API still works and uses a default database and schema:

`typescript
// These methods still work (with deprecation warnings)
const table = await sdk.createTable('users'); // Creates in __default__.public.users
const table = await sdk.getTable('users'); // Gets from __default__.public.users
const tables = await sdk.listTables(); // Lists tables from __default__.public
`

Configuration

$3

`typescript
const sdk = new WeaveStore({
lighthouse: {
apiKey: 'your-lighthouse-api-key',
gateway: 'https://gateway.lighthouse.storage', // Optional custom gateway
},
ipns: {
apiKey: 'your-lighthouse-ipns-api-key',
},
idStrategy: 'snowflake', // or 'uuid' (default: 'snowflake')
cache: {
enabled: true,
ttl: 300000, // 5 minutes in milliseconds
maxSize: 104857600, // 100MB in bytes
},
retry: {
maxAttempts: 3,
backoffMs: 1000,
},
});
`

API Reference

$3

#### initialize()

Initialize the SDK and verify API connections.

`typescript
await sdk.initialize();
`

#### Database Management

##### createDatabase(name: string, config?: DatabaseConfig)

Create a new database.

`typescript
const database = await sdk.createDatabase('my_app', {
cache: { enabled: true, ttl: 600000 },
isolationLevel: 'read-committed',
});
`

##### getDatabase(name: string)

Get an existing database.

`typescript
const database = await sdk.getDatabase('my_app');
`

##### listDatabases(options?: { limit?: number; offset?: number })

List all databases with optional pagination.

`typescript
const databases = await sdk.listDatabases();
const page1 = await sdk.listDatabases({ limit: 10, offset: 0 });
`

##### deleteDatabase(name: string, hardDelete?: boolean)

Delete a database.

`typescript
await sdk.deleteDatabase('old_app'); // Soft delete
await sdk.deleteDatabase('old_app', true); // Hard delete
`

#### Path-Based Access

##### getTableByPath(path: string)

Get a table using full path notation.

`typescript
const table = await sdk.getTableByPath('my_app.public.users');
// Also supports slash notation
const table = await sdk.getTableByPath('my_app/public/users');
`

##### getSchemaByPath(path: string)

Get a schema using path notation.

`typescript
const schema = await sdk.getSchemaByPath('my_app.public');
`

##### getDatabaseByPath(path: string)

Get a database using path notation.

`typescript
const database = await sdk.getDatabaseByPath('my_app');
`

#### Legacy Methods (Deprecated but Supported)

##### createTable(name: string, schema?: Schema) ⚠️ Deprecated

Create a new table in the default database and schema.

`typescript
interface User {
name: string;
email: string;
age: number;
}

const usersTable = await sdk.createTable('users');
// Recommended: Use database.schema.createTable() instead
`

##### getTable(name: string) ⚠️ Deprecated

Get an existing table from the default database and schema.

`typescript
const usersTable = await sdk.getTable('users');
// Recommended: Use database.schema.getTable() or sdk.getTableByPath() instead
`

##### listTables() ⚠️ Deprecated

List all tables in the default database and schema.

`typescript
const tables = await sdk.listTables();
// Recommended: Use database.schema.listTables() instead
`

##### deleteTable(name: string) ⚠️ Deprecated

Delete a table from the default database and schema.

`typescript
await sdk.deleteTable('users');
// Recommended: Use database.schema.deleteTable() instead
`

$3

#### createSchema(name: string, config?: SchemaConfig)

Create a new schema within the database.

`typescript
const schema = await database.createSchema('public', {
validation: { enabled: true, strict: true },
defaultTableSettings: { idStrategy: 'uuid' },
});
`

#### getSchema(name: string)

Get an existing schema.

`typescript
const schema = await database.getSchema('public');
`

#### listSchemas(options?: { limit?: number; offset?: number })

List all schemas with optional pagination.

`typescript
const schemas = await database.listSchemas();
`

#### batchGetSchemas(names: string[])

Load multiple schemas in parallel.

`typescript
const [publicSchema, authSchema] = await database.batchGetSchemas(['public', 'auth']);
`

#### prefetchSchemas(names: string[])

Prefetch schema metadata for faster subsequent access.

`typescript
await database.prefetchSchemas(['public', 'auth']);
`

#### deleteSchema(name: string, hardDelete?: boolean)

Delete a schema.

`typescript
await database.deleteSchema('old_schema'); // Soft delete
await database.deleteSchema('old_schema', true); // Hard delete
`

#### getMetadata()

Get database metadata.

`typescript
const metadata = database.getMetadata();
console.log(metadata.name, metadata.schemas);
`

#### refresh(cascade?: boolean)

Refresh database metadata from IPNS.

`typescript
await database.refresh(); // Refresh just database
await database.refresh(true); // Refresh database and all children
`

$3

#### createTable(name: string, schema?: TableSchema)

Create a new table within the schema.

`typescript
const usersTable = await schema.createTable('users', {
name: { type: 'string', required: true },
email: { type: 'string', required: true },
});
`

#### getTable(name: string)

Get an existing table.

`typescript
const table = await schema.getTable('users');
`

#### listTables(options?: { limit?: number; offset?: number })

List all tables with optional pagination.

`typescript
const tables = await schema.listTables();
`

#### batchGetTables(names: string[])

Load multiple tables in parallel.

`typescript
const [usersTable, postsTable] = await schema.batchGetTables(['users', 'posts']);
`

#### prefetchTables(names: string[])

Prefetch table metadata for faster subsequent access.

`typescript
await schema.prefetchTables(['users', 'posts']);
`

#### deleteTable(name: string, hardDelete?: boolean)

Delete a table.

`typescript
await schema.deleteTable('old_table'); // Soft delete
await schema.deleteTable('old_table', true); // Hard delete
`

#### getMetadata()

Get schema metadata.

`typescript
const metadata = schema.getMetadata();
console.log(metadata.name, metadata.tables);
`

#### refresh(cascade?: boolean)

Refresh schema metadata from IPNS.

`typescript
await schema.refresh(); // Refresh just schema
await schema.refresh(true); // Refresh schema and all tables
`

$3

#### insert(data: T)

Insert a single record.

`typescript
const userId = await usersTable.insert({
name: 'John Doe',
email: 'john@example.com',
age: 30,
});
`

#### insertMany(data: T[])

Insert multiple records in a single operation.

`typescript
const userIds = await usersTable.insertMany([
{ name: 'Alice', email: 'alice@example.com', age: 25 },
{ name: 'Bob', email: 'bob@example.com', age: 30 },
]);
`

#### findById(id: string)

Find a record by ID.

`typescript
const user = await usersTable.findById(userId);
`

#### findAll(options?: QueryOptions)

Find all records.

`typescript
const users = await usersTable.findAll({
limit: 10,
offset: 0,
includeDeleted: false,
sortBy: 'age',
sortOrder: 'desc',
});
`

#### find(filter: Filter, options?: QueryOptions)

Find records matching a filter.

`typescript
const adults = await usersTable.find({ age: { $gte: 18 } });
`

#### findOne(filter: Filter)

Find the first record matching a filter.

`typescript
const user = await usersTable.findOne({ email: 'john@example.com' });
`

#### count(filter?: Filter)

Count records matching a filter.

`typescript
const adultCount = await usersTable.count({ age: { $gte: 18 } });
`

#### updateById(id: string, data: Partial)

Update a record by ID.

`typescript
await usersTable.updateById(userId, { age: 31 });
`

#### update(filter: Filter, data: Partial)

Update multiple records matching a filter. Returns the number of updated records.

`typescript
// Update all inactive users to active
const updatedCount = await usersTable.update(
{ status: 'inactive' },
{ status: 'active' }
);
console.log(Updated ${updatedCount} users);
`

#### deleteById(id: string, hardDelete?: boolean)

Delete a record by ID.

- Soft delete (default): Marks the record as deleted but keeps it in storage
- Hard delete: Actually removes the file from Lighthouse storage (only works for Annual Plan files)

`typescript
// Soft delete (default)
await usersTable.deleteById(userId);

// Hard delete (removes from storage)
await usersTable.deleteById(userId, true);
`

#### delete(filter: Filter, hardDelete?: boolean)

Delete multiple records matching a filter. Returns the number of deleted records.

`typescript
// Soft delete all inactive users
const deletedCount = await usersTable.delete({ status: 'inactive' });
console.log(Deleted ${deletedCount} users);

// Hard delete all users over 65
const hardDeletedCount = await usersTable.delete({ age: { $gt: 65 } }, true);
`

#### healthCheck()

Check the health of IPFS and IPNS connections.

`typescript
const health = await sdk.healthCheck();
console.log(health); // { ipfs: true, ipns: true, timestamp: 1234567890 }
`

#### clearCache()

Clear the cache.

`typescript
sdk.clearCache();
`

$3

#### insert(data: T)

Insert a single record.

`typescript
const userId = await usersTable.insert({
name: 'John Doe',
email: 'john@example.com',
age: 30,
});
`

#### insertMany(data: T[])

Insert multiple records.

`typescript
const userIds = await usersTable.insertMany([
{ name: 'John', email: 'john@example.com', age: 30 },
{ name: 'Jane', email: 'jane@example.com', age: 25 },
]);
`

#### findById(id: string)

Find a record by ID.

`typescript:
const user = await usersTable.findById(userId);
`

#### findAll(options?: QueryOptions)

Find all records with optional pagination.

`typescript
const users = await usersTable.findAll({
limit: 10,
offset: 0,
includeDeleted: false,
});
`

#### find(filter: Filter, options?: QueryOptions)

Find records matching a filter.

`typescript
const adults = await usersTable.find(
{ age: { $gte: 18 } },
{ limit: 10, sortBy: 'age', sortOrder: 'desc' }
);
`

#### findOne(filter: Filter)

Find the first record matching a filter.

`typescript
const user = await usersTable.findOne({ email: 'john@example.com' });
`

#### count(filter?: Filter)

Count records matching a filter.

`typescript
const adultCount = await usersTable.count({ age: { $gte: 18 } });
`

#### updateById(id: string, data: Partial)

Update a record by ID.

`typescript
await usersTable.updateById(userId, { age: 31 });
`

#### deleteById(id: string)

Delete a record by ID (soft delete).

`typescript
await usersTable.deleteById(userId);
`

#### getMetadata()

Get table metadata.

`typescript
const metadata = usersTable.getMetadata();
`

#### refresh()

Refresh table metadata from IPNS.

`typescript
await usersTable.refresh();
`

Query Filtering

The SDK supports MongoDB-style query operators for powerful and flexible querying:

$3

- $eq: Equal to
- $ne: Not equal to
- $gt: Greater than
- $gte: Greater than or equal to
- $lt: Less than
- $lte: Less than or equal to

$3

- $in: Value is in array
- $nin: Value is not in array

$3

- $regex: Regular expression pattern matching (like SQL LIKE)
- $options: Regex options ('i' for case-insensitive, 'g' for global, 'm' for multiline)

$3

- $exists: Check if field exists

$3

- $or: Logical OR - matches if any condition is true
- $and: Logical AND - matches if all conditions are true
- $not: Logical NOT - inverts the condition

$3

`typescript
// Find users older than 25
const users = await usersTable.find({ age: { $gt: 25 } });

// Find users between 18 and 65
const users = await usersTable.find({
age: { $gte: 18, $lte: 65 },
});

// Find users with specific emails
const users = await usersTable.find({
email: { $in: ['john@example.com', 'jane@example.com'] },
});

// Find users not named John
const users = await usersTable.find({
name: { $ne: 'John' },
});
`

$3

`typescript
// Pattern matching - find users whose name contains "john" (case-insensitive)
const users = await usersTable.find({
name: { $regex: 'john', $options: 'i' }
});

// Search across multiple fields with OR
const searchQuery = 'john';
const results = await usersTable.find({
$or: [
{ name: { $regex: searchQuery, $options: 'i' } },
{ username: { $regex: searchQuery, $options: 'i' } },
{ email: { $regex: searchQuery, $options: 'i' } }
]
});

// Complex nested logic with AND and OR
const users = await usersTable.find({
$and: [
{ age: { $gte: 25 } },
{ city: 'NYC' },
{
$or: [
{ status: 'active' },
{ status: 'pending' }
]
}
]
});

// NOT operator - find users NOT in NYC
const users = await usersTable.find({
$not: { city: 'NYC' }
});

// Check field existence
const usersWithEmail = await usersTable.find({
email: { $exists: true }
});
`

$3

`typescript
// Using offset
const page1 = await usersTable.find({}, { limit: 10, offset: 0 });
const page2 = await usersTable.find({}, { limit: 10, offset: 10 });

// Using skip (alias for offset)
const page1 = await usersTable.find({}, { limit: 10, skip: 0 });
const page2 = await usersTable.find({}, { limit: 10, skip: 10 });

// Dynamic pagination
const page = 2;
const limit = 10;
const users = await usersTable.find(
{},
{ skip: (page - 1) * limit, limit: limit }
);

// Pagination with sorting and filtering
const users = await usersTable.find(
{ status: 'active' },
{
sortBy: 'createdAt',
sortOrder: 'desc',
skip: 20,
limit: 10
}
);
`

$3

`typescript
// Search for users with complex criteria
const searchQuery = 'john';
const page = 1;
const limit = 20;

const users = await usersTable.find(
{
$and: [
// Must have email
{ email: { $exists: true } },

// Must be 18 or older
{ age: { $gte: 18 } },

// Must not be banned
{ $not: { status: 'banned' } },

// Search in name or username
{
$or: [
{ name: { $regex: searchQuery, $options: 'i' } },
{ username: { $regex: searchQuery, $options: 'i' } }
]
},

// Must be in NYC or SF
{
$or: [
{ city: 'NYC' },
{ city: 'SF' }
]
}
]
},
{
skip: (page - 1) * limit,
limit: limit,
sortBy: 'name',
sortOrder: 'asc'
}
);
`

For more advanced search and pagination examples, see the Advanced Search and Pagination Guide.

Better Auth Integration

Weavestore includes a built-in adapter for Better Auth, enabling decentralized authentication storage on IPFS/IPNS.

$3

`typescript
import { IPFSStorageSDK, createBetterAuthAdapter } from 'weavestore';
import { betterAuth } from 'better-auth';

// Initialize SDK
const sdk = new IPFSStorageSDK({
lighthouse: {
apiKey: process.env.LIGHTHOUSE_API_KEY!,
},
ipns: {
apiKey: process.env.LIGHTHOUSE_IPNS_API_KEY!,
},
idStrategy: 'uuid',
});

await sdk.initialize();

// Create Better Auth adapter
const adapter = createBetterAuthAdapter({
sdk,
tablePrefix: 'auth_', // Optional: prefix for all tables
usePlural: true, // Optional: use plural table names
});

// Use with Better Auth
export const auth = betterAuth({
database: adapter,
emailAndPassword: {
enabled: true,
},
socialProviders: {
google: {
clientId: process.env.GOOGLE_CLIENT_ID!,
clientSecret: process.env.GOOGLE_CLIENT_SECRET!,
},
},
});
`

$3

- ✅ Full CRUD operations for all Better Auth models
- ✅ Automatic table creation
- ✅ Complex queries with filtering
- ✅ Pagination and sorting
- ✅ Field selection for optimized queries
- ✅ Decentralized storage on IPFS/IPNS
- ✅ No migrations needed

$3

The adapter automatically handles:
- User management (create, read, update, delete)
- Session management
- OAuth account linking
- Verification tokens
- All Better Auth models

$3

See the Better Auth integration example for a complete working example.

$3

For detailed documentation, see Better Auth Adapter README.

Error Handling

The SDK throws IPFSStorageError with specific error codes:

`typescript
import { IPFSStorageError, ErrorCode } from 'weavestore';

try {
await sdk.initialize();
} catch (error) {
if (error instanceof IPFSStorageError) {
switch (error.code) {
case ErrorCode.AUTHENTICATION_FAILED:
console.error('Authentication failed:', error.message);
break;
case ErrorCode.TABLE_NOT_FOUND:
console.error('Table not found:', error.message);
break;
case ErrorCode.NETWORK_ERROR:
console.error('Network error:', error.message);
break;
// ... handle other error codes
}
}
}
`

$3

- AUTHENTICATION_FAILED: API authentication failed
- TABLE_NOT_FOUND: Table does not exist
- TABLE_ALREADY_EXISTS: Table already exists
- RECORD_NOT_FOUND: Record does not exist
- NETWORK_ERROR: Network request failed
- TIMEOUT: Operation timed out
- VALIDATION_ERROR: Input validation failed
- IPFS_UPLOAD_FAILED: IPFS upload failed
- IPNS_PUBLISH_FAILED: IPNS publish failed
- RATE_LIMIT_EXCEEDED: API rate limit exceeded

Performance Considerations

$3

The SDK includes built-in caching with LRU eviction:

- Record data is cached after retrieval
- Record indexes are cached aggressively
- Cache is automatically invalidated on updates
- Configure cache size and TTL based on your needs

$3

Use insertMany() for bulk inserts to reduce IPNS updates:

`typescript
// Good: Single IPNS update
await usersTable.insertMany([user1, user2, user3]);

// Less efficient: Multiple IPNS updates
await usersTable.insert(user1);
await usersTable.insert(user2);
await usersTable.insert(user3);
`

$3

- Records per table: Recommended max 10,000 records
- Record size: Recommended max 1MB per record
- Concurrent operations: Reads are parallel, writes are serialized per table

Getting API Keys

To use this SDK, you need Lighthouse API keys:

1. Visit https://files.lighthouse.storage/
2. Login/Sign up for a free account
3. Generate an API key from the API Key section
4. Update your .env file with the key

📚 Detailed Guide: HOW_TO_GET_API_KEYS.md

Troubleshooting

$3

Ensure your API keys are correct:

`bash


Lighthouse (for IPFS uploads)

export LIGHTHOUSE_API_KEY="your-api-key"

Lighthouse IPNS (for IPNS operations)

export LIGHTHOUSE_IPNS_API_KEY="your-ipns-api-key"
`

$3

The SDK automatically retries failed requests with exponential backoff. If you're experiencing persistent network errors:

1. Check your internet connection
2. Verify API service status
3. Increase retry attempts in configuration

$3

The SDK automatically fails over to alternative IPFS gateways if the primary gateway is unavailable.

Development

$3

`bash
bun run build
`

$3

`bash
bun run typecheck
`

$3

`bash
bun test
``

License

MIT

Contributing

Contributions are welcome! Please open an issue or submit a pull request.

Support

For issues and questions, please open an issue on GitHub.