JoystickDB - A minimalist database server for the Joystick framework
npm install @joystick.js/db-canaryA high-performance, production-ready TCP-based database server built on LMDB with comprehensive client libraries, multi-database support, replication, and enterprise features for Node.js applications.
- Overview
- Features
- Installation
- Quick Start
- Server Setup
- Client Usage
- Multi-Database Operations
- Database Operations
- Indexing
- HTTP API
- Replication
- Administration
- Backup & Restore
- Configuration
- Production Deployment
- API Reference
JoystickDB is a lightweight, blazing-fast database server that provides MongoDB-like operations over TCP using MessagePack for efficient data serialization. Built on top of LMDB (Lightning Memory-Mapped Database), it offers ACID transactions, high performance, minimal resource usage, and enterprise-grade features like multi-database support, replication, and automatic backups.
Think of it as: A simpler, faster alternative to MongoDB that's perfect for applications that need reliable data storage without the complexity of a full database cluster, now with full multi-database support for better data organization.
client.db('database').collection('collection') API pattern``bash`
npm install @joystick.js/db
`bashStart with default settings (port 1983)
npm start
$3
When you first start JoystickDB, it automatically generates a secure API key and displays setup instructions:
`
JoystickDB SetupTo finish setting up your database and enable operations for authenticated users, you will need to create an admin user via the database's admin API using the API key displayed below.
=== STORE THIS KEY SECURELY -- IT WILL NOT BE DISPLAYED AGAIN ===
USGwrwK36RM97xiWs6Df1yFuxPLfo4aY
===
To create a user, send a POST request to the server:
fetch('http://localhost:1984/api/users', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-joystick-db-api-key': 'USGwrwK36RM97xiWs6Df1yFuxPLfo4aY'
},
body: JSON.stringify({
username: 'admin',
password: 'your_secure_password',
role: 'read_write'
})
});
`Important: Save the API key securely - it will not be displayed again and is required for user management operations.
$3
Use the API key to create an admin user via HTTP:
`javascript
// Create admin user via HTTP API
const response = await fetch('http://localhost:1983/api/users', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-joystick-db-api-key': 'your-api-key-from-step-2'
},
body: JSON.stringify({
username: 'admin',
password: 'your_secure_password',
role: 'read_write'
})
});const result = await response.json();
console.log('โ
Admin user created:', result);
`$3
`javascript
import joystickdb from '@joystick.js/db';const client = joystickdb.client({
port: 1983,
authentication: {
username: 'admin',
password: 'your_secure_password'
}
});
client.on('authenticated', async () => {
console.log('๐ Connected to JoystickDB!');
// NEW: Multi-database support with MongoDB-like chaining
const users_db = client.db('user_management');
const users = users_db.collection('users');
// Insert a document
const result = await users.insert_one({
name: 'John Doe',
email: 'john@example.com',
age: 30,
hobbies: ['reading', 'coding', 'gaming']
});
console.log('โ
User created with ID:', result.inserted_id);
// Find the document
const user = await users.find_one({ name: 'John Doe' });
console.log('๐ค Found user:', user);
// Update the document
await users.update_one(
{ name: 'John Doe' },
{ $set: { age: 31 }, $push: { hobbies: 'photography' } }
);
console.log('โ๏ธ User updated!');
// Work with different databases
const inventory_db = client.db('inventory');
const products = inventory_db.collection('products');
await products.insert_one({
name: 'Laptop',
price: 999.99,
category: 'electronics'
});
console.log('๐ฆ Product added to inventory database!');
});
`Server Setup
$3
JoystickDB uses the
JOYSTICK_DB_SETTINGS environment variable for configuration. Set this to a JSON string containing your settings:`bash
export JOYSTICK_DB_SETTINGS='{
"port": 1983,
"database": {
"path": "./data",
"auto_map_size": true
},
"authentication": {
"rate_limit": {
"max_attempts": 5,
"window_ms": 300000
}
},
"primary": true,
"secondary_nodes": [
{ "ip": "192.168.1.100" },
{ "ip": "192.168.1.101" }
],
"secondary_sync_key": "/path/to/sync.key",
"sync_port": 1985,
"backup": {
"enabled": true,
"schedule": "0 2 *",
"retention_days": 30
},
"s3": {
"region": "us-east-1",
"bucket": "my-database-backups"
}
}'
`$3
`bash
Core server configuration via JSON
JOYSTICK_DB_SETTINGS='{"port": 1983, "database": {"path": "./data"}}'Additional server options
WORKER_COUNT=4 # Number of worker processes
NODE_ENV=production # Environment modeAWS S3 for backups (optional)
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
`$3
`javascript
import { create_server } from '@joystick.js/db/server';const server = await create_server();
server.listen(1983, () => {
console.log('๐ JoystickDB server running on port 1983');
});
// Graceful shutdown
process.on('SIGTERM', async () => {
console.log('๐ Shutting down gracefully...');
await server.cleanup();
server.close();
});
`Client Usage
$3
`javascript
const client = joystickdb.client({
host: 'localhost', // Server hostname
port: 1983, // Server port
timeout: 5000, // Request timeout (5 seconds)
reconnect: true, // Auto-reconnect on disconnect
max_reconnect_attempts: 10, // Max reconnection attempts
reconnect_delay: 1000, // Initial reconnect delay (1 second)
authentication: {
username: 'your-username', // Your database username
password: 'your-password' // Your database password
},
auto_connect: true // Connect automatically when created
});
`$3
`javascript
client.on('connect', () => {
console.log('๐ Connected to JoystickDB');
});client.on('authenticated', () => {
console.log('๐ Authentication successful - ready to use!');
});
client.on('error', (error) => {
console.error('โ Client error:', error.message);
});
client.on('disconnect', () => {
console.log('๐ก Disconnected from server');
});
client.on('reconnecting', ({ attempt, delay }) => {
console.log(
๐ Reconnecting... attempt ${attempt}, delay ${delay}ms);
});
`Multi-Database Operations
JoystickDB now supports multiple isolated databases, allowing you to organize your data more effectively. Each database maintains complete isolation with separate collections, indexes, and storage.
$3
JoystickDB uses MongoDB's familiar chaining pattern for multi-database operations:
`javascript
// Connect to different databases
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');
const analytics_db = client.db('analytics');// Each database has its own collections
const users = user_db.collection('users');
const profiles = user_db.collection('profiles');
const products = inventory_db.collection('products');
const categories = inventory_db.collection('categories');
const events = analytics_db.collection('events');
const reports = analytics_db.collection('reports');
// Operations are completely isolated between databases
await users.insert_one({ name: 'John', role: 'admin' });
await products.insert_one({ name: 'Laptop', price: 999 });
await events.insert_one({ type: 'login', user_id: 'john123' });
// Each database maintains separate indexes and statistics
await users.create_index('email');
await products.create_index('category');
await events.create_index('timestamp');
`$3
`javascript
// List all databases on the server
const databases = await client.list_databases();
console.log('๐ Available databases:', databases.databases);// Get database-specific statistics
const user_db = client.db('user_management');
const stats = await user_db.get_stats();
console.log('๐ User DB stats:', stats);
// List collections in a specific database
const collections = await user_db.list_collections();
console.log('๐ Collections in user_management:', collections.collections);
// Create a collection explicitly (optional - collections are created automatically)
await user_db.create_collection('audit_logs', {
// Collection options can be specified here
});
// Drop an entire database (admin operation - use with caution!)
await user_db.drop_database();
console.log('๐๏ธ Database dropped');
`$3
Database names must follow these rules:
- Must be 1-64 characters long
- Can contain letters, numbers, underscores, and hyphens
- Cannot start with a number
- Cannot be reserved names:
admin, config, local`javascript
// Valid database names
const app_db = client.db('my_app');
const user_data = client.db('user-data');
const analytics2024 = client.db('analytics_2024');// Invalid database names (will throw errors)
// client.db('admin'); // Reserved name
// client.db('123invalid'); // Starts with number
// client.db(''); // Empty name
// client.db('a'.repeat(65)); // Too long
`
Database Operations
JoystickDB provides a clean, MongoDB-like chaining API for multi-database operations.
$3
Work with JoystickDB across multiple databases using the intuitive chaining pattern:
`javascript
// Get database and collection references
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');const users = user_db.collection('users');
const products = inventory_db.collection('products');
// Insert documents into different databases
const user = await users.insert_one({
name: 'Alice Smith',
email: 'alice@example.com',
age: 28,
tags: ['developer', 'javascript'],
created_at: new Date()
});
const product = await products.insert_one({
name: 'Wireless Mouse',
price: 29.99,
category: 'electronics',
stock: 150
});
console.log('๐ค Created user in user_management DB:', user.inserted_id);
console.log('๐ฆ Created product in inventory DB:', product.inserted_id);
// Bulk operations across databases
const user_bulk = await users.bulk_write([
{ insert_one: { document: { name: 'Bob', age: 25, role: 'designer' } } },
{ insert_one: { document: { name: 'Carol', age: 30, role: 'manager' } } },
{ insert_one: { document: { name: 'Dave', age: 35, role: 'developer' } } }
]);
const product_bulk = await products.bulk_write([
{ insert_one: { document: { name: 'Keyboard', price: 79.99, category: 'electronics' } } },
{ insert_one: { document: { name: 'Monitor', price: 299.99, category: 'electronics' } } }
]);
console.log('๐ฅ Created users:', user_bulk.inserted_count);
console.log('๐ฆ Created products:', product_bulk.inserted_count);
`$3
`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');const users = user_db.collection('users');
const products = inventory_db.collection('products');
// Find in user database
const alice = await users.find_one({ name: 'Alice Smith' });
console.log('Found Alice in user DB:', alice);
// Find in inventory database
const electronics = await products.find({
category: 'electronics',
price: { $lte: 100 }
});
console.log('Found affordable electronics:', electronics.documents);
// Each database maintains separate query performance
const recent_users = await users.find({}, {
sort: { created_at: -1 },
limit: 10
});
const expensive_products = await products.find({
price: { $gte: 200 }
}, {
sort: { price: -1 },
limit: 5
});
`$3
`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');const users = user_db.collection('users');
const products = inventory_db.collection('products');
// Update in user database
await users.update_one(
{ name: 'Alice Smith' },
{
$set: {
age: 29,
last_updated: new Date(),
status: 'active'
},
$push: { tags: 'senior' }
}
);
// Update in inventory database
await products.update_one(
{ name: 'Wireless Mouse' },
{
$set: { price: 24.99 },
$inc: { stock: -5 }
}
);
// Delete from user database
await users.delete_one({
status: 'inactive',
last_login: { $lt: new Date(Date.now() - 3652460601000) }
});
// Delete from inventory database
await products.delete_one({
stock: { $lte: 0 },
discontinued: true
});
// Delete multiple documents at once
await users.delete_many({
status: 'inactive',
last_login: { $lt: new Date(Date.now() - 3652460601000) }
});
await products.delete_many({
stock: { $lte: 0 }
}, {
limit: 100 // Optional: limit number of deletions
});
`
$3
All query operators work consistently across all databases:
`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');const users = user_db.collection('users');
const products = inventory_db.collection('products');
// Comparison operators work in any database
await users.find({ age: { $gt: 25 } });
await products.find({ price: { $gte: 50, $lte: 200 } });
// Text matching with regular expressions
await users.find({
name: { $regex: '^John' }
});
await products.find({
name: { $regex: 'wireless', $options: 'i' } // Case insensitive
});
// Array operations
await users.find({
tags: { $in: ['developer', 'designer'] }
});
await products.find({
categories: { $all: ['electronics', 'accessories'] }
});
// Complex queries across databases
await users.find({
age: { $gte: 25, $lte: 35 },
tags: { $in: ['developer', 'designer'] },
status: 'active',
name: { $regex: 'Smith$' }
});
await products.find({
price: { $gte: 100 },
category: 'electronics',
stock: { $gt: 0 },
name: { $regex: 'pro', $options: 'i' }
});
`Indexing
Indexes work independently within each database, providing optimal performance for database-specific query patterns.
$3
JoystickDB automatically creates indexes based on query patterns within each database:
`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');const users = user_db.collection('users');
const products = inventory_db.collection('products');
// First query in user database - might be slow
const user1 = await users.find_one({ email: 'john@example.com' });
// JoystickDB creates index for user database email queries
const user2 = await users.find_one({ email: 'jane@example.com' }); // Much faster!
// Separate automatic indexing for inventory database
const product1 = await products.find_one({ sku: 'MOUSE001' });
const product2 = await products.find_one({ sku: 'KEYBOARD002' }); // Auto-indexed!
// Each database maintains its own automatic indexing statistics
`$3
Create and manage indexes independently for each database:
`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');const users = user_db.collection('users');
const products = inventory_db.collection('products');
// Create indexes in user database
await users.create_index('email');
await users.create_index('username', { unique: true });
console.log('๐ User database indexes created');
// Create indexes in inventory database
await products.create_index('sku', { unique: true });
await products.create_index('category');
await products.create_index('price');
console.log('๐ Inventory database indexes created');
// List indexes for each database separately
const user_indexes = await users.get_indexes();
const product_indexes = await products.get_indexes();
console.log('๐ User DB indexes:', user_indexes.indexes);
console.log('๐ Inventory DB indexes:', product_indexes.indexes);
// Drop indexes independently
await users.drop_index('email');
await products.drop_index('category');
`$3
Indexes are completely isolated between databases:
`javascript
const db1 = client.db('database1');
const db2 = client.db('database2');const collection1 = db1.collection('items');
const collection2 = db2.collection('items');
// Create index in database1 only
await collection1.create_index('name');
// This collection in database2 has no indexes yet
const items1 = await collection1.find({ name: 'test' }); // Uses index
const items2 = await collection2.find({ name: 'test' }); // No index, full scan
// Create separate index in database2
await collection2.create_index('name');
// Now both have independent indexes
const items1_fast = await collection1.find({ name: 'test' }); // Uses db1 index
const items2_fast = await collection2.find({ name: 'test' }); // Uses db2 index
`$3
`javascript
const inventory_db = client.db('inventory');
const products = inventory_db.collection('products');// Without index: Searches through ALL products (slow for large datasets)
const expensive_products = await products.find({ price: { $gte: 1000 } });
// Create index on price
await products.create_index('price');
// With index: Jumps directly to expensive products (super fast!)
const expensive_products_fast = await products.find({ price: { $gte: 1000 } });
// Compound queries benefit from multiple indexes
await products.create_index('category');
const gaming_laptops = await products.find({
category: 'laptops',
price: { $gte: 800 }
}); // Uses both category and price indexes
`HTTP API
JoystickDB provides RESTful HTTP endpoints for user management and external integrations. The HTTP API uses API key authentication for secure access.
$3
All HTTP API requests require an API key in the
x-joystick-db-api-key header:`javascript
const headers = {
'Content-Type': 'application/json',
'x-joystick-db-api-key': 'your-api-key-here'
};
`$3
#### Create User
Create a new user with username, password, and role.
`http
POST /api/users
Content-Type: application/json
x-joystick-db-api-key: your-api-key{
"username": "john_doe",
"password": "secure_password123",
"role": "read_write"
}
`Response:
`json
{
"success": true,
"message": "User created successfully",
"username": "john_doe",
"role": "read_write"
}
`Roles:
-
read - Can only read data from all databases
- write - Can only write data to all databases
- read_write - Can read and write data to all databases (admin privileges)#### Get User
Retrieve user information by username.
`http
GET /api/users?username=john_doe
x-joystick-db-api-key: your-api-key
`Response:
`json
{
"success": true,
"user": {
"username": "john_doe",
"role": "read_write",
"created_at": "2024-01-15T10:30:00.000Z"
}
}
`#### Update User
Update user password or role.
`http
PUT /api/users
Content-Type: application/json
x-joystick-db-api-key: your-api-key{
"username": "john_doe",
"password": "new_secure_password456",
"role": "read"
}
`Response:
`json
{
"success": true,
"message": "User updated successfully",
"username": "john_doe",
"updates": ["password", "role"]
}
`#### Delete User
Remove a user from the system.
`http
DELETE /api/users?username=john_doe
x-joystick-db-api-key: your-api-key
`Response:
`json
{
"success": true,
"message": "User deleted successfully",
"username": "john_doe"
}
`$3
#### JavaScript/Node.js
`javascript
const API_KEY = 'your-api-key-here';
const BASE_URL = 'http://localhost:1983';// Create user
const create_user = async (username, password, role) => {
const response = await fetch(
${BASE_URL}/api/users, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-joystick-db-api-key': API_KEY
},
body: JSON.stringify({ username, password, role })
});
return await response.json();
};// Get user
const get_user = async (username) => {
const response = await fetch(
${BASE_URL}/api/users?username=${username}, {
headers: {
'x-joystick-db-api-key': API_KEY
}
});
return await response.json();
};// Update user
const update_user = async (username, updates) => {
const response = await fetch(
${BASE_URL}/api/users, {
method: 'PUT',
headers: {
'Content-Type': 'application/json',
'x-joystick-db-api-key': API_KEY
},
body: JSON.stringify({ username, ...updates })
});
return await response.json();
};// Delete user
const delete_user = async (username) => {
const response = await fetch(
${BASE_URL}/api/users?username=${username}, {
method: 'DELETE',
headers: {
'x-joystick-db-api-key': API_KEY
}
});
return await response.json();
};// Usage examples
const demo = async () => {
// Create a new user
const create_result = await create_user('alice', 'secure123', 'read_write');
console.log('User created:', create_result);
// Get user info
const user_info = await get_user('alice');
console.log('User info:', user_info);
// Update user role
const update_result = await update_user('alice', { role: 'read' });
console.log('User updated:', update_result);
// Delete user
const delete_result = await delete_user('alice');
console.log('User deleted:', delete_result);
};
`#### cURL Examples
`bash
Create user
curl -X POST http://localhost:1983/api/users \
-H "Content-Type: application/json" \
-H "x-joystick-db-api-key: your-api-key-here" \
-d '{"username": "alice", "password": "secure123", "role": "read_write"}'Get user
curl -X GET "http://localhost:1983/api/users?username=alice" \
-H "x-joystick-db-api-key: your-api-key-here"Update user
curl -X PUT http://localhost:1983/api/users \
-H "Content-Type: application/json" \
-H "x-joystick-db-api-key: your-api-key-here" \
-d '{"username": "alice", "role": "read"}'Delete user
curl -X DELETE "http://localhost:1983/api/users?username=alice" \
-H "x-joystick-db-api-key: your-api-key-here"
`$3
The HTTP API returns consistent error responses:
`json
{
"success": false,
"error": "Invalid API key"
}
`Common Error Codes:
-
400 - Bad Request (invalid data)
- 401 - Unauthorized (invalid API key)
- 404 - Not Found (user doesn't exist)
- 409 - Conflict (user already exists)
- 500 - Internal Server ErrorReplication
JoystickDB features a simplified primary/secondary replication system for high availability and read scaling. The replication system maintains database separation across all nodes and uses API key authentication for secure sync operations.
$3
- Primary Node: Main database server that accepts both read and write operations for all databases
- Secondary Nodes: Read-only copies that receive synchronized data from primary for all databases
- API Key Authentication: All sync operations between nodes use API key authentication for security
- Read-only Secondaries: Secondary nodes block write operations from clients, only accepting authenticated sync from primary
- Manual Failover: Admin operations allow promoting a secondary to primary when needed
- Database Isolation: Replication maintains complete database separation across all nodes
$3
#### 1. Configure Primary Node
`bash
Primary server configuration
export JOYSTICK_DB_SETTINGS='{
"port": 1983,
"primary": true,
"secondary_nodes": [
{ "ip": "192.168.1.100" },
{ "ip": "192.168.1.101" }
],
"secondary_sync_key": "/path/to/sync.key",
"sync_port": 1985
}'
`#### 2. Configure Secondary Nodes
`bash
Secondary server configuration
export JOYSTICK_DB_SETTINGS='{
"port": 1983,
"primary": false,
"secondary_sync_key": "/path/to/sync.key",
"sync_port": 1985
}'
`#### 3. Create Sync Key File
The sync key file contains the API key for authenticated sync operations:
`bash
Generate and save sync key (same key for all nodes)
echo "your-secure-sync-api-key" > /path/to/sync.key
chmod 600 /path/to/sync.key
`$3
`javascript
// Connect to primary node
const client = joystickdb.client({
port: 1983,
authentication: {
username: 'admin',
password: 'your-password'
}
});// Check sync system status (admin operation)
const sync_status = await client.admin_operation({
operation: 'get_sync_status'
});
console.log('๐ Sync enabled:', sync_status.enabled);
console.log('๐ก Connected secondaries:', sync_status.connected_nodes);
console.log('๐ Pending operations:', sync_status.pending_count);
// Promote secondary to primary (manual failover)
const promote_result = await client.admin_operation({
operation: 'promote_to_primary',
secondary_ip: '192.168.1.100'
});
console.log('๐ Failover completed:', promote_result.success);
// Force sync to all secondaries
const sync_result = await client.admin_operation({
operation: 'force_sync'
});
console.log('๐ Manual sync completed:', sync_result.synced_nodes);
`$3
`javascript
// Connect to primary for writes (all databases supported)
const primary = joystickdb.client({
host: '192.168.1.10',
port: 1983,
authentication: {
username: 'admin',
password: 'your-password'
}
});// Connect to secondary for reads only (all databases replicated)
const secondary = joystickdb.client({
host: '192.168.1.100',
port: 1983,
authentication: {
username: 'admin',
password: 'your-password'
}
});
// Write to primary (automatically synced to secondaries)
const user_db = primary.db('user_management');
const inventory_db = primary.db('inventory');
await user_db.collection('users').insert_one({
name: 'New User',
email: 'new@example.com'
});
await inventory_db.collection('products').insert_one({
name: 'New Product',
price: 99.99
});
// Read from secondary (reduces load on primary)
const secondary_user_db = secondary.db('user_management');
const secondary_inventory_db = secondary.db('inventory');
const users = await secondary_user_db.collection('users').find({});
const products = await secondary_inventory_db.collection('products').find({});
console.log('๐ฅ Users from secondary:', users.documents.length);
console.log('๐ฆ Products from secondary:', products.documents.length);
// Note: Write operations to secondary will be rejected
try {
await secondary_user_db.collection('users').insert_one({ name: 'Test' });
} catch (error) {
console.log('โ Secondary is read-only:', error.message);
}
`Administration
$3
`javascript
// Get comprehensive server statistics
const stats = await client.get_stats();console.log('๐ฅ๏ธ Server Info:');
console.log(' Uptime:', stats.server.uptime_formatted);
console.log(' Version:', stats.server.version);
console.log(' Node.js:', stats.server.node_version);
console.log('๐พ Memory Usage:');
console.log(' Heap Used:', Math.round(stats.memory.heapUsed / 1024 / 1024), 'MB');
console.log(' Heap Total:', Math.round(stats.memory.heapTotal / 1024 / 1024), 'MB');
console.log('๐๏ธ Database:');
console.log(' Size:', stats.database.used_space_mb, 'MB');
console.log(' Databases:', stats.database.databases);
console.log(' Collections:', stats.database.collections);
console.log('โก Performance:');
console.log(' Operations/sec:', stats.performance.ops_per_second);
console.log(' Avg Response Time:', stats.performance.avg_response_time_ms, 'ms');
console.log('๐ Connections:');
console.log(' Active:', stats.connections.active);
console.log(' Total:', stats.connections.total);
`$3
`javascript
// List all databases on the server
const databases = await client.list_databases();
console.log('๐ Available databases:', databases.databases);// Get statistics for a specific database
const user_db = client.db('user_management');
const user_db_stats = await user_db.get_stats();
console.log('๐ User management DB stats:', user_db_stats);
// List collections in each database
for (const db_name of databases.databases) {
const db = client.db(db_name);
const collections = await db.list_collections();
console.log(
๐ Collections in ${db_name}:, collections.collections);
}// Create collections explicitly in different databases
const analytics_db = client.db('analytics');
await analytics_db.create_collection('events');
await analytics_db.create_collection('reports');
const inventory_db = client.db('inventory');
await inventory_db.create_collection('products');
await inventory_db.create_collection('categories');
`$3
`javascript
// List documents across different databases
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');// List documents in user management database
const user_docs = await client.list_documents('users', {
database: 'user_management',
limit: 50,
skip: 0,
sort_field: 'created_at',
sort_order: 'desc'
});
// List documents in inventory database
const product_docs = await client.list_documents('products', {
database: 'inventory',
limit: 50,
skip: 0,
sort_field: 'price',
sort_order: 'asc'
});
console.log('๐ User documents:', user_docs.documents.length);
console.log('๐ Product documents:', product_docs.documents.length);
// Get specific documents by ID from different databases
const user_doc = await client.get_document('users', 'user-id-here', 'user_management');
const product_doc = await client.get_document('products', 'product-id-here', 'inventory');
// Advanced document querying across databases
const active_users = await client.query_documents('users', {
status: 'active',
last_login: { $gte: new Date(Date.now() - 302460601000) }
}, {
database: 'user_management',
limit: 100,
sort: { last_login: -1 }
});
const expensive_products = await client.query_documents('products', {
price: { $gte: 500 },
stock: { $gt: 0 }
}, {
database: 'inventory',
limit: 50,
sort: { price: -1 }
});
`$3
`javascript
// Simple health check
const ping_result = await client.ping();
if (ping_result.ok === 1) {
console.log('โ
Database is healthy');
} else {
console.log('โ Database is not responding');
}// Reload server configuration without restart
const reload_result = await client.reload();
console.log('๐ Configuration reloaded:', reload_result.message);
// Monitor performance over time
setInterval(async () => {
const stats = await client.get_stats();
console.log(
๐ ${new Date().toISOString()}: ${stats.performance.ops_per_second} ops/sec, ${stats.connections.active} connections, ${stats.database.databases} databases);
}, 30000); // Every 30 seconds
`Backup & Restore
JoystickDB includes automatic backup capabilities with S3 integration. Backups include all databases and maintain database isolation during restore operations.
$3
Configure automatic backups via the
JOYSTICK_DB_SETTINGS environment variable:`bash
export JOYSTICK_DB_SETTINGS='{
"backup": {
"enabled": true,
"schedule": "0 2 *",
"retention_days": 30,
"compression": true
},
"s3": {
"region": "us-east-1",
"bucket": "my-database-backups",
"access_key_id": "your-access-key",
"secret_access_key": "your-secret-key"
}
}'
`$3
`javascript
// Create an immediate backup (includes all databases)
const backup_result = await client.backup_now();
console.log('๐พ Backup created:', backup_result.filename);
console.log('๐ Backup size:', backup_result.size_mb, 'MB');
console.log('โฑ๏ธ Time taken:', backup_result.duration_ms, 'ms');
console.log('๐๏ธ Databases backed up:', backup_result.databases_count);// List all available backups
const backups = await client.list_backups();
console.log('๐ Available backups:');
backups.backups.forEach(backup => {
console.log(
๐ฆ ${backup.filename} (${backup.size_mb} MB) - ${backup.created_at});
console.log( Databases: ${backup.databases_count}, Collections: ${backup.collections_count});
});// Restore from a specific backup (restores all databases)
const restore_result = await client.restore_backup('backup-2024-01-15-02-00-00.tar.gz');
console.log('๐ Restore completed in:', restore_result.duration_ms, 'ms');
console.log('๐๏ธ Restored databases:', restore_result.databases_restored);
console.log('๐ Restored collections:', restore_result.collections_restored);
`$3
`javascript
// Test your backups regularly
const test_backup = async () => {
console.log('๐งช Testing backup system...');
// Create a test backup
const backup = await client.backup_now();
console.log('โ
Backup created successfully');
// Verify backup exists and contains all databases
const backups = await client.list_backups();
const latest = backups.backups.find(b => b.filename === backup.filename);
if (latest && latest.size_mb > 0 && latest.databases_count > 0) {
console.log('โ
Backup verification passed');
console.log(๐ Backup contains ${latest.databases_count} databases);
} else {
console.log('โ Backup verification failed');
}
};// Run backup test monthly
setInterval(test_backup, 30 24 60 60 1000); // 30 days
`Configuration
$3
`json
{
"port": 1983,
"cluster": true,
"worker_count": 4,
"database": {
"path": "./data",
"auto_map_size": true,
"map_size": 1073741824,
"max_dbs": 100
},
"authentication": {
"rate_limit": {
"max_attempts": 5,
"window_ms": 300000,
"lockout_duration_ms": 900000
}
},
"primary": true,
"secondary_nodes": [
{ "ip": "192.168.1.100" },
{ "ip": "192.168.1.101" }
],
"secondary_sync_key": "/path/to/sync.key",
"sync_port": 1985,
"s3": {
"region": "us-east-1",
"bucket": "my-backups",
"access_key_id": "AKIA...",
"secret_access_key": "...",
"endpoint": "https://s3.amazonaws.com"
},
"backup": {
"enabled": true,
"schedule": "0 2 *",
"retention_days": 30,
"compression": true
},
"logging": {
"level": "info",
"file": "./logs/joystickdb.log",
"max_size": "20m",
"max_files": "14d"
},
"performance": {
"connection_pool_size": 1000,
"idle_timeout": 600000,
"request_timeout": 5000,
"auto_index_threshold": 3
}
}
`$3
`bash
Server settings
JOYSTICKDB_PORT=1983
JOYSTICKDB_CLUSTER=true
JOYSTICKDB_WORKER_COUNT=4Database settings
JOYSTICKDB_DATABASE_PATH=./data
JOYSTICKDB_DATABASE_AUTO_MAP_SIZE=true
JOYSTICKDB_DATABASE_MAX_DBS=100Simplified Replication settings
JOYSTICKDB_PRIMARY=true
JOYSTICKDB_SECONDARY_SYNC_KEY=/path/to/sync.key
JOYSTICKDB_SYNC_PORT=1985S3 settings
JOYSTICKDB_S3_REGION=us-east-1
JOYSTICKDB_S3_BUCKET=my-backups
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-keyBackup settings
JOYSTICKDB_BACKUP_ENABLED=true
JOYSTICKDB_BACKUP_RETENTION_DAYS=30Logging
JOYSTICKDB_LOG_LEVEL=info
`Production Deployment
$3
`dockerfile
FROM node:18-alpineWORKDIR /app
Install dependencies
COPY package*.json ./
RUN npm ci --only=productionCopy application code
COPY . .Create data and logs directories
RUN mkdir -p data logsExpose port
EXPOSE 1983Create volumes for persistent data
VOLUME ["/app/data", "/app/logs"]Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD node -e "
const client = require('./src/client/index.js').default.client({port: 1983, timeout: 2000, reconnect: false});
client.ping().then(() => process.exit(0)).catch(() => process.exit(1));
"Start the server
CMD ["npm", "start"]
`$3
`yaml
version: '3.8'services:
joystickdb:
build: .
ports:
- "1983:1983"
volumes:
- ./data:/app/data
- ./logs:/app/logs
environment:
- NODE_ENV=production
- WORKER_COUNT=4
- JOYSTICKDB_LOG_LEVEL=info
- JOYSTICK_DB_SETTINGS={"port": 1983, "database": {"path": "./data", "max_dbs": 100}}
restart: unless-stopped
healthcheck:
test: ["CMD", "npm", "run", "health-check"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
# Optional: Secondary node for replication
joystickdb-secondary:
build: .
ports:
- "1984:1984"
volumes:
- ./data-secondary:/app/data
- ./logs-secondary:/app/logs
environment:
- NODE_ENV=production
- JOYSTICKDB_PORT=1984
- JOYSTICK_DB_SETTINGS={"port": 1984, "database": {"path": "./data", "max_dbs": 100}}
restart: unless-stopped
depends_on:
- joystickdb
`$3
`yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: joystickdb
labels:
app: joystickdb
spec:
replicas: 3
selector:
matchLabels:
app: joystickdb
template:
metadata:
labels:
app: joystickdb
spec:
containers:
- name: joystickdb
image: joystickdb:latest
ports:
- containerPort: 1983
env:
- name: WORKER_COUNT
value: "2"
- name: NODE_ENV
value: "production"
- name: JOYSTICKDB_LOG_LEVEL
value: "info"
- name: JOYSTICK_DB_SETTINGS
valueFrom:
configMapKeyRef:
name: joystickdb-config
key: settings.json
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
volumeMounts:
- name: data
mountPath: /app/data
livenessProbe:
exec:
command:
- npm
- run
- health-check
initialDelaySeconds: 30
periodSeconds: 30
readinessProbe:
exec:
command:
- npm
- run
- health-check
initialDelaySeconds: 5
periodSeconds: 10
volumes:
- name: data
persistentVolumeClaim:
claimName: joystickdb-data
---
apiVersion: v1
kind: ConfigMap
metadata:
name: joystickdb-config
data:
settings.json: |
{
"port": 1983,
"database": {
"path": "./data",
"auto_map_size": true,
"max_dbs": 100
},
"authentication": {
"rate_limit": {
"max_attempts": 5,
"window_ms": 300000
}
},
"backup": {
"enabled": true,
"schedule": "0 2 *",
"retention_days": 30
},
"logging": {
"level": "info"
}
}
---
apiVersion: v1
kind: Service
metadata:
name: joystickdb-service
spec:
selector:
app: joystickdb
ports:
- port: 1983
targetPort: 1983
type: LoadBalancer
`$3
`javascript
// Health check endpoint
const health_check = async () => {
try {
const client = joystickdb.client({
port: 1983,
timeout: 2000,
reconnect: false
});
const result = await client.ping();
client.disconnect();
return result.ok === 1;
} catch (error) {
return false;
}
};// Performance monitoring
const monitor_performance = async () => {
const client = joystickdb.client({
port: 1983,
authentication: {
username: 'admin',
password: process.env.DB_PASSWORD
}
});
const stats = await client.get_stats();
// Log metrics to your monitoring system
console.log('Metrics:', {
uptime: stats.server.uptime,
memory_usage: stats.memory.heapUsed,
ops_per_second: stats.performance.ops_per_second,
active_connections: stats.connections.active,
database_count: stats.database.databases,
total_collections: stats.database.collections,
database_size: stats.database.used_space_mb
});
client.disconnect();
};
`API Reference
$3
#### Connection Management
-
client.connect() - Establish connection to server
- client.disconnect() - Close connection to server
- client.ping() - Test server connectivity and responsiveness#### Authentication & Setup
-
client.setup() - Initial server setup (generates password) - DEPRECATED
- client.authenticate() - Manual authentication (usually automatic)#### Multi-Database Interface
-
client.db(database_name) - Get database interface for method chaining
- client.list_databases() - List all databases on the server#### Database-Level Operations
-
database.collection(collection_name) - Get collection interface within database
- database.list_collections() - List collections in the database
- database.get_stats() - Get database-specific statistics
- database.drop_database() - Drop the entire database (admin operation)
- database.create_collection(collection_name, options) - Create collection explicitly#### CRUD Operations (Collection Chaining API)
-
collection.insert_one(document, options) - Insert single document
- collection.find_one(filter, options) - Find single document
- collection.find(filter, options) - Find multiple documents
- collection.update_one(filter, update, options) - Update single document
- collection.delete_one(filter, options) - Delete single document
- collection.delete_many(filter, options) - Delete multiple documents
- collection.bulk_write(operations, options) - Bulk operations#### Index Management (Collection Chaining API)
-
collection.create_index(field, options) - Create index on field
- collection.upsert_index(field, options) - Create or update index (upsert)
- collection.drop_index(field) - Drop index on field
- collection.get_indexes() - List all indexes for collection#### Auto-Indexing Management
-
client.get_auto_index_stats() - Get automatic indexing statistics#### Simplified Replication Management (via Admin Operations)
-
client.admin_operation({ operation: 'get_sync_status' }) - Get sync system status and statistics
- client.admin_operation({ operation: 'promote_to_primary', secondary_ip: 'ip' }) - Promote secondary to primary (manual failover)
- client.admin_operation({ operation: 'force_sync' }) - Force synchronization with all secondaries#### Administration
-
client.get_stats() - Get comprehensive server statistics
- client.list_collections() - List all collections in default database (backward compatible)
- client.list_documents(collection, options) - List documents with pagination and sorting
- client.get_document(collection, document_id, database) - Get document by ID
- client.query_documents(collection, filter, options) - Advanced document query with filtering
- client.insert_document(collection, document) - Admin insert operation
- client.update_document(collection, document_id, update) - Admin update operation by ID
- client.delete_document(collection, document_id) - Admin delete operation by ID#### Backup & Restore
-
client.backup_now() - Create immediate backup (includes all databases)
- client.list_backups() - List all available backups
- client.restore_backup(backup_name) - Restore from backup (restores all databases)#### Server Management
-
client.reload() - Reload server configuration$3
#### User Management (via HTTP)
-
POST /api/users - Create new user with username, password, and role
- GET /api/users?username= - Get user information by username
- PUT /api/users - Update user password or role
- DELETE /api/users?username= - Delete user by usernameAll HTTP API endpoints require the
x-joystick-db-api-key header for authentication.$3
The client emits these events that you can listen to:
-
connect - Connection established with server
- authenticated - Authentication successful, ready to use
- error - Error occurred (connection, authentication, etc.)
- disconnect - Connection closed
- reconnecting - Reconnection attempt in progress
- response - Unsolicited server response received$3
#### Find Options
`javascript
{
limit: 10, // Maximum number of documents to return
skip: 0, // Number of documents to skip
sort: { field: 1 }, // Sort order (1 = ascending, -1 = descending)
projection: { field: 1 } // Fields to include (1) or exclude (0)
}
`#### Update Options
`javascript
{
upsert: false // Create document if it doesn't exist
}
`#### Index Options
`javascript
{
unique: false, // Enforce uniqueness
sparse: false // Skip documents missing the indexed field
}
`$3
JoystickDB supports these MongoDB-style update operators:
-
$set - Set field values
- $unset - Remove fields
- $inc - Increment numeric values
- $push - Add elements to arrays
- $pull - Remove elements from arrays$3
JoystickDB supports these MongoDB-style query operators:
-
$eq - Equal to
- $ne - Not equal to
- $gt - Greater than
- $gte - Greater than or equal to
- $lt - Less than
- $lte - Less than or equal to
- $in - Value in array
- $nin - Value not in array
- $exists - Field exists
- $regex - Regular expression match$3
`javascript
try {
const user_db = client.db('user_management');
const users = user_db.collection('users');
const result = await users.insert_one({ name: 'John' });
} catch (error) {
if (error.message.includes('Authentication')) {
// Handle authentication error
console.error('Please check your username and password');
} else if (error.message.includes('timeout')) {
// Handle timeout error
console.error('Server is not responding');
} else if (error.message.includes('Connection')) {
// Handle connection error
console.error('Cannot connect to server');
} else if (error.message.includes('Invalid database name')) {
// Handle database naming error
console.error('Database name violates naming rules');
} else {
// Handle other errors
console.error('Database error:', error.message);
}
}
`$3
1. Use Indexes: Create indexes on frequently queried fields in each database
2. Batch Operations: Use
bulk_write for multiple operations
3. Limit Results: Always use limit for large result sets
4. Use Projections: Only fetch fields you need
5. Connection Pooling: Reuse client connections
6. Monitor Performance: Use get_stats() to track performance
7. Database Organization: Use separate databases to organize related data
8. Index Per Database: Create indexes specific to each database's query patterns$3
1. Always Handle Errors: Wrap database calls in try-catch blocks
2. Use Environment Variables: Store passwords and config in env vars
3. Regular Backups: Enable automatic backups for production
4. Monitor Health: Set up health checks and monitoring
5. Use Replication: Set up replication for high availability
6. Index Strategy: Create indexes based on your query patterns per database
7. Connection Management: Properly close connections when done
8. Secure API Keys: Store API keys securely and rotate them regularly
9. User Management: Use appropriate roles (read/write/read_write) for users
10. HTTP API Security: Always use HTTPS in production for HTTP API calls
11. Database Organization: Use meaningful database names and organize data logically
12. Multi-Database Design: Separate concerns across databases (users, inventory, analytics, etc.)
13. Database Naming: Follow naming conventions and avoid reserved names
14. Backward Compatibility: Gradually migrate from single-database to multi-database API
---
For more examples and advanced usage, see the
/test` directory in the repository.Need Help? Check out the comprehensive test suite for real-world usage examples, or refer to the detailed configuration options above.