JoystickDB

A high-performance, production-ready TCP-based database server built on LMDB with comprehensive client libraries, multi-database support, replication, and enterprise features for Node.js applications.

Table of Contents

- Overview
- Features
- Installation
- Quick Start
- Server Setup
- Client Usage
- Multi-Database Operations
- Database Operations
- Indexing
- HTTP API
- Replication
- Administration
- Backup & Restore
- Configuration
- Production Deployment
- API Reference

Overview

JoystickDB is a lightweight, blazing-fast database server that provides MongoDB-like operations over TCP using MessagePack for efficient data serialization. Built on top of LMDB (Lightning Memory-Mapped Database), it offers ACID transactions, high performance, minimal resource usage, and enterprise-grade features like multi-database support, replication, and automatic backups.

Think of it as: A simpler, faster alternative to MongoDB that's perfect for applications that need reliable data storage without the complexity of a full database cluster, now with full multi-database support for better data organization.

Features

$3

- Multi-Database Support: Organize data across multiple isolated databases with MongoDB-like chaining API
- High Performance: Built on LMDB with MessagePack serialization for maximum speed
- MongoDB-like API: Familiar CRUD operations and query syntax - easy to learn
- ACID Transactions: Your data is always consistent and safe
- TCP Protocol: Efficient binary communication over the network
- Automatic Indexing: Smart index creation based on your query patterns
- Secondary Indexes: Create custom indexes for lightning-fast queries
- Database Isolation: Complete data separation between databases with proper key namespacing

$3

- API Key Authentication: Secure API key-based authentication system with user management
- HTTP API: RESTful HTTP endpoints for user management and external integrations
- TCP Replication: Master-slave replication for high availability and read scaling
- Password Authentication: Secure bcrypt password hashing with rate limiting
- Role-Based Access Control: Read, write, and read_write user roles
- Clustering: Multi-process clustering support for better performance
- Backup & Restore: Automatic S3-compatible backup system with scheduling
- Admin Interface: Comprehensive administrative operations and monitoring
- Connection Pooling: Efficient connection management and resource usage
- Health Monitoring: Built-in performance metrics and health checks
- Write Serialization: Ensures data consistency under high concurrent load

$3

- Fluent API: Chain methods for intuitive database operations
- Multi-Database Chaining: client.db('database').collection('collection') API pattern
- Backward Compatibility: Existing single-database API still supported
- Auto-Reconnection: Client automatically reconnects on network issues
- TypeScript Support: Full TypeScript definitions included
- Comprehensive Logging: Detailed logs for debugging and monitoring
- Easy Setup: Get started in minutes with minimal configuration
- Clean Code Architecture: Refactored following Uncle Bob Martin's clean code principles for maintainability
- Modular Design: Small, focused functions with clear responsibilities and minimal dependencies

Installation

``bash
npm install @joystick.js/db
`

Quick Start

$3

`bash


Start with default settings (port 1983)

npm start

Or with custom port

PORT=3000 npm start

With clustering (uses all CPU cores)

WORKER_COUNT=4 npm start
`

$3

When you first start JoystickDB, it automatically generates a secure API key and displays setup instructions:

`
JoystickDB Setup

To finish setting up your database and enable operations for authenticated users, you will need to create an admin user via the database's admin API using the API key displayed below.

=== STORE THIS KEY SECURELY -- IT WILL NOT BE DISPLAYED AGAIN ===
USGwrwK36RM97xiWs6Df1yFuxPLfo4aY
===

To create a user, send a POST request to the server:

fetch('http://localhost:1984/api/users', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-joystick-db-api-key': 'USGwrwK36RM97xiWs6Df1yFuxPLfo4aY'
},
body: JSON.stringify({
username: 'admin',
password: 'your_secure_password',
role: 'read_write'
})
});
`

Important: Save the API key securely - it will not be displayed again and is required for user management operations.

$3

Use the API key to create an admin user via HTTP:

`javascript
// Create admin user via HTTP API
const response = await fetch('http://localhost:1983/api/users', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-joystick-db-api-key': 'your-api-key-from-step-2'
},
body: JSON.stringify({
username: 'admin',
password: 'your_secure_password',
role: 'read_write'
})
});

const result = await response.json();
console.log('✅ Admin user created:', result);
`

$3

`javascript
import joystickdb from '@joystick.js/db';

const client = joystickdb.client({
port: 1983,
authentication: {
username: 'admin',
password: 'your_secure_password'
}
});

client.on('authenticated', async () => {
console.log('🎉 Connected to JoystickDB!');

// NEW: Multi-database support with MongoDB-like chaining
const users_db = client.db('user_management');
const users = users_db.collection('users');

// Insert a document
const result = await users.insert_one({
name: 'John Doe',
email: 'john@example.com',
age: 30,
hobbies: ['reading', 'coding', 'gaming']
});

console.log('✅ User created with ID:', result.inserted_id);

// Find the document
const user = await users.find_one({ name: 'John Doe' });
console.log('👤 Found user:', user);

// Update the document
await users.update_one(
{ name: 'John Doe' },
{ $set: { age: 31 }, $push: { hobbies: 'photography' } }
);

console.log('✏️ User updated!');

// Work with different databases
const inventory_db = client.db('inventory');
const products = inventory_db.collection('products');

await products.insert_one({
name: 'Laptop',
price: 999.99,
category: 'electronics'
});

console.log('📦 Product added to inventory database!');
});
`

Server Setup

$3

JoystickDB uses the JOYSTICK_DB_SETTINGS environment variable for configuration. Set this to a JSON string containing your settings:

`bash
export JOYSTICK_DB_SETTINGS='{
"port": 1983,
"database": {
"path": "./data",
"auto_map_size": true
},
"authentication": {
"rate_limit": {
"max_attempts": 5,
"window_ms": 300000
}
},
"primary": true,
"secondary_nodes": [
{ "ip": "192.168.1.100" },
{ "ip": "192.168.1.101" }
],
"secondary_sync_key": "/path/to/sync.key",
"sync_port": 1985,
"backup": {
"enabled": true,
"schedule": "0 2 *",
"retention_days": 30
},
"s3": {
"region": "us-east-1",
"bucket": "my-database-backups"
}
}'
`

$3

`bash


Core server configuration via JSON

JOYSTICK_DB_SETTINGS='{"port": 1983, "database": {"path": "./data"}}'

Additional server options

WORKER_COUNT=4 # Number of worker processes
NODE_ENV=production # Environment mode

AWS S3 for backups (optional)

AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
`

$3

`javascript
import { create_server } from '@joystick.js/db/server';

const server = await create_server();

server.listen(1983, () => {
console.log('🚀 JoystickDB server running on port 1983');
});

// Graceful shutdown
process.on('SIGTERM', async () => {
console.log('🛑 Shutting down gracefully...');
await server.cleanup();
server.close();
});
`

Client Usage

$3

`javascript
const client = joystickdb.client({
host: 'localhost', // Server hostname
port: 1983, // Server port
timeout: 5000, // Request timeout (5 seconds)
reconnect: true, // Auto-reconnect on disconnect
max_reconnect_attempts: 10, // Max reconnection attempts
reconnect_delay: 1000, // Initial reconnect delay (1 second)
authentication: {
username: 'your-username', // Your database username
password: 'your-password' // Your database password
},
auto_connect: true // Connect automatically when created
});
`

$3

`javascript
client.on('connect', () => {
console.log('🔌 Connected to JoystickDB');
});

client.on('authenticated', () => {
console.log('🔐 Authentication successful - ready to use!');
});

client.on('error', (error) => {
console.error('❌ Client error:', error.message);
});

client.on('disconnect', () => {
console.log('📡 Disconnected from server');
});

client.on('reconnecting', ({ attempt, delay }) => {
console.log(🔄 Reconnecting... attempt ${attempt}, delay ${delay}ms);
});
`

Multi-Database Operations

JoystickDB now supports multiple isolated databases, allowing you to organize your data more effectively. Each database maintains complete isolation with separate collections, indexes, and storage.

$3

JoystickDB uses MongoDB's familiar chaining pattern for multi-database operations:

`javascript
// Connect to different databases
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');
const analytics_db = client.db('analytics');

// Each database has its own collections
const users = user_db.collection('users');
const profiles = user_db.collection('profiles');

const products = inventory_db.collection('products');
const categories = inventory_db.collection('categories');

const events = analytics_db.collection('events');
const reports = analytics_db.collection('reports');

// Operations are completely isolated between databases
await users.insert_one({ name: 'John', role: 'admin' });
await products.insert_one({ name: 'Laptop', price: 999 });
await events.insert_one({ type: 'login', user_id: 'john123' });

// Each database maintains separate indexes and statistics
await users.create_index('email');
await products.create_index('category');
await events.create_index('timestamp');
`

$3

`javascript
// List all databases on the server
const databases = await client.list_databases();
console.log('📚 Available databases:', databases.databases);

// Get database-specific statistics
const user_db = client.db('user_management');
const stats = await user_db.get_stats();
console.log('📊 User DB stats:', stats);

// List collections in a specific database
const collections = await user_db.list_collections();
console.log('📋 Collections in user_management:', collections.collections);

// Create a collection explicitly (optional - collections are created automatically)
await user_db.create_collection('audit_logs', {
// Collection options can be specified here
});

// Drop an entire database (admin operation - use with caution!)
await user_db.drop_database();
console.log('🗑️ Database dropped');
`

$3

Database names must follow these rules:

- Must be 1-64 characters long
- Can contain letters, numbers, underscores, and hyphens
- Cannot start with a number
- Cannot be reserved names: admin, config, local

`javascript
// Valid database names
const app_db = client.db('my_app');
const user_data = client.db('user-data');
const analytics2024 = client.db('analytics_2024');

// Invalid database names (will throw errors)
// client.db('admin'); // Reserved name
// client.db('123invalid'); // Starts with number
// client.db(''); // Empty name
// client.db('a'.repeat(65)); // Too long
`




Database Operations

JoystickDB provides a clean, MongoDB-like chaining API for multi-database operations.

$3

Work with JoystickDB across multiple databases using the intuitive chaining pattern:

`javascript
// Get database and collection references
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');

const users = user_db.collection('users');
const products = inventory_db.collection('products');

// Insert documents into different databases
const user = await users.insert_one({
name: 'Alice Smith',
email: 'alice@example.com',
age: 28,
tags: ['developer', 'javascript'],
created_at: new Date()
});

const product = await products.insert_one({
name: 'Wireless Mouse',
price: 29.99,
category: 'electronics',
stock: 150
});

console.log('👤 Created user in user_management DB:', user.inserted_id);
console.log('📦 Created product in inventory DB:', product.inserted_id);

// Bulk operations across databases
const user_bulk = await users.bulk_write([
{ insert_one: { document: { name: 'Bob', age: 25, role: 'designer' } } },
{ insert_one: { document: { name: 'Carol', age: 30, role: 'manager' } } },
{ insert_one: { document: { name: 'Dave', age: 35, role: 'developer' } } }
]);

const product_bulk = await products.bulk_write([
{ insert_one: { document: { name: 'Keyboard', price: 79.99, category: 'electronics' } } },
{ insert_one: { document: { name: 'Monitor', price: 299.99, category: 'electronics' } } }
]);

console.log('👥 Created users:', user_bulk.inserted_count);
console.log('📦 Created products:', product_bulk.inserted_count);
`

$3

`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');

const users = user_db.collection('users');
const products = inventory_db.collection('products');

// Find in user database
const alice = await users.find_one({ name: 'Alice Smith' });
console.log('Found Alice in user DB:', alice);

// Find in inventory database
const electronics = await products.find({
category: 'electronics',
price: { $lte: 100 }
});
console.log('Found affordable electronics:', electronics.documents);

// Each database maintains separate query performance
const recent_users = await users.find({}, {
sort: { created_at: -1 },
limit: 10
});

const expensive_products = await products.find({
price: { $gte: 200 }
}, {
sort: { price: -1 },
limit: 5
});
`

$3

`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');

const users = user_db.collection('users');
const products = inventory_db.collection('products');

// Update in user database
await users.update_one(
{ name: 'Alice Smith' },
{
$set: {
age: 29,
last_updated: new Date(),
status: 'active'
},
$push: { tags: 'senior' }
}
);

// Update in inventory database
await products.update_one(
{ name: 'Wireless Mouse' },
{
$set: { price: 24.99 },
$inc: { stock: -5 }
}
);

// Delete from user database
await users.delete_one({
status: 'inactive',
last_login: { $lt: new Date(Date.now() - 3652460601000) }
});

// Delete from inventory database
await products.delete_one({
stock: { $lte: 0 },
discontinued: true
});

// Delete multiple documents at once
await users.delete_many({
status: 'inactive',
last_login: { $lt: new Date(Date.now() - 3652460601000) }
});

await products.delete_many({
stock: { $lte: 0 }
}, {
limit: 100 // Optional: limit number of deletions
});
`




$3

All query operators work consistently across all databases:

`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');

const users = user_db.collection('users');
const products = inventory_db.collection('products');

// Comparison operators work in any database
await users.find({ age: { $gt: 25 } });
await products.find({ price: { $gte: 50, $lte: 200 } });

// Text matching with regular expressions
await users.find({
name: { $regex: '^John' }
});

await products.find({
name: { $regex: 'wireless', $options: 'i' } // Case insensitive
});

// Array operations
await users.find({
tags: { $in: ['developer', 'designer'] }
});

await products.find({
categories: { $all: ['electronics', 'accessories'] }
});

// Complex queries across databases
await users.find({
age: { $gte: 25, $lte: 35 },
tags: { $in: ['developer', 'designer'] },
status: 'active',
name: { $regex: 'Smith$' }
});

await products.find({
price: { $gte: 100 },
category: 'electronics',
stock: { $gt: 0 },
name: { $regex: 'pro', $options: 'i' }
});
`

Indexing

Indexes work independently within each database, providing optimal performance for database-specific query patterns.

$3

JoystickDB automatically creates indexes based on query patterns within each database:

`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');

const users = user_db.collection('users');
const products = inventory_db.collection('products');

// First query in user database - might be slow
const user1 = await users.find_one({ email: 'john@example.com' });

// JoystickDB creates index for user database email queries
const user2 = await users.find_one({ email: 'jane@example.com' }); // Much faster!

// Separate automatic indexing for inventory database
const product1 = await products.find_one({ sku: 'MOUSE001' });
const product2 = await products.find_one({ sku: 'KEYBOARD002' }); // Auto-indexed!

// Each database maintains its own automatic indexing statistics
`

$3

Create and manage indexes independently for each database:

`javascript
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');

const users = user_db.collection('users');
const products = inventory_db.collection('products');

// Create indexes in user database
await users.create_index('email');
await users.create_index('username', { unique: true });
console.log('📇 User database indexes created');

// Create indexes in inventory database
await products.create_index('sku', { unique: true });
await products.create_index('category');
await products.create_index('price');
console.log('📇 Inventory database indexes created');

// List indexes for each database separately
const user_indexes = await users.get_indexes();
const product_indexes = await products.get_indexes();

console.log('📋 User DB indexes:', user_indexes.indexes);
console.log('📋 Inventory DB indexes:', product_indexes.indexes);

// Drop indexes independently
await users.drop_index('email');
await products.drop_index('category');
`

$3

Indexes are completely isolated between databases:

`javascript
const db1 = client.db('database1');
const db2 = client.db('database2');

const collection1 = db1.collection('items');
const collection2 = db2.collection('items');

// Create index in database1 only
await collection1.create_index('name');

// This collection in database2 has no indexes yet
const items1 = await collection1.find({ name: 'test' }); // Uses index
const items2 = await collection2.find({ name: 'test' }); // No index, full scan

// Create separate index in database2
await collection2.create_index('name');

// Now both have independent indexes
const items1_fast = await collection1.find({ name: 'test' }); // Uses db1 index
const items2_fast = await collection2.find({ name: 'test' }); // Uses db2 index
`

$3

`javascript
const inventory_db = client.db('inventory');
const products = inventory_db.collection('products');

// Without index: Searches through ALL products (slow for large datasets)
const expensive_products = await products.find({ price: { $gte: 1000 } });

// Create index on price
await products.create_index('price');

// With index: Jumps directly to expensive products (super fast!)
const expensive_products_fast = await products.find({ price: { $gte: 1000 } });

// Compound queries benefit from multiple indexes
await products.create_index('category');
const gaming_laptops = await products.find({
category: 'laptops',
price: { $gte: 800 }
}); // Uses both category and price indexes
`

HTTP API

JoystickDB provides RESTful HTTP endpoints for user management and external integrations. The HTTP API uses API key authentication for secure access.

$3

All HTTP API requests require an API key in the x-joystick-db-api-key header:

`javascript
const headers = {
'Content-Type': 'application/json',
'x-joystick-db-api-key': 'your-api-key-here'
};
`

$3

#### Create User

Create a new user with username, password, and role.

`http
POST /api/users
Content-Type: application/json
x-joystick-db-api-key: your-api-key

{
"username": "john_doe",
"password": "secure_password123",
"role": "read_write"
}
`

Response:
`json
{
"success": true,
"message": "User created successfully",
"username": "john_doe",
"role": "read_write"
}
`

Roles:
- read - Can only read data from all databases
- write - Can only write data to all databases
- read_write - Can read and write data to all databases (admin privileges)

#### Get User

Retrieve user information by username.

`http
GET /api/users?username=john_doe
x-joystick-db-api-key: your-api-key
`

Response:
`json
{
"success": true,
"user": {
"username": "john_doe",
"role": "read_write",
"created_at": "2024-01-15T10:30:00.000Z"
}
}
`

#### Update User

Update user password or role.

`http
PUT /api/users
Content-Type: application/json
x-joystick-db-api-key: your-api-key

{
"username": "john_doe",
"password": "new_secure_password456",
"role": "read"
}
`

Response:
`json
{
"success": true,
"message": "User updated successfully",
"username": "john_doe",
"updates": ["password", "role"]
}
`

#### Delete User

Remove a user from the system.

`http
DELETE /api/users?username=john_doe
x-joystick-db-api-key: your-api-key
`

Response:
`json
{
"success": true,
"message": "User deleted successfully",
"username": "john_doe"
}
`

$3

#### JavaScript/Node.js

`javascript
const API_KEY = 'your-api-key-here';
const BASE_URL = 'http://localhost:1983';

// Create user
const create_user = async (username, password, role) => {
const response = await fetch(${BASE_URL}/api/users, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-joystick-db-api-key': API_KEY
},
body: JSON.stringify({ username, password, role })
});

return await response.json();
};

// Get user
const get_user = async (username) => {
const response = await fetch(${BASE_URL}/api/users?username=${username}, {
headers: {
'x-joystick-db-api-key': API_KEY
}
});

return await response.json();
};

// Update user
const update_user = async (username, updates) => {
const response = await fetch(${BASE_URL}/api/users, {
method: 'PUT',
headers: {
'Content-Type': 'application/json',
'x-joystick-db-api-key': API_KEY
},
body: JSON.stringify({ username, ...updates })
});

return await response.json();
};

// Delete user
const delete_user = async (username) => {
const response = await fetch(${BASE_URL}/api/users?username=${username}, {
method: 'DELETE',
headers: {
'x-joystick-db-api-key': API_KEY
}
});

return await response.json();
};

// Usage examples
const demo = async () => {
// Create a new user
const create_result = await create_user('alice', 'secure123', 'read_write');
console.log('User created:', create_result);

// Get user info
const user_info = await get_user('alice');
console.log('User info:', user_info);

// Update user role
const update_result = await update_user('alice', { role: 'read' });
console.log('User updated:', update_result);

// Delete user
const delete_result = await delete_user('alice');
console.log('User deleted:', delete_result);
};
`

#### cURL Examples

`bash


Create user

curl -X POST http://localhost:1983/api/users \
-H "Content-Type: application/json" \
-H "x-joystick-db-api-key: your-api-key-here" \
-d '{"username": "alice", "password": "secure123", "role": "read_write"}'

Get user

curl -X GET "http://localhost:1983/api/users?username=alice" \
-H "x-joystick-db-api-key: your-api-key-here"

Update user

curl -X PUT http://localhost:1983/api/users \
-H "Content-Type: application/json" \
-H "x-joystick-db-api-key: your-api-key-here" \
-d '{"username": "alice", "role": "read"}'

Delete user

curl -X DELETE "http://localhost:1983/api/users?username=alice" \
-H "x-joystick-db-api-key: your-api-key-here"
`

$3

The HTTP API returns consistent error responses:

`json
{
"success": false,
"error": "Invalid API key"
}
`

Common Error Codes:
- 400 - Bad Request (invalid data)
- 401 - Unauthorized (invalid API key)
- 404 - Not Found (user doesn't exist)
- 409 - Conflict (user already exists)
- 500 - Internal Server Error

Replication

JoystickDB features a simplified primary/secondary replication system for high availability and read scaling. The replication system maintains database separation across all nodes and uses API key authentication for secure sync operations.

$3

- Primary Node: Main database server that accepts both read and write operations for all databases
- Secondary Nodes: Read-only copies that receive synchronized data from primary for all databases
- API Key Authentication: All sync operations between nodes use API key authentication for security
- Read-only Secondaries: Secondary nodes block write operations from clients, only accepting authenticated sync from primary
- Manual Failover: Admin operations allow promoting a secondary to primary when needed
- Database Isolation: Replication maintains complete database separation across all nodes

$3

#### 1. Configure Primary Node

`bash


Primary server configuration


export JOYSTICK_DB_SETTINGS='{
"port": 1983,
"primary": true,
"secondary_nodes": [
{ "ip": "192.168.1.100" },
{ "ip": "192.168.1.101" }
],
"secondary_sync_key": "/path/to/sync.key",
"sync_port": 1985
}'
`

#### 2. Configure Secondary Nodes

`bash


Secondary server configuration


export JOYSTICK_DB_SETTINGS='{
"port": 1983,
"primary": false,
"secondary_sync_key": "/path/to/sync.key",
"sync_port": 1985
}'
`

#### 3. Create Sync Key File

The sync key file contains the API key for authenticated sync operations:

`bash


Generate and save sync key (same key for all nodes)


echo "your-secure-sync-api-key" > /path/to/sync.key
chmod 600 /path/to/sync.key
`

$3

`javascript
// Connect to primary node
const client = joystickdb.client({
port: 1983,
authentication: {
username: 'admin',
password: 'your-password'
}
});

// Check sync system status (admin operation)
const sync_status = await client.admin_operation({
operation: 'get_sync_status'
});

console.log('🔄 Sync enabled:', sync_status.enabled);
console.log('📡 Connected secondaries:', sync_status.connected_nodes);
console.log('📊 Pending operations:', sync_status.pending_count);

// Promote secondary to primary (manual failover)
const promote_result = await client.admin_operation({
operation: 'promote_to_primary',
secondary_ip: '192.168.1.100'
});

console.log('🔄 Failover completed:', promote_result.success);

// Force sync to all secondaries
const sync_result = await client.admin_operation({
operation: 'force_sync'
});

console.log('🔄 Manual sync completed:', sync_result.synced_nodes);
`

$3

`javascript
// Connect to primary for writes (all databases supported)
const primary = joystickdb.client({
host: '192.168.1.10',
port: 1983,
authentication: {
username: 'admin',
password: 'your-password'
}
});

// Connect to secondary for reads only (all databases replicated)
const secondary = joystickdb.client({
host: '192.168.1.100',
port: 1983,
authentication: {
username: 'admin',
password: 'your-password'
}
});

// Write to primary (automatically synced to secondaries)
const user_db = primary.db('user_management');
const inventory_db = primary.db('inventory');

await user_db.collection('users').insert_one({
name: 'New User',
email: 'new@example.com'
});

await inventory_db.collection('products').insert_one({
name: 'New Product',
price: 99.99
});

// Read from secondary (reduces load on primary)
const secondary_user_db = secondary.db('user_management');
const secondary_inventory_db = secondary.db('inventory');

const users = await secondary_user_db.collection('users').find({});
const products = await secondary_inventory_db.collection('products').find({});

console.log('👥 Users from secondary:', users.documents.length);
console.log('📦 Products from secondary:', products.documents.length);

// Note: Write operations to secondary will be rejected
try {
await secondary_user_db.collection('users').insert_one({ name: 'Test' });
} catch (error) {
console.log('❌ Secondary is read-only:', error.message);
}
`

Administration

$3

`javascript
// Get comprehensive server statistics
const stats = await client.get_stats();

console.log('🖥️ Server Info:');
console.log(' Uptime:', stats.server.uptime_formatted);
console.log(' Version:', stats.server.version);
console.log(' Node.js:', stats.server.node_version);

console.log('💾 Memory Usage:');
console.log(' Heap Used:', Math.round(stats.memory.heapUsed / 1024 / 1024), 'MB');
console.log(' Heap Total:', Math.round(stats.memory.heapTotal / 1024 / 1024), 'MB');

console.log('🗄️ Database:');
console.log(' Size:', stats.database.used_space_mb, 'MB');
console.log(' Databases:', stats.database.databases);
console.log(' Collections:', stats.database.collections);

console.log('⚡ Performance:');
console.log(' Operations/sec:', stats.performance.ops_per_second);
console.log(' Avg Response Time:', stats.performance.avg_response_time_ms, 'ms');

console.log('🔌 Connections:');
console.log(' Active:', stats.connections.active);
console.log(' Total:', stats.connections.total);
`

$3

`javascript
// List all databases on the server
const databases = await client.list_databases();
console.log('📚 Available databases:', databases.databases);

// Get statistics for a specific database
const user_db = client.db('user_management');
const user_db_stats = await user_db.get_stats();
console.log('📊 User management DB stats:', user_db_stats);

// List collections in each database
for (const db_name of databases.databases) {
const db = client.db(db_name);
const collections = await db.list_collections();
console.log(📋 Collections in ${db_name}:, collections.collections);
}

// Create collections explicitly in different databases
const analytics_db = client.db('analytics');
await analytics_db.create_collection('events');
await analytics_db.create_collection('reports');

const inventory_db = client.db('inventory');
await inventory_db.create_collection('products');
await inventory_db.create_collection('categories');
`

$3

`javascript
// List documents across different databases
const user_db = client.db('user_management');
const inventory_db = client.db('inventory');

// List documents in user management database
const user_docs = await client.list_documents('users', {
database: 'user_management',
limit: 50,
skip: 0,
sort_field: 'created_at',
sort_order: 'desc'
});

// List documents in inventory database
const product_docs = await client.list_documents('products', {
database: 'inventory',
limit: 50,
skip: 0,
sort_field: 'price',
sort_order: 'asc'
});

console.log('📄 User documents:', user_docs.documents.length);
console.log('📄 Product documents:', product_docs.documents.length);

// Get specific documents by ID from different databases
const user_doc = await client.get_document('users', 'user-id-here', 'user_management');
const product_doc = await client.get_document('products', 'product-id-here', 'inventory');

// Advanced document querying across databases
const active_users = await client.query_documents('users', {
status: 'active',
last_login: { $gte: new Date(Date.now() - 302460601000) }
}, {
database: 'user_management',
limit: 100,
sort: { last_login: -1 }
});

const expensive_products = await client.query_documents('products', {
price: { $gte: 500 },
stock: { $gt: 0 }
}, {
database: 'inventory',
limit: 50,
sort: { price: -1 }
});
`

$3

`javascript
// Simple health check
const ping_result = await client.ping();
if (ping_result.ok === 1) {
console.log('✅ Database is healthy');
} else {
console.log('❌ Database is not responding');
}

// Reload server configuration without restart
const reload_result = await client.reload();
console.log('🔄 Configuration reloaded:', reload_result.message);

// Monitor performance over time
setInterval(async () => {
const stats = await client.get_stats();
console.log(📊 ${new Date().toISOString()}: ${stats.performance.ops_per_second} ops/sec, ${stats.connections.active} connections, ${stats.database.databases} databases);
}, 30000); // Every 30 seconds
`

Backup & Restore

JoystickDB includes automatic backup capabilities with S3 integration. Backups include all databases and maintain database isolation during restore operations.

$3

Configure automatic backups via the JOYSTICK_DB_SETTINGS environment variable:

`bash
export JOYSTICK_DB_SETTINGS='{
"backup": {
"enabled": true,
"schedule": "0 2 *",
"retention_days": 30,
"compression": true
},
"s3": {
"region": "us-east-1",
"bucket": "my-database-backups",
"access_key_id": "your-access-key",
"secret_access_key": "your-secret-key"
}
}'
`

$3

`javascript
// Create an immediate backup (includes all databases)
const backup_result = await client.backup_now();
console.log('💾 Backup created:', backup_result.filename);
console.log('📊 Backup size:', backup_result.size_mb, 'MB');
console.log('⏱️ Time taken:', backup_result.duration_ms, 'ms');
console.log('🗄️ Databases backed up:', backup_result.databases_count);

// List all available backups
const backups = await client.list_backups();
console.log('📋 Available backups:');
backups.backups.forEach(backup => {
console.log( 📦 ${backup.filename} (${backup.size_mb} MB) - ${backup.created_at});
console.log( Databases: ${backup.databases_count}, Collections: ${backup.collections_count});
});

// Restore from a specific backup (restores all databases)
const restore_result = await client.restore_backup('backup-2024-01-15-02-00-00.tar.gz');
console.log('🔄 Restore completed in:', restore_result.duration_ms, 'ms');
console.log('🗄️ Restored databases:', restore_result.databases_restored);
console.log('📊 Restored collections:', restore_result.collections_restored);
`

$3

`javascript
// Test your backups regularly
const test_backup = async () => {
console.log('🧪 Testing backup system...');

// Create a test backup
const backup = await client.backup_now();
console.log('✅ Backup created successfully');

// Verify backup exists and contains all databases
const backups = await client.list_backups();
const latest = backups.backups.find(b => b.filename === backup.filename);

if (latest && latest.size_mb > 0 && latest.databases_count > 0) {
console.log('✅ Backup verification passed');
console.log(📊 Backup contains ${latest.databases_count} databases);
} else {
console.log('❌ Backup verification failed');
}
};

// Run backup test monthly
setInterval(test_backup, 30 24 60 60 1000); // 30 days
`

Configuration

$3

`json
{
"port": 1983,
"cluster": true,
"worker_count": 4,

"database": {
"path": "./data",
"auto_map_size": true,
"map_size": 1073741824,
"max_dbs": 100
},

"authentication": {
"rate_limit": {
"max_attempts": 5,
"window_ms": 300000,
"lockout_duration_ms": 900000
}
},

"primary": true,
"secondary_nodes": [
{ "ip": "192.168.1.100" },
{ "ip": "192.168.1.101" }
],
"secondary_sync_key": "/path/to/sync.key",
"sync_port": 1985,

"s3": {
"region": "us-east-1",
"bucket": "my-backups",
"access_key_id": "AKIA...",
"secret_access_key": "...",
"endpoint": "https://s3.amazonaws.com"
},

"backup": {
"enabled": true,
"schedule": "0 2 *",
"retention_days": 30,
"compression": true
},

"logging": {
"level": "info",
"file": "./logs/joystickdb.log",
"max_size": "20m",
"max_files": "14d"
},

"performance": {
"connection_pool_size": 1000,
"idle_timeout": 600000,
"request_timeout": 5000,
"auto_index_threshold": 3
}
}
`

$3

`bash


Server settings

JOYSTICKDB_PORT=1983
JOYSTICKDB_CLUSTER=true
JOYSTICKDB_WORKER_COUNT=4

Database settings

JOYSTICKDB_DATABASE_PATH=./data
JOYSTICKDB_DATABASE_AUTO_MAP_SIZE=true
JOYSTICKDB_DATABASE_MAX_DBS=100

Simplified Replication settings

JOYSTICKDB_PRIMARY=true
JOYSTICKDB_SECONDARY_SYNC_KEY=/path/to/sync.key
JOYSTICKDB_SYNC_PORT=1985

S3 settings

JOYSTICKDB_S3_REGION=us-east-1
JOYSTICKDB_S3_BUCKET=my-backups
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key

Backup settings

JOYSTICKDB_BACKUP_ENABLED=true
JOYSTICKDB_BACKUP_RETENTION_DAYS=30

Logging

JOYSTICKDB_LOG_LEVEL=info
`

Production Deployment

$3

`dockerfile
FROM node:18-alpine

WORKDIR /app

Install dependencies

COPY package*.json ./
RUN npm ci --only=production

Copy application code

COPY . .

Create data and logs directories

RUN mkdir -p data logs

Expose port

EXPOSE 1983

Create volumes for persistent data

VOLUME ["/app/data", "/app/logs"]

Health check

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD node -e "
const client = require('./src/client/index.js').default.client({port: 1983, timeout: 2000, reconnect: false});
client.ping().then(() => process.exit(0)).catch(() => process.exit(1));
"

Start the server

CMD ["npm", "start"]
`

$3

`yaml
version: '3.8'

services:
joystickdb:
build: .
ports:
- "1983:1983"
volumes:
- ./data:/app/data
- ./logs:/app/logs
environment:
- NODE_ENV=production
- WORKER_COUNT=4
- JOYSTICKDB_LOG_LEVEL=info
- JOYSTICK_DB_SETTINGS={"port": 1983, "database": {"path": "./data", "max_dbs": 100}}
restart: unless-stopped
healthcheck:
test: ["CMD", "npm", "run", "health-check"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s

# Optional: Secondary node for replication
joystickdb-secondary:
build: .
ports:
- "1984:1984"
volumes:
- ./data-secondary:/app/data
- ./logs-secondary:/app/logs
environment:
- NODE_ENV=production
- JOYSTICKDB_PORT=1984
- JOYSTICK_DB_SETTINGS={"port": 1984, "database": {"path": "./data", "max_dbs": 100}}
restart: unless-stopped
depends_on:
- joystickdb
`

$3

`yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: joystickdb
labels:
app: joystickdb
spec:
replicas: 3
selector:
matchLabels:
app: joystickdb
template:
metadata:
labels:
app: joystickdb
spec:
containers:
- name: joystickdb
image: joystickdb:latest
ports:
- containerPort: 1983
env:
- name: WORKER_COUNT
value: "2"
- name: NODE_ENV
value: "production"
- name: JOYSTICKDB_LOG_LEVEL
value: "info"
- name: JOYSTICK_DB_SETTINGS
valueFrom:
configMapKeyRef:
name: joystickdb-config
key: settings.json
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
volumeMounts:
- name: data
mountPath: /app/data
livenessProbe:
exec:
command:
- npm
- run
- health-check
initialDelaySeconds: 30
periodSeconds: 30
readinessProbe:
exec:
command:
- npm
- run
- health-check
initialDelaySeconds: 5
periodSeconds: 10
volumes:
- name: data
persistentVolumeClaim:
claimName: joystickdb-data
---
apiVersion: v1
kind: ConfigMap
metadata:
name: joystickdb-config
data:
settings.json: |
{
"port": 1983,
"database": {
"path": "./data",
"auto_map_size": true,
"max_dbs": 100
},
"authentication": {
"rate_limit": {
"max_attempts": 5,
"window_ms": 300000
}
},
"backup": {
"enabled": true,
"schedule": "0 2 *",
"retention_days": 30
},
"logging": {
"level": "info"
}
}
---
apiVersion: v1
kind: Service
metadata:
name: joystickdb-service
spec:
selector:
app: joystickdb
ports:
- port: 1983
targetPort: 1983
type: LoadBalancer
`

$3

`javascript
// Health check endpoint
const health_check = async () => {
try {
const client = joystickdb.client({
port: 1983,
timeout: 2000,
reconnect: false
});

const result = await client.ping();
client.disconnect();

return result.ok === 1;
} catch (error) {
return false;
}
};

// Performance monitoring
const monitor_performance = async () => {
const client = joystickdb.client({
port: 1983,
authentication: {
username: 'admin',
password: process.env.DB_PASSWORD
}
});

const stats = await client.get_stats();

// Log metrics to your monitoring system
console.log('Metrics:', {
uptime: stats.server.uptime,
memory_usage: stats.memory.heapUsed,
ops_per_second: stats.performance.ops_per_second,
active_connections: stats.connections.active,
database_count: stats.database.databases,
total_collections: stats.database.collections,
database_size: stats.database.used_space_mb
});

client.disconnect();
};
`

API Reference

$3

#### Connection Management
- client.connect() - Establish connection to server
- client.disconnect() - Close connection to server
- client.ping() - Test server connectivity and responsiveness

#### Authentication & Setup
- client.setup() - Initial server setup (generates password) - DEPRECATED
- client.authenticate() - Manual authentication (usually automatic)

#### Multi-Database Interface
- client.db(database_name) - Get database interface for method chaining
- client.list_databases() - List all databases on the server

#### Database-Level Operations
- database.collection(collection_name) - Get collection interface within database
- database.list_collections() - List collections in the database
- database.get_stats() - Get database-specific statistics
- database.drop_database() - Drop the entire database (admin operation)
- database.create_collection(collection_name, options) - Create collection explicitly

#### CRUD Operations (Collection Chaining API)
- collection.insert_one(document, options) - Insert single document
- collection.find_one(filter, options) - Find single document
- collection.find(filter, options) - Find multiple documents
- collection.update_one(filter, update, options) - Update single document
- collection.delete_one(filter, options) - Delete single document
- collection.delete_many(filter, options) - Delete multiple documents
- collection.bulk_write(operations, options) - Bulk operations

#### Index Management (Collection Chaining API)
- collection.create_index(field, options) - Create index on field
- collection.upsert_index(field, options) - Create or update index (upsert)
- collection.drop_index(field) - Drop index on field
- collection.get_indexes() - List all indexes for collection

#### Auto-Indexing Management
- client.get_auto_index_stats() - Get automatic indexing statistics

#### Simplified Replication Management (via Admin Operations)
- client.admin_operation({ operation: 'get_sync_status' }) - Get sync system status and statistics
- client.admin_operation({ operation: 'promote_to_primary', secondary_ip: 'ip' }) - Promote secondary to primary (manual failover)
- client.admin_operation({ operation: 'force_sync' }) - Force synchronization with all secondaries

#### Administration
- client.get_stats() - Get comprehensive server statistics
- client.list_collections() - List all collections in default database (backward compatible)
- client.list_documents(collection, options) - List documents with pagination and sorting
- client.get_document(collection, document_id, database) - Get document by ID
- client.query_documents(collection, filter, options) - Advanced document query with filtering
- client.insert_document(collection, document) - Admin insert operation
- client.update_document(collection, document_id, update) - Admin update operation by ID
- client.delete_document(collection, document_id) - Admin delete operation by ID

#### Backup & Restore
- client.backup_now() - Create immediate backup (includes all databases)
- client.list_backups() - List all available backups
- client.restore_backup(backup_name) - Restore from backup (restores all databases)

#### Server Management
- client.reload() - Reload server configuration

$3

#### User Management (via HTTP)
- POST /api/users - Create new user with username, password, and role
- GET /api/users?username= - Get user information by username
- PUT /api/users - Update user password or role
- DELETE /api/users?username= - Delete user by username

All HTTP API endpoints require the x-joystick-db-api-key header for authentication.

$3

The client emits these events that you can listen to:

- connect - Connection established with server
- authenticated - Authentication successful, ready to use
- error - Error occurred (connection, authentication, etc.)
- disconnect - Connection closed
- reconnecting - Reconnection attempt in progress
- response - Unsolicited server response received

$3

#### Find Options
`javascript
{
limit: 10, // Maximum number of documents to return
skip: 0, // Number of documents to skip
sort: { field: 1 }, // Sort order (1 = ascending, -1 = descending)
projection: { field: 1 } // Fields to include (1) or exclude (0)
}
`

#### Update Options
`javascript
{
upsert: false // Create document if it doesn't exist
}
`

#### Index Options
`javascript
{
unique: false, // Enforce uniqueness
sparse: false // Skip documents missing the indexed field
}
`

$3

JoystickDB supports these MongoDB-style update operators:

- $set - Set field values
- $unset - Remove fields
- $inc - Increment numeric values
- $push - Add elements to arrays
- $pull - Remove elements from arrays

$3

JoystickDB supports these MongoDB-style query operators:

- $eq - Equal to
- $ne - Not equal to
- $gt - Greater than
- $gte - Greater than or equal to
- $lt - Less than
- $lte - Less than or equal to
- $in - Value in array
- $nin - Value not in array
- $exists - Field exists
- $regex - Regular expression match

$3

`javascript
try {
const user_db = client.db('user_management');
const users = user_db.collection('users');
const result = await users.insert_one({ name: 'John' });
} catch (error) {
if (error.message.includes('Authentication')) {
// Handle authentication error
console.error('Please check your username and password');
} else if (error.message.includes('timeout')) {
// Handle timeout error
console.error('Server is not responding');
} else if (error.message.includes('Connection')) {
// Handle connection error
console.error('Cannot connect to server');
} else if (error.message.includes('Invalid database name')) {
// Handle database naming error
console.error('Database name violates naming rules');
} else {
// Handle other errors
console.error('Database error:', error.message);
}
}
`

$3

1. Use Indexes: Create indexes on frequently queried fields in each database
2. Batch Operations: Use bulk_write for multiple operations
3. Limit Results: Always use limit for large result sets
4. Use Projections: Only fetch fields you need
5. Connection Pooling: Reuse client connections
6. Monitor Performance: Use get_stats() to track performance
7. Database Organization: Use separate databases to organize related data
8. Index Per Database: Create indexes specific to each database's query patterns

$3

1. Always Handle Errors: Wrap database calls in try-catch blocks
2. Use Environment Variables: Store passwords and config in env vars
3. Regular Backups: Enable automatic backups for production
4. Monitor Health: Set up health checks and monitoring
5. Use Replication: Set up replication for high availability
6. Index Strategy: Create indexes based on your query patterns per database
7. Connection Management: Properly close connections when done
8. Secure API Keys: Store API keys securely and rotate them regularly
9. User Management: Use appropriate roles (read/write/read_write) for users
10. HTTP API Security: Always use HTTPS in production for HTTP API calls
11. Database Organization: Use meaningful database names and organize data logically
12. Multi-Database Design: Separate concerns across databases (users, inventory, analytics, etc.)
13. Database Naming: Follow naming conventions and avoid reserved names
14. Backward Compatibility: Gradually migrate from single-database to multi-database API

---

For more examples and advanced usage, see the /test` directory in the repository.

Need Help? Check out the comprehensive test suite for real-world usage examples, or refer to the detailed configuration options above.