@camera.ui/rust-detector

A high-performance Rust-based motion detection library with GPU acceleration using WebGPU, optimized for real-time video surveillance and computer vision applications.

Features

- 🚀 GPU Acceleration - Hardware-accelerated motion detection using WebGPU compute shaders
- 📹 Motion Detection - Advanced frame differencing with Gaussian blur and morphological operations
- ⚡ SIMD Optimization - CPU-based SIMD acceleration for fallback operations
- 🎯 Bounding Box Detection - Automatic detection and tracking of moving objects
- 🔧 Multi-Backend Support - Vulkan, Metal, DirectX12, and OpenGL
- 📊 Real-time Performance - Optimized for streaming and live video processing
- 🔄 Adaptive Processing - Automatic fallback from GPU to CPU when needed
- 🎛️ Configurable Parameters - Adjustable threshold, blur, and dilation settings

Installation

``bash
npm install @camera.ui/rust-detector
`

Quick Start

$3

`typescript
import { ImageProcessor } from '@camera.ui/rust-detector';

const processor = new ImageProcessor(1920, 1080);

// Process frame and detect motion
const boundingBoxes = processor.processImage(
frameData, // Uint8Array grayscale frame
30, // threshold (0-255)
5, // blur kernel size
3, // dilation size
100 // minimum area
);

// Each bounding box: [x, y, x+width, y+height]
boundingBoxes.forEach(box => {
console.log(Motion detected at: x=${box[0]}, y=${box[1]}, w=${box[2]-box[0]}, h=${box[3]-box[1]});
});
`

$3

`typescript
import { GpuImageProcessor } from '@camera.ui/rust-detector';

const processor = new GpuImageProcessor(1920, 1080);

// Initialize with preferred GPU backend
const adapterDetails = await processor.initialize('vulkan');
console.log(Using GPU: ${adapterDetails.name} (${adapterDetails.backend}));

// Process frame with GPU acceleration
const boundingBoxes = processor.processImage(
frameData, // grayscale frame buffer
30, // threshold
5, // blur kernel size
3, // dilation size
100, // minimum area
false // debug mode (optional)
);
`

$3

`typescript
import { GpuImageProcessor, ImageProcessor, getGpuBackends } from '@camera.ui/rust-detector';

class MotionDetector {
private gpuProcessor?: GpuImageProcessor;
private cpuProcessor: ImageProcessor;
private useGpu = false;

constructor(width: number, height: number) {
this.cpuProcessor = new ImageProcessor(width, height);
}

async initialize(preferredBackend: string = 'auto') {
try {
const availableBackends = getGpuBackends();
console.log('Available GPU backends:', availableBackends);

this.gpuProcessor = new GpuImageProcessor(1920, 1080);
const adapter = await this.gpuProcessor.initialize(preferredBackend);

console.log(GPU acceleration enabled: ${adapter.name});
this.useGpu = true;
return adapter;
} catch (error) {
console.warn('GPU initialization failed, falling back to CPU:', error);
this.useGpu = false;
return null;
}
}

detectMotion(frame: Uint8Array, config: MotionConfig) {
const processor = this.useGpu ? this.gpuProcessor : this.cpuProcessor;

if (!processor?.initialized) {
throw new Error('Processor not initialized');
}

return processor.processImage(
frame,
config.threshold,
config.blurKernel,
config.dilationSize,
config.minArea,
config.debug
);
}

cleanup() {
this.gpuProcessor?.resetState();
this.cpuProcessor.resetState();
}
}

interface MotionConfig {
threshold: number; // Motion sensitivity (0-255)
blurKernel: number; // Gaussian blur kernel size
dilationSize: number; // Morphological dilation size
minArea: number; // Minimum area for motion detection
debug?: boolean; // Enable debug output
}
`

API Reference

$3

#### Initialization

`typescript
class GpuImageProcessor {
readonly processorType: string; // "GPU"
readonly initialized: boolean;

constructor(width: number, height: number);

// Initialize GPU with preferred backend
initialize(preferredBackend?: string): Promise;

// Reset GPU state and free resources
resetState(): void;
}
`

#### Motion Detection

`typescript
// GPU-accelerated motion detection
processImage(
frame: Buffer, // Grayscale frame data
threshold: number, // Motion threshold (0-255)
kernelSize: number, // Gaussian blur kernel size
dilateSize: number, // Morphological dilation size
minArea: number, // Minimum area for detection
debug?: boolean // Enable debug output
): number[][]; // Array of bounding boxes [x1, y1, x2, y2]
`

$3

#### Initialization

`typescript
class ImageProcessor {
readonly processorType: string; // "CPU"
readonly initialized: boolean;

constructor(width: number, height: number);

// Reset processor state
resetState(): void;
}
`

#### Motion Detection

`typescript
// CPU-based motion detection
processImage(
frame: Uint8Array, // Grayscale frame data
threshold: number, // Motion threshold (0-255)
kernelSize: number, // Gaussian blur kernel size
dilateSize: number, // Morphological dilation size
minArea: number // Minimum area for detection
): number[][]; // Array of bounding boxes [x1, y1, x2, y2]
`

$3

#### GPU Backend Options

`typescript
type GpuBackend = 'vulkan' | 'metal' | 'directx12' | 'opengl' | 'auto';

// Get available GPU backends
getGpuBackends(): string[];
`

#### Adapter Details

`typescript
interface AdapterDetails {
name: string; // GPU name
vendor: number; // Vendor ID
device: number; // Device ID
deviceType: string; // Device type
driver: string; // Driver name
driverInfo: string; // Driver info
backend: string; // Backend type
}
`

#### Bounding Box Format

`typescript
type BoundingBox = [number, number, number, number]; // [x1, y1, x2, y2]
``

Algorithm Details

$3

1. Gaussian Blur - Reduces noise and smooths the image
2. Frame Differencing - Compares current frame with previous blurred frame
3. Thresholding - Converts differences to binary motion mask
4. Morphological Dilation - Fills gaps and expands motion regions
5. Connected Component Analysis - Finds connected motion regions
6. Bounding Box Extraction - Calculates bounding boxes for each region

$3

The GPU implementation uses WebGPU compute shaders with optimized workgroup sizes:

- Horizontal Blur Pass - Separable Gaussian blur (horizontal)
- Vertical Blur Pass - Separable Gaussian blur (vertical)
- Difference & Dilate Pass - Combined frame differencing and morphological dilation

$3

The CPU implementation uses SIMD optimization for:

- Frame Differencing - Vectorized pixel comparison
- Connected Components - Efficient flood-fill algorithm
- Gaussian Blur - SIMD-accelerated convolution

Platform Support

$3

- Windows - x64 (DirectX12, Vulkan, OpenGL)
- macOS - Universal (Metal, Vulkan via MoltenVK)
- Linux - x64, ARM64 (Vulkan, OpenGL)
- FreeBSD - x64 (Vulkan, OpenGL)

$3

| Platform | Vulkan | Metal | DirectX12 | OpenGL |
|----------|--------|-------|-----------|--------|
| Windows | ✅ | ❌ | ✅ | ✅ |
| macOS | ✅* | ✅ | ❌ | ✅ |
| Linux | ✅ | ❌ | ❌ | ✅ |
| FreeBSD | ✅ | ❌ | ❌ | ✅ |

*Vulkan on macOS requires MoltenVK

Requirements

- Modern GPU with WebGPU support (for GPU acceleration)
- Platform-specific requirements:
- Windows: DirectX 12 or Vulkan drivers
- macOS: Metal support (macOS 10.14+)
- Linux: Vulkan or OpenGL drivers

Contributing

Contributions are welcome! Please read our contributing guidelines and submit pull requests to our repository.

License

MIT

---

*Part of the camera.ui ecosystem - A comprehensive camera management solution