High-performance ONNX Runtime bindings for React Native with hardware acceleration support (NNAPI, CoreML, XNNPACK)
npm install react-native-nitro-onnxruntimeHigh-performance ONNX Runtime bindings for React Native, built with Nitro Modules for maximum performance.
- ⚡ Blazing Fast: Built with Nitro Modules for zero-overhead JSI bindings
- 🎯 Hardware Acceleration: Support for NNAPI (Android), CoreML (iOS), and XNNPACK
- 🔄 Modern API: Promise-based async API with TypeScript support
- 📦 Flexible Model Loading: Load models from files, URLs, or buffers
- 🎨 Full Type Support: Complete TypeScript definitions
- 🔧 Configurable: Extensive session options for optimization
``sh`
npm install react-native-nitro-onnxruntime react-native-nitro-modules
> Note: react-native-nitro-modules is required as this library relies on Nitro Modules.
`typescript
import ort from 'react-native-nitro-onnxruntime';
// Load a model
const session = await ort.loadModel('path/to/model.onnx');
// Get input/output information
console.log('Inputs:', session.inputNames);
console.log('Outputs:', session.outputNames);
// Prepare input data
const inputData = new Float32Array(1 3 224 * 224); // Batch=1, Channels=3, Height=224, Width=224
// ... fill inputData with your data
// Run inference
const results = await session.run({
[session.inputNames[0].name]: inputData.buffer
});
// Access output
const outputBuffer = results[session.outputNames[0].name];
const outputData = new Float32Array(outputBuffer);
console.log('Output:', outputData);
`
Models can be loaded using require() for bundled assets. The library automatically copies the model from your app bundle to the device's file system and caches it:
`typescript`
// Load from bundled asset
const session = await ort.loadModel(require('./assets/model.onnx'));
You can also load from a URL:
`typescript`
// Load from URL
const session = await ort.loadModel({
url: 'http://example.com/model.onnx'
});
Or from a file path:
`typescript`
// Load from file path
const session = await ort.loadModel('/path/to/model.onnx');
Note: When using require() or { url }, the model is automatically copied to the app's files directory and cached. Subsequent loads will use the cached file for faster initialization.
#### Android (NNAPI)
`typescript
const session = await ort.loadModel('model.onnx', {
executionProviders: ['nnapi']
});
// Or with options
const session = await ort.loadModel('model.onnx', {
executionProviders: [{
name: 'nnapi',
useFP16: true, // Use FP16 precision
cpuDisabled: true, // Disable CPU fallback
}]
});
`
#### iOS (CoreML)
`typescript
const session = await ort.loadModel('model.onnx', {
executionProviders: ['coreml']
});
// Or with options
const session = await ort.loadModel('model.onnx', {
executionProviders: [{
name: 'coreml',
useCPUOnly: false,
onlyEnableDeviceWithANE: true, // Only use devices with Apple Neural Engine
}]
});
`
#### XNNPACK (Cross-platform)
`typescript`
const session = await ort.loadModel('model.onnx', {
executionProviders: ['xnnpack']
});
`typescript`
const session = await ort.loadModel('model.onnx', {
// Thread configuration
intraOpNumThreads: 4,
interOpNumThreads: 2,
// Graph optimization
graphOptimizationLevel: 'all', // 'disabled' | 'basic' | 'extended' | 'all'
// Memory settings
enableCpuMemArena: true,
enableMemPattern: true,
// Execution mode
executionMode: 'sequential', // 'sequential' | 'parallel'
// Logging
logId: 'MyModel',
logSeverityLevel: 2, // 0=Verbose, 1=Info, 2=Warning, 3=Error, 4=Fatal
// Execution providers
executionProviders: ['nnapi', 'cpu']
});
For advanced use cases, you can load models directly from an ArrayBuffer:
`typescript
import RNFS from 'react-native-fs';
// Option 1: Load from file system
const modelPath = 'path/to/model.onnx';
const base64Data = await RNFS.readFile(modelPath, 'base64');
const arrayBuffer = Uint8Array.from(atob(base64Data), c => c.charCodeAt(0)).buffer;
const session = await ort.loadModelFromBuffer(arrayBuffer, {
executionProviders: ['nnapi']
});
// Option 2: Load from network
const response = await fetch('https://example.com/model.onnx');
const arrayBuffer = await response.arrayBuffer();
const session = await ort.loadModelFromBuffer(arrayBuffer, {
executionProviders: ['coreml']
});
`
Use cases for loadModelFromBuffer:
- Loading models from encrypted storage
- Downloading models from authenticated endpoints
- Processing models before loading (e.g., decompression)
- Dynamic model generation
The library provides a convenient React hook for loading models:
`typescript
import { useLoadModel } from 'react-native-nitro-onnxruntime';
function MyComponent() {
const modelState = useLoadModel(require('./assets/model.onnx'), {
executionProviders: ['nnapi']
});
if (modelState.state === 'loading') {
return
}
if (modelState.state === 'error') {
return
}
// modelState.state === 'loaded'
const session = modelState.model;
// Use session for inference
const runInference = async () => {
const input = new Float32Array(224 224 3);
const results = await session.run({
[session.inputNames[0].name]: input.buffer
});
};
return ;
}
`
Sessions are automatically cleaned up by Nitro Modules when they go out of scope. However, you can manually dispose of a session to free memory immediately:
`typescript`
// Optional: Dispose of session early to free memory immediately
session.dispose();
When to call dispose():
- Loading multiple models and want to free memory between loads
- Memory-constrained environments
- Want immediate cleanup instead of waiting for garbage collection
Note: You don't need to call dispose() in most cases - Nitro Modules will automatically clean up when the session is no longer referenced.
Returns the ONNX Runtime version string.
`typescript`
const version = ort.getVersion();
console.log('ONNX Runtime version:', version);
Load an ONNX model from various sources.
Parameters:
- source: string | number | { url: string } - Model source:string
- : File path on devicenumber
- : require() asset (automatically copied to files directory){ url: string }
- : URL to download from (automatically cached)options
- : SessionOptions (optional) - Configuration options
Returns: Promise
Example:
`typescript
// From bundled asset
const session1 = await ort.loadModel(require('./model.onnx'));
// From file path
const session2 = await ort.loadModel('/data/user/model.onnx');
// From URL
const session3 = await ort.loadModel({ url: 'https://example.com/model.onnx' });
`
Load an ONNX model from an ArrayBuffer.
Parameters:
- buffer: ArrayBuffer - Model dataoptions
- : SessionOptions (optional) - Configuration options
Returns: Promise
Manually copy a model file from a bundled asset or URL to the device's file system. This is useful if you want to copy the file before loading it.
Parameters:
- source: number | { url: string } - Model source to copy
Returns: Promise - Path to the copied file
Example:
`typescript
// Copy bundled asset
const modelPath = await copyFile(require('./model.onnx'));
console.log('Model copied to:', modelPath);
// Now you can load it
const session = await ort.loadModel(modelPath);
// Or copy from URL
const urlPath = await copyFile({ url: 'https://example.com/model.onnx' });
const session2 = await ort.loadModel(urlPath);
`
Note: loadModel() calls this automatically when you pass a require() or { url }, so you typically don't need to call this manually.
React hook for loading models with state management.
Parameters:
- source: Same as loadModel()options
- : SessionOptions (optional)
Returns: OnnxRuntimePlugin`typescript`
type OnnxRuntimePlugin =
| { model: InferenceSession; state: 'loaded' }
| { model: undefined; state: 'loading' }
| { model: undefined; state: 'error'; error: Error };
#### session.inputNames
Array of input tensor information:
`typescript`
type Tensor = {
name: string;
dims: number[]; // Shape, negative values indicate dynamic dimensions
type: string; // 'float32', 'int64', etc.
};
#### session.outputNames
Array of output tensor information (same format as inputNames).
#### session.run(feeds)
Run inference with the given inputs.
Parameters:
- feeds: Record - Map of input names to ArrayBuffers
Returns: Promise - Map of output names to ArrayBuffers
#### session.dispose()
Manually free the session and release resources immediately.
Note: This is optional - sessions are automatically cleaned up by Nitro Modules when they go out of scope. Only call this if you need immediate memory cleanup.
`typescript
type SessionOptions = {
intraOpNumThreads?: number;
interOpNumThreads?: number;
graphOptimizationLevel?: 'disabled' | 'basic' | 'extended' | 'all';
enableCpuMemArena?: boolean;
enableMemPattern?: boolean;
executionMode?: 'sequential' | 'parallel';
logId?: string;
logSeverityLevel?: number;
executionProviders?: (string | ProviderOptions)[];
};
type ProviderOptions = {
name: 'nnapi' | 'coreml' | 'xnnpack';
// NNAPI options (Android)
useFP16?: boolean;
useNCHW?: boolean;
cpuDisabled?: boolean;
cpuOnly?: boolean;
// CoreML options (iOS)
useCPUOnly?: boolean;
useCPUAndGPU?: boolean;
enableOnSubgraph?: boolean;
onlyEnableDeviceWithANE?: boolean;
};
`
- ✅ Android (API 21+)
- ✅ iOS (13.0+)
- float32 (Float)float64
- (Double)int8
- uint8
- int16
- int32
- int64
- bool
-
1. Use Hardware Acceleration: Enable NNAPI (Android) or CoreML (iOS) for better performance
2. Optimize Thread Count: Set intraOpNumThreads based on your device's CPU coresgraphOptimizationLevel: 'all'
3. Enable Graph Optimization: Use for productionuseFP16
4. Reuse Sessions: Create the session once and reuse it for multiple inferences
5. Use FP16: Enable on NNAPI for faster inference with acceptable accuracy loss
The library exports the following:
`typescript
import ort, { useLoadModel, copyFile } from 'react-native-nitro-onnxruntime';
// Using default export object
const session = await ort.loadModel(require('./model.onnx'));
const session2 = await ort.loadModelFromBuffer(arrayBuffer);
// Or using destructured methods
const { loadModel, loadModelFromBuffer } = ort;
const session3 = await loadModel(require('./model.onnx'));
// Utility functions
const modelPath = await copyFile(require('./model.onnx'));
// React hook
function MyComponent() {
const modelState = useLoadModel(require('./model.onnx'));
// ...
}
``
See the example directory for a complete working example with speed comparisons.
See the contributing guide to learn how to contribute to the repository and the development workflow.
MIT
---
Made with Nitro Modules and create-react-native-library