High-performance document scanner with ONNX Runtime + OpenCV
npm install react-native-document-scanner-ai🚀 High-Performance Document Scanner with AI
Advanced document scanning library for React Native with ONNX Runtime + OpenCV integration for real-time document detection, scanning, and processing.
🚀 High Performance
- Native C++ implementation with ONNX Runtime
- Real-time document detection and segmentation
- Optimized for mobile devices with hardware acceleration
- TurboModule + JSI architecture for native performance
🤖 AI-Powered
- Uses ONNX Runtime for ML inference
- Pre-trained YOLOv8 document segmentation models included
- Advanced edge detection and perspective correction
- Intelligent auto-capture functionality
📱 Cross-Platform
- iOS and Android support with consistent APIs
- Automatic native dependency management
- Camera integration with react-native-vision-camera
🎯 Smart Features
- Auto-capture when document is stable
- Real-time preview with quadrilateral overlay
- Multiple enhancement modes (B&W, contrast boost)
- Batch processing support
- Configurable confidence thresholds
``sh`
npm install react-native-document-scanner-aior
yarn add react-native-document-scanner-ai
> ⚠️ Important: Use npx react-native-document-scanner-ai setup commands from your app project, NOT npm run setup:windows. The npm run scripts are only for library development.
The library includes automated setup scripts that configure all required models and dependencies:
`shFor Windows users
npx react-native-document-scanner-ai setup
This will:
- ✅ Download and configure the ONNX model
- ✅ Set up platform-specific dependencies
- ✅ Configure native module linking
- ✅ Verify the installation
$3
After installation, verify everything is working correctly:
`sh
npx react-native-document-scanner-ai verify-setup
`$3
If automatic setup fails, you can manually generate the required ONNX model:
Using NPX commands:
`sh
Generate model (cross-platform)
npx react-native-document-scanner-ai generate-model
`Manual Python approach:
`sh
pip install ultralytics
python -c "from ultralytics import YOLO; model = YOLO('yolov8n.pt'); model.export(format='onnx', imgsz=640)"
`The model will be automatically placed in the correct platform directories (
models/ for general use, ios/ and android/assets/ for platform-specific deployment).Platform-Specific Setup
$3
1. Automatic Dependencies: The library automatically includes OpenCV and ONNX Runtime dependencies via CocoaPods
2. Install iOS Dependencies:
`sh
cd ios && pod install && cd ..
`3. Model Configuration: The ONNX model is automatically bundled with the library
4. Permissions: Add camera permissions to your
Info.plist:
`xml
NSCameraUsageDescription
This app needs access to camera to scan documents
NSPhotoLibraryUsageDescription
This app needs access to photo library to process images
`5. Additional Configuration (if needed):
`xml
NSCameraUsageDescription
Camera access is required to scan documents
NSPhotoLibraryAddUsageDescription
Photo library access is needed to save scanned documents
`$3
1. Automatic Dependencies: The library automatically includes OpenCV and ONNX Runtime dependencies via Gradle
2. Model Configuration: The ONNX model is included in the assets folder and bundled automatically
3. Permissions: Add the following permissions to your
android/app/src/main/AndroidManifest.xml:
`xml
`4. Gradle Configuration: Add to your
android/app/build.gradle (if not already present):
`gradle
android {
packagingOptions {
pickFirst '**/libc++_shared.so'
pickFirst '**/libjsc.so'
pickFirst '**/libfbjni.so'
}
// Increase heap size for ONNX processing
dexOptions {
javaMaxHeapSize "4g"
}
}
`5. ProGuard Configuration (for release builds):
`proguard
Keep ONNX Runtime classes
-keep class ai.onnxruntime.* { ; }
-keep class com.documentscannerai.* { ; }Keep OpenCV classes
-keep class org.opencv.* { ; }
`Post-Installation Steps
$3
Ensure your
metro.config.js includes the following to properly handle ONNX and model files:`javascript
const { getDefaultConfig } = require('metro-config');module.exports = (async () => {
const config = await getDefaultConfig();
// Add support for .onnx files
config.resolver.assetExts.push('onnx', 'pt');
return config;
})();
`$3
For camera functionality, install react-native-vision-camera:
`sh
npm install react-native-vision-camera
or
yarn add react-native-vision-camera
`$3
Run the verification script to ensure everything is properly configured:
`sh
npx react-native-document-scanner-ai verify-setup
`This will check:
- ✅ Package installation
- ✅ Model file presence
- ✅ Platform dependencies
- ✅ Permissions configuration
- ✅ Build configuration
Usage
$3
To avoid the "Tried to access a JS module before the React instance was fully set up" error, use one of these safe import patterns:
#### Option 1: Lazy Loading (Recommended)
`tsx
import React, { useEffect, useState } from 'react';function MyComponent() {
const [scanImage, setScanImage] = useState(null);
const [scannerReady, setScannerReady] = useState(false);
useEffect(() => {
// Load the scanner module after React is ready
const loadScanner = async () => {
try {
const scannerModule = await import('react-native-document-scanner-ai');
setScanImage(() => scannerModule.scanImage);
setScannerReady(true);
} catch (error) {
console.warn('Failed to load document scanner:', error);
}
};
loadScanner();
}, []);
const handleScan = async (imageUri: string) => {
if (!scanImage || !scannerReady) {
throw new Error('Document scanner not ready');
}
return await scanImage(imageUri);
};
// Your component JSX...
}
`#### Option 2: Try-Catch Import
`tsx
let scanImage = null;try {
const scannerModule = require('react-native-document-scanner-ai');
scanImage = scannerModule.scanImage;
} catch (error) {
console.warn('Failed to import react-native-document-scanner-ai:', error);
}
// Use scanImage safely
if (scanImage) {
const result = await scanImage(imageUri);
}
`#### Option 3: Built-in Safe Import Helper
`tsx
import { createSafeImport } from 'react-native-document-scanner-ai';const scanner = createSafeImport();
if (scanner.isAvailable && scanner.scanImage) {
const result = await scanner.scanImage(imageUri);
}
`> 📖 More details: See SAFE_IMPORT.md for comprehensive examples and troubleshooting.
$3
`tsx
import { scanImage, scanFrame } from 'react-native-document-scanner-ai';// Scan a single image
const result = await scanImage('file://path/to/image.jpg');
console.log('Detected corners:', result.quadrilateral);
console.log('Confidence:', result.confidence);
`$3
Scan a static image file for document detection:
`tsx
import { scanImage } from 'react-native-document-scanner-ai';const scanDocument = async (imageUri: string) => {
try {
const result = await scanImage(imageUri, {
enhance: 'contrast',
saveOutput: true,
outputFormat: 'jpg',
outputQuality: 90,
maxSize: 1024, // Resize for faster processing
});
if (result.confidence > 0.8) {
console.log('Document detected with high confidence!');
console.log('Corners:', result.quadrilateral);
console.log('Enhanced image saved at:', result.outputUri);
} else {
console.log('Low confidence detection, manual review needed');
}
} catch (error) {
console.error('Scanning failed:', error);
}
};
`$3
Process camera frames in real-time for live document detection:
`tsx
import React, { useRef, useState } from 'react';
import { View, Text, TouchableOpacity } from 'react-native';
import { Camera, useFrameProcessor, useCameraDevices } from 'react-native-vision-camera';
import { scanFrame } from 'react-native-document-scanner-ai';
import { runOnJS } from 'react-native-reanimated';function DocumentCameraScreen() {
const camera = useRef(null);
const devices = useCameraDevices();
const device = devices.back;
const [detectedDocument, setDetectedDocument] = useState(null);
const frameProcessor = useFrameProcessor((frame) => {
'worklet';
// Convert frame to RGBA array
const frameData = frame.toArrayBuffer();
const rgba = new Uint8Array(frameData);
// Process on JS thread
runOnJS(async (rgbaData: Uint8Array, width: number, height: number) => {
try {
const result = await scanFrame(rgbaData, width, height, {
autoCapture: true,
captureConfidence: 0.85,
captureConsecutiveFrames: 5, // Require 5 stable frames
maxProcessingFps: 10, // Limit processing to 10 FPS
saveOutput: true,
});
if (result.outputUri) {
// Document auto-captured!
setDetectedDocument(result);
console.log('Auto-captured document:', result.outputUri);
}
} catch (error) {
console.error('Frame processing error:', error);
}
})(rgba, frame.width, frame.height);
}, []);
const captureManually = async () => {
if (camera.current) {
const photo = await camera.current.takePhoto({
quality: 90,
enableAutoRedEyeReduction: true,
});
const result = await scanImage(photo.path, {
enhance: 'contrast',
saveOutput: true,
});
setDetectedDocument(result);
}
};
if (!device) {
return Camera not available ;
}
return (
ref={camera}
device={device}
isActive={true}
frameProcessor={frameProcessor}
photo={true}
style={{ flex: 1 }}
/>
onPress={captureManually}
style={{
position: 'absolute',
bottom: 50,
alignSelf: 'center',
backgroundColor: 'white',
padding: 15,
borderRadius: 50,
}}
>
📷 Capture
);
}
`$3
Customize scanning behavior with detailed options:
`tsx
import { scanImage, ScanOptions } from 'react-native-document-scanner-ai';const advancedScanOptions: ScanOptions = {
// Model configuration
onnxModel: 'custom_model.onnx', // Use custom model
threshold: 0.5, // Segmentation sensitivity
// Processing options
maxSize: 1024, // Max image dimension
enhance: 'bw', // Black & white enhancement
returnMask: true, // Get segmentation mask
// Output configuration
saveOutput: true,
outputFormat: 'png',
outputQuality: 95,
// Auto-capture settings
autoCapture: true,
captureConfidence: 0.9, // High confidence required
captureConsecutiveFrames: 8, // More stable frames
maxProcessingFps: 5, // Conservative processing rate
};
const result = await scanImage(imageUri, advancedScanOptions);
// Access additional outputs
if (result.maskUri) {
console.log('Segmentation mask saved at:', result.maskUri);
}
`$3
Process multiple images efficiently:
`tsx
const processBatch = async (imageUris: string[]) => {
const results = await Promise.all(
imageUris.map(uri =>
scanImage(uri, {
enhance: 'contrast',
maxSize: 512, // Smaller size for batch processing
saveOutput: false, // Skip saving for speed
})
)
);
const validDocuments = results.filter(r => r.confidence > 0.7);
console.log(Found ${validDocuments.length} valid documents);
return validDocuments;
};
`Available Scripts
The library provides different commands depending on whether you're a consumer or developer:
$3
Use these NPX commands from your React Native project:
`sh
Setup the library and download models
npx react-native-document-scanner-ai setupVerify installation is working
npx react-native-document-scanner-ai verify-setupGenerate ONNX model (if needed)
npx react-native-document-scanner-ai generate-model
`Note: Do NOT use
npm run setup:windows in your app - that only works during library development.$3
These scripts are available when developing the library itself:
`sh
Setup Scripts (library development only)
npm run setup # Cross-platform setup
npm run setup:windows # Windows-specific setup
npm run setup:unix # macOS/Linux setupModel Management (library development only)
npm run generate:model # Generate ONNX model (Unix)
npm run generate:model:windows # Generate ONNX model (Windows)Verification (library development only)
npm run verify-setup # Verify installationDevelopment Tools
npm run clean # Clean build directories
npm run build:android # Build Android example
npm run build:ios # Build iOS example
npm run auto-publish # Automated publishing
`API Reference
$3
`typescript
export type ScanOptions = {
onnxModel?: string; // Custom model path
threshold?: number; // Segmentation threshold (0-1)
maxSize?: number; // Max image size for processing
enhance?: 'none' | 'bw' | 'contrast'; // Image enhancement
returnMask?: boolean; // Return segmentation mask
saveOutput?: boolean; // Save processed image
outputFormat?: 'jpg' | 'png'; // Output format
outputQuality?: number; // Quality (0-100)
autoCapture?: boolean; // Enable auto-capture
captureConfidence?: number; // Min confidence for auto-capture
captureConsecutiveFrames?: number; // Stable frames required
maxProcessingFps?: number; // Processing throttle
};export type ScanResult = {
quadrilateral: [number, number][]; // Detected document corners
confidence: number; // Detection confidence (0-1)
outputUri?: string; // Processed image path
maskUri?: string; // Segmentation mask path
};
`$3
####
scanImage(uri: string, options?: ScanOptions): PromiseScans a single image file for document detection and processing.
Parameters:
-
uri: File URI of the image to process
- options: Scan configuration optionsReturns: Promise resolving to scan result
####
scanFrame(rgba: Uint8Array, width: number, height: number, options?: ScanOptions): PromiseProcesses a camera frame for real-time document detection.
Parameters:
-
rgba: RGBA pixel data as Uint8Array
- width: Frame width in pixels
- height: Frame height in pixels
- options: Scan configuration optionsReturns: Promise resolving to scan result
Troubleshooting
$3
#### 0. "Tried to access a JS module before the React instance was fully set up"
`
Tried to access a JS module before the React instance was fully set up.
Calls to ReactContext#getJSModule should only happen once initialize() has been called on your native module.
`
Problem: The native module is being accessed before React Native is fully initialized.Solution: Use the safe import patterns described in the Usage section above:
1. Lazy Loading (Recommended):
`tsx
const [scanImage, setScanImage] = useState(null);
useEffect(() => {
import('react-native-document-scanner-ai').then(module => {
setScanImage(() => module.scanImage);
});
}, []);
`2. Try-Catch Import:
`tsx
let scanImage = null;
try {
const module = require('react-native-document-scanner-ai');
scanImage = module.scanImage;
} catch (error) {
console.warn('Scanner not available:', error);
}
`3. Always restart Metro after installing:
npx react-native start --reset-cache#### 1. "Missing script" Error
`
npm error Missing script: "setup:windows"
`
Problem: You're trying to run library development scripts from your app project.Solution: Use NPX commands instead:
`sh
❌ Wrong (only works in library development)
npm run setup:windows✅ Correct (use in your app)
npx react-native-document-scanner-ai setup
`#### 1. Model Not Found Error
`
Error: ONNX model not found at path
`
Solution: Run the setup script to download the model:
`sh
npx react-native-document-scanner-ai setup
`#### 2. Build Errors on Android
`
Error: Failed to resolve: ai.onnxruntime
`
Solution: Ensure your android/app/build.gradle includes the packaging options:
`gradle
android {
packagingOptions {
pickFirst '**/libc++_shared.so'
pickFirst '**/libjsc.so'
}
}
`#### 3. iOS Pod Install Issues
`
[!] CocoaPods could not find compatible versions for pod "ONNX"
`
Solution: Update CocoaPods and clear cache:
`sh
cd ios
rm -rf Pods Podfile.lock
pod install --repo-update
`#### 4. Camera Permission Denied
Solution: Ensure permissions are properly configured in your platform files and request permissions at runtime.
#### 5. Performance Issues
- Reduce
maxSize option for faster processing
- Lower maxProcessingFps for real-time processing
- Use enhance: 'none' to skip post-processing$3
1. Check Setup: Run
npx react-native-document-scanner-ai verify-setup
2. Review Logs: Enable verbose logging in development
3. Platform Issues: Check platform-specific setup instructions
4. Create Issue: GitHub IssuesPerformance Tips
1. Image Size: Use
maxSize option to limit processing resolution (recommended: 1024px)
2. Frame Rate: Set maxProcessingFps to throttle real-time processing (recommended: 5-10 FPS)
3. Model Selection: The included YOLOv8n model is optimized for mobile devices
4. Threading: Processing runs on background threads automatically
5. Memory: Enable returnMask: false unless segmentation masks are needed
6. Auto-capture: Use higher captureConsecutiveFrames for more stable capturesModel Information
$3
- document_segmentation.onnx: Pre-trained YOLOv8n model for document detection
- Input Format: RGB images, normalized to [0,1], 640x640 resolution
- Output Format: Segmentation masks with confidence scores
- Model Size: ~6MB (optimized for mobile)
$3
You can use custom ONNX models by specifying the
onnxModel path:`tsx
const result = await scanImage(imageUri, {
onnxModel: 'path/to/custom_model.onnx',
threshold: 0.5,
});
`Model Requirements:
- Input:
[1, 3, H, W] tensor (RGB image, normalized 0-1)
- Output: [1, 1, H, W] tensor (segmentation mask, 0-1 values)
- Format: ONNX with standard operators
- Optimization: Use ONNX Runtime optimization tools for best performance$3
For training custom document detection models:
1. Datasets: Use DocLayNet, PubLayNet, or custom document datasets
2. Framework: Train with YOLOv8, Detectron2, or similar frameworks
3. Export: Convert to ONNX format with appropriate input/output shapes
4. Optimization: Use ONNX Runtime tools for mobile optimization
Example App
The example app demonstrates all library features:
- ✨ Real-time camera document detection with live preview
- 🎯 Auto-capture functionality with confidence thresholds
- 🎨 Image enhancement modes (B&W, contrast boost, original)
- 📐 Quadrilateral overlay visualization
- 📱 Cross-platform implementation (iOS & Android)
- 🔧 Configuration options and performance tuning
Run the example:
`sh
git clone https://github.com/AaronAnablon/React-Native-Document-Cleaner.git
cd React-Native-Document-Cleaner/example
npm installiOS
cd ios && pod install && cd ..
npx react-native run-iosAndroid
npx react-native run-android
`File Structure
When properly installed, your project should include:
`
your-project/
├── node_modules/
│ └── react-native-document-scanner-ai/
│ ├── lib/ # Compiled TypeScript
│ ├── src/ # Source TypeScript files
│ ├── android/ # Android native code
│ ├── ios/ # iOS native code
│ ├── cpp/ # C++ core implementation
│ ├── models/ # ONNX models
│ │ └── document_segmentation.onnx
│ ├── assets/ # Additional assets
│ │ └── yolov8n.pt # Training checkpoint
│ ├── scripts/ # Setup and utility scripts
│ └── bin/ # CLI tools
├── android/
│ └── app/
│ └── src/main/assets/ # Android model deployment
│ └── document_segmentation.onnx
└── ios/
└── document_segmentation.onnx # iOS model deployment
`Package Information
$3
The npm package includes:
- ✅ Compiled JavaScript/TypeScript libraries
- ✅ Native Android and iOS code
- ✅ C++ core implementation
- ✅ Pre-trained ONNX models
- ✅ Setup and verification scripts
- ✅ CLI tools for easy management
- ✅ Documentation and examples
$3
Check your installed version:
`sh
npm list react-native-document-scanner-ai
`Update to latest:
`sh
npm update react-native-document-scanner-ai
`Quick Reference
$3
`sh
Install package
npm install react-native-document-scanner-aiSetup (automated)
npx react-native-document-scanner-ai setupVerify installation
npx react-native-document-scanner-ai verify-setupGenerate model (if needed)
npx react-native-document-scanner-ai generate-model
`$3
`sh
iOS
cd ios && pod installAndroid (permissions in AndroidManifest.xml)
Build configuration in build.gradle
`$3
`tsx
import { scanImage } from 'react-native-document-scanner-ai';const result = await scanImage('file://path/to/image.jpg', {
enhance: 'contrast',
saveOutput: true,
});
`$3
`tsx
import { scanFrame } from 'react-native-document-scanner-ai';
// Use with react-native-vision-camera frameProcessor
`
Contributing
- Development workflow
- Sending a pull request
- Automated publishing guide
- Code of conduct
Publishing
This package uses automated publishing for releases. See PUBLISHING.md for detailed information.
Quick commands:
`bash
Check for changes and publish automatically
npm run auto-publishPreview what would be published (dry-run)
npm run auto-publish:dry-runWindows users
npm run auto-publish:windows
``MIT
---
Made with create-react-native-library