CSV parser tool with Rust/WASM core for high-performance streaming
npm install @datapos/datapos-tool-rust-csv-core
A high-performance CSV parsing tool with Rust/WebAssembly core for the Data Positioning platform. Provides streaming CSV processing with automatic fallback for browsers without transferable ReadableStream support.
- High Performance: Rust/WASM core using csv-core for efficient parsing
- Dual Mode Processing:
- Stream Mode: For Chromium-based browsers with transferable ReadableStream support
- Chunk Mode: Fallback for Safari (iOS & macOS) without transferable stream support
- Progressive Callbacks: Real-time progress updates during processing
- Single Load: Dynamically loaded once and shared across all connectors
- Memory Efficient: Streaming architecture processes large files without loading entire content into memory
The tool encapsulates:
- Rust Core (rust/datapos_tool_rust_csv_core): Low-level CSV parsing with csv-core
- TypeScript Wrapper (src/index.ts): Browser-friendly API with automatic mode selection
- WASM Binary: Compiled once, bundled with the tool
The tool is designed to be loaded dynamically by connectors:
``typescript
import { loadTool } from '@datapos/datapos-shared/component/tool';
import type { Tool as RustCsvCoreTool } from '@datapos/datapos-tool-rust-csv-core';
// Load the tool (loaded once, shared across all uses)
const csvTool = await loadTool
// Process with transferable streams (Chromium)
const result = await csvTool.processWithTransferableStream(
readableStream,
{ delimiter: ',', hasHeaders: true },
(rowCount) => console.log(Processed ${rowCount} rows)
);
// Process with chunks (Safari fallback)
const result = await csvTool.processWithChunks(
readableStream,
{ delimiter: ',', hasHeaders: true },
(rowCount) => console.log(Processed ${rowCount} rows)`
);
`bashInstall dependencies
npm install
Development
Prerequisites:
- Node.js 18+
- Rust toolchain
-
wasm-pack CLI: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | shStructure:
`
datapos-tool-rust-csv-core/
├── src/
│ └── index.ts # TypeScript API wrapper
├── rust/
│ └── datapos_tool_rust_csv_core/
│ ├── src/
│ │ └── lib.rs # Rust CSV core
│ ├── pkg/ # Generated WASM (gitignored)
│ └── Cargo.toml
├── dist/ # Built package (gitignored)
├── package.json
├── tsconfig.json
└── vite.config.ts
``MIT © 2026 Data Positioning Pty Ltd