WebAssembly bindings for Clay codes, a class of regenerating codes for distributed storage.
npm install @shelby-protocol/clay-codesWebAssembly bindings for Clay codes, a class of regenerating codes for distributed storage.
``bash`
npm install @shelby-protocol/clay-codes
`typescript
import {
createDecoder,
createEncoder,
type EncoderOptions,
} from '@shelby-protocol/clay-codes';
const SAMPLE_OPTIONS: EncoderOptions = {
// Total number of chunks (data + parity)
n: 10,
// Number of systematic data chunks
k: 6,
// Helper chunks contacted during repair
d: 4,
// Size of each chunk in bytes
chunkSizeBytes: 1024,
};
// Input of Uint8Array of length k
const input = Array.from({ length: SAMPLE_OPTIONS.k }, () =>
new Uint8Array(SAMPLE_OPTIONS.chunkSizeBytes),
);
// Encoder + erasure code
const encoder = await createEncoder(SAMPLE_OPTIONS);
const { chunks } = encoder.erasureCode(input);
// Create decoder
const decoder = await createDecoder({
...SAMPLE_OPTIONS,
erasedChunkIndexes: [0, SAMPLE_OPTIONS.n - 1],
});
// Supply every surviving chunk in index order (systematic first, parity next)
const survivors = chunks.filter((_, idx) =>
idx !== 0 && idx !== SAMPLE_OPTIONS.n - 1,
);
const recovered = decoder.decode(survivors);
console.log(recovered.systematic[0]);
`
This package expects the compiled artifact at dist/clay.wasm. Generate it from the C sources with:
`bash`
pnpm run build:wasm
The helper script builds natively on Linux hosts; on macOS/Windows it automatically spins up the repo's C toolchain container to run MACHINE=unknown_clang_wasm32 make and copies the wasm payload into dist/.
`bashInstall dependencies
pnpm install
WASM Integration
The package is designed to work in both Node.js and browser environments. The WASM module is loaded differently depending on the runtime:
- Node.js: Reads the WASM file from disk using
fs/promises
- Browser: Uses fetch() with optional streaming compilationAPI
$3
Creates a new Clay encoder instance.
Parameters:
-
options: EncoderOptions object
- n: Total number of chunks
- k: Number of data chunks
- d: Number of chunks to contact for repair
- chunkSizeBytes: Size of each chunk in bytesReturns:
Promise$3
-
setChunk(idx, data): Copy a systematic chunk into the encoder prior to running
- run(): Execute the Clay encoding routine, filling both data and parity slots
- getChunk(idx): Retrieve either a data or parity chunk after encoding
- erasureCode(input): High-level helper that stages k chunks (either an array
of Uint8Arrays or a flat buffer), runs the encoder, and returns a
ChunkCollection
- free?(): Optional cleanup method (if implemented)$3
Constructs a Clay decoder capable of rebuilding erased chunks.
Parameters:
-
options: DecoderOptions object (extends EncoderOptions)
- n: Total number of chunks
- k: Number of data chunks
- d: Number of chunks to contact for repair
- chunkSizeBytes: Size of each chunk in bytes
- erasedChunksMask: Bit-mask of erased chunk indexes (optional)
- erasedChunkIndexes: Array of erased chunk indexes (optional)
- availableChunkIndexes: Array of surviving chunk indexes (optional)Provide at least one of the erased/available hints so the decoder knows which
chunks must be reconstructed.
Returns:
Promise$3
-
setChunk(idx, data): Stage a surviving chunk (systematic or parity) inside the decoder
- run(): Perform decoding once at least k chunks have been staged
- getChunk(idx): Read back a reconstructed or intact chunk after decoding
- configure(options?): Reconfigure the decoder for a new erasure pattern without reallocating WASM.
Returns available chunk indexes in sorted order. If options is omitted, reuses the last configuration.
- decode(available, config?): High-level helper that stages the surviving chunks (array of Uint8Array),
runs decoding, and returns a ChunkCollection. The optional config parameter allows reconfiguration
for a different erasure pattern
- free?(): Optional cleanup method (if implemented)Decoder Reconfiguration:
The decoder can be reconfigured for different erasure patterns without re-instantiating the WASM module:
`typescript
const decoder = await createDecoder({
n: 4,
k: 2,
d: 3,
chunkSizeBytes: 128,
erasedChunkIndexes: [0, 3],
});// Decode with initial erasure pattern
let recovered = decoder.decode([chunk1, chunk2], { erasedChunkIndexes: [0, 3] });
// Decode with new erasure pattern - the config parameter reconfigures the decoder
recovered = decoder.decode([chunk0, chunk3], { erasedChunkIndexes: [1, 2] });
`The erasure pattern can be specified in three equivalent formats:
-
erasedChunksMask: Bit mask where bit i set to 1 means chunk i is erased
- erasedChunkIndexes: Array of erased chunk indexes
- availableChunkIndexes: Array of available chunk indexes (complement is treated as erased)Note: Coding parameters (
n, k, d, chunkSizeBytes) are fixed per encoder/decoder instance
and cannot be reconfigured. This is because these parameters determine the WASM memory layout and
buffer sizes. To use different parameters, create a new encoder/decoder instance.$3
The package exports custom error classes for better error handling:
-
InvalidChunkIndexError: Thrown when a chunk index is invalid (negative, non-integer, exceeds bounds, or exceeds 32-bit mask width)
- DuplicateChunkIndexError: Thrown when duplicate chunk indexes are providedBoth error classes include the problematic
index as a public readonly property:`typescript
import { InvalidChunkIndexError, DuplicateChunkIndexError } from '@shelby-protocol/clay-codes';try {
decoder.configure({ erasedChunkIndexes: [1, 1, 3] });
} catch (e) {
if (e instanceof DuplicateChunkIndexError) {
console.error(
Duplicate chunk index: ${e.index});
}
}
`$3
-
ChunkCollection: Returned by the high-level helpers and exposes
.chunks (all output in index order), .systematic, and .parity
- flattenSystematic(collection, chunkSizeBytes): Utility that returns a flat
Uint8Array containing just the systematic portion of a ChunkCollection`