@shelby-protocol/clay-codes

WebAssembly bindings for Clay codes, a class of regenerating codes for distributed storage.

Installation

``bash
npm install @shelby-protocol/clay-codes
`

Usage

`typescript
import {
createDecoder,
createEncoder,
type EncoderOptions,
} from '@shelby-protocol/clay-codes';

const SAMPLE_OPTIONS: EncoderOptions = {
// Total number of chunks (data + parity)
n: 10,
// Number of systematic data chunks
k: 6,
// Helper chunks contacted during repair
d: 4,
// Size of each chunk in bytes
chunkSizeBytes: 1024,
};

// Input of Uint8Array of length k
const input = Array.from({ length: SAMPLE_OPTIONS.k }, () =>
new Uint8Array(SAMPLE_OPTIONS.chunkSizeBytes),
);

// Encoder + erasure code
const encoder = await createEncoder(SAMPLE_OPTIONS);
const { chunks } = encoder.erasureCode(input);

// Create decoder
const decoder = await createDecoder({
...SAMPLE_OPTIONS,
erasedChunkIndexes: [0, SAMPLE_OPTIONS.n - 1],
});

// Supply every surviving chunk in index order (systematic first, parity next)
const survivors = chunks.filter((_, idx) =>
idx !== 0 && idx !== SAMPLE_OPTIONS.n - 1,
);
const recovered = decoder.decode(survivors);
console.log(recovered.systematic[0]);
`

Requirements

This package expects the compiled artifact at dist/clay.wasm. Generate it from the C sources with:

`bash
pnpm run build:wasm
`

The helper script builds natively on Linux hosts; on macOS/Windows it automatically spins up the repo's C toolchain container to run MACHINE=unknown_clang_wasm32 make and copies the wasm payload into dist/.

Development

`bash


Install dependencies

pnpm install

Run tests (requires dist/clay.wasm)

pnpm test

Build

pnpm build

Format code

pnpm fmt

Lint code

pnpm lint
`

WASM Integration

The package is designed to work in both Node.js and browser environments. The WASM module is loaded differently depending on the runtime:

- Node.js: Reads the WASM file from disk using fs/promises
- Browser: Uses fetch() with optional streaming compilation

API

$3

Creates a new Clay encoder instance.

Parameters:
- options: EncoderOptions object
- n: Total number of chunks
- k: Number of data chunks
- d: Number of chunks to contact for repair
- chunkSizeBytes: Size of each chunk in bytes

Returns: Promise

$3

- setChunk(idx, data): Copy a systematic chunk into the encoder prior to running
- run(): Execute the Clay encoding routine, filling both data and parity slots
- getChunk(idx): Retrieve either a data or parity chunk after encoding
- erasureCode(input): High-level helper that stages k chunks (either an array
of Uint8Arrays or a flat buffer), runs the encoder, and returns a
ChunkCollection
- free?(): Optional cleanup method (if implemented)

$3

Constructs a Clay decoder capable of rebuilding erased chunks.

Parameters:
- options: DecoderOptions object (extends EncoderOptions)
- n: Total number of chunks
- k: Number of data chunks
- d: Number of chunks to contact for repair
- chunkSizeBytes: Size of each chunk in bytes
- erasedChunksMask: Bit-mask of erased chunk indexes (optional)
- erasedChunkIndexes: Array of erased chunk indexes (optional)
- availableChunkIndexes: Array of surviving chunk indexes (optional)

Provide at least one of the erased/available hints so the decoder knows which
chunks must be reconstructed.

Returns: Promise

$3

- setChunk(idx, data): Stage a surviving chunk (systematic or parity) inside the decoder
- run(): Perform decoding once at least k chunks have been staged
- getChunk(idx): Read back a reconstructed or intact chunk after decoding
- configure(options?): Reconfigure the decoder for a new erasure pattern without reallocating WASM.
Returns available chunk indexes in sorted order. If options is omitted, reuses the last configuration.
- decode(available, config?): High-level helper that stages the surviving chunks (array of Uint8Array),
runs decoding, and returns a ChunkCollection. The optional config parameter allows reconfiguration
for a different erasure pattern
- free?(): Optional cleanup method (if implemented)

Decoder Reconfiguration:

The decoder can be reconfigured for different erasure patterns without re-instantiating the WASM module:

`typescript
const decoder = await createDecoder({
n: 4,
k: 2,
d: 3,
chunkSizeBytes: 128,
erasedChunkIndexes: [0, 3],
});

// Decode with initial erasure pattern
let recovered = decoder.decode([chunk1, chunk2], { erasedChunkIndexes: [0, 3] });

// Decode with new erasure pattern - the config parameter reconfigures the decoder
recovered = decoder.decode([chunk0, chunk3], { erasedChunkIndexes: [1, 2] });
`

The erasure pattern can be specified in three equivalent formats:
- erasedChunksMask: Bit mask where bit i set to 1 means chunk i is erased
- erasedChunkIndexes: Array of erased chunk indexes
- availableChunkIndexes: Array of available chunk indexes (complement is treated as erased)

Note: Coding parameters (n, k, d, chunkSizeBytes) are fixed per encoder/decoder instance
and cannot be reconfigured. This is because these parameters determine the WASM memory layout and
buffer sizes. To use different parameters, create a new encoder/decoder instance.

$3

The package exports custom error classes for better error handling:

- InvalidChunkIndexError: Thrown when a chunk index is invalid (negative, non-integer, exceeds bounds, or exceeds 32-bit mask width)
- DuplicateChunkIndexError: Thrown when duplicate chunk indexes are provided

Both error classes include the problematic index as a public readonly property:

`typescript
import { InvalidChunkIndexError, DuplicateChunkIndexError } from '@shelby-protocol/clay-codes';

try {
decoder.configure({ erasedChunkIndexes: [1, 1, 3] });
} catch (e) {
if (e instanceof DuplicateChunkIndexError) {
console.error(Duplicate chunk index: ${e.index});
}
}
`

$3

- ChunkCollection: Returned by the high-level helpers and exposes
.chunks (all output in index order), .systematic, and .parity
- flattenSystematic(collection, chunkSizeBytes): Utility that returns a flat
Uint8Array containing just the systematic portion of a ChunkCollection`