@napi-rs/webcodecs


WebCodecs API implementation for Node.js using FFmpeg, built with NAPI-RS.

Features

- W3C WebCodecs API compliant - Full implementation of the WebCodecs specification with native DOMException errors
- Video encoding/decoding - H.264, H.265 (with Alpha), VP8, VP9 (with Alpha), AV1
- Encoding Alpha channel - VP9 and HEVC alpha encoding/decoding with transparency support (See canvas-to-video.js example and video.html)
- Audio encoding/decoding - AAC, Opus, MP3, FLAC, Vorbis, PCM variants
- Container muxing/demuxing - MP4, WebM, MKV containers with seeking support
- Image decoding - JPEG, PNG, WebP, GIF, BMP, AVIF, JPEG XL
- Canvas integration - Create VideoFrames from @napi-rs/canvas for graphics and text rendering
- Hardware acceleration - Zero-copy GPU encoding with VideoToolbox (macOS), NVENC (NVIDIA), VAAPI (Linux), QSV (Intel)
- Cross-platform - macOS, Windows, Linux (glibc/musl, x64/arm64/armv7)
- Structured logging - FFmpeg logs redirected to Rust tracing crate for easy integration

Installation

``bash
bun add @napi-rs/webcodecs


or

pnpm add @napi-rs/webcodecs

or

yarn add @napi-rs/webcodecs
`

$3

For creating VideoFrames from canvas content, install @napi-rs/canvas:

`bash
npm install @napi-rs/canvas
`

Quick Start

$3

`typescript
import { VideoEncoder, VideoFrame } from '@napi-rs/webcodecs'

const encoder = new VideoEncoder({
output: (chunk, metadata) => {
console.log(Encoded ${chunk.type} chunk: ${chunk.byteLength} bytes)
},
error: (e) => console.error(e),
})

encoder.configure({
codec: 'avc1.42001E', // H.264 Baseline
width: 1920,
height: 1080,
bitrate: 5_000_000,
hardwareAcceleration: 'prefer-hardware', // Use GPU when available
latencyMode: 'realtime', // Optimize for low latency
})

// Create and encode frames
const frameData = new Uint8Array(1920 1080 4) // RGBA
const frame = new VideoFrame(frameData, {
format: 'RGBA',
codedWidth: 1920,
codedHeight: 1080,
timestamp: 0,
})

encoder.encode(frame)
frame.close()

// Force a keyframe for seeking/streaming
const frame2 = new VideoFrame(frameData, {
format: 'RGBA',
codedWidth: 1920,
codedHeight: 1080,
timestamp: 33333, // 30fps
})
encoder.encode(frame2, { keyFrame: true }) // Force I-frame
frame2.close()

await encoder.flush()
encoder.close()
`

$3

`typescript
import { VideoDecoder, EncodedVideoChunk } from '@napi-rs/webcodecs'

const decoder = new VideoDecoder({
output: (frame) => {
console.log(Decoded frame: ${frame.codedWidth}x${frame.codedHeight})
frame.close()
},
error: (e) => console.error(e),
})

decoder.configure({
codec: 'avc1.42001E',
codedWidth: 1920,
codedHeight: 1080,
})

// Decode chunks
const chunk = new EncodedVideoChunk({
type: 'key',
timestamp: 0,
data: encodedData,
})

decoder.decode(chunk)
await decoder.flush()
decoder.close()
`

$3

`typescript
import { AudioEncoder, AudioData } from '@napi-rs/webcodecs'

const encoder = new AudioEncoder({
output: (chunk, metadata) => {
console.log(Encoded audio: ${chunk.byteLength} bytes)
},
error: (e) => console.error(e),
})

encoder.configure({
codec: 'opus',
sampleRate: 48000,
numberOfChannels: 2,
bitrate: 128000,
})

const audioData = new AudioData({
format: 'f32-planar',
sampleRate: 48000,
numberOfFrames: 1024,
numberOfChannels: 2,
timestamp: 0,
data: new Float32Array(1024 * 2),
})

encoder.encode(audioData)
audioData.close()

await encoder.flush()
encoder.close()
`

$3

`typescript
import { ImageDecoder } from '@napi-rs/webcodecs'
import { readFileSync } from 'fs'

const imageData = readFileSync('image.png')
const decoder = new ImageDecoder({
data: imageData,
type: 'image/png',
})

const result = await decoder.decode()
console.log(Image: ${result.image.codedWidth}x${result.image.codedHeight})
result.image.close()
decoder.close()
`

$3

Read encoded video/audio from MP4, WebM, or MKV containers:

`typescript
import { Mp4Demuxer, VideoDecoder, AudioDecoder } from '@napi-rs/webcodecs'

// Create decoder instances
const videoDecoder = new VideoDecoder({
output: (frame) => {
console.log(Decoded frame: ${frame.timestamp})
frame.close()
},
error: (e) => console.error(e),
})

const audioDecoder = new AudioDecoder({
output: (data) => {
console.log(Decoded audio: ${data.timestamp})
data.close()
},
error: (e) => console.error(e),
})

// Create demuxer with callbacks
const demuxer = new Mp4Demuxer({
videoOutput: (chunk) => videoDecoder.decode(chunk),
audioOutput: (chunk) => audioDecoder.decode(chunk),
error: (e) => console.error(e),
})

// Load from file or buffer
await demuxer.load('./video.mp4')
// or: await demuxer.loadBuffer(uint8Array)

// Configure decoders with extracted configs
videoDecoder.configure(demuxer.videoDecoderConfig)
audioDecoder.configure(demuxer.audioDecoderConfig)

// Get track info
console.log(demuxer.tracks) // Array of track info
console.log(demuxer.duration) // Duration in microseconds

// Demux all packets (calls callbacks)
demuxer.demux()

// Or demux in batches
demuxer.demux(100) // Demux up to 100 packets

// Seek to timestamp (microseconds)
demuxer.seek(5_000_000) // Seek to 5 seconds

demuxer.close()
`

$3

Write encoded video/audio to MP4, WebM, or MKV containers:

`typescript
import { Mp4Muxer, VideoEncoder, AudioEncoder } from '@napi-rs/webcodecs'
import { writeFileSync } from 'fs'

// Create muxer
const muxer = new Mp4Muxer({ fastStart: true })

// Track description will be set from encoder metadata
let videoDescription: Uint8Array | undefined

// Create encoder that feeds into muxer
const videoEncoder = new VideoEncoder({
output: (chunk, metadata) => {
// Capture codec description from first keyframe
if (metadata?.decoderConfig?.description) {
videoDescription = metadata.decoderConfig.description
}
muxer.addVideoChunk(chunk, metadata)
},
error: (e) => console.error(e),
})

videoEncoder.configure({
codec: 'avc1.42001E',
width: 1920,
height: 1080,
bitrate: 5_000_000,
})

// Add tracks before adding chunks
muxer.addVideoTrack({
codec: 'avc1.42001E',
width: 1920,
height: 1080,
})

// Encode frames...
// videoEncoder.encode(frame)

await videoEncoder.flush()

// Finalize and write output
const mp4Data = muxer.finalize()
writeFileSync('output.mp4', mp4Data)
muxer.close()
`

#### Streaming Muxer Mode

For live streaming or large files, use streaming mode:

`typescript
import { Mp4Muxer } from '@napi-rs/webcodecs'

const muxer = new Mp4Muxer({
fragmented: true, // Required for MP4 streaming
streaming: { bufferCapacity: 256 * 1024 },
})

muxer.addVideoTrack({ codec: 'avc1.42001E', width: 1920, height: 1080 })

// Add chunks as they arrive
muxer.addVideoChunk(chunk, metadata)

// Read available data incrementally
const data = muxer.read()
if (data) {
stream.write(data)
}

// Check when finished
if (muxer.isFinished) {
stream.end()
}
`

$3

Create VideoFrames from @napi-rs/canvas for graphics, text rendering, or image compositing:

`typescript
import { VideoFrame } from '@napi-rs/webcodecs'
import { createCanvas } from '@napi-rs/canvas'

const canvas = createCanvas(1920, 1080)
const ctx = canvas.getContext('2d')

// Draw graphics
ctx.fillStyle = '#FF0000'
ctx.fillRect(0, 0, 1920, 1080)
ctx.fillStyle = '#FFFFFF'
ctx.font = '48px sans-serif'
ctx.fillText('Hello WebCodecs!', 100, 100)

// Create VideoFrame from canvas (timestamp required per W3C spec)
const frame = new VideoFrame(canvas, {
timestamp: 0,
duration: 33333, // optional: frame duration in microseconds
})

console.log(frame.format) // 'RGBA'
console.log(frame.codedWidth, frame.codedHeight) // 1920, 1080

// Use with VideoEncoder (see Video Encoding section)
frame.close()
`

Note: Canvas pixel data is copied as RGBA format with sRGB color space.

Supported Codecs

$3

| Codec | Codec String | Encoding | Encoding Alpha | Decoding | Decoding Alpha |
| ----- | ----------------------- | -------- | -------------- | -------- | -------------- |
| H.264 | avc1.* | ✅ | 🙅🏻‍♀️ | ✅ | 🙅🏻‍♀️ |
| H.265 | hev1., hvc1. | ✅ | ✅¹ | ✅ | ✅ |
| VP8 | vp8 | ✅ | 🙅🏻‍♀️ | ✅ | 🙅🏻‍♀️ |
| VP9 | vp09.*, vp9 | ✅ | ✅ | ✅ | ✅ |
| AV1 | av01.*, av01, av1 | ✅ | 🙅🏻‍♀️ | ✅ | 🙅🏻‍♀️ |

Note: Short form codec strings (vp9, av01, av1) are accepted for compatibility with browser implementations.

¹ HEVC Alpha Encoding Limitations:

- Requires software encoder (libx265) - set hardwareAcceleration: 'prefer-software'
- Hardware encoders (VideoToolbox, NVENC, VAAPI, QSV) do not support alpha
- Only YUVA420P (8-bit) and YUVA420P10 (10-bit Main 10 profile) pixel formats supported

Legend:

- ✅ Feature supported
- 🙅🏻‍♀️ Feature not supported by codec format
- ❓ Feature supported by codec format but not yet implemented

$3

| Codec | Codec String | Encoding | Decoding |
| ------ | ------------ | -------- | -------- |
| AAC | mp4a.40.2 | ✅ | ✅ |
| Opus | opus | ✅ | ✅ |
| MP3 | mp3 | ✅ | ✅ |
| FLAC | flac | ✅ | ✅ |
| Vorbis | vorbis | ❌ | ✅ |
| PCM | pcm-* | ❌ | ✅ |

$3

| Format | MIME Type | Decoding |
| ------- | ------------ | -------- |
| JPEG | image/jpeg | ✅ |
| PNG | image/png | ✅ |
| WebP | image/webp | ✅ |
| GIF | image/gif | ✅ |
| BMP | image/bmp | ✅ |
| AVIF | image/avif | ✅ |
| JPEG XL | image/jxl | ✅ |

$3

| Container | Video Codecs | Audio Codecs | Demuxer | Muxer |
| --------- | --------------------------- | ---------------------------- | ------------- | ----------- |
| MP4 | H.264, H.265, AV1 | AAC, Opus, MP3, FLAC | Mp4Demuxer | Mp4Muxer |
| WebM | VP8, VP9, AV1 | Opus, Vorbis | WebMDemuxer | WebMMuxer |
| MKV | H.264, H.265, VP8, VP9, AV1 | AAC, Opus, Vorbis, FLAC, MP3 | MkvDemuxer | MkvMuxer |

Platform Support

Pre-built binaries are available for:

| Platform | Architecture |
| ------------------------ | ------------ |
| macOS | x64, arm64 |
| Windows | x64, arm64 |
| Linux (glibc) | x64, arm64 |
| Linux (musl) | x64, arm64 |
| Linux (glibc, gnueabihf) | armv7 |

W3C Web Platform Tests Compliance

This implementation is validated against the W3C Web Platform Tests for WebCodecs.

$3

| Status | Count | Percentage |
| ----------- | ----- | ---------- |
| Passing | 573 | 99.1% |
| Skipped | 5 | 0.9% |
| Failing | 0 | 0% |

Skipped tests are due to platform-specific features or edge cases.

$3

19 WPT test files require browser APIs unavailable in Node.js:

| Category | Tests | APIs Required |
| ---------------------- | ----- | -------------------------------------- |
| Serialization/Transfer | 5 | MessageChannel, structured clone |
| WebGL/Canvas | 5 | WebGL textures, ImageBitmap, Canvas 2D |
| Cross-Origin Isolation | 8 | COOP/COEP headers |
| WebIDL | 1 | IDL interface validation |

See __test__/wpt/README.md for detailed test status.

Hardware Acceleration

Hardware encoding is fully supported with automatic GPU selection and fallback:

| Platform | Encoders | Features |
| -------- | ------------ | ------------------------------------------------------ |
| macOS | VideoToolbox | H.264, HEVC; realtime mode, allow_sw control |
| NVIDIA | NVENC | H.264, HEVC, AV1; presets p1-p7, spatial-aq, lookahead |
| Linux | VAAPI | H.264, HEVC, VP9, AV1; quality 0-8 |
| Intel | QSV | H.264, HEVC, VP9, AV1; presets, lookahead |

$3

`typescript
encoder.configure({
codec: 'avc1.42001E',
width: 1920,
height: 1080,
// Hardware acceleration preference
hardwareAcceleration: 'prefer-hardware', // 'no-preference' | 'prefer-hardware' | 'prefer-software'
// Latency mode affects encoder tuning
latencyMode: 'realtime', // 'quality' | 'realtime'
// Alpha channel preservation (VP9 and HEVC only)
alpha: 'discard', // 'keep' | 'discard' (default: 'discard')
})
`

- latencyMode: 'realtime' - Enables low-latency encoder options (smaller GOP, no B-frames, fast presets)
- latencyMode: 'quality' - Enables quality-focused options (larger GOP, B-frames, lookahead)
- alpha: 'keep' - Preserves alpha channel (VP9 and HEVC only). For HEVC, requires hardwareAcceleration: 'prefer-software'

The encoder automatically applies optimal settings for each hardware encoder based on the latency mode.

$3

Encode video with transparency using VP9 or HEVC:

`typescript
import { VideoEncoder, VideoFrame } from '@napi-rs/webcodecs'

const encoder = new VideoEncoder({
output: (chunk, metadata) => {
console.log(Alpha chunk: ${chunk.byteLength} bytes)
},
error: (e) => console.error(e),
})

// VP9 alpha - works with hardware or software
encoder.configure({
codec: 'vp09.00.10.08',
width: 1920,
height: 1080,
alpha: 'keep',
})

// HEVC alpha - requires software encoder
encoder.configure({
codec: 'hev1.1.6.L93.B0',
width: 1920,
height: 1080,
alpha: 'keep',
hardwareAcceleration: 'prefer-software', // Required for HEVC alpha
})

// Create frame with alpha channel (I420A format)
const width = 1920
const height = 1080
const frameData = new Uint8Array(width height 1.5 + width * height) // Y + U + V + A
const frame = new VideoFrame(frameData, {
format: 'I420A',
codedWidth: width,
codedHeight: height,
timestamp: 0,
})

encoder.encode(frame)
frame.close()
`

Limitations

$3

All scalability modes (L1Tx, L2Tx, L3Tx, S2Tx, S3Tx, and variants) are accepted and populate metadata.svc.temporalLayerId when temporal layers >= 2.

The W3C WebCodecs spec only defines temporalLayerId in SvcOutputMetadata - there is no spatialLayerId field in the spec. See W3C WebCodecs §6.7.

Note: This implementation computes temporal layer IDs algorithmically from frame index per W3C spec. FFmpeg is not configured for actual SVC encoding, so base layer frames are not independently decodable.

$3

Synchronous errors (e.g., calling encode() on a closed encoder) throw native DOMException instances that pass instanceof DOMException checks per W3C spec:

`typescript
try {
encoder.encode(frame) // on closed encoder
} catch (e) {
console.log(e instanceof DOMException) // true
console.log(e.name) // "InvalidStateError"
}
`

Asynchronous error callbacks receive standard Error objects with the DOMException name in the message:

`typescript
const encoder = new VideoEncoder({
output: (chunk) => {},
error: (e) => {
console.log(e.message) // "EncodingError: ..."
},
})
`

$3

VideoFrame.copyTo() and VideoFrame.allocationSize() support format conversion per W3C WebCodecs spec:

`typescript
const frame = new VideoFrame(i420Data, {
format: 'I420',
codedWidth: 1920,
codedHeight: 1080,
timestamp: 0,
})

// Get allocation size for RGBA output
const rgbaSize = frame.allocationSize({ format: 'RGBA' })

// Copy with format conversion (I420 → RGBA)
const rgbaBuffer = new Uint8Array(rgbaSize)
const layout = await frame.copyTo(rgbaBuffer, { format: 'RGBA' })

frame.close()
`

Supported conversions:

| Source Format | Target Format | Status |
| ---------------------------- | ---------------------- | -------------------- |
| I420, I422, I444, NV12, NV21 | RGBA, RGBX, BGRA, BGRX | ✅ |
| RGBA, RGBX, BGRA, BGRX | RGBA, RGBX, BGRA, BGRX | ✅ |
| RGBA, RGBX, BGRA, BGRX | I420, I422, I444, NV12 | ❌ NotSupportedError |

Per WPT videoFrame-copyTo-rgb.any.js, RGB-to-YUV conversion throws NotSupportedError.

Custom layouts with overflow-inducing values (e.g., offset: 2³²-2) throw TypeError via checked arithmetic. Rect alignment is validated against the source format during conversion.

$3

ImageDecoder supports all W3C spec options:

| Option | Status | Notes |
| ---------------------- | ------ | --------------------------------------------------------------------------------- |
| desiredWidth/Height | ✅ | Scales decoded frames to specified dimensions |
| preferAnimation | ✅ | When false, only decodes first frame for animated formats |
| colorSpaceConversion | ✅ | "default" extracts color space metadata, "none" ignores it (Chromium-aligned) |

Note: Per W3C spec, desiredWidth and desiredHeight must both be specified or both omitted.

$3

- ImageDecoder GIF animation: FFmpeg may return only the first frame. Use VideoDecoder with GIF codec for full animation.

$3

Per-frame quantizer control is available for VP9 and AV1 when using bitrateMode: 'quantizer':

`typescript
encoder.configure({
codec: 'vp09.00.10.08',
width: 1920,
height: 1080,
bitrateMode: 'quantizer',
})

// Per-frame QP control (0-255 range per W3C WebCodecs spec)
encoder.encode(frame, { vp9: { quantizer: 128 } })
`

| Codec | Per-Frame QP | Notes |
| ----- | ------------ | ------------------------------------------------------------ |
| VP9 | ✅ Works | Uses FFmpeg's dynamic qmax update mechanism |
| AV1 | ❌ No effect | FFmpeg's libaom wrapper doesn't support dynamic qmax updates |
| H.264 | ✅ Works | Uses FFmpeg's frame quality mechanism (0-51 range) |
| H.265 | ✅ Works | Uses FFmpeg's frame quality mechanism (0-51 range) |

Note: The 0-255 quantizer range for VP9/AV1 aligns with Chromium's WebCodecs implementation. Internally, values are converted to the 0-63 encoder range using q_index / 4.

Logging

This library uses Rust's tracing crate for structured logging. Enable logging via the WEBCODECS_LOG environment variable:

`bash


Enable all logs at info level

WEBCODECS_LOG=info node your-app.js

Enable FFmpeg logs at debug level

WEBCODECS_LOG=ffmpeg=debug node your-app.js

Enable WebCodecs codec errors at warn, FFmpeg at info

WEBCODECS_LOG=webcodecs=warn,ffmpeg=info node your-app.js

Enable trace-level logging for everything

WEBCODECS_LOG=trace node your-app.js
`

$3

| Target | Description |
| ----------- | ---------------------------------------------------------------------- |
| ffmpeg | FFmpeg internal logs (codec initialization, encoding/decoding details) |
| webcodecs | WebCodecs API logs (codec errors, state transitions) |

$3

| FFmpeg Level | Tracing Level |
| ------------ | ------------- |
| ERROR/FATAL | error |
| WARNING | warn |
| INFO | info |
| VERBOSE | debug |
| DEBUG/TRACE | trace |

Without WEBCODECS_LOG set, all logs are silently discarded.

API Reference

This package implements the W3C WebCodecs API. Key classes:

- VideoEncoder / VideoDecoder - Video encoding and decoding with EventTarget support
- AudioEncoder / AudioDecoder - Audio encoding and decoding with EventTarget support
- VideoFrame - Raw video frame data (supports buffer data, existing VideoFrame, or @napi-rs/canvas Canvas)
- AudioData - Raw audio sample data
- EncodedVideoChunk / EncodedAudioChunk - Encoded media data
- ImageDecoder - Static image decoding
- VideoColorSpace - Color space information
- Mp4Demuxer / WebMDemuxer / MkvDemuxer - Container demuxing with seeking
- Mp4Muxer / WebMMuxer / MkvMuxer - Container muxing with streaming support

All encoders and decoders implement the EventTarget interface with addEventListener(), removeEventListener(), and dispatchEvent().

For full API documentation, see the W3C WebCodecs specification.

Development

$3

- Rust (latest stable)
- Node.js 18+
- pnpm

$3

`bash
pnpm install
pnpm build
`

$3

`bash
pnpm test
`

$3

`bash
pnpm lint
cargo clippy
``

License

MIT