WebCodecs API implementation for Node.js using FFmpeg
npm install @napi-rs/webcodecs
WebCodecs API implementation for Node.js using FFmpeg, built with NAPI-RS.
- W3C WebCodecs API compliant - Full implementation of the WebCodecs specification with native DOMException errors
- Video encoding/decoding - H.264, H.265 (with Alpha), VP8, VP9 (with Alpha), AV1
- Encoding Alpha channel - VP9 and HEVC alpha encoding/decoding with transparency support (See canvas-to-video.js example and video.html)
- Audio encoding/decoding - AAC, Opus, MP3, FLAC, Vorbis, PCM variants
- Container muxing/demuxing - MP4, WebM, MKV containers with seeking support
- Image decoding - JPEG, PNG, WebP, GIF, BMP, AVIF, JPEG XL
- Canvas integration - Create VideoFrames from @napi-rs/canvas for graphics and text rendering
- Hardware acceleration - Zero-copy GPU encoding with VideoToolbox (macOS), NVENC (NVIDIA), VAAPI (Linux), QSV (Intel)
- Cross-platform - macOS, Windows, Linux (glibc/musl, x64/arm64/armv7)
- Structured logging - FFmpeg logs redirected to Rust tracing crate for easy integration
``bash`
bun add @napi-rs/webcodecsor
pnpm add @napi-rs/webcodecsor
yarn add @napi-rs/webcodecs
For creating VideoFrames from canvas content, install @napi-rs/canvas:
`bash`
npm install @napi-rs/canvas
`typescript
import { VideoEncoder, VideoFrame } from '@napi-rs/webcodecs'
const encoder = new VideoEncoder({
output: (chunk, metadata) => {
console.log(Encoded ${chunk.type} chunk: ${chunk.byteLength} bytes)
},
error: (e) => console.error(e),
})
encoder.configure({
codec: 'avc1.42001E', // H.264 Baseline
width: 1920,
height: 1080,
bitrate: 5_000_000,
hardwareAcceleration: 'prefer-hardware', // Use GPU when available
latencyMode: 'realtime', // Optimize for low latency
})
// Create and encode frames
const frameData = new Uint8Array(1920 1080 4) // RGBA
const frame = new VideoFrame(frameData, {
format: 'RGBA',
codedWidth: 1920,
codedHeight: 1080,
timestamp: 0,
})
encoder.encode(frame)
frame.close()
// Force a keyframe for seeking/streaming
const frame2 = new VideoFrame(frameData, {
format: 'RGBA',
codedWidth: 1920,
codedHeight: 1080,
timestamp: 33333, // 30fps
})
encoder.encode(frame2, { keyFrame: true }) // Force I-frame
frame2.close()
await encoder.flush()
encoder.close()
`
`typescript
import { VideoDecoder, EncodedVideoChunk } from '@napi-rs/webcodecs'
const decoder = new VideoDecoder({
output: (frame) => {
console.log(Decoded frame: ${frame.codedWidth}x${frame.codedHeight})
frame.close()
},
error: (e) => console.error(e),
})
decoder.configure({
codec: 'avc1.42001E',
codedWidth: 1920,
codedHeight: 1080,
})
// Decode chunks
const chunk = new EncodedVideoChunk({
type: 'key',
timestamp: 0,
data: encodedData,
})
decoder.decode(chunk)
await decoder.flush()
decoder.close()
`
`typescript
import { AudioEncoder, AudioData } from '@napi-rs/webcodecs'
const encoder = new AudioEncoder({
output: (chunk, metadata) => {
console.log(Encoded audio: ${chunk.byteLength} bytes)
},
error: (e) => console.error(e),
})
encoder.configure({
codec: 'opus',
sampleRate: 48000,
numberOfChannels: 2,
bitrate: 128000,
})
const audioData = new AudioData({
format: 'f32-planar',
sampleRate: 48000,
numberOfFrames: 1024,
numberOfChannels: 2,
timestamp: 0,
data: new Float32Array(1024 * 2),
})
encoder.encode(audioData)
audioData.close()
await encoder.flush()
encoder.close()
`
`typescript
import { ImageDecoder } from '@napi-rs/webcodecs'
import { readFileSync } from 'fs'
const imageData = readFileSync('image.png')
const decoder = new ImageDecoder({
data: imageData,
type: 'image/png',
})
const result = await decoder.decode()
console.log(Image: ${result.image.codedWidth}x${result.image.codedHeight})`
result.image.close()
decoder.close()
Read encoded video/audio from MP4, WebM, or MKV containers:
`typescript
import { Mp4Demuxer, VideoDecoder, AudioDecoder } from '@napi-rs/webcodecs'
// Create decoder instances
const videoDecoder = new VideoDecoder({
output: (frame) => {
console.log(Decoded frame: ${frame.timestamp})
frame.close()
},
error: (e) => console.error(e),
})
const audioDecoder = new AudioDecoder({
output: (data) => {
console.log(Decoded audio: ${data.timestamp})
data.close()
},
error: (e) => console.error(e),
})
// Create demuxer with callbacks
const demuxer = new Mp4Demuxer({
videoOutput: (chunk) => videoDecoder.decode(chunk),
audioOutput: (chunk) => audioDecoder.decode(chunk),
error: (e) => console.error(e),
})
// Load from file or buffer
await demuxer.load('./video.mp4')
// or: await demuxer.loadBuffer(uint8Array)
// Configure decoders with extracted configs
videoDecoder.configure(demuxer.videoDecoderConfig)
audioDecoder.configure(demuxer.audioDecoderConfig)
// Get track info
console.log(demuxer.tracks) // Array of track info
console.log(demuxer.duration) // Duration in microseconds
// Demux all packets (calls callbacks)
demuxer.demux()
// Or demux in batches
demuxer.demux(100) // Demux up to 100 packets
// Seek to timestamp (microseconds)
demuxer.seek(5_000_000) // Seek to 5 seconds
demuxer.close()
`
Write encoded video/audio to MP4, WebM, or MKV containers:
`typescript
import { Mp4Muxer, VideoEncoder, AudioEncoder } from '@napi-rs/webcodecs'
import { writeFileSync } from 'fs'
// Create muxer
const muxer = new Mp4Muxer({ fastStart: true })
// Track description will be set from encoder metadata
let videoDescription: Uint8Array | undefined
// Create encoder that feeds into muxer
const videoEncoder = new VideoEncoder({
output: (chunk, metadata) => {
// Capture codec description from first keyframe
if (metadata?.decoderConfig?.description) {
videoDescription = metadata.decoderConfig.description
}
muxer.addVideoChunk(chunk, metadata)
},
error: (e) => console.error(e),
})
videoEncoder.configure({
codec: 'avc1.42001E',
width: 1920,
height: 1080,
bitrate: 5_000_000,
})
// Add tracks before adding chunks
muxer.addVideoTrack({
codec: 'avc1.42001E',
width: 1920,
height: 1080,
})
// Encode frames...
// videoEncoder.encode(frame)
await videoEncoder.flush()
// Finalize and write output
const mp4Data = muxer.finalize()
writeFileSync('output.mp4', mp4Data)
muxer.close()
`
#### Streaming Muxer Mode
For live streaming or large files, use streaming mode:
`typescript
import { Mp4Muxer } from '@napi-rs/webcodecs'
const muxer = new Mp4Muxer({
fragmented: true, // Required for MP4 streaming
streaming: { bufferCapacity: 256 * 1024 },
})
muxer.addVideoTrack({ codec: 'avc1.42001E', width: 1920, height: 1080 })
// Add chunks as they arrive
muxer.addVideoChunk(chunk, metadata)
// Read available data incrementally
const data = muxer.read()
if (data) {
stream.write(data)
}
// Check when finished
if (muxer.isFinished) {
stream.end()
}
`
Create VideoFrames from @napi-rs/canvas for graphics, text rendering, or image compositing:
`typescript
import { VideoFrame } from '@napi-rs/webcodecs'
import { createCanvas } from '@napi-rs/canvas'
const canvas = createCanvas(1920, 1080)
const ctx = canvas.getContext('2d')
// Draw graphics
ctx.fillStyle = '#FF0000'
ctx.fillRect(0, 0, 1920, 1080)
ctx.fillStyle = '#FFFFFF'
ctx.font = '48px sans-serif'
ctx.fillText('Hello WebCodecs!', 100, 100)
// Create VideoFrame from canvas (timestamp required per W3C spec)
const frame = new VideoFrame(canvas, {
timestamp: 0,
duration: 33333, // optional: frame duration in microseconds
})
console.log(frame.format) // 'RGBA'
console.log(frame.codedWidth, frame.codedHeight) // 1920, 1080
// Use with VideoEncoder (see Video Encoding section)
frame.close()
`
Note: Canvas pixel data is copied as RGBA format with sRGB color space.
| Codec | Codec String | Encoding | Encoding Alpha | Decoding | Decoding Alpha |
| ----- | ----------------------- | -------- | -------------- | -------- | -------------- |
| H.264 | avc1.* | โ
| ๐
๐ปโโ๏ธ | โ
| ๐
๐ปโโ๏ธ |hev1.
| H.265 | , hvc1. | โ
| โ
ยน | โ
| โ
|vp8
| VP8 | | โ
| ๐
๐ปโโ๏ธ | โ
| ๐
๐ปโโ๏ธ |vp09.*
| VP9 | , vp9 | โ
| โ
| โ
| โ
|av01.*
| AV1 | , av01, av1 | โ
| ๐
๐ปโโ๏ธ | โ
| ๐
๐ปโโ๏ธ |
Note: Short form codec strings (vp9, av01, av1) are accepted for compatibility with browser implementations.
ยน HEVC Alpha Encoding Limitations:
- Requires software encoder (libx265) - set hardwareAcceleration: 'prefer-software'
- Hardware encoders (VideoToolbox, NVENC, VAAPI, QSV) do not support alpha
- Only YUVA420P (8-bit) and YUVA420P10 (10-bit Main 10 profile) pixel formats supported
Legend:
- โ
Feature supported
- ๐
๐ปโโ๏ธ Feature not supported by codec format
- โ Feature supported by codec format but not yet implemented
| Codec | Codec String | Encoding | Decoding |
| ------ | ------------ | -------- | -------- |
| AAC | mp4a.40.2 | โ
| โ
|opus
| Opus | | โ
| โ
|mp3
| MP3 | | โ
| โ
|flac
| FLAC | | โ
| โ
|vorbis
| Vorbis | | โ | โ
|pcm-*
| PCM | | โ | โ
|
| Format | MIME Type | Decoding |
| ------- | ------------ | -------- |
| JPEG | image/jpeg | โ
|image/png
| PNG | | โ
|image/webp
| WebP | | โ
|image/gif
| GIF | | โ
|image/bmp
| BMP | | โ
|image/avif
| AVIF | | โ
|image/jxl
| JPEG XL | | โ
|
| Container | Video Codecs | Audio Codecs | Demuxer | Muxer |
| --------- | --------------------------- | ---------------------------- | ------------- | ----------- |
| MP4 | H.264, H.265, AV1 | AAC, Opus, MP3, FLAC | Mp4Demuxer | Mp4Muxer |WebMDemuxer
| WebM | VP8, VP9, AV1 | Opus, Vorbis | | WebMMuxer |MkvDemuxer
| MKV | H.264, H.265, VP8, VP9, AV1 | AAC, Opus, Vorbis, FLAC, MP3 | | MkvMuxer |
Pre-built binaries are available for:
| Platform | Architecture |
| ------------------------ | ------------ |
| macOS | x64, arm64 |
| Windows | x64, arm64 |
| Linux (glibc) | x64, arm64 |
| Linux (musl) | x64, arm64 |
| Linux (glibc, gnueabihf) | armv7 |
This implementation is validated against the W3C Web Platform Tests for WebCodecs.
| Status | Count | Percentage |
| ----------- | ----- | ---------- |
| Passing | 573 | 99.1% |
| Skipped | 5 | 0.9% |
| Failing | 0 | 0% |
Skipped tests are due to platform-specific features or edge cases.
19 WPT test files require browser APIs unavailable in Node.js:
| Category | Tests | APIs Required |
| ---------------------- | ----- | -------------------------------------- |
| Serialization/Transfer | 5 | MessageChannel, structured clone |
| WebGL/Canvas | 5 | WebGL textures, ImageBitmap, Canvas 2D |
| Cross-Origin Isolation | 8 | COOP/COEP headers |
| WebIDL | 1 | IDL interface validation |
See __test__/wpt/README.md for detailed test status.
Hardware encoding is fully supported with automatic GPU selection and fallback:
| Platform | Encoders | Features |
| -------- | ------------ | ------------------------------------------------------ |
| macOS | VideoToolbox | H.264, HEVC; realtime mode, allow_sw control |
| NVIDIA | NVENC | H.264, HEVC, AV1; presets p1-p7, spatial-aq, lookahead |
| Linux | VAAPI | H.264, HEVC, VP9, AV1; quality 0-8 |
| Intel | QSV | H.264, HEVC, VP9, AV1; presets, lookahead |
`typescript`
encoder.configure({
codec: 'avc1.42001E',
width: 1920,
height: 1080,
// Hardware acceleration preference
hardwareAcceleration: 'prefer-hardware', // 'no-preference' | 'prefer-hardware' | 'prefer-software'
// Latency mode affects encoder tuning
latencyMode: 'realtime', // 'quality' | 'realtime'
// Alpha channel preservation (VP9 and HEVC only)
alpha: 'discard', // 'keep' | 'discard' (default: 'discard')
})
- latencyMode: 'realtime' - Enables low-latency encoder options (smaller GOP, no B-frames, fast presets)latencyMode: 'quality'
- - Enables quality-focused options (larger GOP, B-frames, lookahead)alpha: 'keep'
- - Preserves alpha channel (VP9 and HEVC only). For HEVC, requires hardwareAcceleration: 'prefer-software'
The encoder automatically applies optimal settings for each hardware encoder based on the latency mode.
Encode video with transparency using VP9 or HEVC:
`typescript
import { VideoEncoder, VideoFrame } from '@napi-rs/webcodecs'
const encoder = new VideoEncoder({
output: (chunk, metadata) => {
console.log(Alpha chunk: ${chunk.byteLength} bytes)
},
error: (e) => console.error(e),
})
// VP9 alpha - works with hardware or software
encoder.configure({
codec: 'vp09.00.10.08',
width: 1920,
height: 1080,
alpha: 'keep',
})
// HEVC alpha - requires software encoder
encoder.configure({
codec: 'hev1.1.6.L93.B0',
width: 1920,
height: 1080,
alpha: 'keep',
hardwareAcceleration: 'prefer-software', // Required for HEVC alpha
})
// Create frame with alpha channel (I420A format)
const width = 1920
const height = 1080
const frameData = new Uint8Array(width height 1.5 + width * height) // Y + U + V + A
const frame = new VideoFrame(frameData, {
format: 'I420A',
codedWidth: width,
codedHeight: height,
timestamp: 0,
})
encoder.encode(frame)
frame.close()
`
All scalability modes (L1Tx, L2Tx, L3Tx, S2Tx, S3Tx, and variants) are accepted and populate metadata.svc.temporalLayerId when temporal layers >= 2.
The W3C WebCodecs spec only defines temporalLayerId in SvcOutputMetadata - there is no spatialLayerId field in the spec. See W3C WebCodecs ยง6.7.
Note: This implementation computes temporal layer IDs algorithmically from frame index per W3C spec. FFmpeg is not configured for actual SVC encoding, so base layer frames are not independently decodable.
Synchronous errors (e.g., calling encode() on a closed encoder) throw native DOMException instances that pass instanceof DOMException checks per W3C spec:
`typescript`
try {
encoder.encode(frame) // on closed encoder
} catch (e) {
console.log(e instanceof DOMException) // true
console.log(e.name) // "InvalidStateError"
}
Asynchronous error callbacks receive standard Error objects with the DOMException name in the message:
`typescript`
const encoder = new VideoEncoder({
output: (chunk) => {},
error: (e) => {
console.log(e.message) // "EncodingError: ..."
},
})
VideoFrame.copyTo() and VideoFrame.allocationSize() support format conversion per W3C WebCodecs spec:
`typescript
const frame = new VideoFrame(i420Data, {
format: 'I420',
codedWidth: 1920,
codedHeight: 1080,
timestamp: 0,
})
// Get allocation size for RGBA output
const rgbaSize = frame.allocationSize({ format: 'RGBA' })
// Copy with format conversion (I420 โ RGBA)
const rgbaBuffer = new Uint8Array(rgbaSize)
const layout = await frame.copyTo(rgbaBuffer, { format: 'RGBA' })
frame.close()
`
Supported conversions:
| Source Format | Target Format | Status |
| ---------------------------- | ---------------------- | -------------------- |
| I420, I422, I444, NV12, NV21 | RGBA, RGBX, BGRA, BGRX | โ
|
| RGBA, RGBX, BGRA, BGRX | RGBA, RGBX, BGRA, BGRX | โ
|
| RGBA, RGBX, BGRA, BGRX | I420, I422, I444, NV12 | โ NotSupportedError |
Per WPT videoFrame-copyTo-rgb.any.js, RGB-to-YUV conversion throws NotSupportedError.
Custom layouts with overflow-inducing values (e.g., offset: 2ยณยฒ-2) throw TypeError via checked arithmetic. Rect alignment is validated against the source format during conversion.
ImageDecoder supports all W3C spec options:
| Option | Status | Notes |
| ---------------------- | ------ | --------------------------------------------------------------------------------- |
| desiredWidth/Height | โ
| Scales decoded frames to specified dimensions |preferAnimation
| | โ
| When false, only decodes first frame for animated formats |colorSpaceConversion
| | โ
| "default" extracts color space metadata, "none" ignores it (Chromium-aligned) |
Note: Per W3C spec, desiredWidth and desiredHeight must both be specified or both omitted.
- ImageDecoder GIF animation: FFmpeg may return only the first frame. Use VideoDecoder with GIF codec for full animation.
Per-frame quantizer control is available for VP9 and AV1 when using bitrateMode: 'quantizer':
`typescript
encoder.configure({
codec: 'vp09.00.10.08',
width: 1920,
height: 1080,
bitrateMode: 'quantizer',
})
// Per-frame QP control (0-255 range per W3C WebCodecs spec)
encoder.encode(frame, { vp9: { quantizer: 128 } })
`
| Codec | Per-Frame QP | Notes |
| ----- | ------------ | ------------------------------------------------------------ |
| VP9 | โ
Works | Uses FFmpeg's dynamic qmax update mechanism |
| AV1 | โ No effect | FFmpeg's libaom wrapper doesn't support dynamic qmax updates |
| H.264 | โ
Works | Uses FFmpeg's frame quality mechanism (0-51 range) |
| H.265 | โ
Works | Uses FFmpeg's frame quality mechanism (0-51 range) |
Note: The 0-255 quantizer range for VP9/AV1 aligns with Chromium's WebCodecs implementation. Internally, values are converted to the 0-63 encoder range using q_index / 4.
This library uses Rust's tracing crate for structured logging. Enable logging via the WEBCODECS_LOG environment variable:
`bashEnable all logs at info level
WEBCODECS_LOG=info node your-app.js
$3
| Target | Description |
| ----------- | ---------------------------------------------------------------------- |
|
ffmpeg | FFmpeg internal logs (codec initialization, encoding/decoding details) |
| webcodecs | WebCodecs API logs (codec errors, state transitions) |$3
| FFmpeg Level | Tracing Level |
| ------------ | ------------- |
| ERROR/FATAL |
error |
| WARNING | warn |
| INFO | info |
| VERBOSE | debug |
| DEBUG/TRACE | trace |Without
WEBCODECS_LOG set, all logs are silently discarded.API Reference
This package implements the W3C WebCodecs API. Key classes:
-
VideoEncoder / VideoDecoder - Video encoding and decoding with EventTarget support
- AudioEncoder / AudioDecoder - Audio encoding and decoding with EventTarget support
- VideoFrame - Raw video frame data (supports buffer data, existing VideoFrame, or @napi-rs/canvas Canvas)
- AudioData - Raw audio sample data
- EncodedVideoChunk / EncodedAudioChunk - Encoded media data
- ImageDecoder - Static image decoding
- VideoColorSpace - Color space information
- Mp4Demuxer / WebMDemuxer / MkvDemuxer - Container demuxing with seeking
- Mp4Muxer / WebMMuxer / MkvMuxer - Container muxing with streaming supportAll encoders and decoders implement the
EventTarget interface with addEventListener(), removeEventListener(), and dispatchEvent().For full API documentation, see the W3C WebCodecs specification.
Development
$3
- Rust (latest stable)
- Node.js 18+
- pnpm
$3
`bash
pnpm install
pnpm build
`$3
`bash
pnpm test
`$3
`bash
pnpm lint
cargo clippy
``MIT