@mp4e/player

React player for MP4E interactive videos. The WASM engine is bundled inline for universal compatibility.

Installation

``bash
npm install @mp4e/player
`

Usage

`tsx
import { MP4EPlayer } from '@mp4e/player';
import '@mp4e/player/styles.css';

function App() {
return (
src="/video.mp4"
onMetadataLoaded={(metadata) => {
console.log('Loaded:', metadata.schemaVersion);
}}
/>
);
}
`

The player automatically extracts metadata embedded in the MP4 file - you typically only need the src prop.

Features

- Automatic metadata extraction - Reads MP4E metadata from the video file
- WASM engine bundled - Works offline, no external fetches
- Universal bundler support - Vite, webpack, Next.js
- TypeScript types included
- WebGL-accelerated rendering - GPU-powered object highlighting
- Worker mode - Offload engine to Web Worker for better performance

Props

$3

| Prop | Type | Default | Description |
|------|------|---------|-------------|
| src | string | required | Video URL |
| metadata | MP4EMetadata | - | Override embedded metadata |
| controls | boolean | true | Show playback controls |
| autoplay | boolean | false | Auto-start playback |
| muted | boolean | false | Start muted |
| loop | boolean | false | Loop playback |
| interactive | boolean | true | Enable object interactions |
| showBboxes | boolean | false | Show object bounding boxes |
| showCanvasLabels | boolean | false | Show labels on hover |
| showObjectDisplays | boolean | true | Show hover/click displays |
| useWorker | boolean | false | Run engine in Web Worker |
| debug | boolean | false | Show debug UI |

$3

| Prop | Type | Description |
|------|------|-------------|
| onEngineReady | () => void | Engine initialized |
| onEngineError | () => void | Engine failed to load |
| onMetadataLoaded | (metadata: MP4EMetadata) => void | Metadata loaded |
| onTimeUpdate | (time: number, frame: number) => void | Time changed (throttled ~60fps) |
| onFrameChange | (frame: number) => void | Frame changed (every frame) |
| onVideoDuration | (duration: number) => void | Duration available |
| onObjectClick | (objectId: string, data: ObjectData) => void | Object clicked |
| onObjectHover | (objectId: string \| null, data: ObjectData \| null) => void | Object hovered |
| onPlay | () => void | Video started |
| onPause | () => void | Video paused |
| onEnded | () => void | Video ended |
| onSceneChange | (scene: Scene) => void | Scene changed |

$3

| Prop | Type | Description |
|------|------|-------------|
| initialTime | number | Start time in seconds |
| objectGroups | ObjectGroup[] | Object grouping configuration |
| defaultDisplaySettings | ObjectDisplaySettings | Default display behavior |
| runtimeFlags | MP4ERuntimeFlags | Feature flags |
| disableInteraction | boolean | Disable all hover/click interactions |

Ref API

Access player methods via ref:

`tsx
import { useRef } from 'react';
import { MP4EPlayer, MP4EPlayerRef } from '@mp4e/player';

function App() {
const playerRef = useRef(null);

const handleSeek = () => {
playerRef.current?.seek(10); // Seek to 10 seconds
};

return (
<>

Jump to 10s

);
}
`

$3

| Method | Returns | Description |
|--------|---------|-------------|
| play() | Promise | Start playback |
| pause() | void | Pause playback |
| seek(time) | void | Seek to time (seconds) |
| seekToFrame(frame) | void | Seek to frame number |
| seekSilently(time) | void | Seek without triggering rules |

$3

| Method | Returns | Description |
|--------|---------|-------------|
| getCurrentTime() | number | Current time in seconds |
| getCurrentFrame() | number | Current frame number |
| getDuration() | number | Video duration in seconds |
| isPaused() | boolean | Is video paused |
| isMuted() | boolean | Is video muted |
| getVolume() | number | Volume (0-1) |
| setVolume(v) | void | Set volume (0-1) |
| mute() | void | Mute audio |
| unmute() | void | Unmute audio |
| getPlaybackRate() | number | Playback speed |
| setPlaybackRate(r) | void | Set playback speed |

$3

| Method | Returns | Description |
|--------|---------|-------------|
| isFullscreen() | boolean | Is player fullscreen |
| enterFullscreen() | Promise | Enter fullscreen |
| exitFullscreen() | Promise | Exit fullscreen |
| toggleFullscreen() | Promise | Toggle fullscreen |

$3

| Method | Returns | Description |
|--------|---------|-------------|
| getVideoElement() | HTMLVideoElement \| null | Video element |
| getCanvasElement() | HTMLCanvasElement \| null | Canvas element |
| getOverlayContainerRef() | HTMLDivElement \| null | Overlay container |

$3

| Method | Returns | Description |
|--------|---------|-------------|
| getVideoDisplayTransform() | VideoDisplayTransform \| null | Get scale/offset for coordinate conversion |
| getOverlayPositions() | Map | Get current overlay positions |
| getEffectiveMetadata() | MP4EMetadata \| null | Get resolved metadata |

#### VideoDisplayTransform

`typescript
interface VideoDisplayTransform {
scale: number // Scale factor from video-native to display
offsetX: number // Horizontal offset (letterboxing)
offsetY: number // Vertical offset (letterboxing)
displayWidth: number
displayHeight: number
videoWidth: number
videoHeight: number
}
`

Use this to convert between video-native coordinates and screen coordinates:

`typescript
// Convert video-native position to screen position
const transform = playerRef.current?.getVideoDisplayTransform();
if (transform) {
const screenX = videoNativeX * transform.scale + transform.offsetX;
const screenY = videoNativeY * transform.scale + transform.offsetY;
}
`

$3

For smooth 60fps position updates (e.g., during drag operations):

`typescript
// Update overlay position imperatively (no React re-render)
playerRef.current?.updateOverlayStyle('overlay-id', {
x: 100, // video-native coordinates
y: 50,
width: 200,
height: 100,
});
`

This directly updates the DOM element for smooth visual feedback, bypassing React's rendering cycle. Useful for:
- Drag operations
- Real-time position updates
- Animation

Types

$3

`typescript
interface MP4EMetadata {
schemaVersion: string;
videoDetails: VideoDetails;
objects: {
registry: ObjectRegistry;
timeline: FrameData[];
};
layers: Layer[];
variables?: Variable[];
rules?: Rule[];
scenes?: Scene[];
}
`

$3

`typescript
interface ObjectData {
id: string;
label: string;
userLabel?: string;
confidence: number;
trackingType: ObjectTrackingType;
groupIds?: string[];
hidden?: boolean;
data?: Record;
}
`

Development

`bash


Run demo app

npm run dev

Build library

npm run build:lib

Build types

npm run build:types

Publish to npm

npm publish
`

Architecture

`
Player (React) → Engine (WASM) → Pure State Machine
`

- Player is the "body" - handles UI, video, rendering
- Engine is the "brain" - handles events, state, rules
- Zero coupling - player listens to engine events

Performance

The player includes several performance optimizations:

1. Phase 1: Visibility-based re-rendering - only re-render when object visibility changes
2. Phase 2: DOM pooling - reuse overlay elements instead of creating/destroying
3. Phase 3: WebGL rendering - GPU-accelerated bounding box rendering
4. Worker mode: Offload engine computation to Web Worker

Enable worker mode for complex videos:

`tsx

`

Changelog

$3

- Added disableInteraction prop to disable all hover/click interactions
- Added onFrameChange callback (fires on every frame change)
- Added onVideoDuration callback (fires when duration is available)
- Added getVideoDisplayTransform() ref method for coordinate conversion
- Added getOverlayPositions() ref method to get current overlay positions
- Added updateOverlayStyle() ref method for imperative 60fps position updates
- Added getEffectiveMetadata() ref method
- Added getOverlayContainerRef()` ref method

$3

- WebGL-accelerated bounding box rendering (Phase 3 optimization)
- Improved performance for videos with many objects

$3

- DOM pooling for group-bound overlays (Phase 2 optimization)
- Visibility-based re-rendering (Phase 1 optimization)