@etaio/video-analysis

Shared video analysis services for ShotRush - frame extraction, AI analysis, and summary generation.

Features

- Frame Extraction: Extract frames from videos using FFmpeg with quality optimization
- AI Analysis: Analyze frames using OpenAI Vision API for scenes, objects, and actions
- Summary Generation: Generate video summaries and identify key moments
- Scene Detection: Automatic scene change detection
- Relevance Scoring: Score frames for auto-editing purposes

Installation

``bash
pnpm install
pnpm build
`

Usage

$3

`typescript
import { extractFrames, getVideoMetadata } from '@etaio/video-analysis';

// Get video info
const metadata = await getVideoMetadata('/path/to/video.mp4');
console.log(Duration: ${metadata.duration}s, FPS: ${metadata.fps});

// Extract frames
const frames = await extractFrames('/path/to/video.mp4', {
frameInterval: 1, // 1 frame per second
maxFrames: 60, // Max 60 frames
frameQuality: 'medium' // 1280x720 JPEG 80% quality
});

console.log(Extracted ${frames.length} frames);
`

$3

`typescript
import { analyzeFramesBatch } from '@etaio/video-analysis';

const results = await analyzeFramesBatch(
frames.map(f => ({
buffer: f.buffer,
frameNumber: f.frameNumber,
timestamp: f.timestamp,
timestampMs: f.timestampMs
})),
{
apiKey: process.env.OPENAI_API_KEY!,
analyzeObjects: true,
analyzeActions: true,
analyzeScenes: true,
contextDescription: 'Product demo video'
},
(current, total) => {
console.log(Analyzing frame ${current}/${total});
}
);

console.log('Analysis complete:', results);
`

$3

`typescript
import { generateVideoSummary } from '@etaio/video-analysis';

const { summary, keyMoments, tokenUsage } = await generateVideoSummary(
frameAnalyses,
{
apiKey: process.env.OPENAI_API_KEY!,
videoDuration: metadata.duration,
contextDescription: 'Product demo video',
maxKeyMoments: 10
}
);

console.log('Summary:', summary.brief);
console.log('Key Moments:', keyMoments.length);
console.log('Tokens Used:', tokenUsage.totalTokens);
`

Frame Quality Options

- micro: 320x180 JPEG 70% (~5-10 KB per frame)
- low: 640x360 JPEG 75% (~15-25 KB per frame)
- medium: 1280x720 JPEG 80% (~40-60 KB per frame)
- high: 1920x1080 JPEG 85% (~80-120 KB per frame)

Cost Estimation

Using gpt-4o model with medium quality frames:

- Input: ~1,000 tokens per frame (image + prompt)
- Output: ~200 tokens per frame (analysis)
- Summary: ~1,500 input + 500 output tokens

Example (60-second video at 1 fps):
- 60 frames × 1,200 tokens = 72,000 tokens
- Summary: 2,000 tokens
- Total: ~74,000 tokens ≈ $0.38 USD

Architecture

Used in:
- Cloud Run Worker (services/video-analysis-worker): Heavy processing
- Firebase Functions (services/serverless/functions): API gateway
- Frontend Services (apps/web/src/services): Client SDK (future)

Development

`bash


Watch mode

pnpm dev

Build

pnpm build

Clean

pnpm clean
`

Dependencies

- fluent-ffmpeg: Frame extraction
- sharp: Image optimization
- axios: OpenAI API calls
- firebase-admin: Firestore integration
- @etaio/core-types`: Shared TypeScript types