@terminals-tech/embeddings

Ultra-lightweight semantic embeddings for event graphs. Browser-first with swappable providers.

Zero dependencies on the core, <3MB total with embeddings model. Runs entirely in browsers.

Installation

``bash
npm install @terminals-tech/embeddings
`

Quick Start

`javascript
import { EmbeddingProviderFactory, SemanticSearch } from '@terminals-tech/embeddings'

// Create the best available provider (auto-detects)
const provider = await EmbeddingProviderFactory.createBest({
cache: true,
quantizeCache: true // Save 75% memory
})

// Create semantic search
const search = new SemanticSearch(provider)

// Add documents
await search.addDocuments([
{ text: 'User logged in successfully' },
{ text: 'Authentication failed due to invalid password' },
{ text: 'Session expired after timeout' }
])

// Search semantically
const results = await search.search('login problems', { topK: 2 })
// Returns documents about authentication failures and session issues
`

Features

$3

- TransformersJS Provider: all-MiniLM-L6-v2 (22MB model, 384 dimensions)
- Mock Provider: Deterministic embeddings for testing (0 dependencies)
- Memory-Efficient Cache: LRU with optional int8 quantization

$3

`javascript
// Use TransformersJS (browser-optimized)
const transformer = await EmbeddingProviderFactory.create('transformers', {
modelId: 'Xenova/all-MiniLM-L6-v2'
})

// Use mock for testing
const mock = await EmbeddingProviderFactory.create('mock')

// Auto-detect best available
const best = await EmbeddingProviderFactory.createBest()
`

$3


`javascript
import { EmbeddingCache } from '@terminals-tech/embeddings'

const cache = new EmbeddingCache({
maxSize: 1000, // Maximum entries
ttlMs: 3600000, // 1 hour TTL
quantize: true // Int8 quantization (75% memory savings)
})

// Use with any provider
const provider = new TransformersEmbeddingProvider({
cache: true,
cacheSize: 500,
quantizeCache: true
})
`

$3


`javascript
const search = new SemanticSearch(provider)

// Add documents with metadata
await search.addDocuments([
{
text: 'Critical error in payment processing',
metadata: { severity: 'high', timestamp: Date.now() }
}
])

// Search with threshold
const results = await search.search('payment issues', {
topK: 5,
threshold: 0.7 // Minimum similarity score
})

// Find semantic clusters
const clusters = await search.findClusters({
minClusterSize: 3,
similarityThreshold: 0.8
})
`

Integration with @terminals-tech/graph

Enhance your event graphs with semantic understanding:

`javascript
import { TextGraph } from '@terminals-tech/graph'
import { enhanceGraphWithSemantics } from '@terminals-tech/embeddings'

const graph = new TextGraph()
await enhanceGraphWithSemantics(graph)

// Now extract semantic relationships
const relations = await graph.extractSemanticRelations(
'The server crashed because memory usage exceeded the limit'
)
// Understands causal relationship even without explicit keywords

// Find semantic clusters in events
const clusters = await graph.findSemanticClusters(events)
// Groups events by meaning, not just structure
`

Provider Comparison

| Provider | Model Size | Dimensions | Speed | Quality | Dependencies |
|----------|------------|------------|-------|---------|--------------|
| TransformersJS | 22MB | 384 | Fast | High | @huggingface/transformers |
| Mock | 0KB | 64 | Instant | Test | None |

Memory Optimization

With quantization enabled:
- Float32: 384 dimensions × 4 bytes = 1,536 bytes per embedding
- Int8: 384 dimensions × 1 byte = 384 bytes per embedding
- Savings: 75% memory reduction with ~5% accuracy loss

Browser Support

- ✅ Chrome 90+
- ✅ Firefox 89+
- ✅ Safari 14.1+
- ✅ Edge 90+

WebAssembly and SIMD support recommended for best performance.

API Reference

$3


`typescript
interface EmbeddingProvider {
embed(text: string): Promise
embedBatch(texts: string[]): Promise
similarity(a: EmbeddingVector, b: EmbeddingVector): number
dispose?(): void
}
`

$3

- addDocuments(docs) - Add documents to search index
- search(query, options) - Search for similar documents
- findClusters(options) - Find document clusters
- clear() - Clear the search index
- stats() - Get memory usage statistics

Performance

- Embedding Generation: ~10ms per sentence (CPU)
- Similarity Search: <1ms for 1000 documents
- Memory Usage: ~400KB for 1000 cached embeddings (quantized)
- Model Load Time: ~2s first load (cached after)

Examples

See the examples/ directory for:
- full-system.ts` - Complete integration with @terminals-tech suite
- More examples coming soon!

License

MIT © Intuition Labs

---

Built with ❤️ for developers who want semantic understanding without the complexity.