@morphgrid/core

Browser-first WebAssembly runtime and API for HFST/GiellaLT morphology with FST-based token joining.

Overview

The core package provides a complete morphological analysis and generation system with intelligent token joining capabilities. It uses HFST (Helsinki Finite-State Technology) compiled to WebAssembly for high-performance morphological processing.

Primary Environment: Modern web browsers
CLI/Testing: Use Bun for excellent WASM support
Production Servers: Use UralicNLP or HFST CLI tools

Key Features

- HFST WASM Runtime: High-performance morphological analysis and generation
- Web Worker Architecture: Non-blocking processing in browsers
- FST-based Joins: Intelligent token combination using morphological analysis
- Language Pack System: Lazy-loaded transducers with integrity verification
- Comprehensive API: Simple interface for complex morphological operations
- Full Irregular Forms: Proper handling of irregular verbs and nouns via HFST transducers

Quick Start

``typescript
import { morph, configureMorphRuntime, configureMorphHfst } from '@morphgrid/core';

// Configure the runtime
configureMorphRuntime('hfst');
configureMorphHfst({
wasmUrl: '/path/to/hfst.wasm',
packUrl: '/path/to/packs/'
});

// Load a language
await morph.load('fr-FR');

// Analyze words
const analyses = await morph.analyse('aime', 'fr-FR');
// Returns: [{ lemma: 'aimer', surface: 'aime', tags: ['vblex', 'pri', 'p1', 'sg'] }]

// Generate word forms
const forms = await morph.generate({ lemma: 'cheval', tags: ['PL'] }, 'fr-FR');
// Returns: ['chevaux']

// Join tokens intelligently
const decision = await morph.join('je', 'aime', 'fr-FR');
// Returns: { surfacePrev: "j'", surfaceNext: 'aime', joiner: '', noSpace: true, ... }
`

API Reference

$3

#### configureMorphRuntime(mode: 'hfst' | 'rules')
Sets the morphological processing mode. Use 'hfst' for FST-based processing.

#### configureMorphHfst(options: HfstConfig)
Configures HFST-specific settings:
`typescript
interface HfstConfig {
wasmUrl?: string; // Path to hfst.wasm file
packUrl?: string; // Base URL for language packs
}
`

$3

#### morph.load(lang: string): Promise
Loads morphological transducers for a language.

#### morph.analyse(surface: string, lang: string): Promise
Analyzes a surface form into morphological components.

#### morph.generate(input: MorphInput, lang: string): Promise
Generates surface forms from morphological specification.

#### morph.join(prev: string, next: string, lang: string): Promise
Determines how two tokens should be joined using morphological analysis.

FST-based Join System

The join system represents a major advancement in computational morphology for text composition:

$3


`
Custom rule files → Manual maintenance → Language-specific implementations
`

$3

`
Existing GiellaLT/Apertium transducers → Morphological analysis → Feature-based decisions
`

$3

1. Linguistic Accuracy: Based on proven morphological analysis
2. Reduced Maintenance: No custom rule files to maintain
3. Extensibility: Easy to add new languages
4. Robustness: Graceful fallback when FST unavailable

$3

- French: Comprehensive elision (je + aime → j'aime)
- Spanish: Clitics and contractions (de + el → del)
- German: Compound formation (Haus + Tür → Haustür)
- Others: Intelligent default spacing

Architecture

$3


`
Main Thread ←→ Web Worker ←→ HFST WASM
↑ ↑ ↑
Public API Message Bus Morphology
`

$3

- Lazy Loading: Transducers loaded on demand
- Integrity Verification: SHA-256 checksums
- Caching: Browser Cache Storage for performance
- Format: HFST optimized lookup (.hfstol)

Error Handling

The system provides clear error messages instead of silent failures:

`typescript
// When FST analysis unavailable
const result = await morph.analyse('word', 'unsupported-lang');
// Returns: [{ lemma: 'HFST_TRANSDUCER_NOT_LOADED:word', surface: 'word', tags: [] }]

// When generation fails
const forms = await morph.generate({ lemma: 'invalid', tags: ['INVALID'] }, 'fr-FR');
// Returns: ['HFST_GENERATION_FAILED:invalid+INVALID:no_HFST_model_loaded_or_no_results_for_fr-FR']
`

Testing

The package includes comprehensive tests:

`bash
npm test # Run all tests
npm run test:gold # TSV gold standard tests
npm run test:comprehensive # Language-specific join tests
npm run test:hfst # FST functionality tests
``

Browser vs Node.js

The package works in both environments:

- Browser: Uses Web Workers for non-blocking processing
- Node.js: Uses worker_threads with proper WASM loading
- Unified API: Same interface regardless of environment