OmegaSynth - AsterMind Synthetic Data Generator

Label-conditioned synthetic data generation engine for ELM/KELM pipelines with advanced pattern matching and realism improvements.

Features

$3

- retrieval: Simple deterministic retrieval from stored samples (fully realistic formats from curated synthetic examples)
- elm: ELM-based generation with label conditioning (75-85% realistic)
- hybrid: Blends retrieval with ELM jitter for realism + variation (80-90% realistic)
- exact: High-fidelity retrieval with pattern-based variations (typically 95–100% format realism on our internal benchmarks)
- premium: Best-of-all-worlds mode combining exact, hybrid, and improved ELM (85-100% realistic)

$3

- Pattern Correction: Post-processing that learns patterns from training data and corrects generated samples
- Sequence Context: N-gram pattern learning for better character sequence generation
- Confidence-Based Selection: Ranks and selects best candidates based on pattern matching scores
- Character Embeddings: Continuous vector representations for better semantic understanding
- One-Hot Encoding: Optional one-hot encoding for improved accuracy (requires more memory)
- Classification Mode: Optional classification instead of regression for better discrete output handling
- Exact Mode: Zero-jitter option in hybrid mode for high-fidelity realistic data

Installation

$3

``bash
npm install @astermind/astermind-synth
`

$3

`bash
git clone https://github.com/astermindai/astermind-synth.git
cd astermind-synth
npm install
npm run build
`

License Configuration

OmegaSynth requires a valid license token for production use. Set the ASTERMIND_LICENSE_TOKEN environment variable:

`bash
export ASTERMIND_LICENSE_TOKEN="your-license-token-here"
`

A valid license token is required for all use (including evaluation and testing).

For evaluation: Obtain a free 30-day trial key from the license server (see below). Trial keys work exactly like production keys but expire after 30 days.

For testing: Use a trial key, or mock the license runtime in your test setup (see test examples).

$3

👉 Get Your License Key →

Visit our getting started page for step-by-step instructions on:
- Creating a free trial account
- Obtaining your license token
- Setting up your development environment

$3

- Trial Keys: Free 30-day evaluation keys obtained from the license server. Perfect for trying the product before purchasing.
- Production Keys: Full license keys for production use, obtained from AsterMind.

How it works:
- All license keys are validated server-side via JWT verification
- Trial keys expire after 30 days (or as specified in the key)
- All usage requires a valid key (follows industry licensing patterns with server-validated keys)
- License validation happens automatically when you use the library
- Flexible Audience Check: Tokens with different audiences (e.g., "astermind-elm") that include the "astermind-synth" feature will be accepted, allowing multi-product license keys to work across AsterMind products

$3

- ASTERMIND_LICENSE_TOKEN: Your license token (JWT format) - Required for all use

Note: This follows industry licensing patterns with server-validated keys. Free 30-day trial keys are available via the self-service API.

Quick Start

$3

`typescript
import { loadPretrained } from '@astermind/astermind-synth';

// Load a pretrained retrieval/hybrid model and immediately generate values
const synth = loadPretrained('hybrid');
const firstName = await synth.generate('first_name');
const email = await synth.generate('email');
console.log(${firstName} - ${email});
`

$3

`typescript
import { OmegaSynth } from '@astermind/astermind-synth';

const synth = new OmegaSynth({
mode: 'hybrid',
maxLength: 32,
seed: 42
});

const dataset = [
{ label: 'product_name', value: 'Widget A' },
{ label: 'product_name', value: 'Widget B' },
];

await synth.train(dataset);
const product = await synth.generate('product_name');
`

$3

You can load a pretrained model and fine-tune it with your own data:

`typescript
import { loadPretrained } from '@astermind/astermind-synth';

// Load pretrained model
const synth = loadPretrained('retrieval');

// Wait for initial training to complete
await new Promise(resolve => setTimeout(resolve, 100));

// Add your custom data
const customData = [
{ label: 'product_name', value: 'MyProduct A' },
{ label: 'product_name', value: 'MyProduct B' },
{ label: 'custom_label', value: 'Custom Value' },
];

// Fine-tune with additional data
await synth.train(customData);

// Now you can generate from both pretrained and custom labels
const product = await synth.generate('product_name');
const custom = await synth.generate('custom_label');
const email = await synth.generate('email'); // Still works from pretrained data
`

Fine-Tuning Behavior by Mode:
- Retrieval Mode: train() adds new samples to existing data (true fine-tuning)
- ELM/Hybrid Modes: train() retrains on the new data only (replaces previous training). For true fine-tuning, combine old and new data before calling train().

$3

After training with your own data, you can save the model for later use:

`typescript
import { OmegaSynth, saveTrainedModel, loadPretrainedFromVersion } from '@astermind/astermind-synth';
import * as path from 'path';

// Train your model
const synth = new OmegaSynth({ mode: 'hybrid' });
const trainingData = [
{ label: 'product_name', value: 'Widget A' },
{ label: 'product_name', value: 'Widget B' },
// ... more training data
];

await synth.train(trainingData);

// Save the trained model
const outputDir = path.join(process.cwd(), 'my-models');
const modelPath = await saveTrainedModel(
synth,
trainingData, // Training data is required for saving
outputDir,
'1.0.0' // Optional version string
);

// Later, load the saved model
const loadedSynth = loadPretrainedFromVersion(modelPath);
const result = await loadedSynth.generate('product_name');
`

Note: The training data you used must be provided when saving, as it's needed to reconstruct the model when loading.

$3

`typescript
// Premium mode - combines all improvements
const synth = new OmegaSynth({
mode: 'premium',
maxLength: 50,
seed: 42,
usePatternCorrection: true, // Enable pattern correction
useOneHot: false, // Set to true if memory allows
});

await synth.train(dataset);
const result = await synth.generate('first_name');
`

$3

`typescript
// High-fidelity realistic (0% jitter)
const synth = new OmegaSynth({
mode: 'hybrid',
exactMode: true, // 0% jitter = high-fidelity realistic
usePatternCorrection: true,
});
`

API Reference

$3

Main class for synthetic data generation.

#### Constructor

`typescript
const synth = new OmegaSynth({
mode: 'retrieval' | 'elm' | 'hybrid' | 'exact' | 'premium',
maxLength?: number, // Maximum string length (default: 32)
seed?: number, // Random seed for deterministic generation
exactMode?: boolean, // For hybrid: use 0% jitter (default: false)
useOneHot?: boolean, // Use one-hot encoding (default: false)
useClassification?: boolean, // Use classification (default: false)
usePatternCorrection?: boolean, // Enable pattern correction (default: true)
});
`

#### Methods

`typescript
// Train the generator
await synth.train(dataset: LabeledSample[]);

// Generate a single sample
const result = await synth.generate(label: string, seed?: number);

// Generate multiple samples
const batch = await synth.generateBatch(label: string, count: number);

// Get all available labels
const labels = synth.getLabels();

// Check if trained
const isReady = synth.isTrained();

// Save trained model (requires training data)
await saveTrainedModel(synth, trainingData, './my-models', '1.0.0');
`

$3

`typescript
interface LabeledSample {
label: string;
value: string;
}
`

Pretrained Labels

The pretrained model supports the following labels:

- Names: first_name, last_name
- Contact: phone_number, email
- Address: street_address, city, state, country
- Business: company_name, job_title, product_name
- Other: color, uuid, date, credit_card_type (card network names, e.g., Visa®, Mastercard®), device_type

Performance

After implementing all improvements, internal evaluation on the bundled datasets shows:

- Quality: Samples are generated under label-specific format validation rules
- Uniqueness: Standard modes are designed to avoid duplicates within a generation batch
- Realism: 56.30% overall (improved from ~37% baseline)

$3

- phone_number: 82.5%
- email: 82.4%
- uuid: 83.3%
- date: 82.0%
- street_address: 64.3%
- first_name: 59.6%
- city: 53.0%
- product_name: 50.5%
- company_name: 49.3%
- credit_card_type: 47.0%
- device_type: 46.7%
- last_name: 42.7%
- state: 42.7%
- color: 42.4%
- country: 37.4%
- job_title: 35.2%

Architecture Improvements

All 10 architectural improvements have been implemented:

1. ✅ Classification Mode: Optional classification support in ELMGenerator
2. ✅ One-Hot Encoding: Memory-aware one-hot encoding option
3. ✅ Exact Mode: Zero-jitter option for high-fidelity realistic data
4. ✅ Pattern Correction: Post-processing pattern matching and correction
5. ✅ Confidence-Based Selection: Candidate scoring and ranking
6. ✅ Exact Generator: High-fidelity retrieval with pattern variations
7. ✅ Character Embeddings: Character co-occurrence learning
8. ✅ Sequence Context: N-gram pattern learning for better sequences
9. ✅ Premium Generator: Best-of-all-worlds mode combining all improvements
10. ✅ Updated Interface: Full support for all new modes and options

All improvements are integrated into the current codebase.

Memory Optimization

- One-hot encoding defaults to false to prevent memory issues (can be enabled if memory allows)
- PremiumGenerator uses lazy training (only trains generators when needed)
- Pattern correction and sequence context are lightweight
- All improvements are optional and configurable

Examples

$3

`typescript
import { OmegaSynth } from '@astermind/astermind-synth';

const synth = new OmegaSynth({
mode: 'hybrid',
maxLength: 32,
});

await synth.train([
{ label: 'first_name', value: 'John' },
{ label: 'first_name', value: 'Jane' },
]);

const name = await synth.generate('first_name');
console.log(name); // "John" or "Jane" or variation
`

$3

`typescript
const names = await synth.generateBatch('first_name', 10);
console.log(names); // Array of 10 unique names
`

$3

`typescript
// Run evaluation script
npm run evaluate-generated
`

The evaluation script will generate samples, score quality/realism/uniqueness, and write reports.

Development

$3

`bash
npm run build
`

$3

`bash
npm test
`

$3

`bash


Train a new model

npm run train

Test a model

npm run test-elm
`

Project Structure

`
src/omegasynth/
├── core/
│ ├── PatternCorrector.ts # Pattern matching and correction
│ ├── SequenceContext.ts # N-gram pattern learning
│ ├── CharacterEmbeddings.ts # Character embeddings
│ ├── validation.ts # Label-specific validation
│ └── elm_utils.ts # ELM utilities
├── generators/
│ ├── RetrievalGenerator.ts # Simple retrieval
│ ├── ELMGenerator.ts # ELM-based generation
│ ├── HybridGenerator.ts # Hybrid retrieval + jitter
│ ├── ExactGenerator.ts # High-fidelity retrieval
│ └── PremiumGenerator.ts # Best-of-all-worlds
├── encoders/
│ ├── StringEncoder.ts # String encoding/decoding
│ ├── CharVocab.ts # Character vocabulary
│ └── ...
├── examples/
│ ├── quickstart.ts # Quick start examples
│ ├── trainELMFromSynth.ts # ELM training example
│ └── evaluateGeneratedData.ts # Evaluation script
└── OmegaSynth.ts # Main class
`

Testing

All tests are located in src/omegasynth/tests/ and src/omegasynth/**/__tests__/.

Test coverage includes:
- ✅ All generation modes (retrieval, elm, hybrid, exact, premium)
- ✅ Pattern correction
- ✅ Sequence context
- ✅ Exact generator
- ✅ Premium generator (formerly "Perfect")
- ✅ All configuration options
- ✅ Integration tests

Run tests with:
`bash
npm test
`

Usage: Bootstrapping a New AsterMind Project

1) Install dependencies in your project:
`bash
npm install @astermind/astermind-elm
`

2) Install OmegaSynth:
`bash
npm install @astermind/astermind-synth
`

3) Load a pretrained model and generate data for bootstrapping:
`typescript
import { loadPretrained } from '@astermind/astermind-synth';

const synth = loadPretrained('retrieval'); // Fully realistic formats from curated synthetic examples
const samples = await Promise.all([
synth.generate('first_name'),
synth.generate('last_name'),
synth.generate('email'),
synth.generate('phone_number'),
]);
console.log(samples);
`

4) Train AsterMind ELM/KELM using generated data:
`typescript
import { ELM } from '@astermind/astermind-elm';

const categories = ['first_name','last_name','email','phone_number'];
const texts: string[] = []; // your synthesized texts
const labels: string[] = []; // the corresponding labels

const elm = new ELM({
useTokenizer: true,
hiddenUnits: 256,
categories,
maxLen: 50,
});
(elm as any).setCategories?.(categories);

// Train
const labelIndices = labels.map(l => categories.indexOf(l));
(elm as any).trainFromData(
(elm as any).encoder.batchEncode(texts).map((v: number[]) => (elm as any).encoder.normalize(v)),
labelIndices
);

// Predict
const preds = (elm as any).predict('John Doe', 3);
console.log(preds);
`

5) Run the integrated pipeline (training + test + validation + save):
`bash
npm run build
`

Artifacts are saved under dist/models/vX.Y.Z/.

Training Data Statement

OmegaSynth relies on curated and/or synthesized data to bootstrap and validate AsterMind pipelines. The following principles govern training and evaluation datasets:

- Sources and composition:
- Training samples are drawn from internal synthetic corpora and curated, non-PII examples within src/omegasynth/models/*.json.
- The retrieval and exact modes reproduce only the samples present in the curated sets; elm, hybrid, and premium may produce variations guided by learned patterns.
- Privacy and PII:
- Datasets are constructed to avoid real personal data and sensitive attributes. The examples shipped in this repository are synthetic and intended for development and testing.
- If integrating your own data, ensure you have consent, proper governance, and apply de-identification where appropriate.
- Intended use:
- Designed for prototyping, testing, and bootstrapping classification/ELM pipelines. Not intended to replace domain datasets for production without additional validation.
- Ethics and safety:
- Avoid generating or amplifying harmful content. Apply validation rules (validation.ts) and pattern correction to enforce format and policy constraints.
- Reproducibility and versioning:
- Each pipeline run saves a versioned snapshot under dist/models/vX.Y.Z/ containing training_data.json, elm_model.json, and a manifest for traceability.
- Pin a version directory for reproducible experiments and audits.
- Licensing:
- Core AsterMind Synth code is licensed under the EULA in this repository.
- Some supporting tools, examples, or dependencies may be MIT-licensed or under other open-source licenses as noted in their headers.
- Ensure any external data you add complies with its original license/terms.
- Opt-out and removals:
- If you ingest datasets that include opt-out requirements, implement removal tools in your data preparation before running the pipeline.

These guidelines reflect the content previously documented in the training data statement and are now maintained here to keep a single source of truth.

End User License Agreement (EULA)

IMPORTANT: PLEASE READ THIS END USER LICENSE AGREEMENT (“EULA”) CAREFULLY BEFORE USING ASTERMIND SYNTH (“SOFTWARE”). BY INSTALLING, ACCESSING, OR USING THE SOFTWARE, YOU AGREE TO BE BOUND BY THIS EULA. IF YOU DO NOT AGREE, DO NOT INSTALL OR USE THE SOFTWARE.

1) License Grant
- Subject to payment of applicable fees (if any) and your compliance with this EULA, AsterMind AI Corporation (“Licensor”) grants you a limited, non-exclusive, non-transferable, non-sublicensable license to install and use the Software solely for your internal business purposes or personal use, in accordance with the documentation.

2) Restrictions
- You shall not: (a) copy (except for reasonable backup), modify, adapt, translate, or create derivative works of the Software; (b) reverse engineer, decompile, disassemble or otherwise attempt to derive source code except to the extent permitted by law; (c) rent, lease, lend, sell, sublicense, distribute, or make available the Software to any third party; (d) remove, obscure, or alter any proprietary notices; (e) use the Software to build a competing product or service; (f) use the Software in violation of any applicable law or regulation.

3) Ownership
- The Software is licensed, not sold. Licensor and its licensors retain all rights, title, and interest in and to the Software, including all intellectual property rights therein.

4) Fees and Updates
- If the Software is provided on a paid or subscription basis, you agree to pay all fees in accordance with the ordering document or pricing provided by Licensor. Licensor may provide updates, upgrades, or patches at its discretion; any such updates are governed by this EULA unless a separate agreement applies.

5) Data and Privacy
- You are responsible for the data you process with the Software. You agree not to use the Software with data for which you lack rights or consent. The Software may generate or store artifacts (e.g., models, logs) locally; you are responsible for securing and handling such artifacts.

6) Confidentiality
- Any non-public information about the Software, including performance, pricing, and roadmaps, is Licensor’s confidential information. You shall not disclose such information to third parties without prior written consent.

7) Open-Source Components
- The Software may include or interface with third-party open-source components, which are licensed under their own terms. To the extent of any conflict with this EULA, the open-source terms govern the relevant components only. Use of @astermind/astermind-elm remains subject to its published license.

8) Disclaimer of Warranties
- THE SOFTWARE IS PROVIDED “AS IS” AND “AS AVAILABLE.” LICENSOR DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE, INCLUDING WITHOUT LIMITATION WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, AND NON-INFRINGEMENT. LICENSOR DOES NOT WARRANT THAT THE SOFTWARE WILL BE ERROR-FREE OR UNINTERRUPTED.

9) Limitation of Liability
- TO THE MAXIMUM EXTENT PERMITTED BY LAW, LICENSOR SHALL NOT BE LIABLE FOR ANY INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL, EXEMPLARY, OR PUNITIVE DAMAGES, OR FOR LOST PROFITS, REVENUE, DATA, OR BUSINESS, ARISING OUT OF OR RELATED TO THIS EULA OR THE SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. LICENSOR’S TOTAL LIABILITY FOR ANY CLAIMS SHALL NOT EXCEED THE AMOUNTS PAID BY YOU FOR THE SOFTWARE IN THE TWELVE (12) MONTHS PRECEDING THE CLAIM.

10) Termination
- This EULA is effective until terminated. Licensor may suspend or terminate your license for material breach, non-payment, or misuse. Upon termination, you must cease all use of the Software and delete all copies. Sections intended to survive (including Ownership, Restrictions, Confidentiality, Disclaimers, and Limitations of Liability) shall survive termination.

11) Export and Compliance
- You represent and warrant that you are not prohibited from receiving the Software under applicable laws and you will comply with all export control and sanctions laws.

12) Governing Law; Disputes
- This EULA shall be governed by and construed in accordance with the laws of the State of California, without regard to conflict of law principles. Exclusive jurisdiction and venue for any dispute shall lie in state or federal courts located in San Francisco County, California, and the parties consent to personal jurisdiction therein.

13) General
- This EULA constitutes the entire agreement regarding the Software and supersedes all prior or contemporaneous understandings. Any amendments must be in writing and signed by Licensor. If any provision is held unenforceable, the remaining provisions will remain in full force and effect.

Contact: legal@astermind.ai

NOTE ON MIT REFERENCES: Prior references to "MIT" in this repository applied to earlier open-source portions or third-party components only. AsterMind Synth, as a paid/licensed product, is subject to this EULA. Open-source dependencies remain under their respective licenses.

Legal and Supporting Documentation

The following documents are included with this package and govern your use of AsterMind Synth:

- EULA.md - End User License Agreement (full text above)
- TERMS_OF_SERVICE.md - Terms of Service
- PRIVACY_POLICY.md - Privacy Policy
- SUPPORT_SLA.md - Support Service Level Agreement
- ACCEPTABLE_USE_POLICY.md - Acceptable Use Policy
- SECURITY_OVERVIEW.md - Security Overview

These documents are located in the package root directory after installation. For questions or concerns, contact:
- Legal: legal@astermind.ai
- Support: support@astermind.ai

⚠️ Disclaimer

AsterMind Synth generates synthetic data designed to match typical formatting and structure. All outputs are fictional and should not be used to represent real individuals. AsterMind Synth does not guarantee perfect realism and should be validated for fitness within your workflows.

Trademark Notice: Visa® and Mastercard® are registered trademarks of their respective owners. AsterMind Synth is not affiliated with or endorsed by any card networks.

Contributing

1. Ensure all tests pass: npm test
2. Build the project: npm run build
3. Follow TypeScript best practices
4. Add tests for new features

Changelog

$3

$3

Fixed: ReferenceError: __dirname is not defined in ESM modules.

Solution: Removed __dirname usage entirely and replaced with process.cwd() + findPackageRoot() pattern. This is the proper approach since we're locating files relative to the package root, not the current file location.

Changes:
- Removed import.meta.url from source (causes Jest parse errors)
- Uses process.cwd() and findPackageRoot() to locate model files
- Works correctly in both ESM and CommonJS environments
- More reliable file resolution (relative to package root)

Impact: The ESM build (dist/astermind-synth.esm.js) no longer uses __dirname, fixing the error when used in ESM projects.

$3


- Fixed license validation to handle "missing" state when token exists in environment variable but async setLicenseToken() hasn't completed yet
- Now decodes JWT payload directly from ASTERMIND_LICENSE_TOKEN environment variable when state is "missing"
- This allows tokens to work immediately even if async token initialization is still in progress
- Combined with v0.1.4 fixes, now handles all license states: "missing", "invalid", and "expired"

$3

- Fixed license validation to decode JWT payload directly when license-runtime doesn't populate it
- Added JWT payload decoding fallback for cases where state.payload is null but token exists
- Now checks both state.payload.features and decoded JWT payload, allowing tokens with different audiences (e.g., "astermind-elm") that include the "astermind-synth" feature
- This allows multi-product license keys to work across AsterMind products even when license-runtime doesn't expose the payload

$3

- Fixed license validation to check payload directly instead of using hasFeature() which fails when license state is invalid
- Now checks state.payload?.features?.includes("astermind-synth") directly, allowing tokens with different audiences (e.g., "astermind-elm") that include the "astermind-synth" feature
- This allows multi-product license keys to work across AsterMind products
- Improved error handling to only throw "invalid" error when feature is actually missing from payload

$3

- Initial attempt to fix audience validation - required payload-based check in v0.1.3

$3

- file extentions were not included in ESM exports

pre-release updates

$3

- Added exact and premium` generation modes
- Implemented pattern correction system
- Added sequence context for better generation
- Added character embeddings support
- Added confidence-based candidate selection
- Improved realism from ~37% to ~56% overall
- Samples generated under format validation rules; standard modes designed to avoid duplicates

$3

- Basic retrieval, ELM, and hybrid modes
- Pretrained models
- TypeScript support