AST-first specification processor with schema-central architecture
npm install @openuji/speculatorAST-first specification parser and indexer with a schema-central architecture.
Speculator is the core engine of the Speculator ecosystem. It transforms specification documents (Markdown or HTML) into a structured AST and semantic indexes. Unlike traditional tools, it does not render output directly—it provides the semantic foundation for renderers, linters, and other processors.
Speculator operates on a strict three-stage pipeline to ensure determinism and semantic accuracy:
Handles configuration loading and spec composition. It resolves includes (both Markdown directives and HTML markers) into a deterministic CompositeSource.
- Isomorphic IO: Uses FileProvider adapters for Node.js, Web, or Memory environments.
- Cycle Detection: Prevents infinite include loops with actionable diagnostics.
Converts the CompositeSource into the Speculator AST. This stage is powered by dedicated parser modules that ensure Markdown and HTML parity.
- Unified AST: Whether source is ## Heading or Heading
BlockHeading node.
- Source Tracking: Every node preserves its original sourcePos.file, even across multiple includes.
A plugin-based pipeline that refines the AST:
- Transform: Structural normalization.
- Resolve: Semantic binding (connecting cross-references to definitions).
- Index: Extracting derived data (definitions, requirements, issues, examples).
- Compute: Optional generation of TOCs and heading numbering.
The Speculator AST JSON Schema is the single source of truth. All pipeline outputs are validated against this schema, ensuring that downstream tools (like @openuji/render-respec) can rely on a stable, typed data model.
- Document: Root container with metadata and indexes.
- Section: Hierarchical grouping with nested sections.
- Block: Paragraphs, headings, code blocks, lists, tables.
- Inline: Text, emphasis, definitions (dfn), cross-references (xref).
``typescript
import { speculate, corePlugins, NodeFileProvider } from "@openuji/speculator";
const result = await speculate({
entry: "spec/index.md",
plugins: [...corePlugins],
fileProvider: new NodeFileProvider(),
});
console.log(result.workspace.documents[0].ast);
`
Code that maps IR (hast/mdast) to SpecAST lives in src/parse/ following the naming pattern (e.g., HeadingsHtmlParser).
Postprocess plugins implement specific phase handlers (transform, resolve, index, compute`) with deterministic ordering weights.