@llmist/testing

Testing utilities for llmist - mock LLM responses and test agents deterministically.

Installation

``bash
npm install -D @llmist/testing
`

Requires llmist as a peer dependency.

Quick Start

$3

Test gadgets in isolation without any LLM calls:

`typescript
import { testGadget } from '@llmist/testing';
import { Calculator } from './gadgets';

const result = await testGadget(new Calculator(), {
operation: 'add',
a: 5,
b: 3,
});

expect(result.result).toBe('8');
expect(result.error).toBeUndefined();
`

$3

Use the fluent MockBuilder API to script LLM responses:

`typescript
import { mockLLM, createMockClient, resetMocks } from '@llmist/testing';
import { LLMist } from 'llmist';

// Set up mock responses
mockLLM()
.whenMessageContains('hello')
.returns('Hi there! How can I help?')
.register();

mockLLM()
.whenMessageContains('calculate')
.returnsGadgetCall('Calculator', { operation: 'add', a: 1, b: 2 })
.register();

// Create agent with mock client
const agent = LLMist.createAgent()
.withClient(createMockClient())
.withGadgets(Calculator);

const response = await agent.askAndCollect('hello');
// Returns "Hi there! How can I help?" - no API calls made

// Clean up after tests
resetMocks();
`

$3

Match responses based on model, provider, or custom conditions:

`typescript
mockLLM()
.forModel('gpt-4o')
.forProvider('openai')
.whenMessageContains('complex task')
.returns('Handled by GPT-4o')
.register();

mockLLM()
.forModel('haiku')
.whenMessageContains('complex task')
.returns('Handled by Haiku')
.register();
`

$3

Use .once() for responses that should only match once:

`typescript
mockLLM()
.whenMessageContains('first')
.returns('First response')
.once()
.register();

mockLLM()
.whenMessageContains('first')
.returns('Second response')
.register();

// First call returns "First response"
// Subsequent calls return "Second response"
`

API Reference

$3

Test a gadget with given parameters.

$3

Create a new mock builder with fluent API.

$3

Create a mock LLMist client that uses registered mocks.

$3

Clear all registered mocks (call in afterEach).

Documentation

Full documentation at llmist.dev/testing

- Mocking Overview
- Testing Gadgets
- Testing Agents

Related Packages

- llmist - Core library
- @llmist/cli` - Command-line interface

License

MIT