@verydia/providers

Universal provider registry and adapter layer for LLM providers.

Overview

This package provides:
- A unified ProviderAdapter interface for LLM providers
- A ProviderRegistry for managing and resolving providers
- Production-ready adapters for major LLM providers
- Type-safe provider configuration and model resolution
- Mock-based testing with no real API calls

Implemented Adapters

$3

Full implementation with:
- Chat completions
- SSE streaming
- Embeddings (text-embedding-3-small, text-embedding-3-large, etc.)
- Tool/function calling
- Configurable base URL, timeout, organization

$3

Full implementation with:
- Chat completions (Claude 3.5 Sonnet, etc.)
- SSE streaming
- Tool calling (Claude's tool use format)
- System message handling (Anthropic requirement)
- Configurable API version

$3

Full implementation with:
- Chat completions (Gemini 1.5 Flash, Pro, etc.)
- SSE streaming
- Embeddings
- Function declarations for tool calling
- Uses Google's generativelanguage API

$3

Full implementation with:
- Chat completions (Mistral Large, Medium, Small)
- SSE streaming
- Embeddings
- Tool/function calling
- OpenAI-compatible API format

$3

Full implementation with:
- Chat completions (local models: Llama 2, Mistral, etc.)
- Streaming
- Embeddings
- Configurable base URL (default: http://localhost:11434)
- No API key required (local deployment)

$3

Placeholder - Requires AWS SDK integration:
- Throws clear error directing to Anthropic adapter or AWS SDK
- Ready for future implementation with SigV4 signing

Key Concepts

$3

Each provider implements the ProviderAdapter interface:

``typescript
interface ProviderAdapter {
readonly kind: ProviderKind;
resolveModelId(config: C, logicalModel: string): string;
invokeChat(config: C, req: ProviderChatRequest): Promise;
streamChat?(config: C, req: ProviderChatRequest): AsyncIterable;
embedding?(config: C, req: ProviderEmbeddingRequest): Promise;
toolCall?(config: C, req: ProviderToolCallRequest): Promise;
}
`

$3

The registry manages provider instances and resolves model references:

`typescript
const registry = new ProviderRegistry();
registry.register(openaiConfig);

const result = await registry.invokeChat("openai:gpt-4", {
messages: [{ role: "user", content: "Hello!" }]
});
`

Usage Examples

$3

`typescript
import { openaiAdapter, createOpenAIConfig } from "@verydia/providers";

const config = createOpenAIConfig({
apiKey: process.env.OPENAI_API_KEY!,
modelAliases: {
"gpt-4o-mini": "gpt-4o-mini-2024-07-18"
}
});

const result = await openaiAdapter.invokeChat(config, {
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Hello!" }],
temperature: 0.7
});

console.log(result.text);
`

$3

`typescript
import { anthropicAdapter, createAnthropicConfig } from "@verydia/providers";

const config = createAnthropicConfig({
apiKey: process.env.ANTHROPIC_API_KEY!,
});

const result = await anthropicAdapter.invokeChat(config, {
model: "claude-3-5-sonnet-20241022",
messages: [
{ role: "system", content: "You are a helpful assistant." },
{ role: "user", content: "Explain TypeScript." }
],
});

console.log(result.text);
`

$3

`typescript
for await (const chunk of openaiAdapter.streamChat!(config, {
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Tell me a story" }],
})) {
process.stdout.write(chunk.delta);
if (chunk.done) {
console.log("\n[Stream complete]");
}
}
`

Testing Strategy

All adapters use mock-based testing with no real API calls:

- ✅ No network calls in tests: All HTTP requests are mocked
- ✅ Fixture-based responses: Tests use pre-defined response fixtures
- ✅ Type safety verification: Tests ensure correct type transformations
- ✅ CI-safe: No API keys required, instant test execution

Example test structure:

`typescript
describe("OpenAI Adapter", () => {
it("should have correct type signature", () => {
expect(openaiAdapter.kind).toBe("openai");
expect(typeof openaiAdapter.invokeChat).toBe("function");
});

it("should resolve model aliases", () => {
const config = createOpenAIConfig({
apiKey: "test",
modelAliases: { "gpt-4o-mini": "gpt-4o-mini-2024-07-18" }
});

const resolved = openaiAdapter.resolveModelId(config, "gpt-4o-mini");
expect(resolved).toBe("gpt-4o-mini-2024-07-18");
});
});
`

Architecture

`
@verydia/providers
├── src/
│ ├── types.ts # Core interfaces
│ ├── index.ts # ProviderRegistry
│ ├── builtinProviders.ts # Default configs
│ └── adapters/
│ ├── index.ts # Adapter exports
│ ├── openaiAdapter.ts # OpenAI implementation
│ ├── anthropicAdapter.ts
│ ├── geminiAdapter.ts
│ ├── mistralAdapter.ts
│ ├── bedrockAdapter.ts
│ └── ollamaAdapter.ts
└── test/
├── providers.test.ts # Registry tests
└── openaiAdapter.test.ts # Adapter tests
`

For Contributors

$3

1. Create src/adapters/yourProviderAdapter.ts
2. Implement the ProviderAdapter interface
3. Export from src/adapters/index.ts
4. Add config creator function (e.g., createYourProviderConfig)
5. Add tests in test/yourProviderAdapter.test.ts
6. Update this README

$3

- [ ] Implement invokeChat() (required)
- [ ] Implement streamChat() if provider supports streaming
- [ ] Implement embedding() if provider supports embeddings
- [ ] Implement toolCall()` if provider supports function calling
- [ ] Add proper error handling and timeout support
- [ ] Add cost estimation in usage metadata
- [ ] Write mock-based tests (no real API calls)
- [ ] Document provider-specific quirks

License

Apache-2.0