FinOps cost tracking and budget management for @thenoo/agentkit with multi-session support
npm install @thenoo/finops@thenoo/agentkit for better modularity and reusability.
@thenoo/adapter-openai via withFinOps()
cost:update, cost:alert, and budget:reset events
@thenoo/agentkit@0.1.0:
diff
- import { budget, BudgetManager } from '@thenoo/agentkit';
+ import { createBudgetManager, BudgetManager } from '@thenoo/finops';
`
The API remains the same - only the import path changed.
$3
- @thenoo/agentkit - Maintains backward-compatible shim
- @thenoo/adapter-openai - Uses @thenoo/finops for cost tracking
- @thenoo/cli - Generates projects with @thenoo/finops integration
---
Features
- 💰 Budget Management - Session and per-request budget limits
- 📊 Cost Tracking - Token usage and cost calculation
- 🎯 Multi-Session Support - Track costs independently per session
- 🚨 Event-Driven Alerts - Real-time cost alerts and updates
- 🔌 Adapter Integration - FinOpsContext interface for seamless adapter integration
- 🛡️ Type-Safe - Full TypeScript support with Zod validation
- 🌍 Edge Compatible - Works in edge runtimes
Installation
`bash
npm install @thenoo/finops
`
Quick Start
$3
`typescript
import { createBudgetManager } from '@thenoo/finops';
const manager = createBudgetManager({
usd: 10, // Max $10 per session
perRequestCents: 50, // Max 50 cents per request
onExceed: 'halt', // Action when limit exceeded
});
// Track token usage
const state = manager.recordUsage(1000, 0.002); // 1k tokens at $0.002/1M
console.log(Total spend: $${state.totalUsd.toFixed(4)});
console.log(Remaining: $${state.remainingUsd.toFixed(4)});
`
$3
`typescript
import { createFinOpsContext } from '@thenoo/finops';
const context = createFinOpsContext({
budgetConfig: {
usd: 10,
perRequestCents: 50,
onExceed: 'halt',
},
debug: true,
});
// Track cost for different sessions
await context.trackCost({
sessionId: 'user-123',
tokens: 1000,
costPerMTokenUsd: 0.002,
model: 'gpt-4o-mini',
});
await context.trackCost({
sessionId: 'user-456',
tokens: 2000,
costPerMTokenUsd: 0.002,
model: 'gpt-4o-mini',
});
// Get state per session
const state = await context.getState('user-123');
console.log(User 123 spent: $${state.totalUsd.toFixed(4)});
`
Core Concepts
$3
The BudgetManager tracks token usage and enforces spending limits for a single session:
`typescript
import { createBudgetManager } from '@thenoo/finops';
const manager = createBudgetManager({
usd: 5, // Session budget
perRequestCents: 20, // Per-request budget
onExceed: 'halt', // Policy: halt | truncate | degrade_model | alert
});
// Record usage
manager.recordUsage(1000, 0.002); // tokens, cost per 1M tokens
// Check if request would exceed budget
if (manager.wouldExceed(5000, 0.002)) {
console.warn('Request would exceed budget');
}
// Get current state
const state = manager.getState();
console.log(state.totalUsd, state.remainingUsd, state.exceeded);
// Reset budget
manager.reset();
`
$3
The FinOpsContext provides multi-session support and adapter integration:
`typescript
import { createFinOpsContext } from '@thenoo/finops';
const context = createFinOpsContext({
budgetConfig: { usd: 10 },
});
// Track costs per session
await context.trackCost({
sessionId: 'session-123',
tokens: 1000,
costPerMTokenUsd: 0.002,
model: 'gpt-4o-mini',
});
// Check budget before request
const wouldExceed = await context.wouldExceed({
sessionId: 'session-123',
estimatedTokens: 5000,
costPerMTokenUsd: 0.002,
});
// Get all active sessions
const sessions = await context.getSessions();
console.log(Active sessions: ${sessions.length});
`
Budget Policies
Control what happens when budget limits are exceeded:
| Policy | Description |
|--------|-------------|
| halt | Stop execution immediately (default) |
| truncate | Reduce token limits for next request |
| degrade_model | Switch to a cheaper model |
| alert | Emit warning but continue |
`typescript
const manager = createBudgetManager({
usd: 10,
onExceed: 'degrade_model',
});
`
Event Handling
Listen for cost alerts and updates:
`typescript
import { createBudgetManager, createFinOpsContext } from '@thenoo/finops';
// Budget Manager events
const manager = createBudgetManager({ usd: 5 });
manager.onEvent((event) => {
if (event.type === 'cost:alert') {
console.error('Budget exceeded!', event.payload);
} else if (event.type === 'cost:update') {
console.log('Cost update:', event.payload);
}
});
// FinOps Context events
const context = createFinOpsContext({ budgetConfig: { usd: 10 } });
context.onCostAlert((payload) => {
console.error('Cost alert:', payload);
// payload: { sessionId, totalUsd, limitUsd, requestCents, policy, model, timestamp }
});
context.onCostUpdate((payload) => {
console.log('Cost update:', payload);
// payload: { sessionId, totalUsd, requestCents, requestCount, exceeded, model, timestamp }
});
`
Integration with Adapters
$3
`typescript
import { createOpenAIAdapter } from '@thenoo/adapter-openai';
import { createFinOpsContext } from '@thenoo/finops';
const finops = createFinOpsContext({
budgetConfig: { usd: 10, perRequestCents: 50 },
});
const adapter = createOpenAIAdapter({
apiKey: process.env.OPENAI_API_KEY!,
});
// Make a request
const response = await adapter.createChatCompletion({
messages: [{ role: 'user', content: 'Hello!' }],
});
// Track cost
await finops.trackCost({
sessionId: 'user-123',
tokens: response.usage.total_tokens,
costPerMTokenUsd: 2, // Model-specific rate
model: response.model,
});
// Check budget state
const state = await finops.getState('user-123');
if (state.exceeded) {
console.warn('Budget exceeded for user-123');
}
`
$3
Implement FinOpsContext in your adapter:
`typescript
import type { FinOpsContext } from '@thenoo/finops';
class MyAdapter {
constructor(private finops?: FinOpsContext) {}
async generateText(prompt: string, sessionId?: string) {
// Check budget before request
if (this.finops) {
const wouldExceed = await this.finops.wouldExceed({
sessionId,
estimatedTokens: estimateTokens(prompt),
costPerMTokenUsd: 0.002,
});
if (wouldExceed) {
throw new Error('Request would exceed budget');
}
}
// Make API call
const response = await this.makeRequest(prompt);
// Track actual cost
if (this.finops) {
await this.finops.trackCost({
sessionId,
tokens: response.usage.total_tokens,
costPerMTokenUsd: 0.002,
model: response.model,
});
}
return response;
}
}
`
Utility Functions
$3
`typescript
import { estimateTokens } from '@thenoo/finops';
const tokens = estimateTokens('Hello, world!');
console.log(Estimated: ${tokens} tokens); // ~4 tokens (4 chars/token heuristic)
`
$3
`typescript
import { calculateTokens } from '@thenoo/finops';
const usage = calculateTokens(100, 50); // input, output
console.log(usage.total); // 150
console.log(usage.input); // 100
console.log(usage.output); // 50
`
Configuration
$3
`typescript
interface BudgetConfig {
/* Maximum spend per session in USD /
usd?: number;
/* Maximum spend per request in cents /
perRequestCents?: number;
/* Action when budget exceeded /
onExceed?: 'halt' | 'truncate' | 'degrade_model' | 'alert';
}
`
$3
`typescript
interface FinOpsContextConfig {
/* Budget manager instance /
budgetManager?: BudgetManager;
/* Default budget config (used if budgetManager not provided) /
budgetConfig?: BudgetConfig;
/* Enable debug logging /
debug?: boolean;
}
`
TypeScript Support
Full TypeScript support with exported types:
`typescript
import type {
BudgetConfig,
BudgetState,
BudgetPolicy,
TokenUsage,
CostCalculation,
CostAlertPayload,
CostUpdatePayload,
FinOpsContext,
FinOpsContextConfig,
FinOpsEvent,
} from '@thenoo/finops';
`
Backward Compatibility
For @thenoo/agentkit v0.1.0 users, the budget functionality is still available but deprecated:
`typescript
// Old way (still works, but deprecated)
import { budget } from '@thenoo/agentkit';
// New way (recommended)
import { createBudgetManager } from '@thenoo/finops';
`
Related Packages
- @thenoo/agentkit - Core agent framework
- @thenoo/adapter-openai - OpenAI adapter with cost tracking
- @thenoo/adapter-azure - Azure OpenAI adapter
Examples
$3
`typescript
import { createBudgetManager } from '@thenoo/finops';
const manager = createBudgetManager({
usd: 1,
onExceed: 'halt',
});
// Track usage
try {
manager.recordUsage(1_000_000, 2); // $2 (exceeds $1 limit)
} catch (error) {
console.error('Budget exceeded!');
}
`
$3
`typescript
import { createFinOpsContext } from '@thenoo/finops';
const finops = createFinOpsContext({
budgetConfig: { usd: 5 }, // $5 per user
});
async function handleUserRequest(userId: string, prompt: string) {
// Check budget
const wouldExceed = await finops.wouldExceed({
sessionId: userId,
estimatedTokens: estimateTokens(prompt),
costPerMTokenUsd: 0.002,
});
if (wouldExceed) {
throw new Error('User budget exceeded');
}
// Process request...
const tokens = 1000;
await finops.trackCost({
sessionId: userId,
tokens,
costPerMTokenUsd: 0.002,
});
}
`
$3
`typescript
import { createFinOpsContext } from '@thenoo/finops';
const finops = createFinOpsContext({ budgetConfig: { usd: 10 } });
// Real-time cost monitoring
finops.onCostUpdate(async (payload) => {
console.log(Session: ${payload.sessionId});
console.log(Total: $${payload.totalUsd.toFixed(4)});
console.log(Requests: ${payload.requestCount});
// Send to monitoring service
await sendMetric('llm.cost', payload.totalUsd, {
session: payload.sessionId,
model: payload.model,
});
});
// Alert on budget breach
finops.onCostAlert(async (payload) => {
await sendAlert({
severity: 'high',
message: Budget exceeded for ${payload.sessionId},
totalUsd: payload.totalUsd,
limitUsd: payload.limitUsd,
});
});
`
Adapter Integration
The @thenoo/finops package is designed to integrate with LLM adapters like @thenoo/adapter-openai and @thenoo/adapter-azure to provide automatic cost tracking and budget enforcement.
$3
Wrap your OpenAI adapter with withFinOps() from @thenoo/adapter-openai:
`typescript
import { createOpenAIAdapter, withFinOps } from '@thenoo/adapter-openai';
const adapter = createOpenAIAdapter({
apiKey: process.env.OPENAI_API_KEY!,
});
const wrapped = withFinOps(adapter, {
sessionId: 'user-123',
budget: {
usd: 10,
perRequestCents: 50,
policy: 'degrade_model',
},
onAlert: (event) => {
console.warn('Budget alert:', event);
},
});
// Automatic cost tracking
const response = await wrapped.createChatCompletion({
messages: [{ role: 'user', content: 'Hello!' }],
});
`
$3
#### cost:update
`typescript
{
type: 'cost:update',
payload: {
sessionId: string;
totalUsd: number;
requestCount: number;
tokens: number;
model?: string;
timestamp: number;
}
}
`
#### cost:alert
`typescript
{
type: 'cost:alert',
payload: {
sessionId: string;
totalUsd: number;
limitUsd: number;
policy: BudgetPolicy;
exceeded: boolean;
timestamp: number;
}
}
`
#### budget:reset
`typescript
{
type: 'budget:reset',
payload: {
sessionId?: string; // undefined for global reset
timestamp: number;
}
}
`
$3
Export FinOps metrics to OpenTelemetry:
`typescript
import { createFinOpsContext } from '@thenoo/finops';
import { metrics } from '@opentelemetry/api';
const finops = createFinOpsContext({ budgetConfig: { usd: 10 } });
const meter = metrics.getMeter('@thenoo/finops');
const costCounter = meter.createCounter('llm.cost.usd', {
description: 'LLM API costs in USD',
});
finops.onCostUpdate((payload) => {
costCounter.add(payload.totalUsd, {
session: payload.sessionId,
model: payload.model || 'unknown',
});
});
``