Smartly prune massive JSON/strings for LLM context optimization with cost estimation.
npm install smart-data-pruner> Reduce LLM Noise. Save Money. Optimize Context.


Smart Data Pruner is a production-ready utility designed for AI engineering. It intelligently shrinks massive JSON objects and strings to strictly fit within your LLM's context windowโwithout crashing on circular references or losing critical schema structure.
- ๐ง Intelligent Pruning: Multi-stage algorithm (Clean -> Light -> Aggressive -> Nuclear -> Bedrock) adapts to your data.
- ๐ฐ Cost Estimator: Calculate costs for GPT-4o, Claude 3.5, Gemini 1.5, and more.
- ๐ก๏ธ Robust & Safe: Handles circular references, deep nesting, and non-JSON inputs gracefully.
- ๐ CLI & Library: Professional CLI with spinners and pretty-printing.
``bash`
npm install smart-data-pruner
`javascript
const { SmartPruner, estimateCost } = require('smart-data-pruner');
const massiveData = { / ... 50MB of logs ... / };
// 1. Check Cost
try {
const cost = estimateCost(massiveData, 'gpt-4o');
console.log(Potential Cost: $${cost.costUSD});
} catch (err) {
console.error(err);
}
// 2. Prune it!
const pruner = new SmartPruner();
const result = pruner.prune(massiveData, 4000); // Target: 4000 tokens
console.log(Strategy Used: ${result.strategy});`
console.log(result.output);
`bashPrune a file to 4000 tokens (default) and save
npx smart-prune huge-logs.json --out pruned-logs.json
๐ง Pruning Strategies
The pruner applies these strategies sequentially until the token budget is met:
1. Clean: Removes
null, undefined`, empty strings/arrays.- OpenAI: gpt-4o, gpt-4-turbo, gpt-3.5-turbo, gpt-4o-mini
- Anthropic: claude-3-5-sonnet, claude-3-opus, claude-3-haiku
- Google: gemini-1.5-pro, gemini-1.5-flash
MIT