loop-infer

!RLM - Recursive Language Models


A Clojure implementation of Recursive Language Models (RLMs) - a task-agnostic inference paradigm that enables LLMs to handle arbitrarily long contexts by storing data in a sandboxed REPL environment.

Based on the paper "Recursive Language Models" by Zhang, Kraska, and Khattab.

How It Works

Instead of passing large contexts directly to the model, RLMs:

1. Store context in a sandboxed Clojure REPL
2. Let the LLM write code to examine, decompose, and query the context
3. Execute code safely and return results to the LLM
4. Repeat until the LLM produces a final answer

The LLM can also spawn recursive LLM calls for analysis of context chunks.

Installation

$3

``bash


Run directly without installation

npx @unravel-tech/rlm-cli -c document.txt -q "Summarize this"

Or install globally

npm install -g @unravel-tech/rlm-cli
rlm-cli -c document.txt -q "Summarize this"
`

$3

Prerequisites:
- Java 21+
- Clojure CLI

`bash
git clone https://github.com/unravel-team/loop-infer.git
cd loop-infer
`

Set your API key:

`bash
export OPENAI_API_KEY="sk-..."


or


export ANTHROPIC_API_KEY="sk-ant-..."
`

CLI Usage

`bash


Query a file

rlm -c document.txt -q "Summarize the key points"

Query from stdin

cat large-file.txt | rlm -c - -q "Find all dates mentioned"

Inline context

rlm -C "The capital of France is Paris." -q "What is the capital of France?"

Query a directory of data files

rlm -d ./data -e parquet,csv -q "What patterns do you see?"

Use a different model

rlm -m anthropic/claude-sonnet-4-20250514 -c doc.txt -q "Explain the main argument"

Generate an HTML execution report

rlm -c doc.txt -q "Summarize" -H report.html

Verbose output with JSON format

rlm -c doc.txt -q "Summarize" -v -o json
`

Server Mode

RLM can run as an HTTP server with real-time streaming support:

`bash


Start server on default port 8080

rlm --server

Start on custom port

rlm --server --port 3000
`

$3

| Endpoint | Method | Description |
|----------|--------|-------------|
| /health | GET | Health check, returns {"status": "ok", "active-tasks": N} |
| /api/query | POST | Synchronous query, returns JSON result |
| /api/stream | POST | Streaming query with SSE progress events |
| /api/task/:id | GET | Get task status |
| /api/task/:id/report | GET | Get HTML execution report |
| /api/task/:id/live | GET | Live-updating HTML report page |

$3

`bash
curl -X POST http://localhost:8080/api/query \
-H "Content-Type: application/json" \
-d '{
"context": "The quick brown fox jumps over the lazy dog.",
"query": "What animals are mentioned?",
"model": "openai/gpt-4o",
"trace": true
}'
`

Response:
`json
{
"answer": "The animals mentioned are a fox and a dog.",
"costs": {
"total-cost": 0.00123,
"root-llm-calls": 2,
"recursive-llm-calls": 0
},
"trace": { ... }
}
`

$3

`bash
curl -X POST http://localhost:8080/api/stream \
-H "Content-Type: application/json" \
-d '{"context": "Hello world", "query": "What is this?"}'
`

Returns Server-Sent Events (SSE) with real-time progress:
- connected - Task created, returns task ID
- start - Query execution started
- iteration-start - New iteration beginning
- llm-call - Root LLM request/response
- repl-execution - Code execution result
- recursive-llm - Recursive LLM call
- result - Final answer with costs
- done - Execution complete

$3

Open http://localhost:8080/api/task/{task-id}/live in a browser to see a real-time updating execution report.

Library Usage

`clojure
(require '[loop-infer.core :as rlm])

;; Create an RLM instance
(def r (rlm/create-rlm {:model "openai/gpt-4o"
:trace true
:verbose true}))

;; Query with context
(rlm/query r {:context "Paris is the capital of France."
:query "What is the capital of France?"})
;; => "Paris"

;; Check costs
(rlm/cost-summary r)
;; => {:total-cost 0.00123 :root-llm-cost 0.001 :recursive-llm-cost 0.00023 ...}

;; Generate HTML report (if trace was enabled)
(rlm/write-html-report r "report.html")

;; Reset for reuse
(rlm/reset-rlm! r)
`

CLI Options

| Flag | Default | Description |
|------|---------|-------------|
| -m, --model | openai/gpt-4o | LLM model (provider/model format) |
| -r, --recursive-model | (same as --model) | Override model for recursive calls |
| -c, --context | - | Context file path (or - for stdin) |
| -C, --context-string | - | Inline context string |
| -d, --directory | - | Load files from directory as context |
| -p, --pattern | - | Glob pattern for directory files |
| -e, --extensions | - | Comma-separated file extensions |
| -n, --max-files | - | Maximum files to load |
| -q, --query | - | Query to answer (required in CLI mode) |
| -i, --max-iterations | 20 | Max RLM iterations |
| -v, --verbose | false | Enable verbose output |
| -o, --output | text | Output format: text, json, edn |
| -H, --html-report | - | Generate HTML report to file |
| -S, --server | false | Run as HTTP server |
| -P, --port | 8080 | Server port |
| --host | 0.0.0.0 | Server host to bind |
| -k, --api-key | - | API key (or use env vars) |

Supported Models

Any model supported by litellm-clj using provider/model-name format:

- OpenAI: openai/gpt-4o, openai/gpt-4o-mini
- Anthropic: anthropic/claude-sonnet-4-20250514, anthropic/claude-3-5-haiku-20241022

Supported File Types

- Text: txt, md, rst, html, xml, yaml, json, py, js, clj, etc.
- Data: parquet, csv, tsv, json, jsonl, arrow

Building from Source

`bash


Build uberjar

./scripts/build.sh

or

clojure -X:uberjar

Run the jar

java -jar loop-infer.jar --help
`

Publishing to npm

`bash


Build uberjar first

./scripts/build.sh

Install jdeploy and publish

npm install -g jdeploy
jdeploy publish
`

Running Tests

`bash
clojure -M:test
`

Architecture

`
src/loop_infer/
├── core.clj # Public API and CLI
├── rlm.clj # Core RLM iteration algorithm
├── repl.clj # Sandboxed REPL execution
├── prompts.clj # System prompts for LLMs
├── server.clj # HTTP server with SSE streaming
├── trace.clj # Execution tracing and HTML reports
└── file-loader.clj # File and directory loading
``

See implementation.md for detailed architecture documentation.

License

MIT