servitor

stupid simple LLM chatbot framework designed for multiuser chats

currently extremely unstable and not really ready for use

if you just want to make a discord LLM bot, see
ensata, the reference implementation of
servitor

features

- interchangeable inference/embedding drivers
- basilisk (recommended)
- developed in parallel with servitor and supports all the features it uses
- text-generation-webui
and compatible API services (incomplete)
- does not natively support embeddings, which are required for long-term
memory
- OpenAI-compatible API services (incomplete)
- does not currently support token counting, resulting in inefficient
context space allocation
- writing your own driver for something else should be pretty easy
(see src/driver/base.ts)
- dynamic context reallocation
- context is windowed to 2048 tokens (for LLaMA)
- if some piece of information (such as long-term memory) is injected into the
context, then the conversation window shrinks to accomodate it and expands
again once that information is removed
- universal internal monologue system
- allows better planning of replies
- as a side-effect, creates some level of self-consistency for the simulacrum
- vector memory (still ironing this out)
- persistent driver backed by pgvector
- simple in-memory driver

faq

- who is this for? - primarily, myself. however, im releasing this publicly
with the hope that someone else who wants to build a chatbot for i.e. discord
will find it useful.

- why not just use langchain? - it doesnt support multiuser chats, only
one-on-one conversations between "Human" and "AI". i investigated using it,
but ultimately i had to roll every part of the stack for this use case myself
anyway, so it wasnt worth it.

- why "servitor"? - this library is named after the chaos magic concept of
the same name), because
the concepts, motivations, and processes involved in the construction of
servitors operating within the framework of a human psyche has interesting
parallels with that of agent simulacra operating within the framework of
humanity's collective psyche as distilled into LLMs.

and in both cases, as per rule #3 of sudo, with great power comes great
responsibility.