LLM Complete


A command-line tool for generating text completions using local LLM models. Supports direct prompts, and file input/output.

Purpose

LLM completions are continuations of text from a given prompt or existing content. Unlike chat models that answer questions, completion models excel at:

- Continuing partial sentences or paragraphs
- Generating creative writing from prompts
- Adding to existing documentation
- Completing code snippets

$3

``
$ llm-complete -p "export class SillyButton extends HTMLElement {"
`

$3

`javascript
export class SillyButton extends HTMLElement {
constructor() {
super();
this.attachShadow({ mode: 'open' });
const template = document.createElement('template');
template.innerHTML = ;
this.

`

Requirements

You must have NodeJS and GPT4All installed. There are various methods to do this. You could build from source. I installed via AUR.

This tool is based around Mistral 7B by Mistral.AI. The GGUF needs to be installed in the GPT4All path.

`bash
curl -L https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF/blob/main/mistral-7b-v0.1.Q4_K_M.gguf?download=true \
-o ~/.local/share/nomic.ai/GPT4All/mistral-7b-v0.1.Q4_K_M.gguf
`

On Arch, GPT4All stores models in the path above. This may be different in your installation. Also, the node module expects models in ~/.cache/gpt4all/ so we need to link them there for this app to work.

`bash
ln -s ~/.local/share/nomic.ai/GPT4All/mistral-7b-v0.1.Q4_K_M.gguf ~/.cache/gpt4all/
`

Model Choice

This version has specifically been selected for it's balance of light weight and creativity in addition to it's open source license. There are newer and larger versions of this model but they don't perform as well on a laptop with no dedicated GPU. Feel free to use any base LLM instead if you wish, or a larger quant but Chat/Agent trained models will not work as expected with this code. If you do this you will need to supply your own model configuration in models.json

Installation

`bash


Clone repository


git clone https://github.com/besworks/llm-complete.git
cd llm-complete
npm link
`

OR

`bash


Install via npm


npm i -g llm-complete
`

Usage

`bash


No prompt for random output

llm-complete

Direct prompt with quotes

llm-complete -p "This is a test"

Process file to stdout (allows redirection)

llm-complete -f input.txt # output to terminal
llm-complete -f input.txt > output.txt # overwrite
llm-complete -f input.txt >> output.txt # append

Append completion to input file

llm-complete -a story.txt

Select processing device, default gpu, falls back to cpu

DEVICE=cpu llm-complete -f input.txt

Customize buffer size

BUFFER=40 llm-complete

Customize output length

PREDICT=512 llm-complete -p "This is a longer test"

Use a different model

export MODEL="mistral-7b-v0.2-Q6_K.gguf"
export CTX=1024
llm-complete
``

Architecture

For anyone interested, I have written a full breakdown of how this works.