starlight-pipeline

A lightweight neural-network and NLP pipeline for Starlight, inspired by TensorFlow and Keras.
Supports tokenization, vectorization, dense neural networks, SGD & Adam optimizers, model saving/loading, and training progress visualization.

---

Features

* Chainable pipeline API
* Dense neural networks (fully connected layers)
* Optimizers: SGD and Adam
* Training progress bar (TensorFlow-style)
* Text preprocessing (tokenize, stopwords, vectorize)
* Save / Load models (.slm binary format)
* Loss functions: MSE, Cross-Entropy
* Evaluation utilities (accuracy, confusion matrix)

---

Installation

``bash
npm install starlight-pipeline
`

Dependencies:

* starlight-ml
* starlight-vec
* starlight-eval

---

Quick Start

$3

`sl
import fs from "fs"
import { pipeline } from "starlight-pipeline"

let raw = fs.readFileSync("train.json", "utf8")
let data = JSONParse(raw)

let texts = data.texts
let labels = data.labels

define p = pipeline()
.tokenize()
.removeStopwords(["the", "is", "and", "to", "of"])
.vectorize(texts)

let inputSize = p.context.vectorizer.size

p
.dense(inputSize, 16, "relu")
.dense(16, 1, "sigmoid")
.compile({ loss: "mse" })

let X = []
for (let i = 0; i < texts.length; i++) {
X.push(p.context.vectorizer.transform(texts[i]))
}

p.fit(X, labels, {
epochs: 300,
lr: 0.01,
optimizer: "adam"
})

p.saveModel("sentiment.slm")
`

---

Training Output

`
[██████████████████████████████----------] 62.0% Epoch 190/300 - Loss: 0.1824
...
Training complete!
`

---

Loading & Predicting

`sl
import { Pipeline } from "starlight-pipeline"

define model = Pipeline.loadModel("sentiment.slm")

let text = ask("Enter text:")
let vec = model.context.vectorizer.transform(text)
let pred = model.predict(vec)

sldeploy("Prediction:", pred[0])
`

---

Pipeline API

$3

Creates a new pipeline instance.

$3

`js
.tokenize()
.removeStopwords(words)
.vectorize(trainingTexts)
`

$3

`js
.dense(inputSize, outputSize, activation)
`

Supported activations:

* linear
* relu
* sigmoid
* softmax

---

Training

`js
.fit(X, y, {
epochs: 100,
lr: 0.01,
optimizer: "adam" // or "sgd"
})
`

Optimizers:

* SGD – simple and fast
* Adam – adaptive learning rate with momentum

---

Loss Functions

`js
.compile({ loss: "mse" })
`

Available:

* mse
* crossEntropy

---

Model Persistence

$3

`js
pipeline.saveModel("model.slm")
`

$3

`js
Pipeline.loadModel("model.slm")
`

The .slm format stores:

* Network architecture
* Weights & biases
* Activation types
* Vectorizer vocabulary

---

Evaluation

`js
pipeline.evaluate(yTrue, yPred)
``

Returns:

* Accuracy
* Confusion matrix
* Classification report

---

Example Use Cases

* Sentiment analysis
* Text classification
* Lightweight ML in CLI apps
* Educational neural networks
* Offline inference

---

Notes

* Designed for small to medium datasets
* Not GPU-accelerated
* Intended for learning, prototyping, and embedded ML

---

Roadmap

* Mini-batch training
* Dropout layers
* Learning rate schedulers
* Multi-class classification helpers
* Streaming datasets

---

License

MIT License