backprop

An optimized implementation of the backpropagation algorithm in Typescript

Example Usage

``typescript
import { TrainingData, Net } from "./Net";

//example dataset
const XOR: TrainingData = [
[[0, 0], [0]],
[[0, 1], [1]],
[[1, 0], [1]],
[[1, 1], [0]],
];

//create a net with random weights/biases
const net = Net.create(2, 1, 3, 1);

//train the net
const trainConfig = {
gamma: 0.1, //learning rate of weights
gamma_b: 0.1, //learning rate of biases
momentum: 0.1, //momentum of weights/biases
batchSize: 1, //batch size
};

console.log(net.errorDataset(XOR)); //print the average error of the net
net.train(XOR, 100000, trainConfig); //train the net for 100000 epochs
console.log(net.errorDataset(XOR)); //print the new average error of the net
`

Documentation

Table of Contents

Net

- Net.create
- Net.mergeGradients

Net.prototype

- Net.prototype.eval
- Net.prototype.gradient
- Net.prototype.apply
- Net.prototype.train
- Net.prototype.error
- Net.prototype.errorDataset
- Net.prototype.debug

Misc

- NetOptions
- Gradient
- TrainingData
- TrainingOptions

Net

$3

source | TOC
Constructor for the Net class.

#### Arguments

- options {NetOptions} - The weights and biases of the net.

$3

source | TOC
Creates a new net with the given dimensions.

#### Arguments

- inputs {number}: The size of the input layer.
- outputs {number}: The size of the output layer.
- hidden {number}: The size of the hidden layer.
- hiddenLayers {number}: The number of hidden layers.

#### Returns

{Net}: An instance of the Net class, with random weights and biases.

$3

source | TOC
Average several gradients into one.

#### Arguments

- gradients {Gradient[]}: The gradients to average.

#### Returns

{Gradient}: The averaged gradient.

Net.prototype

$3

source | TOC
Run a forward pass of the net using the given input.

#### Arguments

- input {number[]}: The input to the net.

#### Returns

{number[][]}: The state of the net after the input has been passed.

$3

source | TOC
Calculate the required gradient for a given test case

#### Arguments

- input {number[]}: The input to the net.
- expected {number[]}: The expected output of the net.

#### Returns

{Gradient}: The gradient of the net.

$3

source | TOC
Apply a gradient to the net.

#### Arguments

- gradient {Gradient[]}: The gradient to apply.
- gamma {number}: [Optional] The learning rate of the weights.
- gamma_b {number}: [Optional] The learning rate of the biases.

$3

source | TOC
Train the net using the given dataset.

#### Arguments

- dataset {TrainingData[]}: The dataset to train the net on.
- epochs {number}: The number of epochs to train the net for.
- options {TrainingOptions}: The options for training.

$3

source | TOC
Calculate the error of the net for a given test case using MSE

#### Arguments

- input {number[]}: The input to the net.
- expected {number[]}: The expected output of the net.

#### Returns

{number}: The error of the net.

$3

source | TOC
Calculate the average error of the net for a given dataset.

#### Arguments

- dataset {TrainingData[]}: The dataset to use.

#### Returns

{number}: The average error of the net.

$3

source | TOC
Prints the state of the net to the console.

#### Format

`
Weights:
, , ...,
, , ...,
...
, , ...,
Biases:
, , ...,
, , ...,
...
, , ...,
`

Misc

$3

`typescript
{
weights: number[][];
biases: number[][];
dimensions: number[];
};
`

An object describing the weights and biases of a net.

$3

`typescript
[
number[][], //bias gradient
number[][] //weight gradient
]
`

An object describing the gradient of a net.

$3

`typescript
[
[number[], number[]], //input, expected output
[number[], number[]],
[number[], number[]],
...
]
`

An array of training data.

$3

`typescript
{
gamma?: number; //learning rate of the weights
gamma_b?: number; //learning rate of the biases
momentum?: number; //momentum of the weights/biases
batchSize?: number; //the size of the batch
}
`

Notes

Weight Configuration

Weights are internally stored as a 1D array, grouped by origin node. This is so the weights can be read sequentially, increasing the performance of the network.

$3

!A diagram of a simple net

For this net, the weights would be stored as such:

`typescript
[
[0.4, 0.8, 0.52, 0.3], //Layer 0
[0.1, 0.63, 0.48, 0.2], //Layer 1
];
``