Sequential Generalized Likelihood Ratio Tests for A/B-testing
npm install seglirSeGLiR
======
SeGLiR is a javascript library for rapid A/B-testing with Sequential Generalized Likelihood Ratio Tests.
Sequential GLR tests are a family of sequential hypothesis tests, i.e. tests that stop as soon as a significant result has been detected. Compared to classical fixed samplesize tests, sequential GLR tests may give a significant decrease in the needed samplesize, while keeping the same type-1 and type-2 error guarantees. Below is a comparison of the expected samplesize for fixed samplesize and sequential GLR tests at the same levels. For a more detailed explanation of this family of tests (as well as the graph below), see the reference.
!Expected samplesize comparison
SeGLiR currently contain these tests:
* comparing two bernoulli proportions (for instance for comparing conversion on a website)
* comparing two normal means (with equal, known or unknown variance)
* choosing the best arm in a multi-armed bandit setting (with δ-PAC guarantees)
Improvements and additions are welcome, take a look at issues for suggested improvements.
npm install seglir
`
$3
If you're not running SeGLiR in node, note that SeGLiR requires jStat.js available.
`javascript
var glr = require("seglir");
// create an instance of a two-sided test comparing bernoulli proportions, with indifference region with size 0.01, alpha-level = 0.05, beta-level = 0.10
var test = new glr.test("bernoulli", "two-sided", 0.01, 0.05, 0.10);
// add data as it comes in
...
test.addData({x : 0});
test.addData({x : 0, y : 0});
test.addData({y : 1});
...
// when function returns string 'true' or 'false', the test is concluded
test.getResults()
// get bias-adjusted estimates
test.estimate()
`
For a complete function reference, see the reference.
$3
Make sure you have grunt and node installed.
To install the development dependencies run `npm install` and to build it run `grunt`` in the root directory.