sampling

Common library for model sampling.

Installation:

Run

``
yarn add @completion/sampling
`

Usage:

For the most basic usage, you can start from the following snippet:


`ts
import { createSimpleSamplingStream, defaultModelParams } from '@completion/sampling';

createSimpleSamplingStream({
basicAuth: Basic ${someAccessToken},
prompt: "\n\nHuman: What is the meaning of life? \n\nAssistant:",
model: {
// The defaultModelParams object is provided for convenience, with
// a set of sane defaults for model params:
...defaultModelParams,
tokenLimitPerResponse: 256,
stopSequence: ["\n\nHuman:", "\n\nAssistant:"],
name: 'model-name',
},
onSampleComplete: (completion) => {
console.log('Complete sample:', completion);
},
onSampleChange: (completion) => {
console.log('Sample so far:', completion);
},
onError: (err) => {
console.log('Error:', err);
},
});

`


If you're using the interface with conversation and templates instead of raw prompts (not recommended for most users), use:


`ts
import { createSamplingStream, defaultModelParams } from '@completion/sampling';

// createSamplingStream returns an object containing the WebSocket instance:
const { client } = createSamplingStream({
basicAuth: Basic ${someAccessToken},
convo: conversation,
template: 'default',
model: {
// The defaultModelParams object is provided for convenience, with
// a set of sane defaults for model params:
...defaultModelParams,
name: 'model-name',
},
onSampleComplete: (data) => {
console.log('Model says:', data.sampleResult);
},
});
`

Additional options are available for the method:

`ts
createSamplingStream({
basicAuth,
convo,
template,
model,

// A full url to be used for the WebSocket connection:
url: 'https://foo.bar/foo/bar',

// A WebSocket instance. This causes url above to be ignored completely:
ws: new WebSocket('http://foo/bar),

onSampleComplete,

// Called every time new sampling data arrives, with the same signature as onSampleComplete:
//
// Returning false from this method will stop sampling, and cause onSampleComplete to be
// called with stopReason set to stopCallback.
onSampleChange: (data) => {},

// Called with an Error object, if one arises during sampling:
onError: (error) => {},

// Called whenever the client internally logs something. If unset, console.log/info/warn is
// used instead.
onLog: (type: 'info' | 'warn', message) => {},
});
`

A promise-based interface is also available, which resolves once sampling is complete:

`ts
const sampleResult = await createSamplingStreamPromise({
/ same options as above /,
rejectOnError: true
});
``

If provided, callback methods will still be called, before the promise returns/rejects (if appropriate).