Pull through cache

Async pull-through-cache with TTL and updated-value sampling support.

When to use

- You want to reduce async tasks like database queries or external API calls.
- You don't need an external cache, like Redis, yet.
- You still have the memory budget to store things (permanently).
- The cache usage is clearly linked to a specific entity. i.e the provided fetcher can be
simply implemented based on the used cache key.

Install

``shell
npm install --save-dev --exact @lightbase/pull-through-cache
`

Usage

`ts
import { PullThroughCache } from "@lightbase/pull-through-cache";

type MyType = {
id: string;
value: number;
};

const cache = new PullThroughCache()
.withTTL({ ttl: 3000 })
.withFetcher({
async fetcher(cache, key) {
const foo = await myTypeQueryById(key);

return foo;
},
});

cache.get("1"); // => Promise<{ id: "1", value: 5 }>
`

API

$3

Create a new typed pull-through-cache. Note that the cache will also cache undefined
values from the fetcher. So either include undefined in the provided generic argument,
or make sure that the provided fetcher to
.withFetcher
doesn't return undefined.

If no TTL, evict timestamps or sampler is set, the cache stores values indefinitely. In
those cases it might be better to use a new Map() instead.

$3

Expire cached entries once they are in the cache for ttl milliseconds. When
cache.get() is called on an expired entry, the provided fetcher is called.

$3

Sample cache.get() calls to check if the value is outdated. This method supports either
step-based sampling or random-based sampling, based on the uniform distribution of
Math.random().

If the stepValue is provided, each call to cache.get() increments an cache instance
based counter. If $counter % stepValue === 0, the sampler is called.

If randomValue is provided, each call to cache.get() executes a Math.random() call
to retrieve a $random. When $random < $randomValue, the sampler is called.

The sampler is called with both the cache key and cached value. When the sampler returns
expire, the fetcher will be called to retrieve the new value.

$3

Evict all cached entries on the provided hours and minutes combinations. The cache
automatically sorts the provided array and takes the first suitable entry to wait for. It
automatically wraps around to the next day.

Note that it uses a single background timer to execute this functionality. {@link
NodeJS.Timer.unref} is called on the timer, so it doesn't hold up the process.

$3

Set the fetcher to be called when an unknown or expired key is used in cache.get().

If cache.get() is called, before the fetcher is set, the cache will throw an error.

$3

Receive a callback when certain cache events happen, like hits and misses. This can be
used to keep track of the cache performance and the impact on setting and usage changes.

$3

Retrieve a value from the cache. May call both the sampler from .withUpdatedSampler as
well as the fetcher from .withFetcher.

$3

Retrieve all non-expired values in the cache. This acts on expired TTL values if withTTL
is used. However, it ignores the sampler.

$3

Set many entries in the cache. This can be used if you know that certain related keys
should be cached as well in a fetcher call. Note that if you use this call in the
fetcher, you still need to return the individual value. If you use cache.get() in the
fetcher to retrieve this value, you might get an infinite recursive loop if the setMany
call does not include your key.

`ts
import { PullThroughCache } from "@lightbase/pull-through-cache";

const cache = new PullThroughCache()
.withTTL({ ttl: 5 60 1000 })
.withFetcher({
async fetcher(cache, key) {
const allFlags = await retrieveAllFeatureFlagsSomehow();
cache.setMany(flags.map((it) => [it.identifier, it]));

// Make sure that we have the asked for key as well
const thisFlag = allFlags.find((it) => it.identifier === key);
assertNotNil(thisFlag);

return thisFlag;
},
});
`

$3

Retrieve if the cache is enabled or disabled. By default, the cache is enabled. A disabled
cache will use the fetcher on each invocation of cache.get(). Other calls are
practically no-ops:

- cache.getAll() will always return an empty list
- cache.setMany() will ignore the provided entries.

$3

Enable the cache. If the cache is already enabled, this call is a no-op.

$3

Disable the cache. On invocation, the cache will evict all its entries. See
.isEnabled() for behavior of the other methods.

$3

Remove all cached entries from the cache. If withEvictTimestamps is used, the eviction
might be skipped if the eviction happens at the exact moment that this function is called.

License

MIT