Radically Straightforward · Utilities

🛠️ Utilities for Node.js and the browser

Installation

``console
$ npm install @radically-straightforward/utilities
`

Usage

`typescript
import * as utilities from "@radically-straightforward/utilities";
`

$3

`typescript
export function sleep(duration: number): Promise;
`

A promisified version of setTimeout(). Bare-bones: It doesn’t even offer a way to clearTimeout(). Useful in JavaScript that may run in the browser—if you’re only targeting Node.js then you’re better served by timersPromises.setTimeout().

$3

`typescript
export function PromiseWithResolvers(): {
promise: Promise;
resolve: (value: Type | PromiseLike) => void;
reject: (reason?: any) => void;
};
`

A polyfill for Promise.withResolvers() to be used until browser support gets better.

$3

`typescript
export function randomString(): string;
`

A fast random string generator. The generated strings vary in length, but are generally around 10 characters. The generated strings include the characters [a-z0-9]. The generated strings are not cryptographically secure—if you need that, then use crypto-random-string.

$3

`typescript
export function log(...messageParts: string[]): void;
`

Tab-separated logging.

$3

`typescript
export class JSONLinesTransformStream extends TransformStream;
`

A TransformStream to convert a stream of a string with JSON lines into a stream of JSON objects.

Example

`typescript
const reader = new Blob([
\n\n${JSON.stringify("hi")}\n${JSON.stringify({ hello: "world" })}\n,
])
.stream()
.pipeThrough(new TextDecoderStream())
.pipeThrough(new utilities.JSONLinesTransformStream())
.getReader();
(await reader.read()).value; // => "hi"
(await reader.read()).value; // => { hello: "world" }
(await reader.read()).value; // => undefined
`

$3

`typescript
export function capitalize(string: string): string;
`

Capitalizes the first letter of a string. It’s different from Lodash’s capitalize() in that it doesn’t lowercase the rest of the string.

$3

`typescript
export function dedent(
templateStrings: TemplateStringsArray,
...substitutions: any[]
);
`

Removes indentation from a tagged template for more readable code.

This is different from the dedent package in the order of execution: the dedent package resolves interpolations first and removes indentation second, while dedent() removes indentation first and resolves interpolations second.

This different comes in play when the interpolated string contains newlines and indentation: with the dedent package the interpolated string must be aware of the context in which it’s used or it may break the dedenting, but with dedent() the dedenting works regardless of the string being interpolated.

Consider the following example:

`javascript
const exampleOfInterpolatedString =
"example of\n an interpolated string including a newline and indentation";

dedentPackage
Here is an

${exampleOfInterpolatedString}

followed by some more text.
;
// => "Here is an\n\n example of\nan interpolated string including a newline and indentation\n\n followed by some more text."

utilities.dedent
Here is an

${exampleOfInterpolatedString}

followed by some more text.
;
// => "Here is an\n\nexample of\n an interpolated string including a newline and indentation\n\nfollowed by some more text."
`

$3

`typescript
export function tokenize(
text: string,
{
stopWords = new Set(),
stopWordsAction = "delete",
stem = (token) => token,
}: {
stopWords?: Set;
stopWordsAction?: "delete" | "mark";
stem?: (token: string) => string;
} = {},
): {
token: string;
tokenIsStopWord: boolean;
start: number;
end: number;
}[];
`

Process text into tokens that can be used for full-text search.

The part that breaks the text into tokens matches the behavior of SQLite’s Unicode61 Tokenizer.

The stopWords are removed from the text. They are expected to be the result of normalizeToken().

The stem() allows you to implement, for example, SQLite’s Porter Tokenizer.

Reasons to use tokenize() instead of SQLite’s Tokenizers:

1. tokenize() provides a source map, linking each to token back to the ranges in text where they came from. This is useful in highlight(). SQLite’s own highlight() function doesn’t allow you to, for example, do full-text search on just the text from a message, while highlight()ing the message including markup.
2. The stopWords may be removed.
3. The stem() may support other languages, while SQLite’s Porter Tokenizer only supports English.

When using tokenize(), it’s appropriate to rely on the default tokenizer in SQLite, Unicode61.

We recommend using Natural for stopWords and stem().

Example

`typescript
import * as utilities from "@radically-straightforward/utilities";
import natural from "natural";

console.log(
utilities.tokenize(
"For my peanuts allergy peanut butter is sometimes used.",
{
stopWords: new Set(
natural.stopwords.map((stopWord) => utilities.normalizeToken(stopWord)),
),
stem: (token) => natural.PorterStemmer.stem(token),
},
),
);
// =>
// [
// { token: 'peanut', tokenIsStopWord: false, start: 7, end: 14 },
// { token: 'allergi', tokenIsStopWord: false, start: 15, end: 22 },
// { token: 'peanut', tokenIsStopWord: false, start: 23, end: 29 },
// { token: 'butter', tokenIsStopWord: false, start: 30, end: 36 },
// { token: 'sometim', tokenIsStopWord: false, start: 40, end: 49 },
// { token: 'us', tokenIsStopWord: false, start: 50, end: 54 }
// ]
`

$3

`typescript
export function normalizeToken(token: string): string;
`

Normalize a token for tokenize(). It removes accents, for example, turning ú into u. It lower cases, for example, turning HeLlO into hello.

References

- https://stackoverflow.com/a/37511463

$3

`typescript
export function highlight(
text: string,
search: Set,
{
prefix = false,
start = ,
end = ,
...tokenizeOptions
}: {
prefix?: boolean;
start?: string;
end?: string;
} & Parameters[1] = {},
): string;
`

Highlight the search terms in text. Similar to SQLite’s highlight() function, but because it’s implemented at the application level, it can work with text including markup by parsing the markup into DOM and traversing the DOM using highlight() on each Text Node.

search is the token part of the value returned by tokenize().

Example

`typescript
import * as utilities from "@radically-straightforward/utilities";
import natural from "natural";

const stopWords = new Set(
natural.stopwords.map((stopWord) => utilities.normalizeToken(stopWord)),
);

console.log(
utilities.highlight(
"For my peanuts allergy peanut butter is sometimes used.",
new Set(
utilities
.tokenize("peanuts", {
stopWords,
stem: (token) => natural.PorterStemmer.stem(token),
})
.map((tokenWithPosition) => tokenWithPosition.token),
),
{ stopWords, stem: (token) => natural.PorterStemmer.stem(token) },
),
);
// => For my peanuts allergy peanut butter is sometimes used.
`

$3

`typescript
export function snippet(
text: string,
search: Set,
{
surroundingTokens = 5,
...highlightOptions
}: {
surroundingTokens?: number;
} & Parameters[2] = {},
): string;
`

Extract a snippet from a long text that includes the search terms.

Example

`typescript
import * as utilities from "@radically-straightforward/utilities";
import natural from "natural";

const stopWords = new Set(
natural.stopwords.map((stopWord) => utilities.normalizeToken(stopWord)),
);

console.log(
utilities.snippet(
utilities.dedent
Typically mixed in these languages the. Paste extracted from sugarcane or sugar beet was the genesis of contemporary. British brought western style pastry to the spouts mounted on sledges or wagons. Toss their pancakes as well liked by.

Locally e g i aquatica. Hardness whiteness and gloss and.

Extensively planted as ornamental trees by homeowners businesses and. Yh t ritarit poor knights once only a dessert.

A shortbread base and was then only known. Pies of meat particularly beef chicken or turkey gravy and mixed vegetables potatoes. A level the name for an extended time to incorporate. Of soup beer bread and onions before they left for work in restaurants?

For my peanuts allergy peanut butter is sometimes used.

Is transformed from an inferior ovary i e one. They declined in popularity with the correct humidity. Christmas foods to be referred to as xoc l tl. Which part or all of them contain cocoa butter while maintaining.

Potato was called morgenmete and the united states? Used oil in place of. These sandwiches were not as sweet fillings include.

Granola mixed with achiote because. Has undergone multiple changes since then until. Made before making white chocolate they say. Confectionery recipes for them proliferated ' the.

Outdoorsman horace kephart recommended it in central america. Chickpea flour and certain areas of the peter.

Wan are the results two classic ways of manually tempering chocolate. Cost cocoa beans is ng g which is a. Croatian serbian and slovene pala. Km mi further south revealed that sweet potatoes have been identified from grinding. Rabanadas are a range of apple sauce depending on its consistency. Retail value rose percent latin?

Ghee and tea aid the body it is the largest pies of the era. In turkey ak tma in areas of central europe formerly belonging to!
,
new Set(
utilities
.tokenize("peanuts", {
stopWords,
stem: (token) => natural.PorterStemmer.stem(token),
})
.map((tokenWithPosition) => tokenWithPosition.token),
),
{ stopWords, stem: (token) => natural.PorterStemmer.stem(token) },
),
);
// => … work in restaurants? For my peanuts allergy peanut butter is sometimes …
`

$3

`typescript
export function isDate(string: string): boolean;
`

Determine whether the given string is a valid Date, that is, it’s in ISO format and corresponds to an existing date, for example, it is not April 32nd.

$3

`typescript
export const emailRegExp: RegExp;
`

A regular expression that matches valid email addresses. This regular expression is more restrictive than the RFC—it doesn’t match some email addresses that technically are valid, for example, example@localhost. But it strikes a good tradeoff for practical purposes, for example, signing up in a web application.

$3

`typescript
export const ISODateRegExp: RegExp;
`

A regular expression that matches ISO dates, for example, 2024-04-01T14:19:48.162Z.

$3

`typescript
export type Intern = Readonly<
Type & {
[internSymbol]: true;
}
>;
`

Utility type for intern().

$3

`typescript
export type InternInnerValue =
| string
| number
| bigint
| boolean
| symbol
| undefined
| null
| Intern;
`

Utility type for intern().

$3

`typescript
export function intern<
Type extends
| Array
| {
[key: string]: InternInnerValue;
},
>(value: Type): Intern;
`

Interning>) a value makes it unique across the program, which is useful for checking equality with === (reference equality), using it as a key in a Map, adding it to a Set, and so forth:

`typescript
import { intern as $ } from "@radically-straightforward/utilities";

[1] === [1]; // => false
$([1]) === $([1]); // => true

{
const map = new Map();
map.set([1], 1);
map.set([1], 2);
map.size; // => 2
map.get([1]); // => undefined
}

{
const map = new Map, number>();
map.set($([1]), 1);
map.set($([1]), 2);
map.size; // => 1
map.get($([1])); // => 2
}

{
const set = new Set();
set.add([1]);
set.add([1]);
set.size; // => 2
set.has([1]); // => false
}

{
const set = new Set>();
set.add($([1]));
set.add($([1]));
set.size; // => 1
set.has($([1])); // => true
}
`

> Note: We recommend that you alias intern as $ when importing it to make your code less noisy.

> Node: Inner values must be either primitives or interned values themselves, for example, $([1, $({})]) is valid, but $([1, {}]) is not.

> Node: Currently only arrays (tuples) and objects (records) may be interned. In the future we may support more types, for example, Map, Set, regular expressions, and so forth.

> Note: You must not mutate an interned value. Interned values are frozen to prevent mutation.

> Note: Interning a value is a costly operation which grows more expensive as you intern more values. Only intern values when really necessary.

> Note: Interned objects do not preserve the order of the attributes: $({ a: 1, b: 2 }) === $({ b: 2, a: 1 }).

> Note: The pool of interned values is available as intern.pool. The interned values are kept with WeakRefs to allow them to be garbage collected when they aren’t referenced anywhere else anymore. There’s a FinalizationRegistry at intern.finalizationRegistry that cleans up interned values that have been garbage collected.

Related Work

JavaScript Records & Tuples Proposal

A proposal to include immutable objects (Records) and immutable arrays (Tuples) in JavaScript. This subsumes most of the need for intern().

It includes a polyfill which works very similarly to intern() but requires different functions for different data types.

collections-deep-equal

A previous solution to this problem which took a different approach: Instead of interning the values and allowing you to use JavaScript’s Maps and Sets, collections-deep-equal extends Maps and Sets with a different notion of equality.

collections-deep-equal doesn’t address the issue of comparing values with === (reference equality).

collections-deep-equal does more work on every manipulation of the data structure, for example, when looking up a key in a Map, so it may be slower.

collections-deep-equal has different intern pools for each Map and Set instead of intern()’s single global intern pool, which may be advantageous because smaller pools may be faster to traverse.

Immutable.js, collections, mori, TypeScript Collections, prelude-ts, collectable, and so forth

Similar to collections-deep-equal, these libraries implement their own data structures instead of relying on JavaScript’s Maps and Sets. Some of them go a step further and add their own notions of objects and arrays, which requires you to convert your values back and forth, may not show up nicely in the JavaScript inspector, may be less ergonomic to use with TypeScript, and so forth.

The advantage of these libraries over interning is that they may be faster.

immer and icepick

Introduce a new way to create values based on existing values.

seamless-immutable

Modifies existing values more profoundly than freezing.

es6-array-map, valuecollection, @strong-roots-capital/map-objects, and so forth

Similar to collections-deep-equal but either incomplete, or lacking type definitions, and so forth.

Other

-
-
-
-
-
-
-

$3

`typescript
export function backgroundJob(
{
interval,
firstRun = "async",
onStop = () => {},
}: {
interval: number;
firstRun?: "sync" | "async" | "delayed";
onStop?: () => void | Promise;
},
job: () => void | Promise,
): {
run: () => Promise;
stop: () => Promise;
};
`

> Note: This is a lower level utility. See @radically-straightforward/node’s and @radically-straightforward/javascript’s extensions to backgroundJob() that are better suited for their specific environments.

Start a background job that runs every interval.

backgroundJob() is different from setInterval() in the following ways:

1. The interval counts between jobs, so slow background jobs don’t get called concurrently:

`
setInterval()
| SLOW BACKGROUND JOB |
| INTERVAL | SLOW BACKGROUND JOB |
| INTERVAL | ...

backgroundJob()
| SLOW BACKGROUND JOB | INTERVAL | SLOW BACKGROUND JOB | INTERVAL | ...
`

2. You may use backgroundJob.run() to force the background job to run right away. If the background job is already running, calling backgroundJob.run() schedules it to run again as soon as possible (with a wait interval of 0).

3. You may use backgroundJob.stop() to stop the background job. If the background job is in the middle of running, it will finish but it will not be scheduled to run again. This is similar to how an HTTP server may terminate gracefully by stopping accepting new requests but finishing responding to existing requests. After a job has been stopped, you may not backgroundJob.run() it again (calling backgroundJob.run() has no effect).

4. We introduce a random interval variance of 10% on top of the given interval to avoid many background jobs from starting at the same time and overloading the machine.

> Note: If the job throws an exception, the exception is logged and the background job continues.

$3

`typescript
export async function timeout(
duration: number,
function_: () => Promise,
): Promise;
`

Run the given function_ up to the timeout. If the timeout is reached, the returned promise rejects, but there is no way to guarantee that the function_ execution will stop.

$3

`typescript
export function foregroundJob(
job: () => void | Promise,
): (() => Promise) & {
promise: Promise;
};
`

Controls the execution of the given job such that it can’t execute until the previous execution finished.

This is useful, for example, for an autocomplete feature in which an event listener of the input event fetch()es from the server. If the job is called again while it’s running, then it schedules itself to be rerun as soon as it completes.

This is different from backgroundJob() because it doesn’t run periodically—it only runs when it’s called.

This is different from Lodash’s debounce() and throttle() because it doesn’t wait. Instead, it depends on the timing of the job itself to pace the execution, so it works best when the job is slow.

There’s a promise available under the .promise` property of the returned value that resolves when there’s no job running. It’s useful to synchronize actions that must wait on the job to be over.