url-from

A URL generation library that supports type-safe path and query RFC3986 encoding.

- [Usage]
- [API]
- [Tips]

Highlight

- 🔒 Embedding values with type-safe placeholders
- 🌐 Proper RFC3986 encoding for each component such as path and query
- 😊 Flexible management of slashes
- 🧩 Separation of URL definition and generation.
- 🔱 Support for various formats
- Absolute URL https://example.com/
- [Protocol-relative URL] //example.com/
- Root path /path/to
- Relative path path/to

url-from has no external dependencies.

Install

This library can be used with TypeScript 4.7.2 or later.

``
npm install url-from
`

Usage

`js
import urlFrom from "url-from";

// bindUrl: (params?: { tag?: string, "?query": QueryParams, "#fragment"?: string }) => string;
const bindUrl = urlFromhttps://example.com/tags${"/tag?:string"};

const url1 = bindUrl();
console.log(url1); // => "https://example.com/tags"

const url2 = bindUrl({
tag: "🐹", // tag is string type only.
"?query": { foo: "!'", bar: 2, baz: false },
"#fragment": "()*",
});
console.log(url2); // => "https://example.com/tags/%F0%9F%90%B9?foo=%21%27&bar=2&baz=false#%28%29%2A"
`

Please also check the following when using this library:

- Common Patterns for Throwing Exceptions
- Common Patterns for Emitting Warnings
- Literal Parts with % will Emit Warnings

API

- Bind Function
- User Placeholder
- Utility Placeholder
- Placeholder Options
- Argument
- Type Narrowing
- Helper
- Special Values

$3

urlFrom itself is a function that returns a Bind Function.
By calling the Bind Function, a URL is generated.

`js
const bindUrl = urlFromusers/${"userId:number"};
const url1 = bindUrl({ userId: 279 });
const url2 = bindUrl({ userId: 642, "#fragment": "fragment" });
console.log(url1); // => "users/279"
console.log(url2); // => "users/642#fragment"
`

This means that you can create a URL definition file and use the definition to generate URLs in various places.
The Bind Function also allows restricting arguments to literal types through [Type Narrowing], enabling powerful URL management.

$3

- Primitive
- Spread

#### Primitive

Format: ${"xxx"} (support placeholder options [Value Type] & [Optional] & [Conditional Slash])

Value Type: string | number

`js
const url = urlFromhttps://example.com/${"foo"}/to({ foo: "path" });
console.log(url); // => "https://example.com/path/to"
`

#### Spread

Format: ${"...xxx"} (support placeholder options [Value Type] & [Optional] & [Conditional Slash])

Value Type: Array

`js
const url = urlFromhttps://example.com/${"...paths"}({ paths: ["path", "to"] });
console.log(url); // => "https://example.com/path/to"
`

If you want to change the separator:

`js
const url = urlFromhttps://example.com/${"...paths"}({ paths: { value: ["path", "to"], separator: "-" } });
console.log(url); // => "https://example.com/path-to"
`

$3

- Direct
- SchemeHost
- SchemeHostPath
- Scheme
- Userinfo
- Subdomain
- Port

#### Direct

Format: ${["value"]}

Value Type: string | number

This placeholder allows you to directly embed a string that is encoded as a literal string.

`js
const url = urlFromhttps://example.com/path/to/${["white space"]}();
console.log(url); // => "https://example.com/path/to/white%20space"
`

Since this placeholder treats the value as required, you should not pass an empty string. If you pass an empty string, an exception will be thrown.

`js
try {
const bindUrl = urlFromhttps://example.com/path/to/${[""]};
} catch (error) {
console.log(error.message); // => "The value of the index 0 at direct placeholder is empty string."
}
`

#### SchemeHost

Format: ${"scheme://host"} or ${"scheme://authority"} (support placeholder options [Optional])

Value Type: string

- Embeds a string representing the portion from scheme to host (including port).
- It is useful for switching base URLs depending on the environment.

`js
const url = urlFrom${"scheme://host"}/path/to({ "scheme://host": "https://example.com" });
console.log(url); // => "https://example.com/path/to"
`

If you include a path in the value, a warning will be issued. In such cases, use the [SchemeHostPath Placeholder].

`js
// Warn: The value of the placeholder "scheme://host" cannot contain a path.
// Use the placeholder "scheme://host/path" to include paths. Received: https://example.com/path
const url = urlFrom${"scheme://host"}/to({ "scheme://host": "https://example.com/path" });
console.log(url); // => "https://example.com/path/to"
`

⚠ Note:

- Always pass static values prepared in settings or definitions.
- Avoid passing dynamically concatenated strings as values, as it can lead to vulnerabilities.
- If the host part contains Unicode, %HH format, or IPv6, an exception will be thrown (future support may be added).
- If you want to include characters other than the delimiter : or @ in the userinfo part, convert them to %3A and %40, respectively.
- The value of this placeholder should not include a QueryString or Fragment, so any characters after ? or # will be ignored.

#### SchemeHostPath

Format: ${"scheme://host/path"} or ${"scheme://authority/path"}

Value Type: string

- Embeds a string representing the portion from scheme to host or path.
- It is useful for switching base URLs depending on the environment.

`js
const url = urlFrom${"scheme://host/path"}/to({ "scheme://host/path": "https://example.com/path" });
console.log(url); // => "https://example.com/path/to"
`

⚠ Note:

- Always pass static values prepared in settings or definitions.
- Avoid passing dynamically concatenated strings as values, as it can lead to vulnerabilities.
- If the host part contains Unicode, %HH format, or IPv6, an exception will be thrown (future support may be added).
- If you want to include characters other than the delimiter : or @ in the userinfo part, convert them to %3A and %40, respectively.
- The value of this placeholder should not include a QueryString or Fragment, so any characters after ? or # will be ignored.
- This placeholder is available only as required and cannot be optional.

#### Scheme

Format: ${"scheme:"} (support placeholder options [Optional])

Value Type: string

`js
const url = urlFrom${"scheme:"}//example.com/path/to({ "scheme:": "https" });
console.log(url); // => "https://example.com/path/to"
`

#### Userinfo

Using usernames and passwords in URLs is generally a security risk, so please avoid it as much as possible.

Format: ${"userinfo@"} (support placeholder options [Optional])

Value Type: { user?: string; password?: string }

`js
const url = urlFromhttps://${"userinfo@"}example.com/path/to({ "userinfo@": { user: "name", password: "pass" } });
console.log(url); // => "https://name:pass@example.com/path/to"
`

#### Subdomain

Format: ${"subdomain."} (support placeholder options [Optional])

Value Type: string[]

`js
const url = urlFromhttps://${"subdomain."}example.com/path/to({ "subdomain.": ["foo", "bar"] });
console.log(url); // => "https://foo.bar.example.com/path/to"
`

If you want to change the separator:

`js
const url = urlFromhttps://${"subdomain."}example.com/path/to({
"subdomain.": { value: ["foo", "bar"], separator: "-" },
});
console.log(url); // => "https://foo-bar.example.com/path/to"
`

#### Port

Format: ${":port"} (support placeholder options [Optional])

Value Type: number

`js
const url = urlFromhttps://localhost${":port"}/path/to({ ":port": 3000 });
console.log(url); // => "https://localhost:3000/path/to"
`

$3

#### Value Type

This library supports TypeScript-like type specification.

You can specify the type by appending :string or :number after the placeholder name.
If the type is not specified, it accepts string | number as the default type.

`js
const url = urlFromhttps://example.com/users/${"userId:string"}({ userId: "279642" }); // If you pass a number, it will result in a type error
console.log(url); // => "https://example.com/users/279642"
`

When using with [Spread], please use :string[] or :number[] for array types.
If the type is not specified, it accepts Array as the default type.

`js
const url = urlFromhttps://example.com/${"...paths:string[]"}({ paths: ["path", "to"] });
console.log(url); // => "https://example.com/path/to"
`

#### Optional

By appending ? immediately after the placeholder name, it becomes optional.

`js
const bindUrl = urlFromhttps://example.com/users/${"userId?"}; // ${"userId?:string"} is optional string
console.log(bindUrl()); // => "https://example.com/users/"
console.log(bindUrl({ userId: 279642 })); // => "https://example.com/users/279642"
`

#### Conditional Slash

By adding / at the beginning, end, or both ends of the placeholder string, the slash becomes effective only when a value is embedded.

`js
const bindUrl1 = urlFromhttps://example.com/users${"/userId?"};
console.log(bindUrl1()); // => "https://example.com/users"
console.log(bindUrl1({ userId: "279642" })); // => "https://example.com/users/279642"

const bindUrl2 = urlFromhttps://example.com/users${"/userId?/"};
console.log(bindUrl2()); // => "https://example.com/users"
console.log(bindUrl2({ userId: "279642" })); // => "https://example.com/users/279642/"
`

$3

- [Query]
- [Fragment]

#### Query

Key: ?query
Type: QueryParams

`js
const url = urlFromhttps://example.com/path/to({
"?query": {
foo: 1,
bar: ["a", "b"],
},
});
console.log(url); // => "https://example.com/path/to?foo=1&bar=a&bar=b"
`

Ways to specify in the format of [URLSearchParams] or as a string.

`js
// Directly specified in the literal part
const bindUrl = urlFromhttps://example.com/?foo=1&bar=2;

// Adding values with the same keys to the QueryString in the literal part
const url2 = bindUrl({
// Array<[string, Value]>
"?query": [
["foo", 123],
["bar", 234],
],
});
console.log(url2); // => "https://example.com/?foo=1&bar=2&foo=123&bar=234"

// Specifying as a string (Warning: If it contains characters that need to be encoded according to RFC3986, it will be encoded)
const url3 = bindUrl({
// string
"?query": "foo=123&bar=234", // It will produce the same result even with "?foo=123&bar=234"
});
console.log(url3); // => "https://example.com/?foo=123&bar=234"
`

#### Fragment

Key: #fragment
Type: string

`js
const url = urlFromhttps://example.com/path/to({ "#fragment": "fragment" });
console.log(url); // => "https://example.com/path/to#fragment"
`

Method to directly specify in the literal part.

`js
const url = urlFromhttps://example.com/path/to#fragment();
console.log(url); // => "https://example.com/path/to#fragment"
`

$3

If you want Bind Function arguments to be of narrower types, you can use narrowing to make optional mandatory or restrict them to various literal types.

`ts
const bindUrl = urlFrom${"scheme:"}//example.com/users/${"userId"}.narrowing<{
"scheme:": "http" | "https";
}>;

const url1 = bindUrl({ "scheme:": "http", userId: 279642 }); // ✅
const url2 = bindUrl({ "scheme:": "https", userId: 279642 }); // ✅
const url3 = bindUrl({ "scheme:": "ftp", userId: 279642 }); // ❌ TS2322: Type '"ftp"' is not assignable to type '"http" | "https"'.
`

Rules

- Only the parts that exist in the original argument type are targeted.
- The original argument type can be narrowed down.
- Optional types can be made mandatory.
- Mandatory types cannot be made optional.

#### Making specific keys or QueryParams mandatory

`ts
const bindUrl = urlFromhttps://example.com/users/${"userId?"}.narrowing<{
userId: number;
"?query": { foo: string };
// Narrowing down while inheriting free QueryParams
// "?query": { foo: string } & URLFromQueryParams;
}>;

const url1 = bindUrl({ userId: 1, "?query": { foo: "bar" } }); // ✅
const url2 = bindUrl({ userId: 1, "?query": {} }); // ❌ TS2741: Property 'foo' is missing in type '{}' but required in type '{ foo: string; }'.
`

$3

`ts
const bindUrl = urlFromhttps://example.com/theme/${"theme:string"}.narrowing<{
theme: "lighter" | "dark";
}>;

const url1 = bindUrl({ theme: "dark" }); // ✅
const url2 = bindUrl({ theme: "foo" }); // ❌ TS2322: Type '"foo"' is not assignable to type '"lighter" | "dark"'.
`

$3

`ts
const bindUrl = urlFromhttps://example.com/theme/${"theme:string"}${"/color?:string"}.narrowing<
| {
theme: "lighter" | "dark";
}
| {
theme: "original";
color: "red" | "blue";
}
>;

const url1 = bindUrl({ theme: "lighter" }); // ✅
const url2 = bindUrl({ theme: "dark" }); // ✅
const url3 = bindUrl({ theme: "original", color: "red" }); // ✅

// ❌ TS2345: Argument of type '{ theme: "original"; }' is not assignable to parameter of type 'Readonly<{ "?query"?: QueryParams | undefined; "#fragment"?: string | undefined; color?: BindParam; } & ({ theme: "lighter" | "dark"; } | { ...; })>'.
// Property 'color' is missing in type '{ theme: "original"; }' but required in type 'Readonly<{ "?query"?: QueryParams | undefined; "#fragment"?: string | undefined; color?: BindParam; } & { theme: "original"; color: "red" | "blue"; }>'.
const url4 = bindUrl({ theme: "original" });
`

$3

#### encodeRFC3986(string)

Encodes a string using [RFC3986].

string

Type: string

`js
console.log(encodeRFC3986("!'()*")); // => "%21%27%28%29%2A"
`

#### stringifyQuery(query, fragment?)

Generates a string for the QueryString or Fragment portion.

This function always returns the result with a leading "?" regardless of what is passed. If you don't need the "?", you can use stringifyQuery(query).slice(1) to obtain the result without the "?".

query

Type: URLFromQueryParams | QueryDelete

fragment

Type: string | undefined

`js
console.log(stringifyQuery({ foo: 1 }, "fragment")); // => "?foo=1#fragment"
console.log(stringifyQuery({ foo: [1, 2] })); // => "?foo=1&foo=2"
console.log(
stringifyQuery([
["foo", 1],
["foo", 2],
])
); // => "?foo=1&foo=2"
console.log(stringifyQuery(undefined)); // => "?"
console.log(stringifyQuery(undefined, "fragment")); // => "?#fragment"
console.log(stringifyQuery({})); // => "?"
`

#### replaceQuery(url, query?, fragment?)

Performs replacement or deletion of the QueryString or Fragment portion.

url

Type: string

query

Type: URLFromQueryParams | QueryDelete

fragment

Type: string | undefined

Replacement example:

`js
console.log(replaceQuery("https://example.com?foo=1&bar=2#fragment", { bar: "baz" }, "hash")); // => "https://example.com?foo=1&bar=baz#hash"

// Additional examples
console.log(replaceQuery("?a=b", { foo: 1 })); // => "?a=b&foo=1"
console.log(replaceQuery("?a=b", { foo: [1, 2] })); // => "?a=b&foo=1&foo=2"
console.log(
replaceQuery("?a=b", [
["foo", 1],
["foo", 2],
])
); // => "?a=b&foo=1&foo=2"
`

Deletion example:

`js
console.log(replaceQuery("?foo=1&bar=baz#fragment", QueryDelete, "")); // => ""
console.log(replaceQuery("?foo=1&bar=baz#fragment", { foo: QueryDelete })); // => "?bar=baz#fragment"
`

Note that deletion cannot be done with undefined, {}, or "".

`js
console.log(replaceQuery("?foo=1&bar=baz#fragment", undefined, undefined)); // => "?foo=1&bar=baz#fragment"
console.log(replaceQuery("?foo=1&bar=baz#fragment", {}, undefined)); // => "?foo=1&bar=baz#fragment"
console.log(replaceQuery("?foo=1&bar=baz#fragment", "", undefined)); // => "?foo=1&bar=baz#fragment"
`

$3

A list of effects when using special values in placeholders or queries.

| Type or Value | Effect of Placeholder | Effect in Query | Supplement |
| ------------- | --------------------- | --------------- | ---------------------------------------------------- |
| "" | Skip | "key=" | An exception is thrown when used in a required path. |
| number | "0" | "key=0" | |
| NaN | "NaN" | "key=NaN" | A warning is issued when passed as a value. |
| true | - | "key=true" | Cannot be used as a placeholder value. |
| false | - | "key=false" | Cannot be used as a placeholder value. |
| null | Skip | "key" | Represented only by the key in the Query. |
| undefined | Skip | Skip | |
| QueryDelete | - | Delete | Symbol for deleting the key in the Query. |

`js
const bindUrl = urlFromhttps://example.com/${"value?"};

// Placeholder
console.log(bindUrl({ value: "" })); // => "https://example.com/"
console.log(bindUrl({ value: 0 })); // => "https://example.com/0"
// warn: 'The value NaN was passed to the placeholder "value".'
console.log(bindUrl({ value: NaN })); // => "https://example.com/NaN"
console.log(bindUrl({ value: null })); // => "https://example.com/"
console.log(bindUrl({ value: undefined })); // => "https://example.com/"

// Query
console.log(bindUrl({ "?query": { value: "" } })); // => "https://example.com/?value="
console.log(bindUrl({ "?query": { value: 0 } })); // => "https://example.com/?value=0"
// warn: 'Invalid query value for key "value". Received: NaN'
console.log(bindUrl({ "?query": { value: NaN } })); // => "https://example.com/?value=NaN"
console.log(bindUrl({ "?query": { value: null } })); // => "https://example.com/?value"
console.log(bindUrl({ "?query": { value: undefined } })); // => "https://example.com/"
`

Tips

$3

If it is determined that an available URL cannot be generated or is at risk, url-from will throw an exception.

- Common for all placeholders:
- Empty string passed to a required placeholder value.
- Value was passed that does not match the type.
- Individual placeholders:
- Invalid characters included in the value of [Scheme Placeholder].
- [SchemeHost Placeholder] or [SchemeHostPath Placeholder] does not include :// in its value.
- Empty string passed to the value of [Direct Placeholder].
- Value outside the range of 0-65535 or NaN is passed to the [Port Placeholder].
- Others:
- When passed to new URL(), it tried to generate a URL that would throw an exception.

$3

If there is a possible mistake or a more appropriate way to write it, url-from will issue a warning to encourage improvement.

- Literal Part
- When the literal part contains encoded characters according to [RFC3986]
- Solution: Use [Direct Placeholder] to embed such values.
- When the usage of ?=&# in the literal part is inappropriate.
- Common for All Placeholders
- When NaN is passed as the value for each placeholder (except for [Port Placeholder]).
- Individual Placeholders
- When the value of [SchemeHost Placeholder] or [SchemeHostPath Placeholder] contains encoded characters according to [RFC3986].
- Solution: For [SchemeHost Placeholder] or [SchemeHostPath Placeholder], if you need to include encoded characters, pass the percent-encoded (%HH) value.
- When the value of [SchemeHost Placeholder] or [SchemeHostPath Placeholder] contains ? or #.
- Others
- When the content of the URL is completed when passed to new URL().
- When the completion of / occurs.

Note: Square brackets [ ] are used to indicate placeholder names in the original text.

$3

Percent-encoding (%HH) is appropriate according to [RFC3986], but a warning is issued when it is used in the literal part for the following reasons:

- %HH is not easily understandable to humans in its decoded form and can lead to incorrect representations.
- Detecting single % or malformed %HH requires complex processing and rules.

When using % in the literal part, please use [Direct Placeholder].

`js
// warn: The literal part contains an unencoded path string "%". Received: https://example.com/emoji/%F0%9F%90%B9
const url1 = urlFromhttps://example.com/emoji/%F0%9F%90%B9%(); // ❗ Bad
const url2 = urlFromhttps://example.com/emoji/${["🐹%"]}(); // ✅ Good

console.log(url1); // => "https://example.com/emoji/%25F0%259F%2590%25B9%25"
console.log(url2); // => "https://example.com/emoji/%F0%9F%90%B9%25"
`

$3

In url-from, as a path traversal protection measure, if the conditions of /./ or /../ are satisfied through dynamic embedding, a warning is issued and the "." is replaced with a half-width space.

`js
// Assume a path two hierarchy below the tags
// https://example.com/tags//foo
const bindUrl = urlFromhttps://example.com/tags/${"tag:string"}/foo;

// When "." is passed as a value
// Normally, it would result in a path to a different hierarchy than intended... https://example.com/tags/./foo -> https://example.com/tags/foo
// warn: When embedding values in URLs, some dots are replaced with single-byte spaces because we tried to generate paths that include strings indicating the current or parent directory, such as "." or "..".
const url1 = bindUrl({ tag: "." });
// The hierarchy is maintained by replacing spaces with single-byte spaces.
console.log(url1); // => "https://example.com/tags/%20/foo"

// When ".." is passed as a value
// Normally, it would result in a path to a different hierarchy than intended... https://example.com/tags/../foo -> https://example.com/foo
// warn: When embedding values in URLs, some dots are replaced with single-byte spaces because we tried to generate paths that include strings indicating the current or parent directory, such as "." or "..".
const url2 = bindUrl({ tag: ".." });
// The hierarchy is maintained by replacing spaces with single-byte spaces.
console.log(url2); // => "https://example.com/tags/%20%20/foo"
`

The reason for replacing with a half-width space is based on evaluating which risk is lower: changing the directory structure or replacing the "." with a half-width space.

- It is unlikely that a path consisting only of a half-width space has any meaning.
- When used as a tag name, a half-width space is a character subject to trimming, so it is unlikely to have any meaning.
- When it interacts with databases or other storage systems, it is unlikely that a blank-only string would pass through validation.

In this way, while both changing the directory structure due to path traversal and replacing "." with a half-width space are unintended behaviors for implementers, if there is no solution to unintended behavior, it is preferable to choose the option with less risk.

Additional Information

- /.../ is a valid path (e.g., https://en.wikipedia.org/wiki/... is an alias for https://en.wikipedia.org/wiki/Ellipsis).
- Converting /../ to /%2E%2E/ is meaningless because it is interpreted the same as /../.

The replacement of "." with a half-width space only occurs for dynamically embedded values. In cases like the following example, where the /../ occurs due to the static literal ".", the literal "." remains unchanged.

`js
const bindUrl = urlFromhttps://example.com/dot-files/.${"type:string"}/README.md;

console.log(bindUrl({ type: "gitignore" })); // => "https://example.com/dot-files/.gitignore/README.md"
// warn: When embedding values in URLs, some dots are replaced with single-byte spaces because we tried to generate paths that include strings indicating the current or parent directory, such as "." or "..".
console.log(bindUrl({ type: "." })); // => "https://example.com/dot-files/.%20/README.md"
``

$3

- With the url-from mechanism, there is no way to overlook encoding in any part, such as the path or query.
- Strict type checks prevent the inclusion of invalid values.
- Warnings are issued for implementations that need improvement, allowing for refinement into more appropriate and secure implementations.
- There is no risk of path traversal attacks.
- It enables concise and readable code.

NOTE

The sample code in this document has been tested using power-doctest.

LICENSE

@misuken-now/url-from・MIT

[URL]: https://developer.mozilla.org/en-US/docs/Web/API/URL
[URLSearchParams]: https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams
[Protocol-relative URL]: https://en.wikipedia.org/wiki/Wikipedia:Protocol-relative_URL
[RFC3986]: https://datatracker.ietf.org/doc/html/rfc3986
[Usage]: #usage
[API]: #api
[Bind Function]: #bind-function
[User Placeholder]: #user-placeholder
[Primitive]: #primitive
[Spread]: #spread
[Utility Placeholder]: #utility-placeholder
[Direct Placeholder]: #direct
[SchemeHost Placeholder]: #schemehost
[SchemeHostPath Placeholder]: #schemehostpath
[Scheme Placeholder]: #scheme
[Userinfo Placeholder]: #userinfo
[Subdomain Placeholder]: #subdomain
[Port Placeholder]: #port
[Placeholder Options]: #placeholder-options
[Value Type]: #value-type
[Optional]: #optional
[Conditional Slash]: #conditional-slash
[Argument]: #argument
[Query]: #query
[Fragment]: #fragment
[Type Narrowing]: #type-narrowing
[Helper]: #helper
[Special Values]: #special-values
[Tips]: #tips