http-fields

A modern JavaScript library for parsing and serializing HTTP Structured Field Values (RFC 8941 & RFC 9651). Provides bidirectional translation between structured header strings and JSON with full TypeScript support.

See also badgateway/structured-headers

Overview

Structured Fields is a specification that defines common data structures for HTTP header and trailer fields. Instead of each HTTP field having its own custom parsing logic, this specification provides a set of well-defined data types and parsing algorithms that can be shared across multiple fields.

This library implements the complete RFC 8941 and RFC 9651 specifications with strict parsing and serialization according to the standards, including support for dates and Unicode display strings.

Features

- ✅ Complete RFC 8941 & RFC 9651 Implementation - All data types and algorithms
- ✅ TypeScript Support - Comprehensive type definitions and type guards
- ✅ Strict Parsing - Follows RFC's error handling requirements
- ✅ Bidirectional Translation - Parse to JSON and serialize back to strings
- ✅ Extended Data Types - Dates (@) and Display Strings (%) from RFC 9651
- ✅ Type Safety - Full TypeScript definitions with runtime validation
- ✅ Zero Dependencies - Pure JavaScript implementation
- ✅ ES Module Support - Modern JavaScript module format

Installation

``bash
npm install http-fields
`

Or include directly in your project:

`javascript
import * as HTTPFields from "./index.mjs";
`

Quick Start

$3

`javascript
import * as HTTPFields from "http-fields";

// Parse a list
const list = HTTPFields.parse("sugar, tea, rum", "list");
console.log(list);
// Output: [
// { value: { type: 'token', value: 'sugar' }, parameters: {} },
// { value: { type: 'token', value: 'tea' }, parameters: {} },
// { value: { type: 'token', value: 'rum' }, parameters: {} }
// ]

// Serialize back to string
const serialized = HTTPFields.serialize(list, "list");
console.log(serialized); // "sugar, tea, rum"
`

$3

`typescript
import * as HTTPFields from "http-fields";
import type { List, Dictionary, Item } from "http-fields/types";

// Parse with type safety
const list: List = HTTPFields.parse("sugar, tea, rum", "list");
const dict: Dictionary = HTTPFields.parse("a=1, b=2", "dictionary");
const item: Item = HTTPFields.parse("@1672531200", "item");

// RFC 9651 features
const date = HTTPFields.date(new Date());
const displayStr = HTTPFields.displayString("Hello 世界");
`

Data Types

$3

| Type | Description | Example | JSON Representation |
| ----------------------------- | ------------------------------------------------------------ | ------------------ | --------------------------------------------------------- |
| Integer | Signed integer (-999,999,999,999,999 to 999,999,999,999,999) | 42 | 42 |
| Decimal | Decimal number (max 12 integer digits, 3 fractional) | 3.14 | 3.14 |
| String | Quoted ASCII string | "hello world" | "hello world" |
| Token | Unquoted text identifier | application/json | { type: 'token', value: 'application/json' } |
| Byte Sequence | Base64-encoded binary data | :SGVsbG8=: | { type: 'binary', value: 'SGVsbG8=', decoded: 'Hello' } |
| Boolean | True or false value | ?1 or ?0 | true or false |
| Date (RFC 9651) | Unix timestamp with @ prefix | @1672531200 | { type: 'date', value: Date } |
| Display String (RFC 9651) | Unicode string with % prefix | %"Hello 世界" | { type: 'displaystring', value: 'Hello 世界' } |

$3

| Type | Description | Example |
| -------------- | ----------------------------------- | ---------------------- |
| List | Array of items or inner lists | a, b, (c d) |
| Dictionary | Key-value pairs | a=1, b=2, c |
| Inner List | Parenthesized list of items | (a b c) |
| Parameters | Semicolon-separated key-value pairs | ;charset=utf-8;q=0.9 |

API Reference

$3

#### parse(fieldValue, fieldType)

Parses a structured field string into a JSON representation.

Parameters:

- fieldValue (string): The HTTP field value to parse
- fieldType (string): One of 'list', 'dictionary', or 'item'

Returns: Parsed structure as JSON

Example:

`javascript
const result = HTTPFields.parse("a=1, b=2;x=y", "dictionary");
`

#### serialize(data, fieldType)

Serializes a JSON structure back to a structured field string.

Parameters:

- data (any): The data structure to serialize
- fieldType (string): One of 'list', 'dictionary', or 'item'

Returns: Serialized field value string

Example:

`javascript
const fieldValue = HTTPFields.serialize(data, "dictionary");
`

$3

#### token(value)

Creates a token object for use in structured data.

Parameters:

- value (string): The token string

Returns: Token object { type: 'token', value: string }

Example:

`javascript
const tokenObj = HTTPFields.token("application/json");
`

#### binary(base64Value)

Creates a binary object for use in structured data.

Parameters:

- base64Value (string): Base64-encoded string

Returns: Binary object { type: 'binary', value: string }

Example:

`javascript
const binaryObj = HTTPFields.binary("SGVsbG8=");
`

#### date(dateValue) - RFC 9651

Creates a date object for use in structured data.

Parameters:

- dateValue (Date): JavaScript Date object

Returns: Date object { type: 'date', value: Date }

Example:

`javascript
const dateObj = HTTPFields.date(new Date("2024-01-01"));
// Serializes to: @1704067200
`

#### displayString(unicodeValue) - RFC 9651

Creates a display string object for Unicode content.

Parameters:

- unicodeValue (string): Unicode string

Returns: Display string object { type: 'displaystring', value: string }

Example:

`javascript
const displayObj = HTTPFields.displayString("Hello 世界");
// Serializes to: %"Hello %e4%b8%96%e7%95%8c"
`

Examples

$3

`javascript
// Parse a simple list
const simpleList = HTTPFields.parse("a, b, c", "list");

// Parse a list with parameters
const listWithParams = HTTPFields.parse("a;x=1, b;y=2", "list");

// Parse a list with inner lists
const complexList = HTTPFields.parse("a, (b c), d", "list");

// Create and serialize a list
const listData = [
{ value: HTTPFields.token("sugar"), parameters: {} },
{ value: HTTPFields.token("tea"), parameters: { quality: 0.8 } },
{
value: [
{ value: HTTPFields.token("milk"), parameters: {} },
{ value: HTTPFields.token("honey"), parameters: {} },
],
parameters: { organic: true },
},
];
const serialized = HTTPFields.serialize(listData, "list");
// Result: "sugar, tea;quality=0.8, (milk honey);organic"
`

$3

`javascript
// Parse a dictionary
const dict = HTTPFields.parse(
"cache=max-age, public, max-age=3600",
"dictionary"
);

// Parse dictionary with various types
const complexDict = HTTPFields.parse(
'a=1, b="hello", c=?1, d=(x y)',
"dictionary"
);

// Create and serialize a dictionary
const dictData = {
method: { value: HTTPFields.token("GET"), parameters: {} },
timeout: { value: 30, parameters: { unit: HTTPFields.token("seconds") } },
secure: { value: true, parameters: {} },
headers: {
value: [
{ value: HTTPFields.token("authorization"), parameters: {} },
{ value: HTTPFields.token("content-type"), parameters: {} },
],
parameters: {},
},
};
const serialized = HTTPFields.serialize(dictData, "dictionary");
`

$3

`javascript
// Parse a simple item
const item = HTTPFields.parse("42", "item");
// Result: { value: 42, parameters: {} }

// Parse an item with parameters
const itemWithParams = HTTPFields.parse(
'"hello";charset="utf-8";length=5',
"item"
);
// Result: {
// value: "hello",
// parameters: {
// charset: "utf-8",
// length: 5
// }
// }

// Serialize an item
const itemData = {
value: HTTPFields.token("application/json"),
parameters: {
charset: HTTPFields.token("utf-8"),
boundary: "----WebKitFormBoundary7MA4YWxkTrZu0gW",
},
};
const serialized = HTTPFields.serialize(itemData, "item");
// Result: 'application/json;charset=utf-8;boundary="----WebKitFormBoundary7MA4YWxkTrZu0gW"'
`

$3

`javascript
// Parse binary data
const binary = HTTPFields.parse(":SGVsbG8gV29ybGQ=:", "item");
// Result: {
// value: {
// type: 'binary',
// value: 'SGVsbG8gV29ybGQ=',
// decoded: 'Hello World'
// },
// parameters: {}
// }

// Create binary data
const binaryData = {
value: HTTPFields.binary(btoa("Hello World")),
parameters: { encoding: HTTPFields.token("base64") },
};
const serialized = HTTPFields.serialize(binaryData, "item");
// Result: ':SGVsbG8gV29ybGQ=:;encoding=base64'
`

$3

`javascript
// Parse date values
const dateItem = HTTPFields.parse("@1672531200", "item");
// Result: {
// value: {
// type: 'date',
// value: Date(2023-01-01T00:00:00.000Z)
// },
// parameters: {}
// }

// Create and serialize dates
const expiryData = {
value: HTTPFields.date(new Date("2025-12-31T23:59:59Z")),
parameters: { timezone: HTTPFields.token("UTC") },
};
const serialized = HTTPFields.serialize(expiryData, "item");
// Result: '@1767225599;timezone=UTC'

// Dates in dictionaries
const timingDict = {
start: { value: HTTPFields.date(new Date("2024-01-01")), parameters: {} },
end: { value: HTTPFields.date(new Date("2024-12-31")), parameters: {} },
};
const serializedDict = HTTPFields.serialize(timingDict, "dictionary");
// Result: 'start=@1704067200, end=@1735689600'
`

$3

`javascript
// Parse Unicode display strings
const displayItem = HTTPFields.parse('%"Hello 世界"', "item");
// Result: {
// value: {
// type: 'displaystring',
// value: 'Hello 世界'
// },
// parameters: {}
// }

// Create and serialize display strings
const messageData = {
value: HTTPFields.displayString("Welcome to Tokyo 東京へようこそ"),
parameters: { lang: HTTPFields.token("ja") },
};
const serialized = HTTPFields.serialize(messageData, "item");
// Result: '%"Welcome to Tokyo %e6%9d%b1%e4%ba%ac%e3%81%b8%e3%82%88%e3%81%86%e3%81%93%e3%81%9d";lang=ja'

// Mixed content with dates and display strings
const eventList = [
{ value: HTTPFields.displayString("Conference 会議"), parameters: {} },
{ value: HTTPFields.date(new Date("2024-06-15T09:00:00Z")), parameters: {} },
{ value: HTTPFields.token("tokyo-convention-center"), parameters: {} },
];
const serializedList = HTTPFields.serialize(eventList, "list");
// Result: '%"Conference %e4%bc%9a%e8%ad%b0", @1718442000, tokyo-convention-center'
`

Working with Standard Headers Object

The library integrates seamlessly with the standard Web API Headers object used in fetch(), service workers, and other web APIs:

$3

`javascript
// Working with fetch Response headers
async function handleResponse(response) {
const headers = response.headers;

// Parse Cache-Control header (dictionary)
const cacheControlValue = headers.get("cache-control");
if (cacheControlValue) {
try {
const cacheControl = HTTPFields.parse(cacheControlValue, "dictionary");
console.log("Max age:", cacheControl["max-age"]?.value);
console.log("Is private:", cacheControl.private?.value === true);
} catch (error) {
console.warn("Invalid Cache-Control header");
}
}

// Parse Accept-Language header (list)
const acceptLangValue = headers.get("accept-language");
if (acceptLangValue) {
try {
const acceptLang = HTTPFields.parse(acceptLangValue, "list");
const languages = acceptLang.map((item) => ({
language: item.value.value || item.value,
quality: item.parameters.q || 1.0,
}));
console.log("Preferred languages:", languages);
} catch (error) {
console.warn("Invalid Accept-Language header");
}
}

// Parse custom structured header (item)
const apiVersionValue = headers.get("x-api-version");
if (apiVersionValue) {
try {
const apiVersion = HTTPFields.parse(apiVersionValue, "item");
console.log("API Version:", apiVersion.value);
console.log("Deprecated:", apiVersion.parameters.deprecated);
} catch (error) {
console.warn("Invalid X-API-Version header");
}
}
}

// Usage with fetch
fetch("/api/data").then(handleResponse).catch(console.error);
`

$3

`javascript
// Create structured headers for outgoing requests
function createApiRequest(url, options = {}) {
const headers = new Headers(options.headers);

// Set Accept header (list with quality values)
const acceptData = [
{
value: HTTPFields.token("application/json"),
parameters: { q: 0.9 },
},
{
value: HTTPFields.token("application/xml"),
parameters: { q: 0.8 },
},
{
value: HTTPFields.token("/"),
parameters: { q: 0.1 },
},
];
headers.set("accept", HTTPFields.serialize(acceptData, "list"));

// Set custom API preferences (dictionary)
const preferencesData = {
version: { value: 2, parameters: {} },
format: { value: HTTPFields.token("compact"), parameters: {} },
"include-metadata": { value: true, parameters: {} },
};
headers.set(
"x-api-preferences",
HTTPFields.serialize(preferencesData, "dictionary")
);

// Set client info (item with parameters)
const clientInfoData = {
value: HTTPFields.token("MyApp"),
parameters: {
version: "1.2.3",
platform: HTTPFields.token("web"),
},
};
headers.set("x-client-info", HTTPFields.serialize(clientInfoData, "item"));

return fetch(url, {
...options,
headers,
});
}

// Usage
createApiRequest("/api/users", {
method: "GET",
headers: {
authorization: "Bearer token123",
},
});
`

$3

`javascript
import express from "express";
import * as HTTPFields from "http-fields";

const app = express();

// Middleware to parse structured headers
app.use((req, res, next) => {
// Parse client capabilities (list)
const capabilities = req.headers["x-client-capabilities"];
if (capabilities) {
try {
req.clientCapabilities = HTTPFields.parse(capabilities, "list").map(
(item) => item.value.value || item.value
);
} catch (error) {
req.clientCapabilities = [];
}
}

// Parse feature flags (dictionary)
const featureFlags = req.headers["x-feature-flags"];
if (featureFlags) {
try {
req.featureFlags = HTTPFields.parse(featureFlags, "dictionary");
} catch (error) {
req.featureFlags = {};
}
}

next();
});

// Route handler using parsed structured headers
app.get("/api/data", (req, res) => {
// Use parsed capabilities
const supportsCompression = req.clientCapabilities.includes("gzip");
const experimentalFeatures =
req.featureFlags["experimental-ui"]?.value === true;

// Set structured response headers
const cacheData = {
"max-age": { value: 3600, parameters: {} },
public: { value: true, parameters: {} },
};
res.set("cache-control", HTTPFields.serialize(cacheData, "dictionary"));

// Set API metadata (item)
const metadataData = {
value: HTTPFields.token("v2"),
parameters: {
"rate-limit": 1000,
experimental: experimentalFeatures,
},
};
res.set("x-api-metadata", HTTPFields.serialize(metadataData, "item"));

res.json({ data: "response" });
});
`

$3

`javascript
// In service worker
self.addEventListener("fetch", (event) => {
const request = event.request;
const headers = request.headers;

// Parse client preferences
const preferencesValue = headers.get("x-preferences");
let preferences = {};

if (preferencesValue) {
try {
preferences = HTTPFields.parse(preferencesValue, "dictionary");
} catch (error) {
console.warn("Invalid preferences header");
}
}

// Modify response based on preferences
if (preferences.theme?.value === "dark") {
// Serve dark theme assets
event.respondWith(fetch("/assets/dark-theme.css"));
} else {
event.respondWith(fetch(request));
}
});

// Setting headers in service worker
self.addEventListener("fetch", (event) => {
if (event.request.url.includes("/api/")) {
// Add structured headers to API requests
const modifiedRequest = new Request(event.request, {
headers: new Headers([
...event.request.headers.entries(),
[
"x-service-worker",
HTTPFields.serialize(
{
value: HTTPFields.token("active"),
parameters: {
version: "1.0",
"cache-strategy": HTTPFields.token("stale-while-revalidate"),
},
},
"item"
),
],
]),
});

event.respondWith(fetch(modifiedRequest));
}
});
`

$3

`javascript
// Utility class for working with structured headers
import { StructuredHeaders } from "http-fields";

// Usage example
const headers = new StructuredHeaders();

// Set cache control
headers.setStructured(
"cache-control",
{
"max-age": { value: 3600, parameters: {} },
public: { value: true, parameters: {} },
},
"dictionary"
);

// Set accept header
headers.setStructured(
"accept",
[
{ value: HTTPFields.token("application/json"), parameters: { q: 0.9 } },
{ value: HTTPFields.token("text/html"), parameters: { q: 0.8 } },
],
"list"
);

// Use with fetch
fetch("/api/endpoint", {
headers: headers.headers,
});

// Parse response headers
fetch("/api/endpoint").then((response) => {
const responseHeaders = new StructuredHeaders(response.headers);
const cacheControl = responseHeaders.getStructured(
"cache-control",
"dictionary"
);
const rateLimit = responseHeaders.getStructured("x-rate-limit", "item");

console.log("Cache max-age:", cacheControl?.["max-age"]?.value);
console.log("Rate limit:", rateLimit?.value);
});
`

Error Handling

The library follows RFC 8941's strict error handling requirements. Any parsing error causes the entire operation to fail with a descriptive error message:

`javascript
try {
const result = HTTPFields.parse("invalid[syntax", "list");
} catch (error) {
console.error("Parsing failed:", error.message);
// Handle the error - treat as if field was not present
}
`

Common error scenarios:

- Invalid characters in tokens
- Malformed strings (unterminated quotes)
- Numbers out of range
- Invalid base64 encoding
- Malformed inner lists or parameters

JSON Structure Format

$3

`javascript
[
{
value: | ,
parameters: { key: value, ... }
},
...
]
`

$3

`javascript
{
"key1": {
value: | ,
parameters: { key: value, ... }
},
"key2": { ... },
...
}
`

$3

`javascript
{
value: ,
parameters: { key: value, ... }
}
`

$3

`javascript
[
{
value: ,
parameters: { key: value, ... }
},
...
]
`

Real-World Use Cases

$3

`javascript
// Parse Cache-Control as a dictionary
const cacheControl = HTTPFields.parse(
"max-age=3600, private, must-revalidate",
"dictionary"
);

// Create Cache-Control
const cacheData = {
"max-age": { value: 3600, parameters: {} },
private: { value: true, parameters: {} },
"must-revalidate": { value: true, parameters: {} },
};
const header = HTTPFields.serialize(cacheData, "dictionary");
`

$3

`javascript
// Parse Accept header as a list
const accept = HTTPFields.parse(
"text/html;q=0.9, application/json;q=0.8",
"list"
);

// Create Accept header
const acceptData = [
{
value: HTTPFields.token("text/html"),
parameters: { q: 0.9 },
},
{
value: HTTPFields.token("application/json"),
parameters: { q: 0.8 },
},
];
const header = HTTPFields.serialize(acceptData, "list");
`

$3

`javascript
// API versioning header
const version = HTTPFields.parse("v=2;deprecated=?0", "item");

// Feature flags
const features = HTTPFields.parse("feature-a, feature-b;enabled=?1", "list");

// Service metadata
const metadata = HTTPFields.parse(
'service="api", version=2, region="us-west"',
"dictionary"
);
`

Compliance and Testing

This implementation follows RFC 8941 strictly and passes the community test suite available at httpwg/structured-field-tests.

$3

- Follows RFC 8941 size limits (15-digit integers, 3 decimal places, etc.)
- ASCII-only strings (use Byte Sequences for Unicode)
- Strict parsing - any malformed input fails completely
- Parameter keys must be lowercase

Browser Compatibility

This library uses modern JavaScript features:

- ES6 modules
- Array methods (splice, shift, etc.)
- String methods
- Object methods

Supports all modern browsers and Node.js environments.

Testing

Run the comprehensive test suite:

`bash


Run custom tests (47 tests covering all features)

npm test

Run official HTTP Working Group tests (168 tests - ALL PASSING! ✅)

npm run test:official

Run all tests

npm run test:all

Run tests with coverage

npm run test:coverage
``

Test Results:

- ✅ Custom tests: 47/47 passing
- ✅ Official HTTP WG tests: 1711/1711 passing
- ✅ Total: 1758 tests passing

Our implementation now passes the same official test suite used by badgateway/structured-headers while maintaining our developer-friendly API.

Contributing

Contributions welcome! Please ensure:

- All tests pass
- Code follows existing style
- RFC 8941 compliance maintained
- Documentation updated

License

MIT License - see LICENSE file for details.

References

- RFC 8941: Structured Field Values for HTTP
- Community Test Suite
- HTTP Working Group