open-vector-tile

About

The Open Vector Tile specification is a Tile format for storing GIS data. The Vector in the name has become something of a misnomer. This spec and its code are no longer limited to vector data but also supports gridded data and image data inside the same tile.

This also serves as a modified TypeScript implementation of the Mapbox Vector Tile library. So it is backwards compatible but offers a lot of new features and improvements including (but not limited to):

* 🔗 lightweight zero dependency builds.
* 🌴 Proper module treeshake.
* 🦺 Complete TypeScript support / safety.
* 🗜 Pre-Tessellated & Indexed geometries to quickly ship data to the renderer.
* 🧊 Support for 3D geometries.
* ♏ Support for M-Values for each geometry point (used by lines and polygons).
* ♻️ Feature Properties & M-Values are stored as "Shapes" which reuses objects only needing to do lookups on values.
* 🏛 Column encoding of data to make it more compact. Better gzip and brotli compression.
* 🪺 Support nested objects in properties and m-values.
* 📦 All features support first class citizen BBOX data like IDs.
* 🫥 Lines and Polygons support offsets to know the distance it's traveled (useful for correctly rendering dashed lines across tiles).
* 📷 Supports storing multiple images in the tile.
* 🧇 Supports multiple of any gridded data such as elevation, temperature, precipitation, etc.

Inspirations

A very talented Markus Tremmel came up with the idea of migrating away from a row based approach to a column based approach with his COVTiles. I wanted to test the idea of simplifying his project and see if it was worth the effort. Once I saw brotli compression had comperable results, I decided to finish the project.

Motivations

Since another spec is being built at the same time, you may think this is a waste of time or creating an unnecessary conflict of interest. So I wrote my thoughts on this topic and why this spec is being created.

Read The Spec

The first iteration is up. It needs to be revised for more clarity, but includes all the concepts:

open-vector-tile-spec

Install

``bash
#bun
bun add open-vector-tile


pnpm

`ts
const fs = from 'fs';
import { VectorTile } from 'open-vector-tile';

import type {
VectorGeometryType,
VectorMultiLineOffset,
VectorMultiLineString,
VectorMultiPoint,
VectorMultiPolygon,
VectorMultiPolygonOffset,
} from 's2json-spec';

// assume you can read (.pbf | .mvt | .ovt)
const fixture = fs.readFileSync('./x-y-z.vector.pbf');
// Or load with bun:
const fixture = await Bun.file('./x-y-z.vector.pbf').arrayBuffer();
// load the protobuf parsing it directly
const tile = new VectorTile(fixture);

// THE VECTOR API:

// example layer
const { landuse } = tile.layers;

// grab the first feature
const firstFeature = landuse.feature(0);
// grab the geometry
const geometry = firstFeature.loadGeometry();
// get the geometry type
const geoType: VectorGeometryType = firstFeature.geoType();

// OR specifically ask for a geometry type
const points: VectorMultiPoint = firstFeature.loadPoints();
const lines: [VectorMultiLineString, VectorMultiLineOffset] = firstFeature.loadLines();
const polys: [VectorMultiPolygon, VectorMultiPolygonOffset] = firstFeature.loadPolys();

// If you want to take advantage of the pre-tessellated and indexed geometries
// and you're loading the data for a renderer, you can grab the pre-tessellated geometry
const [vertices, indices] = firstFeature.loadGeometryFlat();

// IMAGE API

// example layer
const { satellite } = tile.images;
// grab the image data
const data = satellite.image(); // Uint8Array

// GRIDDED API

// example layer
const { elevation } = tile.grids;
// grab the grid data
const data = elevation.grid(); // number[]
`

$3

`ts
import { writeOVTile, writeMVTile } from 'open-vector-tile'

// Full support for 3D geometries, m-values, complex properties with nested objects, images, grids, etc.
const encodedData = writeOVTile(
vectorTileData, // vector data
images, // compressed images in specs like PNG, JPEG, etc.
grids, // floating point grids like elevation, temperature, wind, precipitation, etc.
); // Uint8Array

// Write the older schema that's fast, light, with fewer feature sets
writeMVTile(
vectorData,
supportMabox // boolean decides if you want to support the old mapbox spec or have improved polygon support
); // Uint8Array
`

General Purpose Information

$3

`ts
type Extents = 512 | 1_024 | 2_048 | 4_096 | 8_192 | 16_384
interface Layer {
// version control helps know what features are available
version: number;
// name of the layer
name: string;
// extent of the vector tile. MUST be one of 512, 1024, 2048, 4096, 8192, or 16384
extent: Extents;
// number of features in the layer
length: number;
}
`

$3

#### Feature Types

`ts
// 6 feature types in total plus the old MapboxVectorFeature
export type VectorFeature =
// points may be a collection of points or single point
| OVectorPointsFeature
// lines may be a collection of lines or single line
| OVectorLinesFeature
// polygons may be a collection of polygons or single polygon
| OVectorPolysFeature
// 3D points may be a collection of 3D points or single 3D point
| OVectorPoints3DFeature
// 3D lines may be a collection of 3D lines or single 3D line
| OVectorLines3DFeature
// 3D polygons may be a collection of 3D polygons or single 3D polygon
| OVectorPolys3DFeature
// Can be any form of points, lines, or polygons without any of the new features
// but all the functions. line offsets and bbox will always be defaults.
| MapboxVectorFeature;
`

#### Feature Properties

`ts
type Extents = 512 | 1_024 | 2_048 | 4_096 | 8_192 | 16_384
interface Feature {
// properties of the feature
properties: any;
// id of the feature
id: number;
// extent of the vector tile. MUST be one of 512, 1_024, 2_048, 4_096, 8_192, or 16_384
extent: Extents;
}
`

#### Get the Feature's Bounding Box

`ts
export type BBox = [left: number, bottom: number, right: number, top: number];
export type BBox3D = [left: number, bottom: number, right: number, top: number, near: number, far: number];

const bbox: BBox | BBox3D = feature.bbox()
`

#### Pull in the geometry as a collection of points

`ts
// supported by all types, points, lines, and polygons
const geometry: Point[] | Point3D[] = feature.loadPoints()
`

#### Pull in the geometry as a collection of lines

`ts
// Supported by any line or polygon type
/// points will return an empty array
interface VectorLineWithOffset {
/* the offset of the line to start processing the dash position /
offset: number;
/* the line data /
geometry: VectorLine;
}
interface VectorLine3DWithOffset {
/* the offset of the line to start processing the dash position /
offset: number;
/* the line data /
geometry: VectorLine3D;
}
const geometry: VectorLineWithOffset[] | VectorLine3DWithOffset[] = feature.loadLines()
`

$3

`ts
const pointFeature: Point[] = (feature as OVectorPointsFeature).loadGeometry()
const lineFeature: VectorLine[] = (feature as OVectorLinesFeature).loadGeometry()
const polyFeature: VectorPoly[] = (feature as OVectorPolysFeature).loadGeometry()
const point3DFeature: Point3D[] = (feature as OVectorPoints3DFeature).loadGeometry()
const line3DFeature: VectorLine3D[] = (feature as OVectorLines3DFeature).loadGeometry()
const poly3DFeature: VectorPoly3D[] = (feature as OVectorPolys3DFeature).loadGeometry()
`

$3

`ts
// works for any polygon or polygon3D type.
// NOTE: If the indices is empty, then the geometry was never pre-earcut and you need to fallback to loadGeometry instead.
const geometry: [geometry: number[], indices: number[]] = feature.loadGeometryFlat()
`

$3

Shapes define the type of data that can be stored in the vector tile. They are explained in the specification.

If you'd like to validate the shape, feel free to use the Ajv library.

`ts
import Ajv from 'ajv';
import { ShapeSchema } from 'open-vector-tile'; // Path to the schema

import type { Shape } from 'open-vector-tile';

const ajv = new Ajv();
const validate = ajv.compile(ShapeSchema);

const shape: Shape = {
a: 'i64',
b: ['string'],
c: {
d: 'f64',
e: 'bool',
f: 'null',
g: 'f32',
h: {
i: 'u64',
},
},
};

validate(shape); // true
`

> [!NOTE]
> Safety Unsafe code is forbidden by a #![forbid(unsafe_code)] attribute in the root of the rust library.

---

Development

$3

You need the tool tarpaulin to generate the coverage report. Install it using the following command:

`bash
cargo install cargo-tarpaulin
`

The bacon coverage tool is used to generate the coverage report. To utilize the pycobertura package for a prettier coverage report, install it using the following command:

`bash
pip install pycobertura
`

$3

To run the tests, use the following command:

`bash


TYPESCRIPT

`bash
cargo tarpaulin


faster