Compact binary encoding for geographic data
npm install geobuf
Geobuf is a compact binary encoding for geographic data.
Geobuf provides _nearly lossless_ compression of GeoJSON data
into protocol buffers.
Advantages over using GeoJSON alone:
- Very compact: typically makes GeoJSON 6-8 times smaller.
- 2-2.5x smaller even when comparing gzipped sizes.
- Very fast encoding and decoding — even faster than native JSON parse/stringify.
- Can accommodate any GeoJSON data, including extensions with arbitrary properties.
The encoding format also potentially allows:
- Easy incremental parsing — get features out as you read them,
without the need to build in-memory representation of the whole data.
- Partial reads — read only the parts you actually need, skipping the rest.
Think of this as an attempt to design a simple, modern Shapefile successor
that works seamlessly with GeoJSON.
Unlike Mapbox Vector Tiles,
it aims for nearly lossless compression of datasets — without tiling, projecting coordinates,
flattening geometries or stripping properties.
Note that the encoding schema is not stable yet —
it may still change as we get community feedback and discover new ways to improve it.
"Nearly" lossless means coordinates are encoded with precision of 6 digits after the decimal point (about 10cm).
#### Sample compression sizes
Data | JSON | JSON (gz) | Geobuf | Geobuf (gz)
------------------- | --------: | --------: | -------: | ----------:
US zip codes | 101.85 MB | 26.67 MB | 12.24 MB | 10.48 MB
Idaho counties | 10.92 MB | 2.57 MB | 1.37 MB | 1.17 MB
``js`
var buffer = geobuf.encode(geojson, new Pbf());
Given a GeoJSON object and a Pbf object to write to,
returns a Geobuf as UInt8Array array of bytes.Buffer.from
In Node@4.5.0 or later, you can use to convert back to a buffer.
`js`
var geojson = geobuf.decode(new Pbf(data));
Given a Pbf object with Geobuf data, return a GeoJSON object. When loading Geobuf data over XMLHttpRequest, you need to set responseType to arraybuffer.
Node and Browserify:
`bash`
npm install geobuf
Browser build CDN links:
- https://unpkg.com/geobuf@3.0.1/dist/geobuf.js
- https://unpkg.com/geobuf@3.0.1/dist/geobuf-dev.js
Building locally:
`bash`
npm install
npm run build-dev # dist/geobuf-dev.js (development build)
npm run build-min # dist/geobuf.js (minified production build)
`bash`
npm install -g geobuf
Installs these nifty binaries:
* geobuf2json: turn Geobuf from stdin or specified file to GeoJSON on stdout
* : turn GeoJSON from stdin or specified file to Geobuf on stdout
* : given a Shapefile filename, send Geobuf on stdout
`bash`
json2geobuf data.json > data.pbf
shp2geobuf myshapefile > data.pbf
geobuf2json data.pbf > data.json
Note that for big files, geobuf2json command can be pretty slow, but the bottleneck is not the decoding,JSON.stringify
but the native on the decoded object to pipe it as a string to stdout.
On some files, this step may take 40 times more time than actual decoding.
* geojsonp — the prototype that led to this project
* pygeobuf — Python implementation of Geobuf
* twkb — a geospatial binary encoding that doesn't support topology
and doesn't encode any non-geographic properties besides id`
* vector-tile-spec
* topojson — an extension of GeoJSON that supports topology
* WKT and WKB — popular in databases
* EWKB — a popular superset of WKB