Compact binary encoding for geographic data
npm install geobuf-published
Geobuf is a compact binary encoding for geographic data.
Geobuf provides _lossless_ compression of GeoJSON data
into protocol buffers.
Advantages over using GeoJSON alone:
- Very compact: typically makes GeoJSON 6-8 times smaller.
- 2-2.5x smaller even when comparing gzipped sizes.
- Very fast encoding and decoding — even faster than native JSON parse/stringify.
- Can accommodate any GeoJSON data, including extensions with arbitrary properties.
The encoding format also potentially allows:
- Easy incremental parsing — get features out as you read them,
without the need to build in-memory representation of the whole data.
- Partial reads — read only the parts you actually need, skipping the rest.
Think of this as an attempt to design a simple, modern Shapefile successor
that works seamlessly with GeoJSON.
Unlike Mapbox Vector Tiles,
it aims for _lossless_ compression of datasets — without tiling, projecting coordinates,
flattening geometries or stripping properties.
Note that the encoding schema is not stable yet —
it may still change as we get community feedback and discover new ways to improve it.
#### Sample compression sizes
| normal | gzipped
------------------- | --------- | --------
us-zips.json | 101.85 MB | 26.67 MB
us-zips.pbf | 12.24 MB | 10.48 MB
idaho.json | 10.92 MB | 2.57 MB
idaho.pbf | 1.37 MB | 1.17 MB
``js`
var buffer = geobuf.encode(geojson, new Pbf());
Given a GeoJSON object and a Pbf object to write to,
returns a Geobuf as a Buffer object in Node or UInt8Array object in browsers.
`js`
var geojson = geobuf.decode(new Pbf(data));
Given a Pbf object with Geobuf data, return a GeoJSON object.
Node and Browserify:
`bash`
npm install geobuf
Making a browser build:
`bash`
npm install
npm run build-dev # dist/geobuf-dev.js (development build)
npm run build-min # dist/geobuf.js (minified production build)
`bash`
npm install -g geobuf
Installs these nifty binaries:
* geobuf2json: turn Geobuf from stdin or specified file to GeoJSON on stdout
* : turn GeoJSON from stdin or specified file to Geobuf on stdout
* : given a Shapefile filename, send Geobuf on stdout
`bash`
json2geobuf data.json > data.pbf
shp2geobuf myshapefile > data.pbf
geobuf2json data.pbf > data.json
Note that for big files, geobuf2json command can be pretty slow, but the bottleneck is not the decoding,JSON.stringify
but the native on the decoded object to pipe it as a string to stdout.
On some files, this step may take 40 times more time than actual decoding.
* geojsonp — the prototype that led to this project
* pygeobuf — Python implementation of Geobuf
* twkb — a geospatial binary encoding that doesn't support topology
and doesn't encode any non-geographic properties besides id`
* vector-tile-spec
* topojson — an extension of GeoJSON that supports topology
* WKT and WKB — popular in databases
* EWKB — a popular superset of WKB