!@thi.ng/rstream-query

!npm downloads

> [!NOTE]
> This is one of 214 standalone projects, maintained as part
> of the @thi.ng/umbrella monorepo
> and anti-framework.
>
> 🚀 Please help me to work full-time on these projects by sponsoring me on
> GitHub. Thank you! ❤️

- About
- Current features
- Status
- Installation
- Dependencies
- Usage examples
- API
- Visualizing a query's dataflow topology
- Authors
- License

About

@thi.ng/rstream
based triple store &
reactive query engine with declarative query specs related to
Datalog /
SPARQL. Inserted triples / facts
are broadcast to multiple indexing streams and any query subscriptions
attached to them. This enables push-based, auto-updating query results,
which are changing each time upstream transformations & filters have
been triggered.

Triples are 3-tuples of
[subject, predicate, object]. Unlike with traditional
RDF
triple stores, any JS data types can be used as subject, predicate or
object (though support for such must be explicitly enabled & this
feature is currently WIP).

$3

- Dynamic & declarative dataflow graph construction via high-level data
specs and/or functions
- Entirely based on stream abstractions provided by
@thi.ng/rstream
- All data transformations done using dynamically composed
tranducers
- Query optimizations
- Extensive re-use of existing sub-query results (via subscriptions)
- Interim result de-duplication / dataflow gates
- Push-based, auto-updating query results

Status

STABLE - used in production

Search or submit any issues for this package

This project is currently still in early development and intended as a
continuation of the Clojure based thi.ng/fabric,
this time built on the streaming primitives provided by
@thi.ng/rstream.

Installation

``bash
yarn add @thi.ng/rstream-query
`

ESM import:

`ts
import * as rsq from "@thi.ng/rstream-query";
`

Browser ESM import:

`html

`

JSDelivr documentation

For Node.js REPL:

`js
const rsq = await import("@thi.ng/rstream-query");
`

Package sizes (brotli'd, pre-treeshake): ESM: 2.58 KB

Dependencies

- @thi.ng/api
- @thi.ng/associative
- @thi.ng/checks
- @thi.ng/equiv
- @thi.ng/errors
- @thi.ng/logger
- @thi.ng/math
- @thi.ng/rstream
- @thi.ng/rstream-dot
- @thi.ng/transducers

Note: @thi.ng/api is in _most_ cases a type-only import (not used at runtime)

Usage examples

One project in this repo's
/examples
directory is using this package:

| Screenshot | Description | Live demo | Source |
|:--------------------------------------------------------------------------------------------------------------------|:--------------------------------------------|:---------------------------------------------------|:--------------------------------------------------------------------------------|
| | Triple store query results & sortable table | Demo | Source |

API

Generated API docs

`ts
import { TripleStore, asTriples } from "@thi.ng/rstream-query";
import { trace } from "@thi.ng/rstream";

// create store with initial set of triples / facts
const store = new TripleStore([
["london", "type", "city"],
["london", "part-of", "uk"],
["portland", "type", "city"],
["portland", "partOf", "oregon"],
["portland", "partOf", "usa"],
["oregon", "type", "state"],
["usa", "type", "country"],
["uk", "type", "country"],
]);

// alternatively, convert an object into a sequence of triples
const store = new TripleStore(asTriples({
london: {
type: "city",
partOf: "uk"
},
portland: {
type: "city",
partOf: ["oregon", "usa"]
},
oregon: { type: "state" },
uk: { type: "country" },
usa: { type: "country" },
});

// compile the below query spec into a dataflow graph
// pattern items prefixed w/ "?" are query variables

// this query matches the following relationships
// using all currently known triples in the store
// when matching triples are added or removed, the query
// result updates automatically...

// currently only "where" and bounded "path" sub-queries are possible
// in the near future, more query types will be supported
// (e.g. optional relationships, pre/post filters etc.)
store.addQueryFromSpec({
q: [
{
// all "where" subqueries are joined (logical AND)
where: [
// match any subject of type "city"
["?city", "type", "city"],
// match each ?city var's "part-of" relationships (if any)
["?city", "partOf", "?country"],
// matched ?country var must have type = "country"
["?country", "type", "country"]
]
}
],
// bind is an (optional) query post-processor and
// allows injection of new variables into the result set
// here we create a new var "answer" whose values are derived from
// the other two query vars
bind: {
answer: (res) => ${res.city} is located in ${res.country}
},
// another post-processing step, only keeps "answer" var in results
select: ["answer"]
})
.subscribe(trace("results"))
// results Set {
// { answer: 'london is located in uk' },
// { answer: 'portland is located in usa' } }

// helper fn to insert new city relationship to the store
const addCity = (name, country) =>
store.into([
[name, "type", "city"],
[name, "partOf", country],
[country, "type", "country"],
]);

addCity("berlin", "germany");
// results Set {
// { answer: 'london is located in uk' },
// { answer: 'portland is located in usa' },
// { answer: 'berlin is located in germany' } }

addCity("paris", "france");
// results Set {
// { answer: 'london is located in uk' },
// { answer: 'portland is located in usa' },
// { answer: 'berlin is located in germany' },
// { answer: 'paris is located in france' } }
`

$3

After setting up the above query and its internal transformations, the
generated dataflow topology then looks as follows:

!graphviz output

- The blue nodes are TripleStore-internal index stream sources,
emitting changes when new triples are added
- The left set of red nodes are the sub-queries of the above where
clause, responsible for joining the individual (S)ubject, (P)redicate
and (O)bject sub-queries.
- The results of these are then further joined (right red node) &
transformed to produce the final solution set and post-process it

Btw. The diagram has been generated using
@thi.ng/rstream-dot
and can be recreated by calling store.toDot() (for the above example)

The source code for the above example is
here

(Many) more features forthcoming...

Authors

- Karsten Schmidt

If this project contributes to an academic publication, please cite it as:

`bibtex
@misc{thing-rstream-query,
title = "@thi.ng/rstream-query",
author = "Karsten Schmidt",
note = "https://thi.ng/rstream-query",
year = 2018
}
``

License

© 2018 - 2026 Karsten Schmidt // Apache License 2.0