A functional interface to join MongoDB collections and add hooks before and after insertions, updates and removals.

Overview

Skip the repetitive glue code for joining collections and wiring related data. Define joins between collections once, then fetch documents in the exact shape you need in a nested tree. The data fetching is optimized to minimize database queries.

Stop repeating business logic all over your code base. Define hooks on collections that will be triggered conditionally before or after insertions, updates or removals: data validation, change propagation, logging, etc.

Table of contents

- Overview
- Table of contents
- Rationale
- Installation and configuration
- setProtocol(protocol)
- Bypassing coll-fns
- Joins and fetch
- Quick start examples
- join(Coll, joinDefinitions)
- Simple array join
- Sub-array joins
- Filtered array-joins
- Object joins
- Function joins
- Recursive joins
- Join additional options
- postFetch
- getJoins
- fetchList(Coll, selector, options)
- fields option and joins
- Examples
- setJoinPrefix(prefix)
- Nested Joins
- Recursion levels
- Documents transformation
- fetchOne(Coll, selector, options)
- fetchIds(Coll, selector, options)
- exists(Coll, selector)
- count(Coll, selector)
- flattenFields(fields)
- Hooks and write operations
- hook(Coll, hooksObj)
- Before hooks
- After hooks
- Hook definition properties
- Examples
- insert(Coll, doc)
- update(Coll, selector, modifier, options)
- remove(Coll, selector)
- Hook best practices
- License

Rationale

Click on any element to unfold it and better understand the rationale behind this library!

Installation and configuration

IMPORTANT: For concision, the examples will use the synchronous Meteor protocol to avoid async/await boilerplate. Of course, your code will have to be adapted when used with an asynchronous protocol.

``bash
npm install coll-fns
`

setProtocol(protocol)

You will have to define which protocol to use before using any of the library's functionality.

`js
import { setProtocol, protocols } from "coll-fns";

/* Built-in protocols include:
* - meteorAsync
* - meteorSync
* - node
*
Can also define a custom protocol! /
setProtocol(protocols.meteorAsync);
`

In a Meteor project, you should probably define a different protocol on client (synchronous) and server (asynchronous).

`js
import { setProtocol, protocols } from "coll-fns";

const protocol = Meteor.isServer ? protocols.meteorAsync : protocols.meteorSync;
setProtocol(protocol);
`

There's also a native NodeJS MongoDB driver protocol built-in (protocols.node).

Bypassing coll-fns

coll-fns intentionally keeps collection instances untouched. It doesn't add any methods nor change exsting ones' behaviour. To bypass any coll-fns functionality, simply use the normal collection methods (ex: Coll.insert, Coll.find().fetchAsync(), Coll.removeOne()). Joins and hooks will only get fired when using the library's functions... and if joins and hooks have been pre-defined, of course! 😉

Joins and fetch

Quick start examples

`js
import { fetchList, join } from "coll-fns";
import { Comments, Posts, Users } from "/collections";

/ Define joins on Posts collection /
join(Posts, {
/ One-to-one join /
author: {
Coll: Users,
on: ["authorId", "_id"],
single: true,
},

/ One-to-many join /
comments: {
Coll: Comments,
on: ["_id", "postId"],
/* single defaults to false,
so joined docs are returned as an array /
},
});

/ Fetch data with nested joined documents in the requested shape. /
fetchList(
Posts,
{},
{
fields: {
title: 1, // <= Own
author: { birthdate: 0 }, // <= Falsy = anything but these fields
comments: { text: 1 },
},
}
);
`

`jsonc
[
{
"title": "Blabla",
"authorId": "foo", // <= Included by join definition
"author": {
"name": "Foo Bar",
"genre": "non-fiction",
},
/ Comments is a one-to-many join, so is returned as a list /
"comments": [{ "text": "Nice!" }, { "text": "Great!" }],
},
]
`

join(Coll, joinDefinitions)

Collections can be joined together with globally pre-registered joins to greatly simplify optimized data fetching.

Joins are not symmetrical by default. Each collection should define its own relationships.

`js
import { join } from "coll-fns";

join(
/ Parent collection /
Coll,

/* Map of joins on children collections.
* Each key is the name of the field
where joined docs will be placed. /
{
joinProp1: {
/ joinDefinition /
},

joinProp2: {
/ joinDefinition /
},
}
);
`

Collections can define as many joins as needed without impacting performance. They will be used only when explicitly fetched. They can be declared in different places (as long as join names don't collide).

In the context of Meteor, joins could (should?) be defined in shared client and server code, but some might only ever be used in one environement or the other. They could also define a set of common joins, but add others in environment specific code.

By default, joins link one document from the parent collection to multiple ones from the child collection. In the case of a one-to-one relationship, the single property should be set to true.

There are three main types of join definitions based on the argument to the on property: array, object and function joins.

$3

on can be defined as an array of [parentProp, childProp] equality.

`js
import { join } from "coll-fns";
import { Comments, Posts, Users } from "/collections";

join(Posts, {
author: {
Coll: Users,
/ post.authorId === user._id /
on: ["authorId", "_id"],
/ Single doc instead of a list /
single: true,
},

comments: {
Coll: Comments,
/ post._id === comment.postId /
on: ["_id", "postId"],
},
});

/ Reversed join from user to posts /
join(Users, {
posts: {
Coll: Posts,
on: ["_id", "authorId"],
},
});
`

$3

Sometimes, the property referencing linked documents is an array (of ids, usually). In that case, the name of the array property should be nested in an array.

`js
import { join } from "coll-fns";
import { Actions, Resources } from "/collections";

/* Each action can be associated with many resources and vice-versa.
Resource's actionIds array is the link between them. /
join(Actions, {
resources: {
Coll: Resources,
on: ["_id", ["actionIds"]],
},
});

/ The reverse join will flip the property names. /
join(Resources, {
actions: {
Coll: Actions,
on: [["actionIds"], "_id"],
},
});
`

$3

Some joins should target only specific documents in the foreign collection. A complementary selector can be passed to the third on array argument.

`js
import { join } from "coll-fns";
import { Actions, Resources } from "/collections";

join(Resources, {
/ Only active tasks (third array element is a selector) /
activeTasks: {
Coll: Tasks,
on: ["_id", "resourceId", { active: true }],
},

/ All tasks associated with a resource /
tasks: {
Coll: Tasks,
on: ["_id", "resourceId"],
},
});
`

$3

The on join definition property can be an object representing a selector. It will always retrieve the same linked documents.

`js
import { join } from "coll-fns";
import { Factory, Workers } from "../collections";

join(Workers, {
/ All workers will have the same factory props. /
factory: {
Coll: Factory,
on: { name: "FACTORY ABC" },
single: true,
},
});
`

$3

When joins are too complex to be defined with an array or object (although rare), a function can be used as the on property. Each parent document will be passed to this function, which should return a selector to use on the child collection.

When using function-based joins, a fields property should be added to the join definition to declare which fields the parent document needs for the join to work:

`js
import { join } from "coll-fns";
import { Comments, Posts } from "/collections";
import { twoMonthsPrior } from "/lib/dates";

join(Posts, {
recentComments: {
Coll: Comments,
on: (post) => {
const { _id: postId, postedAt } = post;

/ This argument must be defined at runtime. /
const minDate = twoMonthsPrior(postedAt);

/ Return a selector for the Comments collection /
return {
createdAt: { $gte: minDate },
postId,
};
},
/ Parent fields needed in the join function /
fields: {
_id: 1, // Optional. _id is implicit in any fetch.
postedAt: 1,
},
},
});
`

$3

A collection can define joins on itself.

`js
import { join } from "coll-fns";
import { Users } from "/collections";

join(Users, {
friends: {
/ Use the same collection in the join definition /
Coll: Users,
on: [["friendIds"], "_id"],
},
});
`

$3

Any additional properties defined on the join (other than Coll, on, single, postFetch) will be treated as options to pass to the nested documents fetchList. It usually includes:

- limit: Maximum joined documents count
- skip: Documents to skip in the fetch
- sort: Sort order of joined documents

$3

Children documents might need to be modified (transformed, ordered, filtered...) after being fetched. The postFetch: (childrenDocs, parentDoc) => childrenDocs join definition property can be used to do so.

The second argument of the function is the parent document. If some of its properties are needed, they should be declared in the fields property to ensure they are not missing from the requested fetched fields.

`js
import { join } from "coll-fns";
import { Actions, Resources } from "/collections";
import { sortTasks } from "/lib/tasks";

join(Resources, {
tasks: {
Coll: Tasks,
on: ["_id", "resourceId"],

/ Ensure tasksOrder will be fetched /
fields: { tasksOrder: 1 },

/ Transform the joined tasks documents based on parent resource. /
postFetch(tasks, resource) {
const { tasksOrder = [] } = resource;
return sortTasks(tasks, tasksOrder);
},
},
});
`

$3

Use getJoins(Coll) to retrieve the complete dictionary of the collection's joins.

fetchList(Coll, selector, options)

Fetch documents with the ability to use collection joins.

Options:

- fields: Field projection object
- limit: Maximum number of documents
- skip: Number of documents to skip
- sort: Sort specification

In its simplest form, fetchList can be used in much the same way as Meteor's Coll.find(...args).fetch().

`js
const users = await fetchList(
Users,
{ status: "active" },
{
fields: { name: 1, email: 1 },
sort: { createdAt: -1 },
limit: 10,
skip: 0,
}
);
`

$3

Contrary to regular projection objects, they can use nested properties { car: { make: 1 } } instead of dot-string ones { car: 1, "car.make": 1 }.

The joins defined on the collections must be explicitly specified in the fields object for the children documents to be fetched. The combined presence of join or own fields determines the shape of the fetched documents.

#### Examples

$3

If this combination approach seems confusing, it is possible to define a prefix that must be explicitly used when joined documents should be used. The prefix will be removed from the returned documents.

Setting the prefix to null or undefined allows using join fields at the document root like any normal field.

`js
import { setJoinPrefix } from "coll-fns";

/ All join fields will have to be prefixed with "+" /
setJoinPrefix("+");

/ Some own fields, some join fields /
fetchList(
Posts,
{},
{
fields: {
title: 1, // <= Own

/ Join fields must be nested under the prefix key /
"+": {
author: { name: 1, birthdate: 1 }, // <= Join sub fields
},
},
}
);
`

This option could also be useful if a document can have some denormalized data with the same property name as the join. The denormalized values or the joined document would then be returned based on the use of the prefix.

If, for some reason, you need to retrieve the prefix, you can do so with getJoinPrefix(Coll).

$3

Joins can be nested to fetch deeply related data. See Hooks best practices for how hooks can be used with nested joins.

`js
import { fetchList } from "coll-fns";

const posts = fetchList(
Posts,
{},
{
fields: {
title: 1,

/ Level 1 : One-to-many join /
comments: {
text: 1,

/ Level 2 : One-to-one join /
user: {
username: 1,
},
},
},
}
);
`

`json
{
"title": "Blabla",
"comments": [
{ "text": "Nice!", "user": { "username": "foo"} },
{ "text": "Great!", "user": { "username": "bar" } }
]
},
`

$3

When a field is declared using a positive number, its value is treated as a recursion limit. This could help preventing infinite loops. The value Infinity can even be used to go as deep as possible (to exhaustion), although it involves a greater risk of infinite loops.

`js
import { join } from "coll-fns";
import { Users } from "/collections";

/ Pre-register recursive join /
join(Users, {
friends: {
Coll: Users,
on: [["friendIds"], "_id"],
},
});

fetchList(
Users,
{},
{
fields: {
name: 1,
/ Join field. Limit to 2 levels deep, reusing parent fields /
friends: 2,
},
}
);
`

$3

Documents can be transformed after fetching. Collection-level transforms are automatically applied if the protocol allows it:

Meteor:

`js
import { Mongo } from "meteor/mongo";

const Users = new Mongo.Collection("users", {
transform: (doc) => ({
...doc,
fullName: ${doc.firstName} ${doc.lastName},
}),
});
`

For a specific fetch, pass a transform option:

`js
const users = await fetchList(
Users,
{ status: "active" },
{
transform: (doc) => ({
...doc,
fullName: ${doc.firstName} ${doc.lastName},
}),
}
);
`

To skip a collection's transform, pass transform: null. Transforms are applied after joins resolve, so they have access to joined data. See Nested Joins for examples of using transforms with complex data structures.

fetchOne(Coll, selector, options)

Fetch a single document from a collection. Same behaviour as fetchList.

`js
import { fetchOne } from "coll-fns";
import { Users } from "/collections";

const user = fetchOne(
Users,
{ _id: userId },
{
fields: {
name: 1,
friends: 1, // <= Join
},
}
);
`

fetchIds(Coll, selector, options)

Fetch only the _id field of matching documents. fields option will be ignored.

`js
import { fetchOne } from "coll-fns";
import { Users } from "/collections";

const userIds = fetchIds(Users, { status: "active" });
`

exists(Coll, selector)

Check if any document matches the selector.

`js
import { fetchOne } from "coll-fns";
import { Users } from "/collections";

const hasActiveUsers = exists(Users, { status: "active" });
// Returns: true or false
`

count(Coll, selector)

Count documents matching the selector.

`js
import { fetchOne } from "coll-fns";
import { Users } from "/collections";

const activeUsersCount = count(UsersCollection, { status: "active" });
// Returns an integer
`

flattenFields(fields)

Flatten a general field specifiers object (which could include nested objects) into a MongoDB-compatible one that uses dot-notation.

`js
import { flattenFields } from "coll-fns";

const flattened = flattenFields({
name: 1,
address: {
street: 1,
city: 1,
},
});
// Result: { name: 1, 'address.street': 1, 'address.city': 1 }
`

Hooks and write operations

Hooks allow you to intercept and react to data mutations (insertions, updates, removals) on collections. They are triggered conditionally before or after write operations, making them ideal for validation, cascading updates, logging, and other side effects.

hook(Coll, hooksObj)

Register hooks on a collection to run before or after specific write operations (insert, update, remove).

The same object argument can define multiple hook types. Each hook type is defined using an array of hook definitions, making it possible to define multiple hooks at once.

Hooks can be defined in multiple places in your codebase. This allows grouping functionally related hooks together.

`js
import { hook } from "coll-fns";
import { Users, Posts } from "/collections";

hook(Users, {
beforeInsert: [
{
fn(doc) {
if (!doc.email) throw new Error("Email is required");

doc.createdAt = new Date();
},
},
],

onInserted: [
{
fn(doc) {
console.log(New user created: ${doc._id});
},
},
],
});
`

$3

These hooks run before the write operation and can prevent the operation by throwing an error.

- beforeInsert: Runs before inserting a document. Receives (doc).
- beforeUpdate: Runs before updating documents. Receives ([...docsToUpdate], modifier).
- beforeRemove: Runs before removing documents. Receives ([...docsToRemove]).

Although arguments can be mutated, it is not the main purpose of these hooks. Mutations are brittle and hard to debug.

beforeUpdate and beforeRemove receive an array of targeted documents, whereas beforeInsert receives a single document.

$3

These hooks run after the write operation completes and are fire-and-forget (not awaited). They should not throw errors that should get back to the caller.

- onInserted: Runs after a document is inserted. Receives (doc).
- onUpdated: Runs after a document is updated. Receives (afterDoc, beforeDoc).
- onRemoved: Runs after a document is removed. Receives (doc).

$3

Each hook definition is an object with the following properties:

`js
{
/* Required. The function to execute.
Arguments depend on the hook type (see above). /
fn(...args) { / ... / },

/* Optional. Fields to fetch for the documents passed to the hook.
* Fields for multiple hooks of the same type are automatically combined.
* If any hook of a type requests all fields with undefined or true,
* all other similar hooks will also get the entire documents.
Has no effect on beforeInsert: the doc to be inserted is the argument. /
fields: { name: 1, email: 1 },

/* Optional (onUpdated only). If true, fetch the document state
* before the update with the same fields value.
* Otherwise, only _id is fetched initially (they would have been
needed anyway to fetch their "after" versions). /
before: true,

/* Optional. Predicate that prevents the hook from running if it
returns a truthy value. Receives the same arguments as fn. /
unless(doc) { return doc.isBot; },

/* Optional. Predicate that allows the hook to run only if it
returns a truthy value. Receives the same arguments as fn. /
when(doc) { return doc.status === "pending"; },

/* Optional handler called if the hook function throws an error.
* A default handler that logs to console.error is defined
* for after-hooks (onInserted, onUpdated, onRemoved)
to prevent an error from crashing the server. /
onError(err, hookDef) { / ... / },
}
`

$3

The data mutation methods below use basically the same arguments as Meteor collection methods.

insert(Coll, doc)

Insert a document into a collection. Returns the document \_id. Runs beforeInsert and onInserted hooks if defined.

`js
const newUser = insert(Users, {
name: "Bob",
email: "bob@example.com",
});
`

Execution flow:

1. Run beforeInsert hooks (can throw to prevent insertion)
2. Insert the document
3. Fire onInserted hooks asynchronously (without awaiting)

update(Coll, selector, modifier, options)

Update documents matching the selector. Returns the number of documents modified. Runs beforeUpdate and onUpdated hooks if defined. Updates multiple documents by default (unlike Meteor's behavior).

`js
update(Users, { status: "pending" }, { $set: { status: "active" } });
`

Execution flow:

1. Fetch target documents with beforeUpdate and onUpdated.before fields
2. Run beforeUpdate hooks with (docsToUpdate, modifier) (can throw to prevent update)
3. Execute the update
4. Fetch updated documents again with onUpdated fields
5. Fire onUpdated hooks asynchronously with (afterDoc, beforeDoc) pairs

Options:

- multi (default: true): Update multiple documents or just the first match;
- arrayFilters: Optional. Used in combination with MongoDB filtered positional operator to specify which elements to modify in an array field.

remove(Coll, selector)

Remove documents matching the selector. Runs beforeRemove and onRemoved hooks if defined.

`js
remove(Users, { inactive: true });
`

Execution flow:

1. Fetch documents matching the selector with beforeRemove and onRemoved fields
2. Run beforeRemove hooks with matched documents (can throw to prevent removal)
3. Remove the documents
4. Fire onRemoved hooks asynchronously with each removed document

Hook best practices

License

MIT