@minikit/mongotools

This project provides 2 wrappers :

- mongodump,
- mongorestore.

There is an autonomous feature called rotation that provide a backup file rotation mechanism

- remove N oldest deprecated backups.

Command line usage

$3

``


get source code

git clone https://github.com/minikit/mongotools.git

install dependencies

npm install

#~ setup environment variables
cp env/initEnv.template.sh env/initEnv.dontpush.sh

you must update env/initEnv.dontpush.sh

`

$3

`


source your options


. ./env/initEnv.dontpush.sh
`

$3

`bash


create a mongo dump

node mt dump

create a encrypted mongo dump

node mt dumpz

list backups

node mt list

restore a mongo local dump

please note that mt restore use following options : dropBeforeRestore: true, deleteDumpAfterRestore: true

node mt restore backup/myDatabase__2020-11-08_150102.gz

rotate backup files

node mt rotation

Helper : show current options values

node mt options
`

$3

You could play with env default options plus in-line command extra options

`bash


create dump of a given 'shippingprices' collection, provide a target backup name as '2023_01_shippingPrices.gz', and show mongodump command

MT_COLLECTION=shippingprices MT_FILENAME=2023_01_shippingPrices.gz MT_SHOW_COMMAND=true node mt dump

show backup in list

node mt list

using mongo: drop a given collection

mongo myDb --eval "db.shippingprices.drop()"

restore collection

MSYS_NO_PATHCONV=1 MT_COLLECTION=shippingprices MT_SHOW_COMMAND=true node mt restore /backup/2023_01_shippingprices.gz

Note that collection option will produce wildcard in nsInclude arg '--nsInclude myDb.*'

`

Library use

$3

You have to import as dependency

`
npm install @minikit/mongotools
`

$3

`
import { MongoTools, MTOptions, MTCommand } from "@minikit/mongotools";

var mongoTools = new MongoTools();
const mtOptions = {
db: 'myDb',
port: 17017,
path: '/opt/myapp/backups',
dropboxToken: process.env.MYAPP_DROPBOX_SECRET_TOKEN
};
`

$3

`
var promiseResult = mongoTools.list(mtOptions);
`

$3

`
var promiseResult = mongoTools.mongodump(mtOptions);
`

$3

`
var promiseResult = mongoTools.mongorestore(mtOptions);
`

$3

`
var promiseResult = mongoTools.rotation(mtOptions);
`

Mongo tools options

Each mongotools feature rely on Mongo tools options (aka. MTOption).

Options precedence is the following:

- take options attribute if set,
- else take related environment variable if any,
- else take default value if any,
- else if it's a mandatory option, throw an error.

TIP: you could also show current options by doing:

`
console.log(new MTOptions());
`

$3

These options are used by dump and restore.

Either uri or host/port/db:

| option | env | required | default value | description |
| ------ | ------------ | -------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| uri | MT_MONGO_URI | true | (none) | mongodump uri, example mongodb+srv://granted-user:MySecretHere@cluster0.xzryx.mongodb.net/tMyDatababse. You could omit database name to dump all databases. |

or

| option | env | required | default value | description |
| ---------- | ---------------- | -------- | ------------- | --------------------------------------------------------------------------------- |
| db | MT_MONGO_DB | true | (none) | mongo database name. For dump only, you could set it to * to dump all databases |
| host | MT_MONGO_HOST | false | 127.0.0.1 | mongo database hostname |
| port | MT_MONGO_PORT | false | 27017 | mongo database port |
| username | MT_MONGO_USER | false | (none) | mongo database username |
| password | MT_MONGO_PWD | false | (none) | mongo database password |
| authDb | MT_MONGO_AUTH_DB | false | admin | mongo auth database |

#### ssl options

Optional ssl related options

| option | env | required | default value | description |
| ------------------- | ----------------------------- | -------- | ------------- | ---------------------------------------------------------------- |
| ssl | MT_MONGO_SSL | false | (none) | if "1" then add --ssl option |
| sslCAFile | MT_MONGO_SSL_CA_FILE | false | (none) | .pem file containing the root certificate chain |
| sslPEMKeyFile | MT_MONGO_SSL_PEM_KEY_FILE | false | (none) | .pem file containing the certificate and key |
| sslPEMKeyPassword | MT_MONGO_SSL_PEM_KEY_PASSWORD | false | (none) | password to decrypt the sslPEMKeyFile, if necessary |
| sslCRLFile | MT_MONGO_SSL_CRL_FILE | false | (none) | pem file containing the certificate revocation list |
| sslFIPSMode | MT_MONGO_SSL_FIPS | false | (none) | if "1" then use FIPS mode of the installed openssl library |
| tlsInsecure | MT_MONGO_TLS_INSECURE | false | (none) | if "1" then bypass the validation for server's certificate chain |

$3

| option | env | required | default value | description |
| ------------------------ | ---------------------- | -------- | ----------------------- | --------------------------------------------------------------------- |
| path | MT_PATH | false | backup | dump target directory, created if it doesn't exist |
| dumpCmd | | false | mongodump | mongodump binary |
| fileName | MT_FILENAME | false | | dump target filename |
| encrypt | | false | false | encrypt the dump using secret |
| secret | MT_SECRET | false | null | secret to use if encrypt is enabled |
| encryptSuffix | | false | .enc | encrypt file suffix |
| includeCollections | | false | (none) | Deprecated - please use collection |
| collection | MT_COLLECTION | false | (none) | Collection to include, if not specified all collections are included |
| excludeCollections | MT_EXCLUDE_COLLECTIONS | false | (none) | Collections to exclude, if not specified all collections are included |
| numParallelCollections | | false | 4 | Number of collections mongodump should export in parallel. |
| viewsAsCollections | | false | false | When specified, mongodump exports read-only views as collections. |

Simple example:

`
import { MongoTools, MTOptions, MTCommand } from "node-mongotools";
var mongoTools = new MongoTools();

mongoTools.mongodump({
db:'myDatabase',
path:'backup',
username:'root', password:'mypass', authDb:'admin'
})
.then((success) => console.info("success", success) )
.catch((err) => console.error("error", err) );
`

$3

| option | env | required | default value | description |
| ------------------------ | ------------ | -------- | -------------- | -------------------------------------------------------------- |
| dumpFile | MT_DUMP_FILE | true | (none) | dump file to restore |
| restoreCmd | | false | mongorestore | mongorestore binary |
| dropBeforeRestore | | false | false | set it to true to append --drop option |
| deleteDumpAfterRestore | | false | false | set it to true to remove restored backup file |
| decrypt | | false | false | decrypt the dump using secret. Activated if suffix is detected |
| secret | MT_SECRET | false | null | secret to use if decrypt is enabled |

Simple example:

`
import { MongoTools, MTOptions, MTCommand } from "node-mongotools";
var mongoTools = new MongoTools();

mongoTools.mongorestore({
dumpFile:'backup/myDatabase__2020-11-8_160011.gz',
username:'root', password:'mypass', authDb:'admin'
})
.then((success) => {
console.info("success", success.message);
if (success.stderr) {
console.info("stderr:\n", success.stderr);// mongorestore binary write details on stderr
}
})
.catch((err) => console.error("error", err) );
`

$3

A safe time windows is defined by :

- now - rotationWindowsDays day(s) ===> now
where backups can't be removed.

Backup out of safe time windows are called deprecated backup.

- rotationMinCount: minimum deprecated backups to keep,
- rotationCleanCount: number of (oldest) deprecated backups to delete.

| option | env | required | default value | description |
| --------------------- | ------------------------ | -------- | ------------- | ------------------------------------------------------ |
| rotationDryMode | MT_ROTATION_DRY_MODE | false | false | dont do delete actions, just print it |
| rotationWindowsDays | MT_ROTATION_WINDOWS_DAYS | true | 15 | safe time windows in days since now |
| rotationMinCount | MT_ROTATION_MIN_COUNT | true | 2 | minimum deprecated backups to keep. |
| rotationCleanCount | MT_ROTATION_CLEAN_COUNT | true | 10 | number of (oldest first) deprecated backups to delete. |

Simple example:

`
MT_ROTATION_CLEAN_COUNT=2 \
MT_ROTATION_DRY_MODE=true \
MT_ROTATION_WINDOWS_DAYS=3 \ node mt rotation
``

Example details: if there is a backup that is more than 3 days old, keep 2 newer ones and delete the 10 oldest.