A mongo driver thin wrapper to be used within custom applications for Itential Automation Platform
This 'dbmanager' module makes it easy to connect to mongo from a custom-built
app within Itential Automation Platform (IAP).
This is an _extremely_ thin wrapper over mongo's native node driver:
https://docs.mongodb.com/ecosystem/drivers/node/. The only reason this exists
is so that:
1. You have a standardized way to store/manage mongo config within Itential Automation Platform (IAP)
1. You don't have to know/research any best practices for connecting to mongo
1. You can copy/paste code examples from example-cog.js for most common use cases
* downsides of a mongo adapter
* data type serialization (no native Date objects for example)
* very unscalable for large IAP ecosystems
* limited API
* upsides of a mongo adapter
* available in workflow builder
Inside your custom app's code, run npm install --save
@itentialopensource/dbmanager
Also, the app or service using dbmanager must install the desired version of
mongodb. Running npm install --save mongodb will install latest version of
mongodb driver.
Then make sure to follow the "Configuring dbmanager" section below.
Then see the example cog file for examples on how to use
dbmanager from your app's javascript code.
You can view the API docs
There are two things the mongo node driver (and therefore dbmanager) needs to
connect to mongo:
* a connection string
* connection options (an object)
If you configure your connection settings in the right place (see below
sections), then dbmanager will automatically detect those settings without you
having to specify where they are.
Here's an example dbmanager configuration if your mongo server is using ssl and
replica sets. You'll notice that the password is encrypted using IAP's
encrypt.js script. To use that script, log into a machine that has IAP
installed, then cd into
/opt/itential/current/node_modules/@itential/pronghorn-core/utils, and then runnode encrypt.js your-password-here. The script will print an encrypted value
of that password which you can copy/paste.
Where exactly you put this config depends on the version of IAP you're running
(see sections below).
``
"dbManagerConfig" {
"default": {
"connectionString": 'mongodb://node1:27017,node2:27017,node2:27017?replicaSet=rs0',
"connectionOptions": {
"auth": {
"user": "customerMongoUser",
"password": "$ENC93eb9439537ae34196db49409dd0261a8b87218fafd0419"
}
"ssl": true,
"sslValidate": false,
"sslCA": './keys/mongodb-server.pem',
"checkServerIdentity": true
}
},
}
You do not need to specify the database name being used. When you connect to
this from javascript code, you specify there which database name you want to
connect to. This way you can use multiple databases from within the same
connection.
Notice that this format differs from what you would put into IAP's mongoProps.
The reason it differs is because we want to be a pass-through to mongo's native
node API, and MongoProps data format is a customized and different than the
format mongodb's node driver expects. Because dbmanager is a pass-through to
mongodb, this allows you, as the user of dbmanager, to have the full power of
mongo's node driver at your disposal.
See http://mongodb.github.io/node-mongodb-native/3.2/reference/connecting/ for
more details about how 'connectionString' and 'connectionOptions' is formatted.
If you have IAP version 2019.2 or later, then you can edit properties specific to each app you write.
* Open up Itential
* Log in as an admin
* Go to Settings > Services
* Select your app from the services list that appears on the left
add a new "dbManagerConfig" section to the _properties*_ object
Therefore, when you're finished adding "dbManagerConfig" to your app's service
properties, the entire service config you see on screen will look something
like below. Notice that "dbManagerConfig" is _inside_ the "properties" section.
`
{
"loggerProps": {
"description": "Logging",
"log_max_files": 100,
"log_max_file_size": 1048576,
"log_level": "warn",
"log_directory": "/var/opt/logs",
"log_filename": "MyApp.log",
"console_level": "trace"
},
"isEncrypted": true,
"model": "@custom/my-app",
"name": "MyApp",
"type": "Application",
"properties": {
"dbManagerConfig": {
"default": {
"connectionString": "mongodb://127.0.0.1:27017",
"connectionOptions": {}
},
"connection2": {
"connectionString": "mongodb://some.other.host",
"connectionOptions": {}
}
}
},
"rabbitmq": {
"protocol": "amqp",
"port": 5672,
"username": "username",
"password": "password",
"locale": "en_US",
"frameMax": 0,
"heartbeat": 0,
"vhost": "/",
"certPath": "",
"keyPath": "",
"passphrase": "passphrase",
"caPath": "",
"hosts": [
"localhost"
]
}
}
For IAP versions prior to 2019.2, IAP reads all configuration from a file
called properties.json. The dbmanager tool will look inside properties.json for
a top-level property called "dbManagerConfig".
Therefore, you must edit properties.json file to include a "dbManagerConfig"
section. You must edit the properties.json file for _every_ IAP server (if you
are using Itential HA, this will be multiple machines). The default location is
/opt/itential/current/properties.json .
When you are finished adding dbManagerConfig to properties.json, the
properties.json file will look something like this:
`
{
"dbManagerConfig": {
"default": {
"connectionString": 'mongodb://node1:27017,node2:27017,node2:27017?replicaSet=rs0'
"connectionOptions": {
"auth": {
"user": "customerMongoUser"
"password": "$ENC:abcdef12345"
}
"ssl": true,
"sslValidate": false,
"sslCA": './keys/mongodb-server.pem'
"checkServerIdentity": true,
"dbName": 'customer_database',
}
},
"connection2": {
"connectionString": "mongodb://127.0.0.1:27017",
"connectionOptions": {},
}
},
"applicationProps": {
"directory": "./node_modules/"
},
"uiProps": {
"description": "UI",
"layout": "node_modules/@itential/pronghorn-core/ui/views/layout.jade",
"home": "node_modules/@itential/pronghorn-core/ui/views/home.jade",
"login": "node_modules/@itential/pronghorn-core/ui/views/login.jade",
"profile": "node_modules/@itential/pronghorn-core/ui/views/profile.jade",
"user_config": "node_modules/@itential/pronghorn-core/ui/views/user_config.jade",
"group_config": "node_modules/@itential/pronghorn-core/ui/views/group_config.jade",
"new_user": "node_modules/@itential/pronghorn-core/ui/views/dialogs/new_user.jade",
"edit_user": "node_modules/@itential/pronghorn-core/ui/views/dialogs/edit_user.jade",
"new_group": "node_modules/@itential/pronghorn-core/ui/views/dialogs/new_group.jade",
"fav_icon": "node_modules/@itential/pronghorn-core/ui/img/favicon.ico"
},
...
`
2019.2 fundamentally improved how all configuration works in IAP.
Therefore, if you are _migrating_ from 2019.1 (or earlier) to 2019.2 or later,
IAP will come with a migration script. This script will take any properties
inside properties.json and add them to a "profile" (which resides directly in
mongo; not a file anymore). If you have a running instance of IAP using 2019.2
or later, you can log in as an admin and navigate to Settings > Profiles to see
profile config.
The dbmanager tool is smart enough to "just work" in this situation. If you
already configured dbmanager for 2019.1, and you run the migration script,
dbamanager will know to look inside the active "profile" and find all your
pre-existing dbManagerConfig configuration without you having to do anything
(other than run the migration script).
However, the intent of 2019.2 is to have every app ("service") contain its own
configuration. So you will probably want to update each of your apps'
configuration with dbManagerConfig at some point.
Any dbManagerConfig you set _for that specific service_ (that is, set within a
service's config) will override anything migrated into the "profile" config.
There are a few more advanced ways to use this module, but they require a bit
more understanding to ensure you don't negate any value from caching
connections. It's expensive to keep opening/closing connections, so dbmanager
caches connections. Therefore, you should never use the following function
unless you understand how it affects caching.
You can attach to mongo events via
`
const client = getConnection();
client.on(...)
If you want to manually close a client, use the closeConnection function.
Your code contribution is welcomed! To contribute:
* You must run npm test` locally (with docker installed to run integration
tests) and everything must pass before being accepted into master. The
pipeline will fail if tests do not pass.
* See development notes
Itential, LLC proprietary