ElasticSearch 9 Node.js ODM library
Basic information
- This library maps ES documents and functions to JS classes
$3
- ODM models are represented via ES aliases
- Calling functions like
createIndex() will create an underlying index and also sets an alias into it
- Most of the communication is accomplished via aliases, real indices are used only when necessary
- This is to simplify some processes like reindexing/cloning/migrating from source (old) index into the new one
- Thanks to this, we can create new index, migrate data from the original one, than switch the aliases and delete the original one.
- And the index are still in human-readable format
- Alias consist from two parts:
- tenant (AKA prefix) - defaults to
*
- name - main index name
- Final alias looks like this:
-
tenant_name
#### Work with aliases
- Alias name is specified when model is created, like this -
createClass('myIndex')
- Tenant can be specified using
createClass('myIndex', void 0, 'myTenant')
- or later by using
.in('myTenant')
- Function
in('tenant') (and
immediateRefresh(bool) as well) creates new ODM model with updated values, original ODM remains unchanged
- example:
-
createClass('myIndex').in('default') ->
default_myIndex
-
createClass('myIndex') ->
*_myIndex
-
createClass('myIndex', void 0, 'tenant') ->
tenant_myIndex
-
createClass('myIndex', void 0, 'tenant').in('changedTenant') ->
changedTenant_myIndex
- When creating a new instance (or performing some operations), the alias cannot contain any wildcard
- it means you must fully specify the tenant
$3
- Underlying index is in a very similar format, just also contains unique identifier:
-
tenant_name-id
- This library is made to use aliases, but technically speaking it should be also working with ES indices directly (or even with mixture of indices and aliases)
- It is necessary to avoid any conflicts between indices / aliases
- Newly created indices will always be created as aliases with underlying indices
#### Usage
-
import { createClass, BulkArray, BaseModel, JointModel, esClient, setClient, esErrors, setLoggerConfig, setLoggerUidFunction, setConfiguration } from 'es-odm';
-
createClass function creates new class
- Each class contains several properties
-
schema,
alias,
_tenant,
_name and
_immediateRefresh
- Class static functions profit from those properties
- e.g.
static search() uses them to get alias
- You may write your own static function, which will use those properties
- Or rewrite existing functions
- Which may influence another functions
- Or redefine class properties
- Instances have access to those properties / functions
-
myInstance.constructor.alias
- And they use them during ES operations
- eq. to validate data, ...
- Whenever we need new / independent class, we must either create a new one or clone an existing one
- Additionally, library provides
BulkArray
- Class inherited from
Array
- Contains methods which uses ES Bulk API
-
save,
delete and
reload
- Plus some additional methods to dynamically mark and remove ODM instance - see in code
- Exported
BaseModel should be used mainly for
instanceof checks
- Do not create instances from it / do not change its parameters (unless you really know what you are doing)
-
JointModel is a special class used to "record" searches from multiple distinct ODM classes and then perform single search with different queries above multiple indices
-
setClient replaces ES client singleton
- Should be called once at application startup
- New client is then used - even in already created classes
-
esErrors - Errors from underlying ES library
-
setLoggerConfig - Replaces default logger configuration
-
setLoggerUidFunction - Pass custom function to be used to generate ID, which is passed to generated logs
-
setConfiguration - Pass some additional configuration
#### Class usage
##### Create new class
- To create new class, call
createClass(name, ?schema, ?tenant):
-
name - essential, it must be specified
-
schema - may be undefined or Joi object, it is there, so we can do additional validations
-
tenant - Serves as part of the index (prefix)
- This is required and cannot contain underscores (
_)
- By default, it is
*
- You can rewrite it later by using
.in('newTenant')
- It will create inherited class
-
const SpecificTenantClass = MyClass.in('newTenant');
- You must specify it before creating new instance
- When searching for records, found records will have correctly set tenant
##### Modify / clone existing class
- Class can be modified / cloned using:
-
in(tenant)
- Returns class clone with tenant changed to given value
-
immediateRefresh(bool)
- Returns class clone with immediate refresh set to given value
- By default, new classes (and its instances) does perform
refresh: true with all write operations -> You can use this to disable it
-
clone(?changes)
- Clones class
-
changes is optional object to set cloned class properties
- This method is internally used by
in() and
immediateRefresh()
- You should not need to use it
- Cloning class means:
- New inherited class is created
- Changes made afterwards to cloned class are NOT transferred to original one
#### Instances
- Instances are made from prepared class
- Manually:
- You prepare class and then you can call
new MyClass(?data, ?_id, ?_version, ?_highlight, ?_primary_term, ?_seq_no, ?_score, ?_sort)
-
data is optional object whose properties are placed to instance
-
_id is optional ES _id
-
_version is optional ES _version
-
_highlight is optional ES highlight
-
_primary_term is optional ES _primary_term
-
_seq_no is optional ES _seq_no
-
_score is optional ES _score
-
_sort is optional ES sort
- From static functions:
- When you call functions like
findAll(),
search(), ...
- Instance is made from class and correct data is loaded from ES
- Instance contains only ES data and methods to save / reload / validate / ...
- All ES properties, search functions, ... are saved in class
- NOTE:
_id,
_version,
_highlight,
_primary_term,
_seq_no,
_score and
_sort are not enumerable
#### BulkArray
- For ES Bulk API, you can use
BulkArray
- Class inherited from
Array
- All static search functions in BaseModel class returns
BulkArray of instances instead of
Array
- Provides bulk functions:
-
async save(?useVersion)
- Saves all items to ES
- Not existing ids are generated and pushed to instances
- Returns ES response
-
useVersion - Checks if version match
- uses sequence numbers internally, if not presented, it will fetch them and checks version automatically
-
async delete(?useVersion)
- Deletes all items from ES
- Returns ES response
-
useVersion - Checks if version match
- uses sequence numbers internally, if not presented, it will fetch them and checks version automatically
-
async reload()
- Reloads all instances in bulk array
#### Examples:
``
class Counter extends createClass('counter', Joi.object()) {
async customFunction() {
return 666;
}
static customStatic() {
return new this({ a: 4 }, myId
);
}
static async search(...args) {
// Rewritting static function -> affects search, find and findAll
const results = await super.search(...args);
if (results.length <= 0) {
throw Error(Nothing
);
} else {
return results;
}
}
}
...
const MyCounter = Counter.in('default'); //Index is 'default_counter'
const allCounters = await MyCounter.findAll(); //BulkArray with all counters in index
await allCounters.delete(); //Everything deleted from ES
const instanceCounter = new MyCounter({ counter: 15 }, 'myId'); //Prepared new instance
await instanceCounter.save(); //New instance saved
`
#### Functions / Methods
##### Internal getters
- static get alias
- Returns class full alias, usable for ES queries
- It consists of tenant and name
##### Class level API
- static async search(body, ?from, ?size, ?additional)
- Performs ES search
- Returns BulkArray
of instances
- Used by another static functions
- Redefining this function will affect their behavior
- User must specify body
- alias
is already defined in the class
- from
and size
are optional
- Returns all results if from
/ size
are not specified, no matter how many there are
- additional
is an optional object with additional data
- static async *bulkIterator(body, ?source, ?bulkSize, ?additional)
- Async generator for iterating over bulks of documents
- body
, source
and additional
arguments are the same as in the search
function
- bulkSize
is optional custom size of a bulk
- static async *itemIterator(body, ?source, ?bulkSize, ?additional)
- Async generator for iterating over documents
- Arguments are the same as in the bulkIterator
generator
- static async findAll()
- Finds all entries in ES, matching class alias
- Uses this.search()
- Returns BulkArray
- static async find(ids)
- Performs ES 'search' query
- Always returns BulkArray
of instances
- Uses this.search()
- static async get(ids)
- Performs ES 'get'
- If 'ids' is strings, returns single instance
- Else if 'ids' is an array of strings, returns BulkArray
of instances
- static async head(ids)
- Performs ES 'mget' without '_source' field
- Throws when any of the IDs is not found
- If 'ids' is strings, returns single object
- Else if 'ids' is an array of strings, returns array of objects
- static async delete(ids, ?version)
- Performs ES 'delete'
- Uses bulk API
- Class alias must be fully specified
- Returns ES response
- If version
is specified, only one id is allowed
- static async exists(ids)
- Performs ES 'exists'
- If 'ids' is strings, returns single boolean
- Else if 'ids' is array of strings, returns array of booleans
- true if exists, else otherwise
- static async update(ids, body)
- Performs ES 'update'
- Uses bulk API
- Returns ES response
- static async count(?body)
- Performs ES 'count'
- Returns number
- static async updateByQuery(body)
- Performs ES 'update_by_query'
- Returns ES response
- static async deleteByQuery(body)
- Performs ES 'delete_by_query'
- Returns ES response
##### Indices API
- static async createIndex(?body, ?setAlias = true)
- Creates index given by current class
- body
is optional settings
- setAlias
is used when we do not want automatically set an alias to newly create index
- static async getIndex()
- Returns ES index of ODM alias
- Returns string
- static async aliasIndex(index)
- Puts alias of current ODM onto selected index
- static async deleteAlias()
- Deletes ODM alias from ES, underlying index remains unchanged
- static async aliasExists()
- Checks ODM alias existence
- Returns boolean
- static async indexExists()
- Checks ODM index existence
- Returns boolean
- static async deleteIndex()
- Deletes alias and index given by current class
- static async getMapping()
- Gets mapping of index given by current class
- static async putMapping(mapping)
- Puts mapping to index given by current class
- static async getSettings(?includeDefaults = false)
- Gets settings of index given by current class
- static async putSettings(settings)
- Puts settings to index given by current class
- static async reindex(destinationModel, ?script)
- Reindex from current model class to selected one
- Destination may be chosen by ODM or by alias/index string
- script
specifies optional painless script to be used
- static async cloneIndex(?settings)
- Creates clone of ODM index
- Current index has to be manually blocked for write (be made read-only)
- settings
is optional settings to be used for newly created index
- Returns newly created index
- static async refresh()
- Refreshed current index / indices
- static async openPIT()
- Opens Point in Time in given index
- static async closePIT(id)
- Closes given Point in Time
##### Instance level API
- async save(?useVersion)
- saves or re-saves instance
- it uses specified _id
or generates new one if not specified
- it uses ES 'index' function
- useVersion
- checks if version matches, _id
and _version
must be specified
- sequence number will be fetched automatically in not presented
- async reload()
- Reloads instance data from ES
- Uses get
and new this.constructor()
internally
- async delete(?useVersion)
- Deletes an instance from ES
- _id
must be specified and entry must exist in ES
- useVersion
- checks if version matches, _version
must be specified
- sequence number will be fetched automatically in not presented
- clone(?_preserveAttributes)
- Returns clone of current instance
- Deep copy
- Cloned instance is created from the same class
- preserveAttributes
(defaults true) can be used to preserve attributes (_id
, _version
, ...)
- async validate()
- Validates instance using Joi
- Throws in case of error / incorrect data
##### Class copy
- static clone(?changes)
- Creates class copy
- In fact, it creates new class which inherits from the current one
- changes
is an optional object with properties to be set
- static in(newTenant)
- Clones class using clone()
- Sets given tenant
- static immediateRefresh(newRefresh)
- Clones class using clone()
- Sets given refresh mode
#### Other
- static _parseIndex(index)
- Parses given index (or even alias) and returns its parts
- static __checkIfFullySpecified(functionName)
- Checks if current class is fully specified, meaning it doesn't contain any wildcard in alias
- Throws otherwise
- static __getConstructor(searchResult, constructorCache)
- Returns the correct class (constructor) for given search result
- static _alterSearch(body)
- Alters the search function query (and also query of other functions that use the query)
- By default, just passes the body without any change, but can be rewritten
- static _unpackData(source)
- Alters the fetched ES documents before the instance is created from them
- By default, just passes the source without any change, but can be rewritten
- async _packData(cache)
- Alters the instance data before it is saved to ES
- By default, just passes the instance data without any change, but can be rewritten
- static async _afterSearch(instances, ?cache)
- Special function called for each record founded by search / find / findAll / get
- By default, it is empty, but you can overwrite it in your code
- static async _getBulkSize()
- Returns optimal size of bulk for given model
- Used in search` function for pagination