ElasticSearch 9 Node.js ODM library

Basic information

- This library maps ES documents and functions to JS classes

$3

- ODM models are represented via ES aliases
- Calling functions like createIndex() will create an underlying index and also sets an alias into it
- Most of the communication is accomplished via aliases, real indices are used only when necessary
- This is to simplify some processes like reindexing/cloning/migrating from source (old) index into the new one
- Thanks to this, we can create new index, migrate data from the original one, than switch the aliases and delete the original one.
- And the index are still in human-readable format

- Alias consist from two parts:
- tenant (AKA prefix) - defaults to *
- name - main index name
- Final alias looks like this:
- tenant_name

#### Work with aliases
- Alias name is specified when model is created, like this - createClass('myIndex')
- Tenant can be specified using createClass('myIndex', void 0, 'myTenant')
- or later by using .in('myTenant')
- Function in('tenant') (and immediateRefresh(bool) as well) creates new ODM model with updated values, original ODM remains unchanged
- example:
- createClass('myIndex').in('default') -> default_myIndex
- createClass('myIndex') -> *_myIndex
- createClass('myIndex', void 0, 'tenant') -> tenant_myIndex
- createClass('myIndex', void 0, 'tenant').in('changedTenant') -> changedTenant_myIndex

- When creating a new instance (or performing some operations), the alias cannot contain any wildcard
- it means you must fully specify the tenant

$3

- Underlying index is in a very similar format, just also contains unique identifier:
- tenant_name-id

- This library is made to use aliases, but technically speaking it should be also working with ES indices directly (or even with mixture of indices and aliases)
- It is necessary to avoid any conflicts between indices / aliases
- Newly created indices will always be created as aliases with underlying indices

#### Usage
- import { createClass, BulkArray, BaseModel, JointModel, esClient, setClient, esErrors, setLoggerConfig, setLoggerUidFunction, setConfiguration } from 'es-odm';
- createClass function creates new class
- Each class contains several properties
- schema, alias, _tenant, _name and _immediateRefresh
- Class static functions profit from those properties
- e.g. static search() uses them to get alias
- You may write your own static function, which will use those properties
- Or rewrite existing functions
- Which may influence another functions
- Or redefine class properties
- Instances have access to those properties / functions
- myInstance.constructor.alias
- And they use them during ES operations
- eq. to validate data, ...
- Whenever we need new / independent class, we must either create a new one or clone an existing one
- Additionally, library provides BulkArray
- Class inherited from Array
- Contains methods which uses ES Bulk API
- save, delete and reload
- Plus some additional methods to dynamically mark and remove ODM instance - see in code
- Exported BaseModel should be used mainly for instanceof checks
- Do not create instances from it / do not change its parameters (unless you really know what you are doing)
- JointModel is a special class used to "record" searches from multiple distinct ODM classes and then perform single search with different queries above multiple indices
- setClient replaces ES client singleton
- Should be called once at application startup
- New client is then used - even in already created classes
- esErrors - Errors from underlying ES library
- setLoggerConfig - Replaces default logger configuration
- setLoggerUidFunction - Pass custom function to be used to generate ID, which is passed to generated logs
- setConfiguration - Pass some additional configuration


#### Class usage

##### Create new class
- To create new class, call createClass(name, ?schema, ?tenant):
- name - essential, it must be specified
- schema - may be undefined or Joi object, it is there, so we can do additional validations
- tenant - Serves as part of the index (prefix)
- This is required and cannot contain underscores (_)
- By default, it is *
- You can rewrite it later by using .in('newTenant')
- It will create inherited class
- const SpecificTenantClass = MyClass.in('newTenant');
- You must specify it before creating new instance
- When searching for records, found records will have correctly set tenant

##### Modify / clone existing class
- Class can be modified / cloned using:
- in(tenant)
- Returns class clone with tenant changed to given value
- immediateRefresh(bool)
- Returns class clone with immediate refresh set to given value
- By default, new classes (and its instances) does perform refresh: true with all write operations -> You can use this to disable it
- clone(?changes)
- Clones class
- changes is optional object to set cloned class properties
- This method is internally used by in() and immediateRefresh()
- You should not need to use it

- Cloning class means:
- New inherited class is created
- Changes made afterwards to cloned class are NOT transferred to original one

#### Instances
- Instances are made from prepared class
- Manually:
- You prepare class and then you can call new MyClass(?data, ?_id, ?_version, ?_highlight, ?_primary_term, ?_seq_no, ?_score, ?_sort)
- data is optional object whose properties are placed to instance
- _id is optional ES _id
- _version is optional ES _version
- _highlight is optional ES highlight
- _primary_term is optional ES _primary_term
- _seq_no is optional ES _seq_no
- _score is optional ES _score
- _sort is optional ES sort
- From static functions:
- When you call functions like findAll(), search(), ...
- Instance is made from class and correct data is loaded from ES
- Instance contains only ES data and methods to save / reload / validate / ...
- All ES properties, search functions, ... are saved in class
- NOTE: _id, _version, _highlight, _primary_term, _seq_no, _score and _sort are not enumerable

#### BulkArray
- For ES Bulk API, you can use BulkArray
- Class inherited from Array
- All static search functions in BaseModel class returns BulkArray of instances instead of Array
- Provides bulk functions:
- async save(?useVersion)
- Saves all items to ES
- Not existing ids are generated and pushed to instances
- Returns ES response
- useVersion - Checks if version match
- uses sequence numbers internally, if not presented, it will fetch them and checks version automatically

- async delete(?useVersion)
- Deletes all items from ES
- Returns ES response
- useVersion - Checks if version match
- uses sequence numbers internally, if not presented, it will fetch them and checks version automatically

- async reload()
- Reloads all instances in bulk array

#### Examples:
``
class Counter extends createClass('counter', Joi.object()) {
async customFunction() {
return 666;
}

static customStatic() {
return new this({ a: 4 }, myId);
}

static async search(...args) {
// Rewritting static function -> affects search, find and findAll
const results = await super.search(...args);
if (results.length <= 0) {
throw Error(Nothing);
} else {
return results;
}
}
}

...

const MyCounter = Counter.in('default'); //Index is 'default_counter'

const allCounters = await MyCounter.findAll(); //BulkArray with all counters in index
await allCounters.delete(); //Everything deleted from ES

const instanceCounter = new MyCounter({ counter: 15 }, 'myId'); //Prepared new instance
await instanceCounter.save(); //New instance saved

`


#### Functions / Methods

##### Internal getters
- static get alias
- Returns class full alias, usable for ES queries
- It consists of tenant and name

##### Class level API
- static async search(body, ?from, ?size, ?additional)
- Performs ES search
- Returns BulkArray of instances
- Used by another static functions
- Redefining this function will affect their behavior
- User must specify body
- alias is already defined in the class
- from and size are optional
- Returns all results if from / size are not specified, no matter how many there are
- additional is an optional object with additional data


- static async *bulkIterator(body, ?source, ?bulkSize, ?additional)
- Async generator for iterating over bulks of documents
- body, source and additional arguments are the same as in the search function
- bulkSize is optional custom size of a bulk

- static async *itemIterator(body, ?source, ?bulkSize, ?additional)
- Async generator for iterating over documents
- Arguments are the same as in the bulkIterator generator

- static async findAll()
- Finds all entries in ES, matching class alias
- Uses this.search()
- Returns BulkArray

- static async find(ids)
- Performs ES 'search' query
- Always returns BulkArray of instances
- Uses this.search()

- static async get(ids)
- Performs ES 'get'
- If 'ids' is strings, returns single instance
- Else if 'ids' is an array of strings, returns BulkArray of instances

- static async head(ids)
- Performs ES 'mget' without '_source' field
- Throws when any of the IDs is not found
- If 'ids' is strings, returns single object
- Else if 'ids' is an array of strings, returns array of objects

- static async delete(ids, ?version)
- Performs ES 'delete'
- Uses bulk API
- Class alias must be fully specified
- Returns ES response
- If version is specified, only one id is allowed

- static async exists(ids)
- Performs ES 'exists'
- If 'ids' is strings, returns single boolean
- Else if 'ids' is array of strings, returns array of booleans
- true if exists, else otherwise

- static async update(ids, body)
- Performs ES 'update'
- Uses bulk API
- Returns ES response

- static async count(?body)
- Performs ES 'count'
- Returns number

- static async updateByQuery(body)
- Performs ES 'update_by_query'
- Returns ES response

- static async deleteByQuery(body)
- Performs ES 'delete_by_query'
- Returns ES response

##### Indices API
- static async createIndex(?body, ?setAlias = true)
- Creates index given by current class
- body is optional settings
- setAlias is used when we do not want automatically set an alias to newly create index

- static async getIndex()
- Returns ES index of ODM alias
- Returns string

- static async aliasIndex(index)
- Puts alias of current ODM onto selected index

- static async deleteAlias()
- Deletes ODM alias from ES, underlying index remains unchanged

- static async aliasExists()
- Checks ODM alias existence
- Returns boolean

- static async indexExists()
- Checks ODM index existence
- Returns boolean

- static async deleteIndex()
- Deletes alias and index given by current class

- static async getMapping()
- Gets mapping of index given by current class

- static async putMapping(mapping)
- Puts mapping to index given by current class

- static async getSettings(?includeDefaults = false)
- Gets settings of index given by current class

- static async putSettings(settings)
- Puts settings to index given by current class

- static async reindex(destinationModel, ?script)
- Reindex from current model class to selected one
- Destination may be chosen by ODM or by alias/index string
- script specifies optional painless script to be used

- static async cloneIndex(?settings)
- Creates clone of ODM index
- Current index has to be manually blocked for write (be made read-only)
- settings is optional settings to be used for newly created index
- Returns newly created index

- static async refresh()
- Refreshed current index / indices

- static async openPIT()
- Opens Point in Time in given index

- static async closePIT(id)
- Closes given Point in Time

##### Instance level API
- async save(?useVersion)
- saves or re-saves instance
- it uses specified _id or generates new one if not specified
- it uses ES 'index' function
- useVersion - checks if version matches, _id and _version must be specified
- sequence number will be fetched automatically in not presented

- async reload()
- Reloads instance data from ES
- Uses get and new this.constructor() internally

- async delete(?useVersion)
- Deletes an instance from ES
- _id must be specified and entry must exist in ES
- useVersion - checks if version matches, _version must be specified
- sequence number will be fetched automatically in not presented

- clone(?_preserveAttributes)
- Returns clone of current instance
- Deep copy
- Cloned instance is created from the same class
- preserveAttributes (defaults true) can be used to preserve attributes (_id, _version, ...)

- async validate()
- Validates instance using Joi
- Throws in case of error / incorrect data

##### Class copy
- static clone(?changes)
- Creates class copy
- In fact, it creates new class which inherits from the current one
- changes is an optional object with properties to be set

- static in(newTenant)
- Clones class using clone()
- Sets given tenant

- static immediateRefresh(newRefresh)
- Clones class using clone()
- Sets given refresh mode

#### Other
- static _parseIndex(index)
- Parses given index (or even alias) and returns its parts

- static __checkIfFullySpecified(functionName)
- Checks if current class is fully specified, meaning it doesn't contain any wildcard in alias
- Throws otherwise

- static __getConstructor(searchResult, constructorCache)
- Returns the correct class (constructor) for given search result

- static _alterSearch(body)
- Alters the search function query (and also query of other functions that use the query)
- By default, just passes the body without any change, but can be rewritten

- static _unpackData(source)
- Alters the fetched ES documents before the instance is created from them
- By default, just passes the source without any change, but can be rewritten

- async _packData(cache)
- Alters the instance data before it is saved to ES
- By default, just passes the instance data without any change, but can be rewritten

- static async _afterSearch(instances, ?cache)
- Special function called for each record founded by search / find / findAll / get
- By default, it is empty, but you can overwrite it in your code

- static async _getBulkSize()
- Returns optimal size of bulk for given model
- Used in search` function for pagination