Strapi Plugin App Search

To run this plugin it must be installed into a strapi application.

Usage

Install in any strapi application pnpm add strapi-plugin-app-search
Then add the proper config to config/hook.js
Example:

``
module.exports = ({ env }) => ({
settings: {
'strapi-app-search': {
enabled: true,
applicationId: '0000000000',
apiEndpoint: 'https://123.ent-search.us-east-2.aws.elastic-cloud.com/api/as/v1/',
apiKey: 'asdfihweq123125432',
debug: env('DEBUG_APP_SEARCH', false),
formatIndex: (apiName, env) => ${apiName}-${env},
apisToSync: [
{
name: 'api::content-type.content-type',
indexName: 'es-index-name',
processEsObj: (obj) => ({ ...obj, newProp: true }),
populate: ['relation'],
},
]
}
}
});
`

processEsObj is used to format the object before saving

You can also choose to individually upload the content type entries to App Search after creating instead of using the Sync button in the App Search plugin home page.
Example: src/api/content-type/content-types/content-type/lifecycles.js

`
/**
* Lifecycle callbacks for the content-type model.
*/

const APP_SEARCH_ENGINE = content-type-${strapi.config.environment};

const saveAppSearchObj = (model) => {
const { apisToSync } = getConfig();
const apiToSync = (apisToSync || []).find(({ name }) => name === content-type);
const indexName = getIndex(model.locale);

if (strapi.appSearch) {
strapi.appSearch.saveObject(apiToSync && apiToSync.processEsObj ? apiToSync.processEsObj(model) : model, indexName);
}
}

const shouldUseAppSearch = () => {
const { modelsToIgnore = [] } = getConfig();

return !modelsToIgnore.includes(MODEL_NAME);
}

const getIds = (params, ids = []) => {
if (!params || typeof params !== 'object') {
return ids;
}

if (Array.isArray(params)) {
params.forEach((item) => getIds(item, ids));
return ids;
}

Object.entries(params).forEach(([key, entry]) => {
if (key === 'id') {
if (typeof entry !== 'object') {
ids.push(entry);
} else if (entry['$in']) {
ids.push(...entry['$in']);
}
} else {
getIds(entry, ids);
}
});

return ids;
};

module.exports = {
async afterCreate({ result }) {
if (shouldUseAppSearch()) {
saveAppSearchObj({ ...result });
}
},

async afterUpdate({ result }) {
if (shouldUseAppSearch()) {
saveAppSearchObj({ ...result });
}
},

async afterUpdateMany({ params }) {
const ids = getIds(params);
const service = strapi.service('api::report.report');

if (shouldUseAppSearch()) {
service.find({ filters: { id: { $in: ids } }, populate: [] }).then(({ results }) => {
results.forEach((entry) => {
saveAppSearchObj({ ...entry });
});
});
}
},

async afterDelete({ result }) {
if (shouldUseAppSearch()) {
strapi.appSearch.deleteObject(result.id, getIndex(result.locale));
}
},

async beforeDeleteMany({ params, state }) {
const ids = getIds(params);
const knex = strapi.db.connection;

state.ids = ids;

state.entries = await knex
.select('id', 'locale')
.from('report')
.where('id', 'in', ids);
},

async afterDeleteMany({ state }) {
const { ids, entries } = state;

if (shouldUseAppSearch()) {
ids.forEach((id) => {
const entry = entries.find(({ id: entryId }) => entryId === id);
strapi.appSearch.deleteObject(id, getIndex(entry?.locale));
});
}
}
};
`

Dev

You need to create a new strapi project or just use an existing example one:

`
pnpm dev
`

Publishing

Publishing should already be setup. Just follow these steps to publish the project:

- After code merged to main/master
- Checkout the main/master branch
- Run pnpm version [patch|minor|major]
- Push to remote with git push --tags to trigger the tag pipeline

Migration: Elasticsearch App Search to Regular Elasticsearch

$3

This migration utility migrates data from Elastic App Search (hidden .ent-search-engine-documents-* indices) to regular Elasticsearch indices with aliases for zero-downtime cutover.

$3

Set the following environment variables before running the migration:

- ELASTICSEARCH_URL (required): Elasticsearch cluster endpoint
- Example: https://your-cluster.es.region.aws.elastic-cloud.com
- ELASTICSEARCH_API_KEY (required): Elasticsearch API key for authentication
- MIGRATION_DRY_RUN (optional): Set to true to preview changes without executing
- Default: false
- MIGRATION_CONCURRENCY (optional): Number of indices to migrate concurrently
- Default: 3
- MIGRATION_STATE_FILE (optional): Path to store migration state (for rollback)
- Default: .migration-state.json in repo root

$3

#### 1. Dry Run (Preview Only)

`bash
export ELASTICSEARCH_URL=https://your-cluster.es.region.aws.elastic-cloud.com
export ELASTICSEARCH_API_KEY=your-api-key
export MIGRATION_DRY_RUN=true
node scripts/migrate-ent-search-production.js
`

This will list all indices to be migrated and show what alias changes would occur, without making any actual changes.

#### 2. Full Migration

`bash
export ELASTICSEARCH_URL=https://your-cluster.es.region.aws.elastic-cloud.com
export ELASTICSEARCH_API_KEY=your-api-key
node scripts/migrate-ent-search-production.js
`

The migration:
- Creates new destination indices with timestamped names (e.g., media-production__migrated_20231219120000)
- Reindexes documents from source indices using the Elasticsearch _reindex API with automatic slicing
- Verifies document counts match between source and destination
- Atomically flips aliases to point to new indices
- Saves migration state to .migration-state.json for rollback capability

For large migrations (hundreds of thousands of documents), the script may take a while. You can monitor progress with the status check script in a separate terminal.

$3

While the migration is running (or to check on in-progress migrations), use the status check script:

`bash
export ELASTICSEARCH_URL=https://your-cluster.es.region.aws.elastic-cloud.com
export ELASTICSEARCH_API_KEY=your-api-key
node scripts/check-migration-status.js
`

Options:
- POLL_INTERVAL: Time in milliseconds between status checks (e.g., 5000 for 5 seconds). If 0 or not set, checks once and exits.
- MAX_WAIT: Maximum time in milliseconds to keep polling (e.g., 3600000 for 1 hour). After this time, polling stops even if migrations are incomplete.

Example - Poll every 5 seconds for up to 1 hour:

`bash
export ELASTICSEARCH_URL=https://your-cluster.es.region.aws.elastic-cloud.com
export ELASTICSEARCH_API_KEY=your-api-key
export POLL_INTERVAL=5000
export MAX_WAIT=3600000
node scripts/check-migration-status.js
`

The status check displays:
- Document counts for source and destination indices
- Reindex task progress (in-progress, pending, or complete)
- Percentage completion for each migration
- Summary of total complete/in-progress/pending migrations

$3

If some indices have empty destination indices (reindex failed/timed out), use the retry script:

`bash
export ELASTICSEARCH_URL=https://your-cluster.es.region.aws.elastic-cloud.com
export ELASTICSEARCH_API_KEY=your-api-key
node scripts/retry-failed-migrations.js
`

The retry script:
- Scans for failed migrations (destination indices with 0 documents but non-empty source)
- Deletes incomplete destination indices
- Creates new destination indices with fresh timestamps
- Reindexes from scratch with wait_for_completion: true
- Flips aliases to the new indices
- Updates migration state file

Dry run:
`bash
export ELASTICSEARCH_URL=https://your-cluster.es.region.aws.elastic-cloud.com
export ELASTICSEARCH_API_KEY=your-api-key
export MIGRATION_DRY_RUN=true
node scripts/retry-failed-migrations.js
`

$3

If needed, restore aliases to previous indices:

`bash
export ELASTICSEARCH_URL=https://your-cluster.es.region.aws.elastic-cloud.com
export ELASTICSEARCH_API_KEY=your-api-key
node scripts/rollback-migration.js
``

The rollback script:
- Reads the migration state file
- Atomically flips aliases back to previous targets
- Preserves the state file for audit purposes (delete manually after confirming)

$3

The migration only migrates production indices (allowlist):
- media: media-production, media-production-ar-sa, media-production-de-de, media-production-es-es, media-production-fr-fr, media-production-it-it, media-production-ja-jp, media-production-ko-kr, media-production-pt-br, media-production-ru-ru, media-production-zh-cn
- news: news-production, news-production-ar-sa, news-production-de-de, news-production-es-es, news-production-fr-fr, news-production-it-it, news-production-ja-jp, news-production-ko-kr, news-production-pt-br, news-production-ru-ru, news-production-zh-cn
- reports: reports-production, reports-production-ar-sa, reports-production-de-de, reports-production-es-es, reports-production-fr-fr, reports-production-it-it, reports-production-ja-jp, reports-production-ko-kr, reports-production-pt-br, reports-production-ru-ru, reports-production-zh-cn

$3

- Never hardcode API keys; always use environment variables
- The migration is idempotent; running it multiple times is safe (creates new timestamped indices)
- Source indices are never modified
- Large migrations may take several hours; monitor logs for progress
- Network interruptions are handled gracefully; the migration can be resumed