Headless node.js test runner for Repeato mobile UI tests
npm install @repeato/cli-testrunnerRepeato CLI is a headless test runner that allows you to execute test batches designed with Repeato Studio.
We try to rely as much a possible on open standards, so all the workspace data is stored in either image formats or text based formats (mostly JSON).
That makes it possible to generate tests not just with Repeato Studio, but also with tools of your choice.
That's also true for the test run artefacts: All data is stored as image data (screenshots), XML (JUnit export, can be used for Jenkins) and JSON.
Repeato CLI is built using Node.js which makes it possible to run it on almost any system. However, it has dependencies to native modules, which makes it necessary to build parts of it for the target OS and platform.
Node.js version 20 or 22 is strongly recommended.
You can use nvm to switch to the right Node.js version:nvm use 20.
You can also choose to use another Node.js version, but this requires compiling our native modules on your end.
The recommended way of using this package is via npx, which downloads and runs Repeato-CLI right away:
npx @repeato/cli-testrunner
If you run Repeato as part of your CI/CD, you might want to fix the version of the test runner, so you don't run into a situation where your setup breaks because of updates we ship. (Although those should usually not cause any problems!)
npx @repeato/cli-testrunner@1.4.95
npm -g @repeato/cli-testrunner
...and then run it like this:
cli-testrunner --help
The testrunner essentially needs two inputs:
1. The path to your workspace containing you batch
2. The batch ID of the batch you want to execute
cli-testrunner --help will print all the required and optional params:
``
Runs a selected test batch of your Repeato workspace
Usage via npx:
$ npx @repeato/cli-testrunner [params]
Usage when installed globally via npm:
$ cli-testrunner [params]
Required params:
--accessToken: Your Repeato access token. Can be generated here: https://clients.repeato.app/
--workspaceDir: Path to the workspace directory. See also https://www.repeato.app/documentation/test-workspaces/
--batchId: Which batch should be executed.
Just execute cli-testrunner --workspaceDir [workspace-dir] to see a list of contained batches
Optional params:
--deviceId: The device ID of the device you want to connect to
List Android devices via 'adb devices' or iOS simulators via 'xcrun simctl list'
--runMode: "AllTests" | "OnlyFailed" (default: "AllTests")
--timeoutFactor: Since some CI servers sometimes don't provide as much performance as local machines,
it can be useful to increase the timeout when running on a server by a factor. E.g pass 1.5 to increase the timeouts by 50% (default: 1.0)
--waitDurationBetweenSteps: Additional wait time between step executions can help during debugging or to increase test stability.
(default: 0, milliseconds)
--enableUtf8Keyboard: "true" | "false". Enable the Repeato test automation keyboard.
Set this to "false" if you want to use the default keyboard of your device. (default: "true")
--outputDir: Override the default export path for batch run reports (default is: [workspaceDir]/batchRuns)
--logLevel: "WARN" | "INFO" | "DEBUG" (default: "INFO")
--force: Ignore errors (DB version mismatch) and execute anyway
Examples:
$ npx @repeato/cli-testrunner --workspaceDir C:/path/to/your/workspace
List all batches in your workspace
$ npx @repeato/cli-testrunner --workspaceDir C:/path/to/your/workspace --batchId 278962-W73KOl2-SIUSZ
Executes all the tests in the given batch
`
1. A JUnit xml file is created in the root folder of Repeato CLI
2. For each test a testRun object is created in [workspaceDir]/[testId]/test.json
3. For each test step a screenshot is created in
4. An html batch report is created in the directory. The exported directory can be shared via a webserver to make the report accessible to your QA team.[workspaceDir]/batchReports/[your batch report]
5. A batchRun.json file containing all the batch run information in directory
6. Returns with exit code 0 if batch runner managed to run all tests, even if a test failed. Returns different exit code if some error occured during execution.
Supported Node.js versions: 20.x, 22.x.
You can use nvm to switch to the right Node.js version:
nvm use 20.
Supported operating systems: MacOS, Windows, Linux
Simply execute
npx @repeato/cli-testrunner [args]
We have a couple of guides on how to install Android emulators or iOS simulators on your target system here:
https://www.repeato.app/documentation/virtual-test-devices/
On Android you can use ADB to push your APK to the virtual device.
1. Install ADB
2. adb push /path/to/myApp.apk
On iOS you can use IDB
1. Install IDB
2. idb install /path/to/myApp.app
We tested our CLI test runner with following services:
- Github Actions
- Bitrise
- Codemagic
You can find some example workflows here:
https://www.repeato.app/documentation/continuous-integration/
In most cases Repeato should deliver the same results as Repeato Studio. We are actually using the same code base for both.
However, it really also depends on how the tests and the app is designed.
However, there are limits: Here is an example where Studio and CLI could deliver different results: Let's say your app is designed in a way, that a button appears on stage, but it's not clickable for 100ms (because it was coded this way. Maybe there is a server request involved, ...). Let's further assume Repeato-Studio (running on an older workstation) executes slower than the CLI (which is running on a fast server).
Repeato Studio (being slower) would click the button quite late (150ms delay), the test would run through.
Repeato CLI (being fast) would click the button quite early (50ms delay), but the button would not be ready and therefore not trigger an action.
Here are some options of what you can try to solve the issue:
- Make sure you are using the same device on Repeato Studio and Repeato CLI
- Slow down the CLI testrunner: There is a --waitDurationBetweenSteps` flag that could be used to make it wait between steps.
- Figure out the problematic steps and add a wait step before them
- Send us your batch report (support@repeato.app) and we can have a look at it