C8 Cross-Component End-to-End Test Suite

Overview

The C8 Cross-Component End-to-End Test Suite represents an extensive collection of end-to-end tests meticulously tailored for Camunda C8, encompassing the entire product stack. This specialized suite is dedicated to test scenarios that traverse multiple C8 components, harnessing the power of Playwright with TypeScript. It adheres to the Page Object Model design pattern, ensuring effortless test creation and seamless maintenance. The focus of the test suite is to align with the C8 persona profiles and their user flows as well as the most important cross-component user flows for the product stack determined by each product team. The persona profiles that will be focused on can be observed in this link

Supported Testing Platforms

$3

This test suite supports testing across three primary platforms:

1. C8 SaaS
2. C8 Self-Managed with Helm Chart
3. C8 Run

Supported Cluster Versions for SaaS

For the Camunda SaaS offering, the test suite supports following cluster versions:

- 8.6.X
- 8.7.X
- 8.8.X
- 8.9.X

By default, the test suite is configured to run tests on the latest snapshot versions. If you wish to run tests on a different generation, you can adjust the configurations in the .github/c8_versions.json file.

Supported Versions for SM with Helm

For the Camunda SM Helm offering, the test suite supports following versions:

- 8.6.X
- 8.7.X
- 8.8.X
- 8.9.X

The test suite primarily uses Elasticsearch as the default search engine. However, it also supports OpenSearch, with dedicated workflows explicitly named to indicate OpenSearch as the selected engine.

Supported Versions for C8Run

For the Camunda C8Run offering, the test suite supports following versions:

- 8.6.X
- 8.7.X
- 8.8.X
- 8.9.X

$3

GitHub Actions can be used to facilitate testing across various cluster versions and components:

#### SaaS

For SaaS, a test run can be manually triggered for specific minor versions and their associated cluster generation. This can be done through the SaaS On-Demand GitHub Action.

#### SM with Helm

For SM using Helm, a test run can manually trigger by specifying any component version as an input. TThis can be done through the following workflows:

- SM On-Demand GitHub Action for versions ≤ 8.7.X
- SM On-Demand GitHub Action for versions ≥ 8.8.X

#### C8Run

For C8Run, a test run can be manually triggered for specific minor versions, for each supported Operating System. This can be done through the following workflows:

- C8Run On-Demand with Linux GitHub Action
- C8Run On-Demand with macOS GitHub Action
- C8Run On-Demand with Windows GitHub Action

Prerequisites

1. Before running the test suite, make sure you have the following prerequisites installed:

- Node.js (with npm, >= v14)

Getting Started

$3

To run the tests, you need your personal Vault token. You should normally have access to this token. If not, you’ll need to reach out to IT or Infra teams.

1. Clone the repository:

``bash
git clone https://github.com/camunda/c8-cross-component-e2e-tests.git
cd c8-cross-component-e2e-tests
`

2. Install dependencies and auto-configure Vault CLI for Mac, Linux, and Windows:

`bash
npm install
npx playwright install
`

- This will automatically download the correct Vault CLI for your system (Mac, Windows, Linux) into the bin/ directory.
- No manual Vault CLI install or admin rights required.

3. Create your .env file automatically:

`bash
npm run setup
`

- When prompted, paste your Vault token from vault.int.camunda.com/ui/vault/access.
- The script fetches all necessary credentials and secrets, applies in-place transformations to the required email usernames, and generates your .env file in the project root.

4. Set the following variables manually in .env:

After running npm run setup, open your .env file and fill these:

- TASKLIST_VERSION (Set to v1 or v2 for 8.8+)
- PLAYWRIGHT_BASE_URL (Base_url for Helm chart testing, or remove it completely)
- CLUSTER_VERSION (Cluster version on INT to test, or leave blank)
- MINOR_VERSION (Minor version, e.g. SM-8.8, corresponds to the test folders in this repo)

The setup script will leave these blank and clearly mark them for manual input.

If you do not have immediate Vault access or are otherwise blocked, you can copy the required .env values from Keeper:

- In Keeper, go to Shared Engineering All → C8 Cross Component E2E Test Suite Credentials
- Copy all provided environment variable values
- Paste them into your local .env file

Setting up your own organization (optional)

Request the creation of an organisation identical to Automation QA
Org ID: 3626cf46-8f97-4711-81b3-ecfc341348c2
Environment: INT

Requirements:
- You must be set as the Owner
- Name the organisation: Automation QA - Your_Name

Inviting test users

Within the Automation QA - Your_Name organisation, invite the following users:

- Alias of C8_USERNAME
- Alias of C8_USERNAME_TEST
- Alias of C8_USERNAME_TEST_2
- Alias of C8_USERNAME_TEST_3
- Alias of C8_USERNAME_TEST_4 (v8.8+)
- Alias of C8_USERNAME_TEST_5 (v8.8+)
- Alias of C8_USERNAME_TEST_6 (v8.8+)
- Alias of C8_USERNAME_TEST_7 (v8.8+)
- Alias of C8_USERNAME_TEST_8 (v8.8+)
- Alias of C8_USERNAME_TEST_9 (v8.8+)

Passwords & invitation acceptance

1. Retrieve the Gmail passwords from Keeper:
Shared → Engineering All → C8 Cross Component E2E Test Suite Credentials
2. Use these emails to sign up and accept the invitations.
3. Align the passwords with:
- C8_PASSWORD
- C8_PASSWORD_TEST

Running Smoke Tests in CI

$3

To trigger the SaaS smoke tests in CI, use the workflow:

SaaS On-Demand Generation Creation Smoke

- You’ll be prompted to input the version you want to test for each component.
- The minor version is simply the minor version your PR is targeting.
- Example: If your PR targets 8.9, then the minor version is 8.9.

$3

For Helm chart tests (if you want to run only the smoke test), use the workflow:

SM On-Demand Smoke 8.8+

- Provide the component images you want to test against as workflow inputs.

Running Tests Locally

1. Run Tests in Headless Mode

To run the test suite in headless mode (without a graphical user interface), use the following command:

`bash
npm run test:local
`

To run one specific test file in headless mode (without a graphical user interface), use the following command:

`bash
npx playwright test --project=chromium tests/your-test-folder/your-test-spec
`

2. Run Tests with UI (Interactive Mode)

To run the test suite with the UI (interactive mode), append --ui at the end of the npx playwright test command (note the test 'Create Cluster' must be run first):

`bash
npx playwright test --ui
`

$3

#### C8 SaaS
For a full test run, no additional configuration is required. However, if you are running a single test, ensure you have a healthy and active cluster.

If you want to run tests against the production environment, add the following to your local .env file:

`bash
IS_PROD=true
`

#### C8 Self-Managed (SM) with Helm
To test locally on the SM Helm environment, you need an active SM instance. You can create one through any of the following GitHub Actions workflows:

- Basic Helm Chart for versions ≤ 8.7.X
- Basic Helm Chart for versions ≥ 8.8.X
- Helm Chart With RBA Enabled for versions ≤ 8.7.X
- Helm Chart With RBA Enabled for versions ≥ 8.8.X
- Helm Chart With Multi Tenancy Enabled for versions ≤ 8.7.X
- Helm Chart With Multi Tenancy Enabled for versions ≥ 8.8.X
- Basic Helm Chart with OpenSearch for versions ≤ 8.7.X
- Basic Helm Chart with OpenSearch for versions ≥ 8.8.X

Once the SM instance is created, copy the generated test URL and input it into your .env file under the PLAYWRIGHT_BASE_URL variable.

Note: For workflows ≥ 8.8.x, if you only need to retrieve the test URL without running tests, simply uncheck the Enable tests checkbox.

If you want to run tests with Resource-Based Access (RBA) enabled, add the following to your .env file, otherwise SM tests default to RBA disabled:

`bash
IS_RBA=true
`
If you want to run tests with Multi-Tenancy (MT) enabled, add the following to your .env file, otherwise SM tests default to MT disabled:

`bash
IS_MT=true
`
#### C8 Run
To test locally against C8 Run, you must set up a local instance of C8 Run. Please follow the instructions provided in the C8 Run README.md. By default, the BASE_URL is localhost:8080, however if this is configured as something differently, tests can be run against this by inputting the new BASE_URL into your .env file under the PLAYWRIGHT_BASE_URL variable.

Automated Nightly Test Runs

The test suite is configured to run automatically every night. The results of these nightly test runs are published on TestRail. If any test fails during these automated runs, the Designated Responsible Individual (DRI) of the project is notified via email for immediate attention and resolution. On a nightly basis, the test suite is configured to run the following tests:

- C8 SaaS: Tests are run against a SNAPSHOT generation cluster.
- C8 SM with Helm: Tests are run against the alpha branch of the camunda-platform-helm repository, using SNAPSHOT images for each component. This is peformed with both Elasticsearch and OpenSearch.
- C8 Run: Tests are run against the main branch of the camunda/camunda repository.

Test Suite Structure

The test suite is organized using the Page Object Model design pattern, where each test file contains test scenarios related to a specific component or feature of the Camunda C8 product stack. Additionally, it leverages separate page objects for each web page or component being tested.

You can find the page objects in the pages directory, and the test files in the tests directory.

For example:

Page files:

LoginPage.ts - Page object for the login functionality.

Test files:

login.spec.ts - Tests related to the login functionality using the LoginPage object.

Feel free to add more test files or page objects to expand the test coverage based on your specific requirements.

GitHub Actions Workflow Naming Convention

To maintain consistency and clarity in workflow names displayed in the GitHub Actions UI, we use the following naming pattern:

$3