testrail-reporter-playwright

The testrail-reporter-playwright package can be used to integrate to the cloud version of testrail (testrail.com or testrail.net). This package is built as a reporter to be incorporated in the playwright config file. This integrates with TestRail and automatically creates TestRail Runs and add test results on matching testrail test case IDs.

Installation

bash
npm install testrail-reporter-playwright


Features

The package supports the below features:
- Integrates to Playwright with provisions to be addded as a reporter.
- Creates a New test run in testrail or use an existing testrail run.
- Update the test cases in a test run after a test is complete.
- Update the Error log too to the testrail test case on failures.
- Attach the screenshot too to the Test run level on failures.
- Print the test run on the test completion.
- Support for a single test run for multiple playwright spec files run.
Usage

1. Install the package.

2. Add 'testrail-reporter-playwright' as a reporter to the playwright config file. This should be added under `"module.exports = defineConfig({"`block. Sample code below: reporter: ['line', 'html', 'testrail-reporter-playwright'],

3. Create a 'testrail.config.json' file in your project's root directory. Below should be the data under it:`json { "TESTRAIL_HOST": "https://<>.testrail.com", "TESTRAIL_USERNAME": "your testrail login email", "TESTRAIL_PASSWORD": { "encrypted": "", "iv": " "testrailPassword": process.env.TESTRAIL_PWD }, "TESTRAIL_PROJECT_ID": "", "TESTRAIL_SUITE_ID": { "": "test suite id in testrail", "": "test suite id in testrail" }, "TESTRAIL_API_ATTACHMENT_PATH": "/index.php?/api/v2/add_attachment_to_run/", "TESTRAIL_RUN_NAME": "", "TESTRAIL_RUN_ID": "" }`- TESTRAIL_HOST - TestRail instance domain name e.g`https://.testrail.com`- TESTRAIL_USERNAME - This should be a valid testrail login email id; can be a service account email id created for the team or a personal email based on security policies - TESTRAIL_PASSWORD - Here, the combination of 'encrypted' and 'iv' OR the 'testrailPassword' will be used. Recommended to encrypt the testrail plain text password using`https://www.npmjs.com/package/encrypt-decrypt-crypto`, the same package that is used for decrypting to a plain text password. In case the encrypted form is not used, then the user can send it from command line to the TESTRAIL_PWD environment variable. Sample is: TESTRAIL_PWD=password123 npm run test - TESTRAIL_PROJECT_ID - Numeric number/Id of the project - TESTRAIL_SUITE_ID - Multiple line items, each line item contains a short name followed by its test suites id. - TESTRAIL_API_ATTACHMENT_PATH - Does not change. Should stay the same. - TESTRAIL_RUN_NAME - Sample name can be ' Regression tests' and then it will be followed by the date and time when a new test run is created. - TESTRAIL_RUN_ID - Should stay blank - "" in most cases to create a new test run for every test cycle. In case you need to use an existing test run id, then update this field with the numerical testrail test run id.

4. Create/Update the test cases in your playwright suites including the TestRail test IDs in the test names or descriptions. For example:`javascript test(' ', async () => { // Test steps here });`Note: Please have the testrailid followed by a space and then the test case name/title

5. Run your tests with the same commands as before. - The current reporter will create a new run if needed, update the test results and publish the test run url at the end of the tests. In case of failures, it will log the console error to the test case and the screenshot to the test run. - Note: Mandatory** Please add a ENV to the command line run command with a value matching the testrail.config.js -> TESTRAIL_SUITE_ID -> 'project short name 1'. This is used to match the test case suite id and create a test run accordingly. Sample is: TESTRAIL_PWD=password123 ENV="int" npm run test => here, "password123" refers to the testrail plain text password and "int" matches the suiteId name in testrail.config.json file.

`TestRail Configuration`


Navigate to Testrail UI -> Administration -> Site Settings -> Check both the checkboxes for 'Enable API' and 'Enable session authentication for API'. These settings needs to be enabled for the reporter to upload results to testrail.
Sample testrail config js file

json
{
    "TESTRAIL_HOST": "https://test.testrail.com",
    "TESTRAIL_USERNAME": "test@test.com",
    "TESTRAIL_PASSWORD": {
        "encrypted": "",
        "iv": ""
    },
    "TESTRAIL_PROJECT_ID": 159,
    "TESTRAIL_SUITE_ID": {
        "AR": 13104,
        "RAPI": 13314,
        "ER": 13044
    },
    "TESTRAIL_API_ATTACHMENT_PATH": "/index.php?/api/v2/add_attachment_to_run/",
    "TESTRAIL_RUN_NAME": "Front End_Regression Automation_Playwright - Created on ",
    "TESTRAIL_RUN_ID": ""
}

Installation

bash
npm install testrail-reporter-playwright

Features

The package supports the below features:
- Integrates to Playwright with provisions to be addded as a reporter.
- Creates a New test run in testrail or use an existing testrail run.
- Update the test cases in a test run after a test is complete.
- Update the Error log too to the testrail test case on failures.
- Attach the screenshot too to the Test run level on failures.
- Print the test run on the test completion.
- Support for a single test run for multiple playwright spec files run.

Usage

1. Install the package.

2. Add 'testrail-reporter-playwright' as a reporter to the playwright config file. This should be added under `"module.exports = defineConfig({"`

 block. Sample code below:  
reporter: ['line', 'html', 'testrail-reporter-playwright'],

3. Create a 'testrail.config.json' file in your project's root directory. Below should be the data under it:

json
{
    "TESTRAIL_HOST": "https://<>.testrail.com",
    "TESTRAIL_USERNAME": "your testrail login email",
    "TESTRAIL_PASSWORD": {
        "encrypted": "",
        "iv": "        "testrailPassword": process.env.TESTRAIL_PWD
    },
    "TESTRAIL_PROJECT_ID": "",
    "TESTRAIL_SUITE_ID": {
        "": "test suite id in testrail",
        "": "test suite id in testrail"
    },
    "TESTRAIL_API_ATTACHMENT_PATH": "/index.php?/api/v2/add_attachment_to_run/",
    "TESTRAIL_RUN_NAME": "",
    "TESTRAIL_RUN_ID": ""
}


- TESTRAIL_HOST - TestRail instance domain name e.g

`https://.testrail.com`


- TESTRAIL_USERNAME - This should be a valid testrail login email id; can be a service account email id created for the team or a personal email based on security policies
- TESTRAIL_PASSWORD - Here, the combination of 'encrypted' and 'iv' OR the 'testrailPassword' will be used. Recommended to encrypt the testrail plain text password using

`https://www.npmjs.com/package/encrypt-decrypt-crypto`

, the same package that is used for decrypting to a plain text password. In case the encrypted form is not used, then the user can send it from command line to the TESTRAIL_PWD environment variable. Sample is: TESTRAIL_PWD=password123 npm run test
- TESTRAIL_PROJECT_ID - Numeric number/Id of the project
- TESTRAIL_SUITE_ID - Multiple line items, each line item contains a short name followed by its test suites id.
- TESTRAIL_API_ATTACHMENT_PATH - Does not change. Should stay the same.
- TESTRAIL_RUN_NAME - Sample name can be ' Regression tests' and then it will be followed by the date and time when a new test run is created.
- TESTRAIL_RUN_ID - Should stay blank - "" in most cases to create a new test run for every test cycle. In case you need to use an existing test run id, then update this field with the numerical testrail test run id.

4. Create/Update the test cases in your playwright suites including the TestRail test IDs in the test names or descriptions. For example:

javascript
test(' ', async () => {
    // Test steps here
});


Note: Please have the testrailid followed by a space and then the test case name/title

5. Run your tests with the same commands as before. 
- The current reporter will create a new run if needed, update the test results and publish the test run url at the end of the tests. In case of failures, it will log the console error to the test case and the screenshot to the test run.
- Note: Mandatory** Please add a ENV to the command line run command with a value matching the testrail.config.js -> TESTRAIL_SUITE_ID -> 'project short name 1'. This is used to match the test case suite id and create a test run accordingly. Sample is: TESTRAIL_PWD=password123 ENV="int" npm run test => here, "password123" refers to the testrail plain text password and "int" matches the suiteId name in testrail.config.json file.

TestRail Configuration

Navigate to Testrail UI -> Administration -> Site Settings -> Check both the checkboxes for 'Enable API' and 'Enable session authentication for API'. These settings needs to be enabled for the reporter to upload results to testrail.

Sample testrail config js file

json
{
"TESTRAIL_HOST": "https://test.testrail.com",
"TESTRAIL_USERNAME": "test@test.com",
"TESTRAIL_PASSWORD": {
"encrypted": "",
"iv": ""
},
"TESTRAIL_PROJECT_ID": 159,
"TESTRAIL_SUITE_ID": {
"AR": 13104,
"RAPI": 13314,
"ER": 13044
},
"TESTRAIL_API_ATTACHMENT_PATH": "/index.php?/api/v2/add_attachment_to_run/",
"TESTRAIL_RUN_NAME": "Front End_Regression Automation_Playwright - Created on ",
"TESTRAIL_RUN_ID": ""
}