Slack Test Results Notification

> Send Slack message with test results from popular testing frameworks

A Slack test reporting tool that supports all major testing frameworks.
Generate, publish and alert your team with detailed test results, including
summaries, in-depth reports, failed test analyses, flaky test detection and AI
analyses directly to your chosen Slack channel.

Features

- Send Test Results to Slack: Automatically send test results to a Slack channel.
- Send Flaky Test Details to Slack: Automatically send flaky test details to a Slack channel.
- Send AI Test Summary to Slack: Automatically send AI test summary to a Slack channel.
- Send Failed Test Details to Slack: Automatically send failed test details to a Slack channel.
- Build your own Slack message: Create and customize your own Slack test reports with our flexible templating system.
- Tagging: Tag users, channels and groups in the message.
- Conditional Notifications: Use the --onFailOnly option to send notifications only if tests fail.

!Example view

Setup

You'll need a CTRF report generated by your testing framework. CTRF reporters are available for most testing frameworks and easy to install.

No CTRF reporter? No problem!

Use junit-to-ctrf to convert a JUnit report to CTRF.

$3

1. Go to the Slack API: Incoming Webhooks page.
2. Click on "Create a Slack App" or use an existing app.
3. Add the "Incoming Webhooks" feature to your app.
4. Activate the Incoming Webhook and add a new webhook to your workspace.
5. Copy the webhook URL provided.

Set the webhook URL as an environment variable in your shell or CI environment:

``sh
export SLACK_WEBHOOK_URL='https://hooks.slack.com/services/your/webhook/url'
`

Make sure to replace 'https://hooks.slack.com/services/your/webhook/url' with your actual webhook URL.

You might want to store the webhook URL as a secret.

You can also use the --webhook-url option to set the webhook URL.

$3

1. Go to the Slack API: Slack Apps page.
2. Follow the instructions to create a Slack App.
3. Add the 'chat:write' scope to your app.
4. Generate a Bot OAuth Token.
5. Invite the bot to your channel.
6. Copy the Bot User OAuth Token.

Set the OAuth Token as an environment variable in your shell or CI environment:

`sh
export SLACK_OAUTH_TOKEN='xoxb-your/bot/user/oauth/token'
`

You must also set the channel ID as an environment variable when using the OAuth Token:

`sh
export SLACK_CHANNEL_ID='C0123456789'
`

You can also use the --oauth-token and --channel-id options to set the OAuth Token or channel ID respectively.

Usage

You can use a glob pattern or a single file path to send the test results summary to Slack.

$3

To send the test results summary to Slack:

`sh
npx slack-ctrf results /path/to/ctrf-report.json
`

You can use a glob pattern with multiple files which will be merged together:

`sh
npx slack-ctrf results "ctrf/*.json"
`

!Results view

$3

To send AI failed test summary to Slack:

`sh
npx slack-ctrf ai /path/to/ctrf-report.json
`

Use --consolidated argumement to send a single message with all AI summaries

`sh
npx slack-ctrf ai /path/to/ctrf-report.json --consolidated
`

!AI view

See the AI Test Reporter to add AI summaries to your CTRF report

$3

To send flaky test report to Slack:

`sh
npx slack-ctrf flaky /path/to/ctrf-report.json
`

!Flaky view

Failed

To send failed test report to Slack:

`sh
npx slack-ctrf failed /path/to/ctrf-report.json
`

Use --consolidated argumement to send a single message with all AI summaries

`sh
npx slack-ctrf failed /path/to/ctrf-report.json --consolidated
`

!Failed view

Build your own Slack message

The custom method lets you build your own report using a Handlebars
template allowing you to leverage data from your CTRF report to build a dynamic Slack message. The template can be a Block Kit JSON or Slack flavored markdown context block.

To send a message to Slack using a custom Handlebars template:

`sh
npx slack-ctrf custom /path/to/ctrf-report.json /path/to/template.hbs --blockkit
`

`sh
npx slack-ctrf custom /path/to/ctrf-report.json /path/to/template.hbs --markdown
`

The template has access to the entire CTRF report data through the ctrf variable. For example:

`handlebars
Tests: {{ctrf.summary.tests}}
`

`markdown
Tests: 100
`

$3

Provide a Block Kit JSON template to the --blockkit option. See the Slack Block Kit Documentation and Slack Block Kit Builder for help creating a template.

The Block Kit JSON template can include Handlebars syntax to access test report data. For example:

`json
{
"blocks": [
{
"type": "header",
"text": {
"type": "plain_text",
"text": "Test Results: {{ctrf.summary.total}} tests run"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "Passed: {{ctrf.summary.passed}}\nFailed: {{ctrf.summary.failed}}"
}
}
]
}
`

See the templates directory for examples.

$3

Provide a Slack flavored markdown template to the --markdown option. See the Slack Flavored Markdown Documentation for help creating a template.

See the templates directory for examples.

$3

Handlebars is a simple templating language that lets you insert data into your
message in a declarative way. You can use placeholders, conditionals, and loops
to dynamically generate content based on your test results.

CTRF Properties

The ctrf object provides access to your test results data. Here are the common
properties:

$3

- tests: Total number of tests
- passed: Number of passed tests
- failed: Number of failed tests
- skipped: Number of skipped tests
- start: Test suite start time
- stop: Test suite end time

$3

An array of test results, each containing:

- name: Test name
- status: Test status ("passed", "failed", "skipped")
- message: Test output/error message
- duration: Test duration in milliseconds
- retries: Number of retries (for flaky tests)

Example accessing test data:

`hbs
{{#each ctrf.tests}}
Test: {{this.name}} - Status: {{this.status}}
{{/each}}
`

$3

The template can include helpers to format the data. There is a wide range of helpers available from CTRF custom helpers handlebars-helpers-ctrf, handlebars built-in helpers and slack-ctrf helpers

Send Only on Failures

To send the test results summary to Slack only if there are failed tests, use the --onFailOnly option:

`sh
npx slack-ctrf results /path/to/ctrf-file.json --onFailOnly
`

or using the alias:

`sh
npx slack-ctrf results /path/to/ctrf-file.json -f
`

Custom Notification Title

You can choose a custom title for your notification, use the --title option:

`sh
npx slack-ctrf results /path/to/ctrf-file.json --title "Custom Title"
`

or using the alias:

`sh
npx slack-ctrf results /path/to/ctrf-file.json -t "Custom Title"
`

Add Prefix and Suffix

Add custom markdown text to as a prefix, or suffix to the message:

`sh
npx slack-ctrf results /path/to/ctrf-file.json -p "prefix" -s "suffix"
`

Read about compatible markdown

Tagging a user, channel or group

You can tag users, channels or groups in the message by using the --prefix or --suffix options.

You need to a Slack user ID, channel ID or group ID.

You can find your user ID by:

- Clicking on your profile picture in Slack
- Clicking the three dots (...)
- Selecting "Copy member ID"

Once you have your user ID (which will look something like U01ABC123DE), use that in your command instead of your display name:

`sh
npx slack-ctrf results /path/to/ctrf-file.json -s "<@U0123456789> please review the results"
`

You can also tag a channel by using the # symbol with the channel ID:

`sh
npx slack-ctrf results /path/to/ctrf-file.json -s "<#C0123456789> please review the results"
`

And finally, you can tag a group by using the \!subteam^ symbol with the group ID:

`sh
npx slack-ctrf results /path/to/ctrf-file.json -s "<\!subteam^0123456789> please review the results"
`

Options

- --onFailOnly, -f: Send notification only if there are failed tests.
- --title, -t: Title of the notification.
- --prefix, -p: Custom markdown text to add as a prefix to the message
- --suffix, -s: Custom markdown text to add as a suffix to the message
- --webhook-url, -w: Incoming webhook URL
- --oauth-token, -o: OAuth token
- --channel-id, -ch: Channel ID

Merge reports

You can merge reports if your chosen reporter generates multiple reports through design, parallelisation or otherwise.

If you use a glob pattern, the reports will be merged automatically.

Otherwise, the ctrf-cli package provides a method to merge multiple ctrf json files into a single file.

After executing your tests, use the following command:

`sh
npx ctrf merge
`

Replace directory with the path to the directory containing the CTRF reports you want to merge.

Programmatic Usage

You can use the package programmatically to send notifications to Slack. To install the package, run:

`sh
npm install slack-ctrf
`

see API documentation for more details.

Configuration

You can skip features by using environment variables.

`sh
export CTRF_SKIP_FOOTER=true # Skip the footer
export CTRF_SKIP_CHART=true # Skip the chart
export CTRF_SKIP_WARNINGS=true # Skip the build warnings
`

Development

Contributions are very welcome!

$3

`sh
npm run all
`

$3

If you have a Slack account to test with, set the environment variables test:

`sh
export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/your/webhook/url


or


export SLACK_OAUTH_TOKEN=xoxb-your/bot/user/oauth/token
export SLACK_CHANNEL_ID=C0123456789
`

Run the E2E tests:

`sh
npm run e2e
`

or test a single command:

`sh
node dist/cli.js results "ctrf-report.json"
``

What is CTRF?

CTRF is a universal JSON test report schema that addresses the lack of a standardized format for JSON test reports.

Consistency Across Tools: Different testing tools and frameworks often produce reports in varied formats. CTRF ensures a uniform structure, making it easier to understand and compare reports, regardless of the testing tool used.

Language and Framework Agnostic: It provides a universal reporting schema that works seamlessly with any programming language and testing framework.

Facilitates Better Analysis: With a standardized format, programatically analyzing test outcomes across multiple platforms becomes more straightforward.

Support Us

If you find this project useful, consider giving it a GitHub star ⭐ It means a lot to us.