Write BDD tests in Markdown.
npm install @bifravst/bdd-markdown
Write BDD tests in Markdown.
Writing BDD tests should be more comfortable
than this,
so why not use Markdown? It can look
like this.
- it is a well supported document format, many tools like auto-formatters
already exist
- it provide good tools to structure a hierarchical document
- it has support for embedding source code / JSON payloads, and even tables
- front matter can be used for feature-level configuration
Work on the original BDD e2e feature runner began in 2018, and the project has
been proved very useful for testing cloud-native solutions. Read more about the
original idea
here.
However, the implementation had some shortcomings. Especially understanding test
results and the way state and retries were handled was not optimal. In addition
was the old codebase itself not sufficiently covered with tests. Therefore this
project was initiated in 2022, with four years of experience authoring and
running tests. With a fresh set of eyes, the way to write test was complete
changed from Gherkin to Markdown which called for releasing it as a standalone
project.
- Demo of supported syntax
- Gherkin Rule keyword
- Mars Rover Kata (this
demonstrates the Soon keyword which retries steps)
Run: $(set -o pipefail && npx tsx examples/mars-rover/tests.ts | npx tsx reporter/console-cli.ts)
- Firmware UART log assertions
(this demonstrates the use of the Context, which is a global object
available to provide run-time settings to the test run, which replace
placeholders in step titles and codeblocks.)
Run: $(set -o pipefail && npx tsx examples/firmware/tests.ts | npx tsx reporter/console-cli.ts)
Soon keywordLet's have a look at this scenario:
``markdownTo Do List
Given I create a new task named My item
Then the list of tasks should contain My item
What if you are testing a todo list system, that is eventually consistent?
More specifically: creating a new task happens through a POST request to an202 Accepted
API that returns a status code.
The system does not guarantee that task you've just created is _immediately_
available.
The Then assertion will fail, because it is executed immediately.
For testing eventual consistent systems, we need to either wait a reasonable
enough time or retry the assertion.
However, if there are many similar assertions in your test suite will quickly
add up to long run times.
Therefore the most efficient solution is to retry the assertion until it passes,
or times out. This way a back-off algorithm can be used to wait increasing
longer times and many tries during the test run will have the least amount of
impact on the run time.
Implementing the appropriate way of retrying is left to the implementing step,
however you are encourage to mark these eventual consisted steps using the
Soon keyword.
By default the features are loaded in no particular order. You _may_ attempt to
order them using a naming convention, however this can enforce a forced ranking
of all features, and over time files might need to get renamed to make room for
new features.
In this project, features can specify a dependency to one or more other features
in their front matter, and after parsing all features files, they will be sorted
topologically.
Features can define their dependencies via the needs keyword:
`markdown
---
needs:
- First feature
---
Given this is the first step
`
This feature will be run after a feature with the name First feature
In addition, features can specify whether they should be run before all other
features, or after all. Multiple keywords can have this flag, but dependencies
will take precedence.
`markdown
---
order: first
---
Given this is the first step
`
`markdown
---
order: last
---
Given this is the first step
`
Features can be skipped, this will also skip all dependent and transiently
dependent features.
`markdown
---
run: never
---
Given this is the first step
`
Features can be run exclusively, this will also run all dependent and
transiently dependent features. All other features not marked as run: only
will be skipped.
`markdown
---
run: only
---
Given this is the first step
`
Variants (defined in the frontmatter of a feature) can be used to run the same
feature in different variants. For every entry in variants, the feature file is
run. (Example)
For JSON code-blocks there is a special notation to replace number placeholders,
while still maintaining the JSON syntax and allow for formatters like prettier
to format the code-block.
``markdownv
Given is the number 42
And I store this in result
`json`
{ "foo": "$number{v}" }
Then result should match
`json`
{ "foo": 42 }
It includes a markdown reporter, which will turn the suite result into markdown,
suitable for displaying it as
GitHub Actions job summaries.
Example: Mars Rover Report