E2E Test Runner

A powerful, flexible end-to-end testing framework for API and database testing. Write tests in YAML or TypeScript, validate against multiple data stores, and generate comprehensive reports.

Features

- Multi-format tests: Write tests in YAML for simplicity or TypeScript for full control
- Database adapters: Built-in support for PostgreSQL, MongoDB, Redis, and Azure EventHub
- HTTP testing: Native HTTP adapter for REST API testing with JSONPath assertions
- Parallel execution: Run tests concurrently with configurable parallelism
- Multiple reporters: Console, JUnit XML, HTML, and JSON output formats
- Variable interpolation: Dynamic values with built-in functions (${uuid()}, ${now()}, etc.)
- Flexible filtering: Filter tests by tags, priority, or name patterns

Installation

$3

``bash
npm install -g @liemle3893/e2e-runner
`

$3

`bash
npm install --save-dev @liemle3893/e2e-runner
`

$3

`bash
npx @liemle3893/e2e-runner --help
npx @liemle3893/e2e-runner run --config ./e2e.config.yaml
`

$3

Install database adapters as needed:

`bash


PostgreSQL

npm install pg

MongoDB

npm install mongodb

Redis

npm install ioredis

Azure EventHub

npm install @azure/event-hubs
`

Quick Start

$3

`bash
e2e init
`

This creates e2e.config.yaml in your current directory.

$3

Create my-api.test.yaml:

`yaml
name: Get Users API
description: Test the users endpoint
tags: [smoke, api]
priority: P1

execute:
- id: get-users
adapter: http
action: GET
params:
url: /api/users
assertions:
- path: $.status
operator: equals
expected: 200
- path: $.body.data
operator: isArray
`

$3

`bash
e2e run
`

CLI Commands

$3

Execute E2E tests.

`bash


Run all tests

e2e run

Run with specific config

e2e run -c ./config/e2e.config.yaml

Run tests from a specific directory

e2e run -d ./tests/integration

Filter by tags

e2e run --tag smoke --tag regression

Filter by priority

e2e run --priority P0 --priority P1

Filter by name pattern

e2e run -g "user*"

Parallel execution

e2e run -p 4

Stop on first failure

e2e run --bail

Dry run (show what would run)

e2e run --dry-run
`

$3

Validate test file syntax without running.

`bash
e2e validate
e2e validate -d ./tests
`

$3

List discovered tests.

`bash
e2e list
e2e list --tag smoke
`

$3

Check adapter connectivity.

`bash
e2e health
e2e health --adapter postgresql
`

$3

Initialize configuration file.

`bash
e2e init
`

CLI Options

| Option | Short | Description | Default |
|--------|-------|-------------|---------|
| --config | -c | Config file path | e2e.config.yaml |
| --env | -e | Environment name | local |
| --test-dir | -d | Test directory | . (current) |
| --report-dir | | Report output directory | ./reports |
| --verbose | -v | Verbose output | false |
| --quiet | -q | Errors only | false |
| --parallel | -p | Parallel test count | 1 |
| --timeout | -t | Timeout in ms | 30000 |
| --retries | -r | Retry count | 0 |
| --bail | | Stop on first failure | false |
| --grep | -g | Filter by name pattern | |
| --tag | | Filter by tag (repeatable) | |
| --priority | | Filter by P0/P1/P2/P3 | |
| --reporter | | Reporter type (repeatable) | console |
| --output | -o | Report output path | |
| --dry-run | | Show tests without running | false |

Environment Variables

| Variable | Description |
|----------|-------------|
| E2E_CONFIG | Config file path |
| E2E_ENV | Environment name |
| E2E_TEST_DIR | Test directory |
| E2E_REPORT_DIR | Report output directory |
| E2E_VERBOSE | Enable verbose output (1 or true) |
| NO_COLOR | Disable colored output (1 or true) |

Configuration File

`yaml
version: "1.0"

environments:
local:
baseUrl: "http://localhost:3000"
adapters:
postgresql:
connectionString: "${POSTGRESQL_CONNECTION_STRING}"
mongodb:
connectionString: "${MONGODB_CONNECTION_STRING}"
redis:
connectionString: "${REDIS_CONNECTION_STRING}"

defaults:
timeout: 30000
retries: 1
retryDelay: 1000
parallel: 4

variables:
testPrefix: "e2e_test_"

reporters:
- type: console
verbose: true
- type: junit
output: "./reports/junit.xml"
`

Writing Tests

$3

`yaml
name: Create User Flow
description: Test user creation and retrieval
tags: [integration, users]
priority: P1
timeout: 60000

variables:
testEmail: "test-${uuid()}@example.com"

setup:
- id: cleanup-existing
adapter: postgresql
action: exec
params:
query: "DELETE FROM users WHERE email LIKE 'test-%@example.com'"

execute:
- id: create-user
adapter: http
action: POST
params:
url: /api/users
body:
email: "${testEmail}"
name: "Test User"
capture:
userId: $.body.id
assertions:
- path: $.status
operator: equals
expected: 201

- id: get-user
adapter: http
action: GET
params:
url: "/api/users/${captured.userId}"
assertions:
- path: $.body.email
operator: equals
expected: "${testEmail}"

verify:
- id: check-database
adapter: postgresql
action: query
params:
query: "SELECT * FROM users WHERE id = $1"
params: ["${captured.userId}"]
assertions:
- path: $.rows[0].email
operator: equals
expected: "${testEmail}"

teardown:
- id: delete-user
adapter: postgresql
action: exec
params:
query: "DELETE FROM users WHERE id = $1"
params: ["${captured.userId}"]
`

$3

`typescript
// users.test.ts
import type { TestDefinition } from '@liemle3893/e2e-runner';

export default {
name: 'Create User Flow',
tags: ['integration', 'users'],
priority: 'P1',

async execute(ctx) {
// Create user via API
const createRes = await ctx.http.post('/api/users', {
email: test-${ctx.uuid()}@example.com,
name: 'Test User',
});

ctx.expect(createRes.status).toBe(201);
ctx.capture('userId', createRes.body.id);

// Verify in database
const dbResult = await ctx.postgresql.query(
'SELECT * FROM users WHERE id = $1',
[ctx.captured.userId]
);

ctx.expect(dbResult.rows).toHaveLength(1);
ctx.expect(dbResult.rows[0].email).toContain('test-');
},

async teardown(ctx) {
await ctx.postgresql.exec(
'DELETE FROM users WHERE id = $1',
[ctx.captured.userId]
);
},
} as TestDefinition;
`

Variable Interpolation

$3

| Function | Description | Example |
|----------|-------------|---------|
| ${uuid()} | Generate UUID v4 | 550e8400-e29b-41d4-a716-446655440000 |
| ${now()} | Current timestamp (ISO) | 2024-01-15T10:30:00.000Z |
| ${timestamp()} | Unix timestamp (ms) | 1705315800000 |
| ${random()} | Random number 0-1 | 0.7234 |
| ${randomInt(min, max)} | Random integer | 42 |

$3

Access environment variables with ${env.VAR_NAME}:

`yaml
params:
apiKey: "${env.API_KEY}"
`

$3

Access values captured from previous steps:

`yaml
params:
url: "/api/users/${captured.userId}"
`

Adapters

$3

`yaml
- adapter: http
action: POST # GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS
params:
url: /api/endpoint
headers:
Authorization: "Bearer ${token}"
body:
key: value
timeout: 5000
`

$3

`yaml
- adapter: postgresql
action: query # or 'exec'
params:
query: "SELECT * FROM users WHERE id = $1"
params: [123]
`

$3

`yaml
- adapter: mongodb
action: find # insertOne, updateOne, deleteOne, find, etc.
params:
collection: users
filter:
email: "test@example.com"
`

$3

`yaml
- adapter: redis
action: get # set, del, hgetall, etc.
params:
key: "user:123"
`

Assertions

| Operator | Description |
|----------|-------------|
| equals | Strict equality |
| notEquals | Not equal |
| contains | String/array contains |
| notContains | Does not contain |
| greaterThan | Numeric > |
| lessThan | Numeric < |
| matches | Regex match |
| isArray | Is array type |
| isObject | Is object type |
| hasLength | Array/string length |
| hasProperty | Object has property |

Reporters

$3

Default output to terminal with colors and progress.

$3

XML output for CI/CD integration:

`yaml
reporters:
- type: junit
output: "./reports/junit.xml"
`

$3

Interactive HTML report:

`yaml
reporters:
- type: html
output: "./reports/report.html"
`

$3

Machine-readable JSON:

`yaml
reporters:
- type: json
output: "./reports/results.json"
`

Programmatic API

`typescript
import { runTests, validateTests, listTests, checkHealth } from '@liemle3893/e2e-runner';

// Run tests
const result = await runTests({
config: './e2e.config.yaml',
testDir: './tests',
tag: ['smoke'],
parallel: 4,
});

console.log(Passed: ${result.passed}, Failed: ${result.failed});

// Validate test syntax
const validation = await validateTests({ testDir: './tests' });
if (!validation.valid) {
console.error('Validation errors:', validation.errors);
}

// List available tests
const tests = await listTests({ tag: ['integration'] });
tests.forEach(t => console.log(- ${t.name} (${t.tags.join(', ')})));

// Check adapter health
const health = await checkHealth();
console.log('All adapters healthy:', health.healthy);
``

License

MIT