Quality Intelligence Engine

Deterministic Test Failure Classification & Analysis for Playwright

A rule-based intelligent system that automatically classifies test failures, provides root cause analysis, and delivers actionable insights—without relying on AI black boxes.

---

🎯 Overview

The Quality Intelligence Engine analyzes Playwright test failures and classifies them into:

- API_BUG: Backend/API failures (authentication errors, data mismatches, server errors)
- UI_BUG: Frontend/UI issues (visibility problems, rendering failures, element issues)

Each classification comes with:
- ✅ Confidence score (0-100%) - How certain we are about the classification
- ✅ Root cause analysis - What went wrong and why
- ✅ Evidence - Signals that led to the classification
- ✅ Diagnosis - Step-by-step breakdown of the failure
- ✅ Suggested actions - Concrete next steps to fix the issue

---

🚀 Quick Start

$3

``bash
npm install
`

$3

`bash


Run Playwright tests

npx playwright test

Run Quality Intelligence Engine

npm start
`

The engine will:
1. Read test results from playwright-report/report.json
2. Classify failures using the taxonomy
3. Generate detailed analysis reports
4. Output results to output/ directory

---

📁 Project Structure

`
quality-intelligence-engine/
├── src/
│ ├── intelligence/ # Classification algorithms
│ ├── pipeline/ # Data processing
│ ├── utils/ # Utilities
│ ├── types.ts # Type definitions
│ ├── normalizer.ts # Input normalization
│ ├── configLoader.ts # Configuration
│ └── reporter.ts # Output generation
├── tests/ # Example test cases
├── config/ # Configuration files
└── playwright.config.ts # Playwright configuration
`

---

⚙️ Configuration

Edit config/agent.config.json:

`json
{
"engine": {
"mode": "standard",
"confidenceThresholds": {
"fail": 0.85,
"passRisk": 0.6
}
}
}
`

---

🏷️ Taxonomy System

Tests are tagged with their expected failure type:

`typescript
test('Data | NUMERIC_MISMATCH | inventory count wrong', async ({ page }) => {
// Test code...
});
`

---

📊 Confidence Scoring

Deterministic, explainable confidence calculations:

- Base Confidence: 0.6 - 0.95
- Adjustments: +0.05 - +0.1 based on signals
- Maximum: 0.95 (never 100% certain)

---

🔧 Development

`bash


Run tests

npx playwright test

Type check

npx tsc --noEmit
``

---

Version: 1.0.0 (Refactored)
Last Updated: 2026-01-29