**Deterministic Test Failure Classification & Analysis for Playwright**
npm install quality-intelligence-engineDeterministic Test Failure Classification & Analysis for Playwright
A rule-based intelligent system that automatically classifies test failures, provides root cause analysis, and delivers actionable insightsβwithout relying on AI black boxes.
---
The Quality Intelligence Engine analyzes Playwright test failures and classifies them into:
- API_BUG: Backend/API failures (authentication errors, data mismatches, server errors)
- UI_BUG: Frontend/UI issues (visibility problems, rendering failures, element issues)
Each classification comes with:
- β
Confidence score (0-100%) - How certain we are about the classification
- β
Root cause analysis - What went wrong and why
- β
Evidence - Signals that led to the classification
- β
Diagnosis - Step-by-step breakdown of the failure
- β
Suggested actions - Concrete next steps to fix the issue
---
``bash`
npm install
`bashRun Playwright tests
npx playwright test
The engine will:
1. Read test results from
playwright-report/report.json
2. Classify failures using the taxonomy
3. Generate detailed analysis reports
4. Output results to output/ directory---
π Project Structure
`
quality-intelligence-engine/
βββ src/
β βββ intelligence/ # Classification algorithms
β βββ pipeline/ # Data processing
β βββ utils/ # Utilities
β βββ types.ts # Type definitions
β βββ normalizer.ts # Input normalization
β βββ configLoader.ts # Configuration
β βββ reporter.ts # Output generation
βββ tests/ # Example test cases
βββ config/ # Configuration files
βββ playwright.config.ts # Playwright configuration
`---
βοΈ Configuration
Edit
config/agent.config.json:`json
{
"engine": {
"mode": "standard",
"confidenceThresholds": {
"fail": 0.85,
"passRisk": 0.6
}
}
}
`---
π·οΈ Taxonomy System
Tests are tagged with their expected failure type:
`typescript
test('Data | NUMERIC_MISMATCH | inventory count wrong', async ({ page }) => {
// Test code...
});
`---
π Confidence Scoring
Deterministic, explainable confidence calculations:
- Base Confidence: 0.6 - 0.95
- Adjustments: +0.05 - +0.1 based on signals
- Maximum: 0.95 (never 100% certain)
---
π§ Development
`bash
Run tests
npx playwright testType check
npx tsc --noEmit
``---
Version: 1.0.0 (Refactored)
Last Updated: 2026-01-29