Ultimate QA MCP Server - AI-Driven Automated Testing

An advanced MCP (Model Context Protocol) server that converts natural language QA scenarios into fully automated test execution. This server supports comprehensive testing across UI (Playwright), APIs (HTTP), Databases (MySQL/PostgreSQL/MongoDB), and Performance monitoring.

🎯 Overview

The Ultimate QA MCP Server is a production-ready intelligent testing framework that:
- ✅ Converts plain English QA scenarios to automated tests
- ✅ Executes tests across UI, API, and Database layers
- ✅ Supports multiple LLM providers (OpenAI, Anthropic, Custom)
- ✅ Manages test context, variables, and extracted data
- ✅ Provides intelligent error recovery with retry logic
- ✅ Generates comprehensive test reports

$3

- Natural Language QA - Describe tests in plain English
- Multi-Executor Architecture - UI, API, DB, Performance testing
- Multi-LLM Support - OpenAI GPT-4, Anthropic Claude, Custom models
- Context Management - Session tracking, variables, data extraction
- Error Recovery - Exponential backoff retry with heuristics
- Cross-Browser Testing - Chromium, Firefox, WebKit

---

✨ Core Features

$3

1. parse_scenario - Convert English QA descriptions to structured test steps
2. execute_scenario - Parse and execute complete test scenarios
3. launch_browser - Start Playwright browser instance (chromium/firefox/webkit)
4. close_browser - Clean up browser resources
5. get_context - Retrieve session variables and extracted data
6. initialize_llm - Setup LLM provider (OpenAI, Anthropic, Custom)
7. get_status - Server status and active browsers

$3

- LLM Integration - Multi-model abstraction (OpenAI, Anthropic, Custom)
- Natural Language Parser - English to JSON step conversion
- UI Executor - Playwright-based browser automation (17 action types)
- API Executor - HTTP testing with auth, validation, extraction
- Database Executor - MySQL, PostgreSQL, MongoDB support
- Context Manager - Session tracking, variables, data extraction
- Action Router - Orchestrates multi-executor workflows
- Error Handler - Retry logic with exponential backoff
- Configuration - Environment-based setup
- Type System - Full TypeScript type safety
- MCP Server - Protocol implementation

---

📦 Installation

$3

- Node.js 17 or higher
- npm 8 or higher
- TypeScript 5.3.3+
- Playwright 1.48.0+

$3

1. Clone the repository:
``bash
git clone https://github.com/rashaideh93/Ultimate-QA-MCP-Server-AI-Driven-Automated-Testing.git
cd test
`

2. Install dependencies:
`bash
npm install
`

3. Configure environment:
`bash
cp .env.example .env
# Edit .env with your settings:
# LLM_PROVIDER=openai (or anthropic, custom)
# LLM_MODEL_NAME=gpt-4
# LLM_API_KEY=your-api-key
`

4. Build the project:
`bash
npm run build
`

5. Start the server:
`bash
npm start
`

---

🚀 Usage

$3

Simply describe your test in plain English:

`
"Navigate to https://example.com/login,
fill email with admin@test.com,
fill password with Pass123!,
click the Login button,
wait 2 seconds for page load,
verify Welcome message appears,
take a screenshot"
`

The server will automatically:
1. ✅ Parse the scenario into structured steps
2. ✅ Launch a Playwright browser
3. ✅ Execute each action in sequence
4. ✅ Capture screenshots and logs
5. ✅ Return detailed results

$3

`bash

Build the project

npm run build

Start the server

npm start

Run tests (in another terminal)

npm test

Watch mode for development

npm run dev
`

$3

#### Claude Desktop

Add to claude_desktop_config.json:
`json
{
"mcpServers": {
"ultimate-qa": {
"command": "node",
"args": ["/path/to/test/build/index.js"],
"env": {
"LLM_PROVIDER": "openai",
"LLM_MODEL_NAME": "gpt-4",
"LLM_API_KEY": "sk-..."
}
}
}
}
`

#### VS Code Copilot

1. Press Cmd+Shift+P (Mac) or Ctrl+Shift+P (Windows/Linux)
2. Select MCP: Add server...
3. Configure with node ./build/index.js

#### Other MCP Clients

Configure stdio transport connecting to node ./build/index.js

---

📝 Usage Examples

$3

Natural Language Input:
`
"Navigate to example.com/login,
fill email with test@example.com,
fill password with TestPass123,
click Login button,
verify Dashboard page appears"
`

Automatic Execution:
- ✅ Parse scenario
- ✅ Launch browser
- ✅ Fill form fields
- ✅ Click button
- ✅ Verify page state
- ✅ Generate report

$3

Scenario:
`
"Create user via POST /api/users with name: John, email: john@test.com
Extract userId from response
Navigate to /users/dashboard
Search for user by email
Verify user appears in list"
`

Result:
- ✅ API call execution
- ✅ Data extraction
- ✅ UI navigation
- ✅ Element verification
- ✅ Test report

$3

Scenario:
`
"Add product to cart via UI
Complete purchase
Verify order status in database
Check inventory was updated"
`

Result:
- ✅ UI actions
- ✅ API calls
- ✅ Database queries
- ✅ Cross-layer validation

$3

See USAGE_EXAMPLES.md for:
- 7+ detailed real-world scenarios
- Multi-tab testing
- Data-driven testing
- Conditional flows
- Performance monitoring
- CI/CD integration examples

---

🏗️ Architecture

$3

`
┌─────────────────────────────────────────┐
│ MCP Server (7 Tools) │
├─────────────────────────────────────────┤
│ Application Layer │
│ - Action Router │
│ - Context Manager │
│ - Error Handler │
├─────────────────────────────────────────┤
│ Execution Layer (Multi-Executor) │
│ - UI Executor (Playwright) │
│ - API Executor (HTTP) │
│ - DB Executor (MySQL/Postgres/MongoDB) │
│ - Performance Monitor │
├─────────────────────────────────────────┤
│ Intelligence Layer │
│ - LLM Provider (OpenAI/Anthropic) │
│ - Natural Language Parser │
├─────────────────────────────────────────┤
│ Infrastructure │
│ - Playwright │
│ - Axios HTTP Client │
│ - Database Drivers │
└─────────────────────────────────────────┘
`

$3

| Module | Lines | Purpose |
|--------|-------|---------|
| index.ts | 290 | MCP server entry point |
| config.ts | 140 | Environment configuration |
| types.ts | 320 | TypeScript type definitions |
| llm.ts | 290 | LLM provider abstraction |
| nlParser.ts | 365 | Natural language to JSON |
| contextManager.ts | 330 | Session & variable tracking |
| uiExecutor.ts | 415 | Playwright automation |
| apiExecutor.ts | 160 | HTTP testing |
| dbExecutor.ts | 230 | Database operations |
| actionRouter.ts | 275 | Orchestration |
| errorHandler.ts | 205 | Error recovery |

$3

- Exponential Backoff - Automatic retry with increasing delays
- Selector Heuristics - Multiple fallback strategies
- Error Recovery - Graceful degradation
- Comprehensive Logging - Full execution trace

$3

`bash
npm test # Run all tests
npm run test:watch # Watch mode
npm run test:coverage # Coverage report
npm run build # Build TypeScript
`

For detailed testing guide, see the test files in src/__tests__/

---

📁 Project Structure

`
ultimate-qa-mcp-server/
├── src/ # Source code
│ ├── index.ts # MCP server entry
│ ├── config.ts # Configuration
│ ├── types.ts # Type definitions
│ ├── llm.ts # LLM abstraction
│ ├── nlParser.ts # NL parser
│ ├── contextManager.ts # Session manager
│ ├── uiExecutor.ts # Playwright
│ ├── apiExecutor.ts # HTTP testing
│ ├── dbExecutor.ts # Database
│ ├── actionRouter.ts # Orchestration
│ ├── errorHandler.ts # Error handling
│ └── __tests__/ # Test suites
│
├── build/ # Compiled JS (auto-generated)
│
├── 📚 Documentation
│ ├── README.md # This file
│ ├── ULTIMATE_QA_GUIDE.md # Complete user guide (2000+ lines)
│ ├── USAGE_EXAMPLES.md # Real-world examples
│ └── API_REFERENCE.md # MCP tool reference
│
├── ⚙️ Configuration
│ ├── package.json # Dependencies
│ ├── tsconfig.json # TypeScript config
│ ├── jest.config.cjs # Test config
│ ├── .env.example # Environment template
│ └── qodana.yaml # Code analysis
│
└── 📋 Version Control
├── .git/ # Git repository
├── .gitignore # Git exclusions
└── .npmignore # NPM exclusions
`

$3

`bash
npm run build # Compile TypeScript to JavaScript
npm run dev # Development mode with watch
npm start # Start the server
`

$3

`bash

Run all tests

npm test

Test specific files

npm test -- actionRouter

Use MCP Inspector for interactive testing

npx @modelcontextprotocol/inspector node ./build/index.js
`

The MCP Inspector provides:
- Interactive tool testing
- Schema inspection
- Output visualization
- Debugging interface

---

🤖 AI Integration

$3

Configure in claude_desktop_config.json:
`json
{
"mcpServers": {
"ultimate-qa": {
"command": "node",
"args": ["/path/to/test/build/index.js"],
"env": {
"LLM_PROVIDER": "openai",
"LLM_MODEL_NAME": "gpt-4",
"LLM_API_KEY": "sk-..."
}
}
}
}
`

$3

1. Install MCP extension for VS Code
2. Configure with node ./build/index.js
3. Use natural language for test scenarios

$3

Compatible with:
- LM Studio - Local LLM deployment
- Cursor - Code editor with AI
- Windsurf - AI-powered IDE
- Cline - Command-line interface
- Custom clients - Any MCP-compatible application

$3

OpenAI:
- GPT-4 (recommended)
- GPT-3.5-turbo

Anthropic:
- Claude 3 Opus
- Claude 3 Sonnet
- Claude 2

Custom:
- On-premise OpenAI-compatible API
- Self-hosted models

---

⚙️ Configuration

$3

Create .env file with:

`bash

LLM Configuration

LLM_PROVIDER=openai # openai | anthropic | custom
LLM_MODEL_NAME=gpt-4 # Model to use
LLM_API_KEY=sk-... # Your API key
LLM_BASE_URL= # For custom models
LLM_TIMEOUT=30000 # Request timeout (ms)

Database Configuration (Optional)

DB_TYPE=mysql # mysql | postgres | mongodb
DB_HOST=localhost
DB_PORT=3306
DB_USERNAME=user
DB_PASSWORD=password
DB_NAME=testdb

Server Configuration

SCREENSHOTS_ENABLED=true # Capture screenshots
SCREENSHOTS_PATH=./screenshots
REPORTS_ENABLED=true
REPORTS_PATH=./qa-reports
REPORT_FORMATS=json,markdown,html

Performance

PERFORMANCE_CHECKS=true
PAGE_LOAD_THRESHOLD_MS=3000
API_RESPONSE_THRESHOLD_MS=1000

Retries

MAX_RETRY_ATTEMPTS=3
RETRY_DELAY_MS=1000
RETRY_BACKOFF_MULTIPLIER=1.5
`

$3

- browserType - "chromium" (default), "firefox", "webkit"
- headless - true (default) or false for headed mode
- url - Initial URL to navigate to

$3

- MySQL - Version 5.7+, Driver: mysql2
- PostgreSQL - Version 10+, Driver: pg
- MongoDB - Version 4.0+, Driver: mongodb

---

🔧 Troubleshooting

$3

- ✅ Ensure Node.js 17+ installed: node --version
- ✅ Check dependencies: npm install
- ✅ Verify build: npm run build
- ✅ Check .env file exists with LLM_API_KEY

$3

- ✅ Verify absolute path in MCP config is correct
- ✅ Confirm build/index.js exists
- ✅ Check stderr output: npm start 2>&1
- ✅ Test MCP connection: npx @modelcontextprotocol/inspector node ./build/index.js

$3

- ✅ Verify element selectors are correct (CSS or XPath)
- ✅ Check elements exist before interaction
- ✅ Review Playwright docs for specific elements
- ✅ Enable screenshots for debugging: SCREENSHOTS_ENABLED=true
- ✅ Check logs in ./qa-reports directory

$3

- ✅ Use headless mode (default): faster execution
- ✅ Close browsers when done: close_browser
- ✅ Use appropriate waits: avoid excessive delays
- ✅ Enable connection pooling for databases
- ✅ Monitor network requests in DevTools

$3

- ✅ Verify DB connection string in .env
- ✅ Check database is running and accessible
- ✅ Ensure user has appropriate permissions
- ✅ Test connection: mysql -u user -p -h host

For more help, see ULTIMATE_QA_GUIDE.md

---

📖 MCP Tools Reference

$3

Initialize LLM provider for natural language parsing
`json
{
"provider": "openai|anthropic|custom",
"modelName": "gpt-4|claude-3-opus|...",
"apiKey": "your-api-key",
"baseUrl": "optional-for-custom"
}
`

$3

Convert English QA scenario to structured steps
`json
{
"scenarioText": "Navigate to login, fill email with test@test.com, click Login",
"extractVariables": true
}
`

$3

Parse and execute complete scenario
`json
{
"browserId": "browser-1",
"scenarioText": "Your test scenario...",
"generateReport": true
}
`

$3

Start Playwright browser instance
`json
{
"browserType": "chromium|firefox|webkit",
"headless": true,
"url": "https://example.com"
}
`

$3

Clean up browser resources
`json
{
"browserId": "browser-1"
}
`

$3

Retrieve session variables and data
`json
{
"browserId": "browser-1"
}
`

$3

Server status and active browsers
`json
{}
`

For detailed tool documentation, see API_REFERENCE.md

---

📦 Dependencies

$3

- @modelcontextprotocol/sdk (2.0+) - MCP protocol
- playwright (1.48+) - Browser automation
- axios (1.6+) - HTTP client
- zod (3.22+) - Type validation
- dotenv (16.3+) - Environment config

$3

- mysql2 (3.6+) - MySQL driver
- pg (8.11+) - PostgreSQL driver
- mongodb (6.3+) - MongoDB driver

$3

- typescript (5.3+) - Language support
- jest (29+) - Testing framework
- @types/node - Type definitions

---

📊 Project Statistics

| Metric | Value |
|--------|-------|
| Core Modules | 11 |
| MCP Tools | 7 |
| Total Lines | ~3,500 |
| Type Coverage | 100% |
| Compilation Errors | 0 |
| Test Coverage | Included |

---

📄 Documentation

Complete documentation available:

1. ULTIMATE_QA_GUIDE.md (2000+ lines)
- Complete user guide
- Detailed examples
- Best practices
- Advanced features

2. USAGE_EXAMPLES.md
- 7+ real-world scenarios
- Multi-system testing
- Performance monitoring
- CI/CD integration

3. API_REFERENCE.md
- MCP tool documentation
- Parameter details
- Response formats

---

🔐 Security & Best Practices

✅ 100% TypeScript - Full type safety
✅ Input Validation - Zod schema validation
✅ Error Handling - Comprehensive error recovery
✅ Secrets Management - Environment variable based
✅ Connection Pooling - Database optimization
✅ Retry Logic - Exponential backoff

---

📜 License

MIT License - See LICENSE file for details

---

🤝 Contributing

Contributions welcome! Please:
1. Fork the repository
2. Create a feature branch
3. Submit a pull request
4. Include test coverage

---

📚 Resources

- MCP Documentation
- Playwright Documentation
- OpenAI API
- Anthropic Claude

---

💬 Support

For issues and questions:

1. Check Documentation - See guides above
2. Review Examples - See USAGE_EXAMPLES.md
3. Check Logs - Enable debug logging
4. Run Tests - npm test`
5. Open Issue - GitHub repository

---

Version: 2.0.0