Foundry QA Agent MCP Server

AI-powered QA testing via Model Context Protocol (MCP). Uses GLM-4.7 for test execution and Gemini 3 Flash for visual UX/UI critique.

Overview

This MCP server provides a single tool (run_qa_test) that executes browser-based QA tests using an autonomous AI agent. The agent:

- Navigates and interacts with your web application using Playwright
- Validates functionality and behavior
- Captures screenshots at key checkpoints
- Uses Gemini 3 Flash for visual UX/UI critique
- Reports findings with severity levels and reproduction steps

Setup

$3

``bash
npm install -g @diyor28/qa-mcp-server
npx playwright install chromium
`

$3

`bash
git clone https://github.com/iota-uz/foundry
cd foundry
pnpm install
pnpm build:qa-core && pnpm build:qa-mcp-server
cd packages/qa-core && npx playwright install chromium
`

$3

Required:

`bash
export CEREBRAS_API_KEY="your-cerebras-api-key"
export GEMINI_API_KEY="your-gemini-api-key"
`

Optional configuration:

`bash
export HEADLESS="true" # Run browser in headless mode
export ENABLE_TRACE="false" # Capture Playwright traces
export TRACE_DIR="/tmp/qa-traces" # Where to save traces
export MAX_ITERATIONS="15" # Max agent loop iterations
export DEFAULT_VIEWPORT_WIDTH="1280"
export DEFAULT_VIEWPORT_HEIGHT="800"
`

$3

Edit ~/Library/Application Support/Claude/claude_desktop_config.json:

If installed from npm:

`json
{
"mcpServers": {
"foundry-qa": {
"command": "npx",
"args": ["-y", "@diyor28/qa-mcp-server"],
"env": {
"CEREBRAS_API_KEY": "your-key",
"GEMINI_API_KEY": "your-key"
}
}
}
}
`

If installed from source:

`json
{
"mcpServers": {
"foundry-qa": {
"command": "node",
"args": ["/absolute/path/to/foundry/packages/qa-mcp-server/dist/index.js"],
"env": {
"CEREBRAS_API_KEY": "your-key",
"GEMINI_API_KEY": "your-key"
}
}
}
}
`

Restart Claude Desktop.

Usage

$3

Ask Claude:

> "Use run_qa_test to validate the login flow on http://localhost:3000"

$3

Provide a detailed scenario:

> "Run a QA test with this scenario:
>
> `
> {
> "baseUrl": "http://localhost:3000",
> "scenario": {
> "name": "Checkout Flow",
> "content": "1. Add item to cart\n2. Proceed to checkout\n3. Fill invalid card details\n4. Verify error message\n5. Fill valid card\n6. Complete order"
> }
> }
> `

$3

> "Critique the UX/UI of http://localhost:3000/dashboard, focusing on layout clutter, color contrast, and navigation intuitiveness"

How It Works

$3

`
MCP Server (qa-mcp-server)
↓
QA Core Library (qa-core)
├── BrowserController (Playwright)
├── VisionAnalyzer (Gemini 3 Flash)
├── AgentLoop (GLM-4.7 via Cerebras)
└── ReportBuilder
`

$3

1. Initialization: Launch Playwright browser with configured viewport
2. Agent Execution: GLM-4.7 agent follows scenario instructions, using tools:
- navigate - Go to URLs
- click - Click elements
- fill - Fill form fields
- screenshot - Capture screenshots
- analyze_ui_ux - Request visual critique from Gemini
- assert_visible, assert_text - Validate behavior
3. Visual Analysis: Gemini 3 Flash analyzes screenshots for UX/UI issues
4. Report Generation: Findings aggregated with severity levels
5. Artifact Collection: Screenshots and traces packaged in report

$3

The agent uses Foundry's @foundry/context library for intelligent context management:

- Token budgeting: Automatically fits within GLM-4.7's 200K context window
- Auto-compaction: Old tool outputs are pruned when context overflows
- Structured blocks: System prompt, scenario, browser state, history

Debugging

$3

See the browser in action:

`bash
export HEADLESS="false"
`

$3

Enable Playwright tracing for detailed debugging:

`bash
export ENABLE_TRACE="true"
export TRACE_DIR="/tmp/qa-traces"
`

View traces with:

`bash
pnpm playwright show-trace /tmp/qa-traces/trace-*.zip
`

$3

Progress events are logged to stderr. When running locally:

`bash
node dist/index.js 2> qa-debug.log
`

Report Structure

The tool returns a JSON report with:

`json
{
"summary": "Test completed with 2 issue(s) found: 1 high, 1 medium.",
"status": "issues_found",
"duration": 45230,
"findings": [
{
"severity": "high",
"title": "Poor color contrast on login button",
"description": "WCAG AA violation: contrast ratio 2.1:1 (requires 4.5:1)",
"reproSteps": ["Navigate to /login", "Observe primary CTA button"],
"category": "accessibility",
"screenshotName": "login-page"
}
],
"artifacts": [
{
"type": "screenshot",
"name": "login-page",
"contentType": "image/png",
"base64Data": "iVBORw0KGgoAAAANS..."
}
],
"metadata": {
"scenarioName": "Login Flow",
"baseUrl": "http://localhost:3000",
"viewport": { "width": 1280, "height": 800 },
"timestamp": "2026-01-27T12:00:00.000Z"
}
}
`

$3

- critical: Blocks core functionality, immediate fix required
- high: Significant issue affecting user experience
- medium: Noticeable issue, should be fixed
- low: Minor issue, cosmetic or edge case

$3

- functional: Core functionality bugs
- ux: User experience issues
- ui: Visual design issues
- accessibility: WCAG compliance violations

Integration with Backend

The qa-core library can be imported into the Foundry backend:

`typescript
import { QaRunner } from '@diyor28/qa-core';

const runner = new QaRunner({
browser: { headless: true, viewport: { width: 1280, height: 800 } },
models: {
cerebrasApiKey: process.env.CEREBRAS_API_KEY!,
geminiApiKey: process.env.GEMINI_API_KEY!,
},
baseUrl: 'http://localhost:3000',
scenario: {
name: 'Test Scenario',
content: '1. Navigate to homepage\n2. Verify title',
},
});

const report = await runner.run();
`

This enables the existing QA service to use the same agent logic without MCP overhead.

Limitations

- Local browser only: Uses local Playwright, not Browserbase
- Single scenario per run: No test suite execution
- English only: Agent prompts in English

Troubleshooting

$3

Install Chromium:

`bash
pnpm playwright install chromium
`

$3

Set environment variables before starting:

`bash
export CEREBRAS_API_KEY="your-key"
export GEMINI_API_KEY="your-key"
`

$3

Try increasing max iterations:

`bash
export MAX_ITERATIONS="20"
`

Or provide more specific scenario instructions.

$3

The agent automatically captures screenshots. If none appear in artifacts, check:

1. Browser launched successfully
2. Navigation succeeded
3. Agent loop completed without errors

Development

$3

`bash
pnpm dev # Run with tsx for development
`

$3

`bash


Start the server

node dist/index.js

In another terminal, send a test request (requires MCP client)

``

License

Private - Foundry internal use only