runhuman


> AI-orchestrated human QA testing from your terminal

What is Runhuman?

Runhuman is an AI-orchestrated human QA testing service. Get real human testing of your web applications on-demand, with structured results extracted automatically.

Perfect for:
- AI coding agents that need human verification
- CI/CD pipelines with human QA gates
- Visual/UX testing requiring real human judgment
- On-demand QA without hiring/managing teams

Installation

``bash


Install globally

npm install -g runhuman

Or use with npx

npx runhuman --help
`

Quick Start

`bash


1. Login with your API key

runhuman login

2. Create your first test

runhuman create https://myapp.com -d "Test the checkout flow"

3. Check the status

runhuman status

4. Get results

runhuman results
`

Commands

$3

#### create [url]

Create a new QA test job.

`bash


Basic usage

runhuman create https://myapp.com -d "Test checkout flow"

With template

runhuman create https://myapp.com --template tmpl_abc123

Synchronous (wait for result)

runhuman create https://myapp.com -d "Test" --sync

With custom schema

runhuman create https://myapp.com -d "Test" --schema ./schema.json
`

Options:
- -d, --description - Test instructions for the human tester
- --template - Use a pre-defined template
- --duration - Target test duration
- --screen-size - Screen size (desktop/mobile/tablet)
- --schema - Path to JSON schema for structured output
- --sync - Wait for result before exiting
- --json - Output as JSON

#### status

Check the status of a test job.

`bash
runhuman status job_abc123
`

#### wait

Wait for a job to complete and display results.

`bash
runhuman wait job_abc123
runhuman wait job_abc123 --timeout 300
`

Options:
- --timeout - Maximum wait time (default: 600)

#### results

Display detailed test results.

`bash
runhuman results job_abc123
runhuman results job_abc123 --json
`

#### list

List all your test jobs.

`bash


List all jobs

runhuman list

Filter by status

runhuman list --status completed

Filter by project

runhuman list --project proj_abc123

Limit results

runhuman list --limit 20
`

Options:
- --status - Filter by status (pending/claimed/in_progress/completed/failed)
- --project - Filter by project
- --limit - Maximum number of jobs to return
- --json - Output as JSON

$3

#### projects list

List all your projects.

`bash
runhuman projects list
runhuman projects ls # alias
`

#### projects create

Create a new project.

`bash
runhuman projects create "My App"
runhuman projects create "My App" -d "Production QA testing"
`

Options:
- -d, --description - Project description
- --default-url - Default URL to test
- --github-repo - Link GitHub repository

#### projects show

Show details of a specific project.

`bash
runhuman projects show proj_abc123
runhuman projects info proj_abc123 # alias
`

#### projects update

Update a project.

`bash
runhuman projects update proj_abc123 --name "New Name"
runhuman projects update proj_abc123 --description "New description"
`

Options:
- --name - New project name
- -d, --description - New description
- --default-url - New default URL
- --github-repo - New GitHub repository

#### projects delete

Delete a project permanently.

`bash
runhuman projects delete proj_abc123 --force
`

Options:
- --force - Skip confirmation prompt

$3

#### keys list

List all API keys for a project.

`bash
runhuman keys list --project proj_abc123
runhuman keys ls --project proj_abc123 # alias
`

Options:
- --project - Project ID (required)
- --show-keys - Show full API keys (default: masked)

#### keys create

Create a new API key.

`bash
runhuman keys create "CI/CD Key" --project proj_abc123
runhuman keys new "CI/CD Key" --project proj_abc123 # alias
`

Options:
- --project - Project ID (required)
- --copy - Copy key to clipboard

#### keys delete

Delete an API key permanently.

`bash
runhuman keys delete key_abc123 --force
runhuman keys rm key_abc123 --force # alias
`

Options:
- --force - Skip confirmation prompt

$3

#### templates list

List all test templates for a project.

`bash
runhuman templates list --project proj_abc123
runhuman templates ls --project proj_abc123 # alias
`

Options:
- --project - Project ID (required)

#### templates create

Create a new test template.

`bash
runhuman templates create "Checkout Flow" --project proj_abc123
runhuman templates create "Login Test" --project proj_abc123 \
-d "Test login functionality" \
--duration 180 \
--screen-size mobile \
--schema ./schema.json
`

Options:
- --project - Project ID (required)
- -d, --description - Template description
- --duration - Target test duration
- --screen-size - Default screen size
- --schema - Path to JSON schema file

#### templates show

Show details of a specific template.

`bash
runhuman templates show tmpl_abc123
runhuman templates info tmpl_abc123 # alias
`

#### templates update

Update a template.

`bash
runhuman templates update tmpl_abc123 --name "New Name"
runhuman templates update tmpl_abc123 --description "New description"
runhuman templates update tmpl_abc123 --schema ./new-schema.json
`

Options:
- --name - New template name
- -d, --description - New description
- --duration - New target duration
- --screen-size - New default screen size
- --schema - Path to new JSON schema file

#### templates delete

Delete a template permanently.

`bash
runhuman templates delete tmpl_abc123 --force
runhuman templates rm tmpl_abc123 --force # alias
`

Options:
- --force - Skip confirmation prompt

$3

#### github link

Link a GitHub repository to your project.

`bash
runhuman github link owner/repo --project proj_abc123
runhuman gh link volter-ai/runhuman --project proj_abc123 # alias
`

Options:
- --project - Project ID (required)

#### github repos

List linked GitHub repositories.

`bash
runhuman github repos
runhuman github repos --project proj_abc123 # filter by project
`

Options:
- --project - Filter by project

#### github issues

List GitHub issues for a repository.

`bash
runhuman github issues owner/repo
runhuman github issues owner/repo --state open
runhuman github issues owner/repo --labels "bug,needs-qa"
`

Options:
- --state - Filter by state (open/closed/all, default: open)
- --labels - Filter by comma-separated labels

#### github test

Create a QA test job for a GitHub issue.

`bash
runhuman github test 123 --repo owner/repo --url https://myapp.com
runhuman github test 123 --url https://myapp.com --sync # wait for result
`

Options:
- --repo - Repository (or use default from config)
- --url - URL to test (required)
- --template - Template ID to use
- --sync - Wait for result before exiting

#### github bulk-test

Create QA test jobs for multiple GitHub issues.

`bash


Test all open issues

runhuman github bulk-test --repo owner/repo --url https://myapp.com

Test issues with specific labels

runhuman github bulk-test --repo owner/repo --url https://myapp.com \
--labels "bug,needs-qa" \
--limit 5
`

Options:
- --repo - Repository (or use default from config)
- --url - URL to test (required)
- --labels - Filter issues by comma-separated labels
- --state - Filter by state (open/closed/all, default: open)
- --template - Template ID to use
- --limit - Maximum number of jobs to create (default: 10)

$3

#### login

Login to Runhuman and save your API key.

`bash
runhuman login


Prompts for API key

Or provide directly


runhuman login --api-key qa_live_xxxxxxxxxxxxx
`

Get your API key at: runhuman.com/dashboard

#### logout

Logout and clear saved credentials.

`bash
runhuman logout
`

#### whoami

Display current user info and account balance.

`bash
runhuman whoami
`

$3

#### config get

Get a configuration value.

`bash
runhuman config get apiUrl
runhuman config get project
`

#### config set

Set a configuration value.

`bash


Set globally

runhuman config set apiUrl https://api.runhuman.com --global

Set for current project

runhuman config set project proj_abc123
runhuman config set defaultUrl https://myapp.com
`

Options:
- --global - Save to global config instead of project config

#### config list

List all configuration values.

`bash
runhuman config list
`

Shows configuration from all sources:
- Global config (~/.config/runhuman/config.json)
- Project config (.runhumanrc)
- Environment variables
- Effective (merged) config

#### config reset

Reset configuration to defaults.

`bash


Reset project config

runhuman config reset project

Reset global config

runhuman config reset global

Reset all

runhuman config reset all
`

#### init

Initialize a new Runhuman project with interactive prompts.

`bash
runhuman init
`

Creates a .runhumanrc file in the current directory with your project configuration.

Configuration

$3

Configuration is loaded from multiple sources with the following priority (highest to lowest):

1. CLI flags - --api-key, --api-url, etc.
2. Environment variables - RUNHUMAN_API_KEY, RUNHUMAN_API_URL, etc.
3. Project config - .runhumanrc in current directory
4. Global config - ~/.config/runhuman/config.json
5. Defaults

$3

`bash
RUNHUMAN_API_KEY # API key for authentication
RUNHUMAN_API_URL # API base URL (default: https://runhuman.com)
RUNHUMAN_PROJECT # Default project ID
RUNHUMAN_DEFAULT_URL # Default URL to test
RUNHUMAN_DEFAULT_DURATION # Default test duration in seconds
RUNHUMAN_DEFAULT_SCREEN_SIZE # Default screen size (desktop/mobile/tablet)
RUNHUMAN_OUTPUT_FORMAT # Output format (pretty/json/compact)
RUNHUMAN_NO_COLOR=1 # Disable colored output
`

$3

Create a .runhumanrc file in your project root:

`json
{
"project": "proj_abc123",
"defaultUrl": "https://myapp.com",
"defaultDuration": 300,
"defaultScreenSize": "desktop",
"githubRepo": "owner/repo",
"outputFormat": "pretty",
"color": true
}
`

$3

Stored at ~/.config/runhuman/config.json:

`json
{
"apiUrl": "https://runhuman.com",
"apiKey": "qa_live_xxxxxxxxxxxxx",
"color": true
}
`

JSON Output

All commands support --json flag for machine-readable output:

`bash
runhuman create https://myapp.com -d "Test" --json
runhuman status job_abc123 --json
runhuman list --json
`

JSON output format:

`json
{
"success": true,
"data": { ... },
"error": null
}
`

Exit Codes

- 0 - Success
- 1 - General error
- 2 - Authentication error
- 3 - Not found error
- 4 - Validation error
- 5 - Timeout error

Examples

$3

`bash


1. Login

runhuman login

2. Initialize project

runhuman init

3. Create a test

runhuman create https://myapp.com -d "Test the login flow"

4. Wait for results

runhuman wait job_abc123

5. View detailed results

runhuman results job_abc123
`

$3

`bash


Create test and wait for result (fail on error)

runhuman create https://staging.myapp.com \
-d "Smoke test on staging" \
--sync \
--json > result.json

Exit code will be non-zero if test fails

`

$3

`bash


1. Link your repo

runhuman github link owner/repo --project proj_abc123

2. List issues needing QA

runhuman github issues owner/repo --labels "needs-qa"

3. Test a specific issue

runhuman github test 42 --url https://myapp.com --sync

4. Bulk test multiple issues

runhuman github bulk-test --url https://myapp.com --labels "needs-qa" --limit 5
`

$3

`bash


1. Create a template

runhuman templates create "Login Flow Test" --project proj_abc123 \
-d "Test login with valid and invalid credentials" \
--duration 180 \
--schema ./schemas/login-test.json

2. Use the template

runhuman create https://myapp.com --template tmpl_abc123

3. List all templates

runhuman templates list --project proj_abc123
`

MCP Server for AI Agents

If you're using AI agents (Claude, GPT, etc.), check out our MCP server package:

`bash
npm install -g @runhuman/mcp
``

See @runhuman/mcp for details.

Learn More

- Website: runhuman.com
- Documentation: runhuman.com/docs
- GitHub: github.com/volter-ai/runhuman

Support

- Email: hey@runhuman.com
- Issues: GitHub Issues

How It Works

1. You create a job - Define test instructions and optional output schema
2. Job posted to testers - Request goes to human tester pool
3. Human performs test - Real person tests with video/screenshot recording
4. AI extracts data - GPT-4o processes feedback into structured JSON
5. You get results - Clean, typed data ready for automation

Pricing: Pay-per-second ($0.0018/sec of tester time). No monthly fees, no minimums.

---

Built with ❤️ by the Volter AI team