Automated testing for MCP servers and agents

Headless MCP client for running natural-language test scripts that are fast, repeatable, and CI-friendly.

test-mcp demo

$ npm install -g @loadmill/test-mcp

$ test-mcp

🤖 Using: openai/gpt-4o-mini

🧪 Running tests (default mode)

🚀 Found 1 test file(s)

📋 Running test: Dice roll with remote MCP server integration test

💬 Step 1: Roll a dice for me

🔧 Tool: dice/rollDice

✅ Response (1900ms): You rolled a 4!

✅ PASSED - All tests completed successfully

Why test-mcp?

📝

YAML Test Scripts

Write natural language test flows with simple prompts and assertions

🔌

STDIO + HTTP Support

Connect to local and remote MCP servers with both transport types

🤖

Multiple LLM Providers

Works with Anthropic Claude and OpenAI GPT models out of the box

🔄

CI Ready

Fast, headless operation perfect for automated testing pipelines

Quick Start

1. Install

npm install -g @loadmill/test-mcp

2. Configure

Create mcp.config.json:

{
  "mcpClient": {
    "provider": "openai",
    "model": "gpt-4o-mini",
    "api_key": "${env:OPENAI_API_KEY}"
  },
  "mcpServers": {
    "myserver": {
      "type": "stdio",
      "command": "npx",
      "args": ["@my/mcp-server"]
    }
  }
}

3. Write Tests

Create tests/example.test.yaml:

description: "Test my MCP server"

steps:
  - prompt: "Use the weather tool to get temperature in NYC"
  - assert: "The weather data was retrieved successfully"

4. Run

test-mcp

Help shape the future of test-mcp

We’re building this together. Contribute ideas, code, or feedback on GitHub.

Contribute on GitHub