Measurement instrument · Model Context Protocol

Grade your MCP server
in one command.

Models pick the wrong tools. Token bills balloon. Servers break their own protocol. None of it is visible from your server's code — so jig measures it on the wire and hands you a report card you can send to someone.

$ npx -y @shodh/jig check "npx -y your-mcp-server"

Apache-2.0 No account, no telemetry Single checksummed binary

Report cardrubric-v1.5

grade —

mcp-servers/everything v2.0.0

Protocol90

Context cost92

Schema hygiene87

Descriptions100

Robustness100

Fig. 01

Every run writes a report card.

Not a wall of terminal output that scrolls away — a self-contained HTML document with the score, the ranked fixes, and where every token goes. It lands next to you by default, and it survives being forwarded.

The HTML report card jig writes by default: score hero with 93, five dimension bars, a context-bill callout, and a per-tool token chart

./jig-report-mcp-servers-everything.html — written by default, zero JavaScript, opens anywhere.

Fig. 02

The problem is measurable, and measured.

An MCP server "works" only if a model understands its tools, picks the right one, and fills the arguments. That surface is probabilistic — here is how often it goes wrong.

50k+

tokens burned before the user's first word, with 5–10 servers installed

independent 2026 measurements

~30

tools before a model's selection accuracy measurably falls off

arXiv:2606.30317

50%

of 127 public servers never answered initialize

jig census v2, 2026-07-21

97%

of 856 surveyed tool descriptions carry at least one quality smell

arXiv:2602.14878

Fig. 03

One session. Every question answered.

Fourteen commands over stdio and Streamable HTTP. Every number is exact, or labelled as an approximation — never silently wrong.

01jig checkIs my server well-built? One graded verdict over five weighted dimensions, ranked fixes, HTML report card.

02jig contextWhat does the model actually see? The exact tool-use request body, token-annotated, per provider dialect. No API key — nothing is sent anywhere.

03jig budgetWhat does it cost? Per tool, per model tokenizer, scored against a real ecosystem distribution.

04jig benchWhich tool does a real model pick? N-run selection distributions with argument validation. Your host’s model via MCP sampling, a local endpoint, or your own key.

05jig evalDid my change regress selection? .jig suites in git — selection rates, never single-run booleans — with CI gates.

06jig authIs the OAuth surface conformant? RFC-cited checks of metadata, challenge, PKCE and DCR — or --login to run the authorization-code flow for real.

07jig serversWhat's already configured here? Claude Desktop, Claude Code, Cursor, VS Code and project configs — merged, with secrets redacted.

08jig serveCan my agent do this itself? Jig as an MCP server: its own capabilities as tools, graded A by its own rubric.

09inspect · call · read · promptPoke the protocol directly — every byte captured in a raw JSONL tap you can replay and assert on.

Fig. 04

See exactly what the model sees.

Developers write tool descriptions blind. jig context renders the precise request body a model API receives from your server — token-priced, section by section — and sends nothing to anyone.

jig context output showing the system prompt at 42 tokens, server instructions at 312 tokens, and 13 tools at 1,159 tokens with a per-tool breakdown

Anchored to what jig bench actually sends — proven byte-identical by a parity test.

Fig. 05

Measure behaviour, then regression-test it.

Selection is probabilistic, so jig reports distributions — never a bare "pass". Promote any bench run into a versioned suite with one flag, then gate CI on the rate.

jig bench showing make_reservation selected 5 of 5 runs with per-run latency and token usage

jig bench — N runs, outcome taxonomy, argument validation.

jig eval showing four suite cases with pass, flaky, fail and errored verdicts and a failed CI gate

jig eval — flakiness surfaced, gate exits non-zero.

Fig. 06

OAuth conformance, without logging in.

Authorization is the ecosystem's loudest unmet need — and roughly 8.5% of public servers implement it. jig probes the discoverable surface and cites the clause behind every verdict.

jig auth conformance table with thirteen passing checks, each citing an RFC clause, verdict CONFORMANT

RFC 9728 · 8414 · 7591 · 8707 · 9207 — no credentials sent, nothing installed on the server.

Fig. 07

The standards behind the grade are public.

Every rule jig enforces is written down, cited to primary sources, and paired with the exact command that verifies it — including honest labels where nothing can verify it yet.

33 SOPs

Design, protocol discipline, auth, deployment, testing, remediation — each with its rule, its evidence, and its verification command.

Read the standards →

The census

50 servers priced, then 127 graded dimension by dimension — the distribution jig grades against, the dataset its weights are fitted to, and a public correction where our own first reading was wrong.

Read the census →

rubric-v1.5

Weights and penalty tables in the open — now fitted to a 63-server spread census rather than asserted, with every rejected candidate weighting recorded. Deliberately arguable: if the rubric is wrong, that is a fixable, public thing.

Read the rubric →

Fig. 08

What this doesn't do yet.

A tool that grades honesty should be held to it.

Known limits

The desktop workbench builds and runs, but has no installer. All four panes work against a live server; packaging, code signing, and auto-update are not done, so you build it from source today.
jig auth --login mints one token, then stops. The authorization-code + PKCE flow ships and proves the token against a live session; refresh-token rotation and the device-code grant are still on the roadmap.
Description quality is deterministic heuristics by default, never an LLM verdict, and every report says so. An opt-in labelled judge (jig check --judge) asks a model the three questions a heuristic cannot — does the description state its purpose, distinguish its siblings, and document its parameters — but it is off by default, reported in its own section, and never scored: the composite is byte-identical with and without it.
Only bench and eval ever need a model. check, context, budget, inspect and auth have never needed a key. For the two that do, there are now three keyless-or-not options: your host’s model over MCP sampling, a local endpoint via --base-url --no-auth, or your own key.
Sampling cannot pin the model. A host chooses which model answers, so a sampling-backed bench reports whatever identity the host returns — or unknown — host-selected. Use --base-url or your own key when you need to name the model under test.
The ecosystem leaderboard is live at /leaderboard/ — and it took two blockers to get there. The first, a stdout-polluting server still reading A, shipped as the protocol ceiling. The second only appeared when we re-graded the fleet: robustness is the only dimension with a live-measured input (boot timing), and it drifts with machine load — enough to move a letter grade. Publishing per-vendor grades that move when our laptop is busy would be the dishonesty this tool exists to catch, so every grade there is the median of three complete runs with the observed range printed beside it. Median spread came out at 1 point, worst case 5; the four lowest-scoring servers scored identically in all three runs.
Corrections are published as loudly as findings — see the census for the first one.

What grade is yours?

One command. No account, no signup, no data leaves your machine.