mcp-server-surface-test — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited mcp-server-surface-test (Agent Skill) and scored it 92/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 2 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 2 flagged
The text {match} tells the agent to skip the normal "ask the user first" gate. Used adversarially it removes the human-in-the-loop check before destructive or sensitive actions, turning a normally-gated agent into a fire-and-forget executor.
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Run a Roslyn MCP server audit against a loaded C# repo. The skill bundles two prompt files — prompts/quick.md for the bounded read-only tier and prompts/full.md for the comprehensive run — and routes between them based on $ARGUMENTS.
This skill is a null-op without the Roslyn MCP server. The audit's entire purpose is to exercise the server's live surface — without it, the run produces no audit-grade evidence.
mcp__roslyn__server_info appears in your current tool surface and call it. The response must include connection.state: "ready".connection.state is initializing / degraded / absent, stop and report:"This skill requires the Roslyn MCP server (`mcp__roslyn__tools must be callable,connection.statemust beready). Start the server — for exampledotnet tool run roslynmcpor ensure the plugin's stdio entry is active in your client config — confirmmcp__roslyn__server_inforeturnsready`, then re-invoke this skill."*
Do not substitute Read, Grep, Bash: dotnet build, or any other host-side fallback. There is no generic non-MCP audit fallback in this skill — a broken server precondition halts the run.
$ARGUMENTS may include any combination of the following tokens, in any order. Tokens are space-separated.
The audit needs a C# workspace to load via mcp__roslyn__workspace_load. By default the audit targets the current session's repo root. Override the target in two equivalent ways:
--target=<absolute-path-or-repo-root> — explicit flag form.$ARGUMENTS that resolves to an existing directory containing a .sln/.slnx/.csproj file.Resolution rules:
--target=<path> and a bare path appear, stop and report the conflict — pick one form. Do not silently prefer one over the other..sln/.slnx/.csproj, or is not a git working tree, stop and report — the audit cannot proceed without a loadable workspace and a tree the disposable-worktree step can branch from.workspace_load argument; the prompt's Isolation row in the report header records the resolved target path.--full is the default. Read prompts/full.md and run it phase by phase.--quick — read-only smoke pass. Read prompts/quick.md instead. Target runtime ≤15 minutes vs --full's 90–180 minutes. No apply-mode mutations, no disposable worktree, no test runs, no nuget_vulnerability_scan (network-dependent).--full — explicit form of the default; routes to prompts/full.md.--cleanup-only — recovery mode for an interrupted full run. Resolve the target repo, perform the disposable-worktree cleanup from Step 3, report any persisted checkpoint summary you find, then stop without running audit phases or emitting findings. This mode is intended for crash recovery before a fresh --full rerun.--quick and --full together → stop and report; pick one.--cleanup-only with --quick or --full together → stop and report; cleanup mode is a standalone operation.--single-agent (full tier only)audit-phase-runner subagents per the prompt's Phase 0.5: Subagent dispatch plan. The orchestrator owns workspace lifecycle, the disposable worktree's try/finally, all report-file writes, and finding emission; subagents return compact structured summaries the orchestrator pastes in. This is the only way the --full tier achieves its 250+ tool-call coverage without burning through a single agent's context window.--single-agent — opt out of the dispatch plan and run every phase in the orchestrator's own context. Use only when (a) the host environment cannot spawn subagents, or (b) the operator wants a one-shot run and accepts that long phases may surface as phase-failed-budget in the coverage summary. The completion gate (no silent skipped-budget truncation) still applies — partial runs surface as P1 audit defects, not as quiet ledger entries.--no-worktree (full tier only)--no-worktree — degraded mode for environments that genuinely cannot create a git worktree. Phase 6 sub-phases that require a worktree are marked skipped-safety — --no-worktree. The promotion scorecard still emits, but writer recommendations default to needs-more-evidence for any tool whose round-trip evidence depended on the disposable worktree. Has no effect under `--quick` (the quick tier already skips Phase 6); reject the combination with a one-line message.--output-mode=findings|fragments (both tiers)Default: findings — Phase 19 emits stdout / GitHub Issues per the maintainer-aware routing below.
--output-mode=fragments — Phase 19 instead emits one <audited-repo-root>/backlog.d/<finding-id>.md per actionable finding, for /backlog-intake to consume. Intended for maintainer-managed repos that participate in a backlog.d/ ingestion pipeline. Bypasses --auto-file / --no-auto-file entirely — those flags have no effect under fragments mode (the file-system handoff IS the destination). The prose .md report still writes to the audited repo's audit-reports/ directory; /backlog-intake reads the fragment's source_audit field to back-reference the prose report. See prompts/full.md Phase 19 Fragments path for the full contract./mcp-server-stress skill (maintainer-only repo-local alias) is a thin invocation of /mcp-server-surface-test --output-mode=fragments against the current Claude Code session's repo — it exists for muscle-memory continuity and to advertise the fragments-mode default for maintainer audits.--auto-file / --no-auto-file (findings mode only)The auto-file default is maintainer-aware: the skill probes the operator's GitHub identity and auto-files when the operator owns the upstream repo. Findings always go to https://github.com/darylmcd/Roslyn-Backed-MCP regardless of the audited repo, since findings are about the MCP server itself and the audited repo is just a fixture. These flags have no effect under --output-mode=fragments — fragments mode skips the GitHub-Issues handoff entirely.
Maintainer detection (run once per invocation, before Phase 19):
Dot-source ${CLAUDE_PLUGIN_ROOT}/skills/mcp-server-surface-test/lib/render-finding.ps1 and call Test-IsMaintainer. It wraps gh api user --jq .login and compares against the maintainer login derived from the single $script:UpstreamRepo constant — that one literal in lib/render-finding.ps1 is the source of truth for "who is the maintainer" and "where do auto-filed Issues land." Any failure mode (gh missing, unauthenticated, network failure, login mismatch) returns $false and is treated as not the maintainer.
Default (no flag):
gh api user --jq .login == darylmcd): auto-file each non-refused finding via gh issue create (same call as the explicit --auto-file path).Explicit overrides:
--auto-file — force-enable auto-filing regardless of detected identity. Still subject to the gh-available + authenticated requirement (falls back to stdout-print with one warning line if either is missing); still subject to the P0/security refusal contract below.--no-auto-file — force-disable auto-filing. Always emit to stdout, even when the maintainer is detected. Use this for a dry-run on the maintainer's own machine.--auto-file and --no-auto-file together → stop and report; pick one.Refusal contract — load-bearing pre-disclosure safeguard: the skill does not call gh issue create for any finding whose severity == P0 or whose area == security. Such findings print to stdout with this header line: **SECURITY / P0 finding — DO NOT FILE PUBLICLY.** Escalate via GitHub security advisories at https://github.com/darylmcd/Roslyn-Backed-MCP/security/advisories/new. The refusal applies regardless of detected identity, --auto-file, or --no-auto-file.
Reject any token that is neither a valid target path, --target=<path>, --quick, --full, --cleanup-only, --output-mode=findings, --output-mode=fragments, --no-worktree, --single-agent, --auto-file, nor --no-auto-file. One-line message; ask the user to fix or drop the offending token. Conflict: --auto-file or --no-auto-file combined with --output-mode=fragments → emit a one-line warning explaining that the auto-file flag is ignored under fragments mode, then proceed with fragments mode.
The audit is read-only against the audited repository's `main` branch. Phase 6 of the full tier writes apply-mode mutations, but only inside a disposable worktree the prompt creates and tracks. The flow is:
compile_check after apply, build_workspace + test_run after apply). The point is to exercise the write path of the MCP server, not to ship product changes.dotnet build-server shutdown followed by git worktree remove --force (the Windows lock-release sequence is mandatory; dotnet build-server shutdown releases testhost.exe / VBCSCompiler.exe locks on the worktree's bin/ dirs). Teardown runs even on apply failure — the prompt wraps the Phase 6 chain in try/finally discipline.main branch is never directly mutated. No PR is opened from the audit run. No commits land in the audited repo's history.The --no-worktree flag opts into a degraded mode where the disposable worktree is skipped — see Step 2 for the contract and the report-header record requirement. The --quick tier always skips the disposable worktree (no apply-mode phases run at all).
For --cleanup-only, run only the teardown half of this step against residual surface-test worktrees and branches. Do not delete arbitrary worktrees: restrict candidates to the surface-test naming pattern recorded in the audit prompt, verify the path is under the target repo's worktree area before removal, run dotnet build-server shutdown, then remove the worktree and delete matching local branches that are not checked out. Leave the primary checkout clean or report the exact remaining paths.
--quick: read ${CLAUDE_PLUGIN_ROOT}/skills/mcp-server-surface-test/prompts/quick.md and run it phase by phase.--full (default): read ${CLAUDE_PLUGIN_ROOT}/skills/mcp-server-surface-test/prompts/full.md and run it phase by phase.Persist the audit draft after each phase as the prompt instructs — the canonical report path lives in the prompt's Output Format section: <audited-repo-root>/audit-reports/<timestamp>_<repo-id>_mcp-server-surface-test.md.
After the audit phases complete, the prompt's final finding-emission phase walks every actionable finding (entries in MCP server issues or Improvement suggestions with a concrete fix sketch) and renders each through one shared envelope:
--no-auto-file is passed): print each finding to stdout as a ready-to-paste GitHub Issue body. The render block is ## TITLE, label list, and the structured body fields (id, source-repo, severity, area, server-version, anchors, finding, repro, proposed-fix).gh api user --jq .login == darylmcd — or when --auto-file is passed): call gh issue create --repo darylmcd/Roslyn-Backed-MCP --title <title> --label <area:X,severity:Y> --body-file <tempfile> per finding, except those refused by the P0/security refusal contract (Step 2) and those filtered by the dedup pre-check (Step 5a). Refused findings still print to stdout with the security-advisory escalation banner.Before each gh issue create invocation, the auto-file path must query the issue tracker for a near-match and skip filing when one exists. This prevents the auto-filer from re-filing the same bug across audit runs, or from filing a finding that has already been promoted to good first issue via the maintainer-curated path. Without this gate, the auto-filer is the primary source of duplicate-issue noise (the gap was first observed on 2026-05-11 when the IT-Chat-Bot audit auto-filed a duplicate of an existing maintainer-promoted goto_type_definition issue).
For each finding about to be filed:
[audit] goto_type_definition throws for BCL/metadata-only types instead of returning structured no-source result, the key is goto_type_definition. For findings that span multiple tools or skills, use the first tool name plus one or two distinguishing symptom keywords. gh issue list --repo darylmcd/Roslyn-Backed-MCP \
--state all \
--search "<search-key> in:title" \
--json number,state,title,labels --limit 10→ skipped: duplicate of #<N> (open) — <existing-title>. Add the finding to the run's "deduped" audit-trail list.→ skipped: previously resolved as #<N> (closed) — <existing-title>. Do not refile; if the operator believes the finding represents a regression of a prior fix, they will re-open the existing issue manually.gh issue create as before.The match threshold is intentionally conservative: better to skip a borderline case and emit a warning than to file a true duplicate. Operators reviewing the skipped-duplicate list can re-file manually if the dedup was wrong. The audit-trail entry for each skipped finding includes the candidate existing issue number so the operator can verify quickly.
scripts/archive-old-reports.ps1Reports written to the audit-reports/ directory accumulate over time. The skill ships a small PowerShell wrapper at ${CLAUDE_PLUGIN_ROOT}/skills/mcp-server-surface-test/scripts/archive-old-reports.ps1 that moves *.md files older than N days (default 30) into a year-stamped archive/<YYYY>/ subdirectory. The reports directory path defaults to audit-reports and can be overridden via -ReportsRelativePath.
Invocation (any shell with pwsh on path):
# Preview the archive plan without mutating anything.
pwsh -NoProfile -File ${CLAUDE_PLUGIN_ROOT}/skills/mcp-server-surface-test/scripts/archive-old-reports.ps1 -DryRun
# Archive reports older than 60 days under the default reports directory.
pwsh -NoProfile -File ${CLAUDE_PLUGIN_ROOT}/skills/mcp-server-surface-test/scripts/archive-old-reports.ps1 -OlderThanDays 60Behavior contract:
README.md stays in place regardless of age.The script is invoked manually (no automatic scheduler).
mcp__roslyn__server_info is not callable or connection.state is not ready, halt.try/finally so teardown executes even on apply failure. dotnet build-server shutdown always precedes git worktree remove --force on Windows.--auto-file, or passed --no-auto-file — those findings are stdout-only with the security-advisory escalation banner.~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.