migrate-vstest-to-mtp — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited migrate-vstest-to-mtp (Agent Skill) and scored it 45/100 (orange). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 7 high-severity and 1 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 8 flagged
A base64 string of 128+ characters appears in a documentation file. Encoded prompt injection hides the hostile instruction in base64 — invisible to keyword filters — and relies on the agent's ability to decode it at runtime. There is no normal authoring reason to embed a multi-hundred-byte base64 blob in skill docs.
*.sig, SIGNATURES) outside the documentation.A base64 string of 128+ characters appears in a documentation file. Encoded prompt injection hides the hostile instruction in base64 — invisible to keyword filters — and relies on the agent's ability to decode it at runtime. There is no normal authoring reason to embed a multi-hundred-byte base64 blob in skill docs.
*.sig, SIGNATURES) outside the documentation.A base64 string of 128+ characters appears in a documentation file. Encoded prompt injection hides the hostile instruction in base64 — invisible to keyword filters — and relies on the agent's ability to decode it at runtime. There is no normal authoring reason to embed a multi-hundred-byte base64 blob in skill docs.
*.sig, SIGNATURES) outside the documentation.A base64 string of 128+ characters appears in a documentation file. Encoded prompt injection hides the hostile instruction in base64 — invisible to keyword filters — and relies on the agent's ability to decode it at runtime. There is no normal authoring reason to embed a multi-hundred-byte base64 blob in skill docs.
*.sig, SIGNATURES) outside the documentation.A base64 string of 128+ characters appears in a documentation file. Encoded prompt injection hides the hostile instruction in base64 — invisible to keyword filters — and relies on the agent's ability to decode it at runtime. There is no normal authoring reason to embed a multi-hundred-byte base64 blob in skill docs.
*.sig, SIGNATURES) outside the documentation.A base64 string of 128+ characters appears in a documentation file. Encoded prompt injection hides the hostile instruction in base64 — invisible to keyword filters — and relies on the agent's ability to decode it at runtime. There is no normal authoring reason to embed a multi-hundred-byte base64 blob in skill docs.
*.sig, SIGNATURES) outside the documentation.A base64 string of 128+ characters appears in a documentation file. Encoded prompt injection hides the hostile instruction in base64 — invisible to keyword filters — and relies on the agent's ability to decode it at runtime. There is no normal authoring reason to embed a multi-hundred-byte base64 blob in skill docs.
*.sig, SIGNATURES) outside the documentation.The text {match} tells the agent to skip the normal "ask the user first" gate. Used adversarially it removes the human-in-the-loop check before destructive or sensitive actions, turning a normally-gated agent into a fire-and-forget executor.
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Migrate a .NET test solution from VSTest to Microsoft.Testing.Platform (MTP). The outcome is a solution where all test projects run on MTP, dotnet test works correctly, and CI/CD pipelines are updated.
Inspect the supplied project, Directory.Build.props, global.json, and CI files before searching the web or answering from memory. Resolve the framework and SDK mode first: .NET 9 and earlier use the compatibility property plus the -- separator; .NET 10 native MTP uses global.json, removes that property, and passes MTP arguments without the separator. For central properties, never condition on IsTestProject in Directory.Build.props; use a property already available there, such as MSBuildProjectName.
Important: Do not mix VSTest-based and MTP-based .NET test projects in the same solution or run configuration -- this is an unsupported scenario.
dotnet run / dotnet watch / direct executable execution for test projectsvstest.console.exe with dotnet test on MTPdotnet test arguments from VSTest syntax to MTP syntaxmigrate-mstest-v1v2-to-v3 or migrate-mstest-v3-to-v4| Input | Required | Description |
|---|---|---|
| Project or solution path | No | The .csproj, .sln, or .slnx entry point containing test projects. Discover it yourself by globbing the working directory; ask only when nothing is found or the choice is genuinely ambiguous |
| Test framework | No | MSTest, NUnit, xUnit.net v2, or xUnit.net v3. Auto-detected from package references |
| .NET SDK version | No | Determines dotnet test integration mode. Prefer the repository's global.json or explicitly stated CI SDK; use host dotnet --version only when the repository does not pin or state one |
| CI/CD pipeline files | No | Paths to pipeline definitions that invoke vstest.console or dotnet test |
global.json, and pipeline files in the current working directory. Open literal search results; the skill directory is not the user's repository. Continue after skill activation and do not ask for a discoverable path.--no-build, new settings files, or unrelated package upgrades. Never retain --settings <file> unless that file exists.-- separator. For .NET 10 native MTP mode, explicitly say to remove it.--ignore-exit-code 8, TestingPlatformCommandLineArguments, and TESTINGPLATFORM_EXITCODE_IGNORE=8.--filter-class/--filter-method/--filter-trait command and explain AND behavior. When the resolved xUnit version supports it, include --filter-query path syntax for complex expressions and link the xUnit query-filter documentation.platform-detection skill for the package-to-framework mapping. Key indicators:MSTest or MSTest.TestAdapter, or uses MSTest.Sdk (with <IsTestApplication> not set to false). Note: MSTest.TestFramework alone is a library dependency, not a test project.NUnit3TestAdapterxunit and xunit.runner.visualstudioglobal.json or explicit user context. Fall back to dotnet --version only when neither exists; the agent host SDK must not silently override a stated .NET 8/9 CI target.Directory.Build.props file exists at the solution or repo root -- all MTP properties should go there for consistencyvstest.console.exe usage in CI scripts or pipeline definitionsdotnet test arguments in CI scripts: --filter, --logger, --collect, --settings, --blame*dotnet test to establish a baseline of test pass/fail countsCritical: Set MTP runner properties inDirectory.Build.propsat the solution or repo root whenever possible, rather than per-project. This prevents inconsistent configuration where some projects use VSTest and others use MTP (an unsupported scenario). Note: MTP also requires test projects to have<OutputType>Exe</OutputType>. OnlyMSTest.Sdksets this automatically. For all other setups (MSTest NuGet packages withEnableMSTestRunner, NUnit withEnableNUnitRunner, xUnit.net withYTest.MTP.XUnit2), prefer setting<OutputType>Exe</OutputType>centrally inDirectory.Build.propswith a condition that targets only test projects. If you cannot reliably target only test projects fromDirectory.Build.props, setting<OutputType>Exe</OutputType>per-project is an acceptable exception.
>
Conditioning in `Directory.Build.props`: Do NOT useCondition="'$(IsTestProject)' == 'true'"--IsTestProjectis set by the test SDK targets later in evaluation and is not available whenDirectory.Build.propsis imported. Use a property that is available early, such asMSBuildProjectName, to target test projects by naming convention. For example, if all test projects end in.Tests:
>
``xml <PropertyGroup Condition="$(MSBuildProjectName.EndsWith('.Tests'))"> <OutputType>Exe</OutputType> </PropertyGroup> ``>
Adjust the condition (e.g.,.EndsWith('Tests'),.Contains('.Test')) to match the test project naming convention used in the repository.
>
Put every applicable central property in that same condition:OutputType, the framework runner opt-in, andTestingPlatformDotnetTestSupporton .NET 9 and earlier. Do not leave an unconditional runner property that still affects production projects.
Each framework has its own opt-in property. Add these in Directory.Build.props for consistency.
#### MSTest
Option A -- MSTest NuGet packages (3.2.0+):
<PropertyGroup>
<EnableMSTestRunner>true</EnableMSTestRunner>
<OutputType>Exe</OutputType>
</PropertyGroup>Ensure the project references MSTest 3.2.0 or later. If the version is already 3.2.0+, no MSTest version upgrade is needed for MTP migration.
Option B -- MSTest.Sdk:
When using MSTest.Sdk, MTP is enabled by default -- no EnableMSTestRunner or OutputType Exe property is needed (the SDK sets both automatically). The only action is: if the project has <UseVSTest>true</UseVSTest>, remove it. That property forces the project to use VSTest instead of MTP.
#### NUnit
Requires NUnit3TestAdapter 5.0.0 or later.
NUnit3TestAdapter to 5.0.0+:<PackageReference Include="NUnit3TestAdapter" Version="5.0.0" /><PropertyGroup>
<EnableNUnitRunner>true</EnableNUnitRunner>
<OutputType>Exe</OutputType>
</PropertyGroup>#### xUnit.net
Add a reference to YTest.MTP.XUnit2 -- this package provides MTP support for xUnit.net v2 projects without requiring an upgrade to xunit.v3. You must also set OutputType to Exe:
<PackageReference Include="YTest.MTP.XUnit2" Version="0.4.0" /><PropertyGroup>
<OutputType>Exe</OutputType>
</PropertyGroup>Note:YTest.MTP.XUnit2preserves the VSTest--filtersyntax, so no filter migration is needed for xUnit.net v2. It also supports--settingsfor runsettings (xunit-specific configurations only),xunit.runner.json, TRX reporting via--report-trx, and--treenode-filter.
#### xUnit.net v3
xUnit.net v3 (xunit.v3 package) has built-in MTP support. Enable it with:
<PropertyGroup>
<UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner>
</PropertyGroup>Important: xUnit.net v3 on MTP does NOT support the VSTest --filter syntax. You must translate filters to xUnit.net v3's native filter options (see Step 5).The dotnet test integration depends on the .NET SDK version.
#### .NET 10 SDK and later (recommended)
Use the native MTP mode by adding a test section to global.json:
{
"sdk": {
"version": "10.0.100"
},
"test": {
"runner": "Microsoft.Testing.Platform"
}
}In this mode, dotnet test arguments are passed directly -- for example, dotnet test --report-trx.
Important: global.json does not support trailing commas. Ensure the JSON is strictly valid.#### .NET 9 SDK and earlier
Use the VSTest mode of dotnet test command to run MTP test projects by adding this property in Directory.Build.props:
<PropertyGroup>
<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
</PropertyGroup>Important: In this mode, you must use--to separatedotnet testbuild arguments from MTP arguments. For example:dotnet test --no-build -- --list-tests.
VSTest-specific arguments must be translated to MTP equivalents. Build-related arguments (-c, -f, --no-build, --nologo, -v, etc.) are unchanged.
| VSTest argument | MTP equivalent | Notes | |
|---|---|---|---|
--test-adapter-path | Not applicable | MTP does not use external adapter discovery | |
--blame | Not applicable | ||
--blame-crash | --crashdump | Requires Microsoft.Testing.Extensions.CrashDump NuGet package | |
--blame-crash-dump-type <TYPE> | --crashdump-type <TYPE> | Requires CrashDump extension | |
--blame-hang | --hangdump | Requires Microsoft.Testing.Extensions.HangDump NuGet package | |
--blame-hang-dump-type <TYPE> | --hangdump-type <TYPE> | Requires HangDump extension | |
--blame-hang-timeout <TIMESPAN> | --hangdump-timeout <TIMESPAN> | Requires HangDump extension | |
--collect "Code Coverage;Format=cobertura" | --coverage --coverage-output-format cobertura | Per-extension arguments | |
| `-d\ | --diag <LOG_FILE>` | --diagnostic | |
--filter <EXPRESSION> | --filter <EXPRESSION> | Same syntax for MSTest, NUnit, and xUnit.net v2 (with YTest.MTP.XUnit2). For xUnit.net v3, see filter migration below | |
| `-l\ | --logger trx` | --report-trx | Requires Microsoft.Testing.Extensions.TrxReport NuGet package |
--results-directory <DIR> | --results-directory <DIR> | Same | |
| `-s\ | --settings <FILE>` | --settings <FILE> | MSTest and NUnit still support .runsettings |
| `-t\ | --list-tests` | --list-tests | Same |
-- <RunSettings args> | --test-parameter | Applicable only to MSTest and NUnit |
#### Filter migration
MSTest, NUnit, and xUnit.net v2 (with `YTest.MTP.XUnit2`): The VSTest --filter syntax is identical on both VSTest and MTP. No changes needed.
xUnit.net v3 (native MTP): xUnit.net v3 does NOT support the VSTest --filter syntax on MTP. You must translate filters to xUnit.net v3's native filter options.
#### xUnit.net v3 filter flags
| Flag | Description |
|---|---|
--filter-class "name" | Run all tests in a given class. Supports wildcards (*). |
--filter-not-class "name" | Exclude all tests in a given class |
--filter-method "name" | Run a specific test method |
--filter-not-method "name" | Exclude a specific test method |
--filter-namespace "name" | Run all tests in a namespace |
--filter-not-namespace "name" | Exclude all tests in a namespace |
--filter-trait "name=value" | Run tests with a matching trait |
--filter-not-trait "name=value" | Exclude tests with a matching trait |
Multiple values can be specified with a single flag: --filter-class Foo Bar.
#### VSTest → xUnit.net v3 filter translation table
VSTest --filter syntax | xUnit.net v3 MTP equivalent | Notes |
|---|---|---|
FullyQualifiedName~ClassName | --filter-class *ClassName* | Wildcards required for substring match |
FullyQualifiedName=Ns.Class.Method | --filter-method Ns.Class.Method | Exact match on fully qualified method |
Name=MethodName | --filter-method *MethodName* | Wildcards for substring match |
Category=Value (trait) | --filter-trait "Category=Value" | Filter by trait name/value pair |
| Complex expressions | --filter-query "expr" | Uses xUnit.net query filter language (see below) |
#### xUnit.net v3 query filter language
For complex expressions, use --filter-query with a path-segment syntax:
/<assemblyFilter>/<namespaceFilter>/<classFilter>/<methodFilter>[traitName=traitValue]Each segment matches against: assembly name, namespace, class name, method name. Use * for "match all" in any segment. Documentation: <https://xunit.net/docs/query-filter-language>
Prefer the directly corresponding --filter-class / --filter-method / --filter-trait flags when they express the original filter. Use --filter-query only after validating the query with --list-tests; a syntactically accepted query that selects zero tests is not a successful translation.
#### Translation example
# VSTest
dotnet test --filter "FullyQualifiedName~IntegrationTests&Category=Smoke"
# xUnit.net v3 MTP -- using individual filters (AND behavior)
dotnet test -- --filter-class *IntegrationTests* --filter-trait "Category=Smoke"
# xUnit.net v3 MTP -- using query language (assembly/namespace/class/method[trait])
dotnet test -- --filter-query "/*/*/*IntegrationTests*/*[Category=Smoke]"Note: When combining--filter-classand--filter-trait, both conditions must match (AND behavior). For complex expressions, use--filter-querywith the path-segment syntax. See the xUnit.net query filter language docs for full reference.
If CI scripts use TRX reporting, crash dumps, hang dumps, or coverage, add the owning packages. Under Central Package Management, the project references are:
<ItemGroup>
<PackageReference Include="Microsoft.Testing.Extensions.TrxReport" />
<PackageReference Include="Microsoft.Testing.Extensions.CrashDump" />
<PackageReference Include="Microsoft.Testing.Extensions.HangDump" />
<PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" />
</ItemGroup>Add matching PackageVersion entries in Directory.Packages.props. Without Central Package Management, put exact versions resolved from the configured feed on these references. Keep the versions compatible with the selected MTP stack; do not copy fixed versions from migration guidance.
#### Azure DevOps
If using the VSTest task (`VSTest@3`): Replace with the .NET Core CLI task (DotNetCoreCLI@2):
# Before (VSTest task)
- task: VSTest@3
inputs:
testAssemblyVer2: '**/*Tests.dll'
runSettingsFile: 'test.runsettings'
# After (.NET Core CLI task)
- task: DotNetCoreCLI@2
displayName: Run tests
inputs:
command: 'test'
arguments: '--no-build --configuration Release'If already using DotNetCoreCLI@2: Update arguments per Step 5 translations. Remember the -- separator on .NET 9 and earlier:
- task: DotNetCoreCLI@2
displayName: Run tests
inputs:
command: 'test'
arguments: '--no-build -- --report-trx --results-directory $(Agent.TempDirectory)'#### GitHub Actions
Update dotnet test invocations in workflow files with the same argument translations from Step 5.
#### Replace vstest.console.exe
If any script invokes vstest.console.exe directly, replace it with dotnet test. The test projects are now executables and can also be run directly.
#### Zero tests exit code
VSTest silently succeeds when zero tests are discovered. MTP fails with exit code 8. Options:
--ignore-exit-code 8 when running testsDirectory.Build.props:<PropertyGroup>
<TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
</PropertyGroup>TESTINGPLATFORM_EXITCODE_IGNORE=8Once migration is complete and verified, remove packages that are only needed for VSTest:
Microsoft.NET.Test.Sdk -- not needed for MTP (MSTest.Sdk v4 already omits it by default)xunit.runner.visualstudio -- only needed for VSTest discovery of xUnit.net (not needed when using YTest.MTP.XUnit2)NUnit3TestAdapter VSTest-only features -- the adapter is still needed but only for the MTP runnerNote: If you need to maintain VSTest compatibility during a transition period, keep these packages.
dotnet build -- confirm zero errorsdotnet test -- confirm all tests pass./bin/Debug/net8.0/MyTests.exe) -- confirm it worksdotnet build completes with zero errorsdotnet test passes all tests and test counts match pre-migration baseline./bin/Debug/net8.0/MyTests.exe)vstest.console.exe invocations remain in CI scripts<OutputType>Exe</OutputType> is set for all non-MSTest.Sdk test projects| Pitfall | Solution |
|---|---|
| Mixing VSTest and MTP projects in the same solution | Migrate all test projects together -- mixed mode is unsupported |
dotnet test arguments ignored on .NET 9 and earlier | Use -- to separate build args from MTP args: dotnet test -- --report-trx |
| Exit code 8 on CI without failures | MTP fails when zero tests run; use --ignore-exit-code 8 or fix test discovery |
| MSTest.Sdk v4 + vstest.console no longer works | MSTest.Sdk v4 no longer adds Microsoft.NET.Test.Sdk -- add it explicitly or switch to dotnet test |
Missing <OutputType>Exe</OutputType> | Required for all setups except MSTest.Sdk (which sets it automatically) |
Using Condition="'$(IsTestProject)' == 'true'" in Directory.Build.props | IsTestProject is not yet defined when Directory.Build.props is evaluated -- use $(MSBuildProjectName.EndsWith('.Tests')) (or a similar name-based check) instead |
run-tests for running tests on the new MTP platformmtp-hot-reload for iterative test fixing with hot reload on MTP~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.