CategoryWeightCategory scoreContribution

Securityprompt, exec, net, exfil, eval

35%

30.8 pts

Supply chainhash, typosquat, maintainer, lockfile

20%

100

20.0 pts

Maintenancestaleness, pinning, CI

15%

100

15.0 pts

TransparencySKILL.md, perms, README

15%

100

15.0 pts

Communityinstalls, verify, response

15%

100

15.0 pts

Findings & checks · 1 flagged

Securityscore 88 · 1 finding

MEDIUMInstruction telling the agent not to ask for approvalSS-SKILL-INJECT-DONT-ASK-01 · Prompt injection · plugins/sagemaker-ai/skills/model-selection/SKILL.md

MEDIUM — it fires on intent; the real damage depends on the host agent's own approval-gating.

Why it matters

The text {match} tells the agent to skip the normal "ask the user first" gate. Used adversarially it removes the human-in-the-loop check before destructive or sensitive actions, turning a normally-gated agent into a fire-and-forget executor.

The exact value spotted

excerptplugins/sagemaker-ai/skills/model-selection/SKILL.md· markdown

392. From the results, filter out any hub whose `HubDescription` contains "AI Registry" — thes

… (34 chars elided on L39)

403. The remaining hubs are eligible (e.g., `SageMakerPublicHub` and any private hubs).

414. If exactly one eligible hub exists, use it automatically — do not ask the user.

425. If multiple eligible hubs exist, present them to the user and ask which one to use. Examp

… (3 chars elided on L42)

Occurrences

1 occurrence · at L41

How to fix

Remove the approval-skipping instruction, or scope it narrowly to a specific safe, reversible action.

Delete blanket "don't ask / no need to confirm" directives from the skill.
If the skill is a genuine autonomous job, restrict the opt-out to a named non-destructive action rather than all actions.

Framework references

OWASPLLM01 ATLASAML.T0051

Trace & refs

ruleSS-SKILL-INJECT-DONT-ASK-01sha25612211d7a427c799a…rubric 365aacaView on GitHub

Supply chainscore 100 · 0 findings

✓—All supply chain checks passedNo findings in this category for the latest scan.pass

Maintenancescore 100 · 0 findings

✓—All maintenance checks passedNo findings in this category for the latest scan.pass

Transparencyscore 100 · 0 findings

✓—All transparency checks passedNo findings in this category for the latest scan.pass

Communityscore 100 · 0 findings

✓—All community checks passedNo findings in this category for the latest scan.pass

The primary manifest — the file an agent reads to learn what this artifact does.

plugins/sagemaker-ai/skills/model-selection/SKILL.md · 1 file

plugins/sagemaker-ai/skills/model-selection/SKILL.md3.3 KB · Markdown

name: model-selection

description: Selects a base model for the user's use case by querying SageMaker Hub. Use when the user asks which model to use, wants to select or change their base model, mentions a model name or family (e.g., "Llama", "Mistral", "Nova"), or wants to evaluate a base model — always activate even for known model names because the exact Hub model ID must be resolved. Queries available models, presents benchmarks and licenses, and confirms selection.

metadata:

version: "1.0.0"

Model Selection

Guides the user through selecting a base model based on their use case.

When to Use

User asks which model to use
User wants to select or change their base model
User mentions a model name or family (e.g., "Llama", "Mistral", "Nova") — the exact Hub model ID still needs to be resolved
User wants to evaluate a base model before deciding whether to finetune

Prerequisites

A use_case_spec.md file exists. If not, activate the use-case-specification skill to generate it first.

Workflow

Step 1: Check Region

Run:

python -c "import boto3; print(boto3.session.Session().region_name)"

None → STOP. Tell user: "Set your region via export AWS_DEFAULT_REGION=us-west-2 or aws configure."
Set → store REGION in context, continue.

Step 2: Discover Hub

List all available SageMaker Hubs in the user's region by calling the SageMaker ListHubs API using the aws___call_aws tool.
From the results, filter out any hub whose HubDescription contains "AI Registry" — these do not contain JumpStart models.
The remaining hubs are eligible (e.g., SageMakerPublicHub and any private hubs).
If exactly one eligible hub exists, use it automatically — do not ask the user.
If multiple eligible hubs exist, present them to the user and ask which one to use. Example:

   I found the following model hubs:
   - SageMakerPublicHub — SageMaker Public Hub
   - Private-Hub-XYZ — Private Hub models
   Which hub would you like to use?

Store the selected hub name for use in subsequent steps.

Step 3: Select Base Model

First, retrieve all available SageMaker Hub model names by running: python model-selection/scripts/get_model_names.py <hub-name>.

Present all available models to the user with their licenses before making any recommendations. Cross-reference the model list with references/model-licenses.md and display each as <model name> - [<license>](<url>). For example: "Qwen3-4B - Apache 2.0"

If you already know the model the user wants to use (from conversation context or planning files), confirm that it's in the list, display its license, and move on. Otherwise, help the user pick a model following the instructions in references/model-selection.md. Important: Make sure to remember this list of available models when helping with model selection. Don't recommend a model that's not available to the user.

Step 4: Confirm Selection

Present a summary to the user:

Here's what we've selected:
- Base model: [model name]

Ask if they'd like to proceed with this model.

References

references/model-selection.md — Model selection instructions and benchmark descriptions
references/model-licenses.md — Model license information for display during model selection

model-selection— agent skill

Is model-selection safe to install?

Model Selection

When to Use

Prerequisites

Workflow

Step 1: Check Region

Step 2: Discover Hub

Step 3: Select Base Model

Step 4: Confirm Selection

References

Audit the pieces. Scan the whole. Decide.