mlflow — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited mlflow (Agent Skill) and scored it 96/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 1 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 1 flagged
The text {match} tells the agent to skip the normal "ask the user first" gate. Used adversarially it removes the human-in-the-loop check before destructive or sensitive actions, turning a normally-gated agent into a fire-and-forget executor.
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Use MLflow to make experiments, evaluation evidence, model packages, and release decisions inspectable and reproducible. Adapt to the project's stack and maturity; do not impose an entire platform on a task that only needs a correctly instrumented training loop.
production deployments, shared registry alias changes, schema migrations, or access-policy changes.
Reuse the project's framework, orchestration, dependency manager, and logging conventions.
identifiers, credentials, or environment dumps. Review autologging and tracing before enabling them.
approved artefacts with known provenance; do not disable TLS checks to make a connection work.
[TODO]for unknown project-specific values; never invent successful runs, model quality, or API support.
and services, but must not depend on another file in the skill's source repository.
Infer what is already evident, then ask only for missing information that changes the solution:
Classical ML, deep learning, GenAI/agents, or a custom pipeline? Who consumes the result?
versions; Windows, WSL, container, or cluster; CPU/GPU; package and network restrictions.
mechanism, tenant boundary, retention, and data classification. Never ask for secret values.
held-out strategy, primary metric and direction, baseline, meaningful improvement, guardrails, and acceptable operational cost. Which data is allowed to leave the current environment?
infrastructure configuration? Which actions are authorised and which need an approval gate?
Do not block a harmless local investigation on every unanswered production question. Record bounded assumptions and proceed with the smallest useful, reversible step.
Resolve and lock exact versions for executable work; do not silently upgrade a project.
project. Adapt and validate them using the consuming project's environment and dependency manager.
latest documentation is adiscovery source, not proof that an installed client, server, R package, or managed service supports an API. Check versioned documentation and function signatures where necessary.
Treat legacy registry stages and older API examples as migration inputs, not defaults.
translate Python GenAI, tracing, registry, or packaging calls into imagined R equivalents.
WSL are separate environments; do not reuse a virtual environment across their filesystems.
Resolve these paths relative to this skill folder, not the project working directory. Load only the guides needed for the current task; examples are opt-in demonstrations, not automatic setup steps.
| Task | Resource |
|---|---|
| Scope a cross-project integration | Project contract |
| Instrument runs, provenance, autologging or distributed execution | Tracking and reproducibility |
| Design splits, compare models and validate metrics | Evaluation and selection and protocol template |
| Package models, manage versions, or prepare inference | Models and release and release record |
| Instrument deep-learning training, checkpoints and inference | Deep learning |
| Trace or evaluate GenAI, RAG, prompts and agents | GenAI and agents |
| Configure local/Windows/WSL, OpenShift 4 or AWS operations | Platforms and operations |
| Use R or bridge R/Python workflows | R and interoperability |
| Diagnose failures or check version-sensitive claims | Troubleshooting and sources |
| Inspect and adapt implementation patterns | Reference examples and project validation |
| Apply scalar checks to existing evaluation evidence | Metric comparator, policy example, evidence template |
Find the code that controls training, evaluation, logging, or serving. State a falsifiable local hypothesis and the cheapest useful check. Choose the relevant workflow; load only its supporting material. An MLflow question does not automatically require changing the model or infrastructure.
or chronological windows with gaps/embargoes where necessary. Fit preprocessing on training data only. Tune on validation data; reserve the final test for the agreed release decision.
construction, feature availability, seed, code revision, environment, and dependency lock. A seed or dataset digest alone does not establish reproducibility.
selection, and non-quality constraints. Do not claim superiority from an isolated best run.
metrics with clear units and steps. Use parent/child runs when the hierarchy adds meaning.
accidental nested runs, high-cardinality tags, and distributed workers racing on one run.
tables, input examples, checkpoints, dependency files, and traces for sensitive content.
selection, and resumability. MLflow records the work; it does not make training deterministic.
and evaluation dataset. Capture quality, latency, cost, safety, and failure evidence without exposing sensitive traces. External judges require approved providers and data handling.
cases, missing values, uncertainty, and slice coverage. A missing required metric is not a pass.
forecasting use backtesting; for unsupervised tasks justify proxy metrics and external checks.
example, dependencies, and required artefacts. Test reloading and inference, not just saving.
reconstruction. Loading successfully does not prove training reproducibility or serving parity.
Record immutable identifiers in release evidence; a mutable alias is not an immutable release.
operational readiness, and the required approval. An alias is a pointer, not an approval system.
health checks, concurrency, observability, rollback, and cleanup before shared deployment.
Do not assume MLflow supplies authentication, enterprise RBAC, orchestration, drift detection, or a complete production serving platform without separately configured components.
Run the smallest relevant check immediately after a change, repair local failures, then widen verification only for the remaining risks. Do not install heavyweight frameworks or call paid providers just to exercise an unrelated path.
Return only the sections useful to the task:
evaluation data/split identity, and artefact locations. Redact sensitive endpoint details.
rollback considerations, and untested environments or optional integrations.
For reviews, lead with severity-ranked findings and concrete evidence. For implementation, leave an executable, validated increment rather than a catalogue of possible MLflow features.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.