Machine Learning Engineer (MLE): get your model trained with just a sentence.
SaferSkills independently audited MLE (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Based on the user's request (usually like "Train / fine-tune a certain model on a certain dataset"), fill in the template codebase by implementing the abstract functions in the engine.
"mle/engine/evaluate.py", "mle/engine/check_dataset.py", "pyproject.toml", and "README.md".
the user and skill instructions.
to the "pyproject.toml" file.
prompt the user to specify.
decide whether to download the dataset. Note that this path is only a local path, which SHOULD NEVER appear in the codebase.
mle installed. If none can befound, prompt the user to specify one.
most suitable format based on the context.
check_dataset and check_preprocessed_dataset functions. Youmay find an example implementation in the appendix of this document.
preprocess function that converts the dataset to the desiredformat (if different) and applies the preprocessing transformations and analysis if applicable.
official documentation of the model. If none can be found, infer the best setup based on other similar models.
train function that trains the model.online for the official documentation of the dataset and model. If none can be found, infer the best setup based on other similar datasets and models.
infer function that generates the predictions.online for the official documentation of the dataset. If none can be found, infer the best evaluation protocol based on the dataset content.
evaluate function that evaluates the model.from os.path import exists
from mle.vars import ExpConfig
def check_dataset(config: ExpConfig) -> str:
if not exists(config.dataset_dir):
return f"Not found: {config.dataset_dir}"
if not exists(f"{config.dataset_dir}/training"):
return f"Invalid: missing the training split"
if not exists(f"{config.dataset_dir}/validation_public"):
return f"Invalid: missing the public validation split"
extras = []
if exists(f"{config.dataset_dir}/validation_hidden"):
extras.append("hidden validation")
if exists(f"{config.dataset_dir}/testing"):
extras.append("testing")
return f"OK with {"and".join(extras)}" if extras else "OK"
def check_preprocessed_dataset(config: ExpConfig) -> str:
return "OK" if exists(config.preprocessed_dataset_dir) else f"Not found: {config.preprocessed_dataset_dir}"~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.