Querying Isabl Data

You are helping the user query data from the Isabl platform using the Python SDK.

Checklist

Work through these steps systematically:

1. [ ] Understand the query goal (what data is needed)
2. [ ] Identify the entity type (experiments, analyses, samples, individuals)
3. [ ] Determine filter criteria (project, status, dates, etc.)
4. [ ] Choose appropriate operators (exact match, contains, ranges)
5. [ ] Build the query using ii.get_* functions
6. [ ] Execute and verify results
7. [ ] Format output as needed (DataFrame, JSON, etc.)
8. [ ] When showing IDs for experiments/samples/individuals, prefer `system_id` (or identifier if system_id is not available); avoid pk unless explicitly requested
9. [ ] For analysis summaries, choose scope-aware identifiers:

• individual_level_analysis -> show individual system_id (e.g., IID_H210514)
• project_level_analysis -> show project pk
• avoid listing long targets/references experiment arrays unless explicitly requested

1. [ ] When building frontend links from `ISABL_API_URL`, remove `/api/v1/` before appending query params (e.g., ?analysis=<pk>)
2. [ ] Before any table, add a one-line plain-language summary of what the table contains (scope, filters, and count if known)

Step 1: Understand the Query Goal

Ask clarifying questions if needed:

• What entity type? (experiments, analyses, samples, individuals)
• What filters? (project, application, status, dates)
• What output format? (list, DataFrame, count only)

Step 2: Identify Entity Type

import isabl_cli as ii

# Main entity types and their query functions
experiments = ii.get_experiments(...)      # Sequencing data
analyses = ii.get_analyses(...)            # Pipeline results
samples = ii.get_instances("samples", ...) # Tissue specimens
individuals = ii.get_instances("individuals", ...)  # Patients/subjects
applications = ii.get_instances("applications", ...)  # Pipelines
projects = ii.get_instances("projects", ...)  # Project groupings

Step 3: Determine Filter Criteria

# Filter by project
experiments = ii.get_experiments(projects=102)

# Filter by status
analyses = ii.get_analyses(status="SUCCEEDED")

# Filter by application
analyses = ii.get_analyses(application__name="MUTECT")

# Filter by date range
analyses = ii.get_analyses(created__gte="2024-01-01")

# Combine multiple filters
experiments = ii.get_experiments(
    projects=102,
    sample__category="TUMOR",
    technique__method="WGS"
)

Step 4: Choose Appropriate Operators

Traverse Relationships

Use double underscore to traverse related objects:

import isabl_cli as ii

# Experiments for a specific individual
experiments = ii.get_experiments(sample__individual__pk=500)

# Analyses by application name
analyses = ii.get_analyses(application__name="variant_caller")

# Experiments with specific technique
experiments = ii.get_experiments(technique__method="WGS")

Step 5: Build the Query

Find experiments by project

import isabl_cli as ii

experiments = ii.get_experiments(
    projects=102,
    sample__category="TUMOR"
)
for exp in experiments:
    print(f"{exp.system_id}: {exp.sample.identifier}")

ID display convention (important)

When users ask for an experiment/sample/individual "ID", prefer the human-readable identifier (system_id or identifier) and omit numeric pk unless asked.

import isabl_cli as ii

experiments = ii.get_experiments(projects=102, fields=["system_id", "sample"])
for exp in experiments:
    print(f"experiment_id={exp.system_id}")

For MCP isabl_query, prefer:

isabl_query(
    endpoint="experiments",
    filters={"projects": 102},
    output_fields=["system_id"]
)

Analysis scope convention (important)

When listing analyses, use identifiers that match the analysis scope:

• For individual-level analyses (for example WGTS_REPORT), display the individual id (IID_H...).
• For project-level analyses, display the project pk.
• Do not show large targets/references experiment lists unless the user asks for them.

Table response convention (important)

Before rendering a table, include one concise summary line that states what is being shown.

Examples:

• "Showing 3 failed WGTS_REPORT analyses in project 267, with individual IDs and stderr links."
• "Showing the latest 10 succeeded MUTECT analyses with target experiment IDs."

Find successful analyses for an application

import isabl_cli as ii

analyses = ii.get_analyses(
    application__name="variant_caller",
    application__version="2.0.0",
    status="SUCCEEDED"
)
for a in analyses:
    print(f"Analysis {a.pk}: {a.results}")

Find failed analyses

import isabl_cli as ii

failed = ii.get_analyses(
    status="FAILED",
    created__gt="2024-01-01"
)
for a in failed:
    print(f"{a.pk}: {a.application.name}")

Get single instance by ID

import isabl_cli as ii

# By system_id
exp = ii.Experiment("SAMPLE_001")

# By primary key
analysis = ii.Analysis(12345)

Count records without fetching

import isabl_cli as ii

count = ii.get_instances_count("experiments", projects=102)
print(f"Total experiments: {count}")

Get individual's full tree

import isabl_cli as ii

individual = ii.get_tree(individual_pk)
for sample in individual.sample_set:
    print(f"Sample: {sample.identifier}")
    for exp in sample.experiment_set:
        print(f"  Experiment: {exp.system_id}")

Step 6: Execute and Verify Results

Get results from analyses

import isabl_cli as ii
from isabl_cli import utils

# Get specific result from an experiment
vcf_path, analysis = utils.get_result(
    experiment=experiment,
    application_key=123,
    result_key="vcf"
)
print(f"VCF: {vcf_path}")

# Get all results for an application
results = utils.get_results(
    experiment=experiment,
    application_name="variant_caller"
)

Performance Tips

import isabl_cli as ii

# Only fetch needed fields
experiments = ii.get_experiments(
    projects=102,
    fields=["pk", "system_id", "sample"]
)

# Stop after N results
experiments = ii.get_experiments(
    projects=102,
    count_limit=100
)

# Use cursor pagination for very large datasets
experiments = ii.get_instances(
    "experiments",
    projects=102,
    paginator="cursor"
)

Step 7: Format Output

Convert to DataFrame

import isabl_cli as ii
import pandas as pd

experiments = ii.get_experiments(projects=102)
df = pd.DataFrame([
    {
        "system_id": e.system_id,
        "sample": e.sample.identifier,
        "category": e.sample.category,
        "technique": e.technique.method,
    }
    for e in experiments
])
df.head()

Data Model Reference

Individual
├── identifier (unique)
├── species (HUMAN, MOUSE, etc.)
├── gender
└── sample_set → [Sample]

Sample
├── identifier
├── individual → Individual
├── category (TUMOR, NORMAL, etc.)
└── experiment_set → [Experiment]

Experiment
├── system_id (unique)
├── sample → Sample
├── technique → Technique
├── platform → Platform
├── raw_data (dict)
├── bam_files (dict)
└── analysis_set → [Analysis]

Analysis
├── pk
├── application → Application
├── targets → [Experiment]
├── references → [Experiment]
├── status (CREATED, STAGED, SUBMITTED, STARTED, SUCCEEDED, FAILED)
├── storage_url (path to results)
└── results (dict)

Application
├── name
├── version
├── assembly
├── species
└── settings (dict)