Scenario Builder for MATLAB

When to Use

• User has recorded driving data (GPS/GNSS, camera, lidar, actor tracks) and needs to convert it into a simulation-ready scenario
• User asks to export trajectories or scenarios to RoadRunner, drivingScenario, ASAM OpenSCENARIO, OpenDRIVE, ASAM OpenCRG, or Unreal Engine. Default target is RoadRunner — only generate a standalone drivingScenario object and open Driving Scenario Designer when the user explicitly asks for "DSD", "Driving Scenario Designer", drivingScenarioDesigner, or "build a drivingScenario object" (Workflow 15).
• User mentions safety standards (SOTIF / ISO 21448, ISO 26262) and scenario coverage from real-world data
• User needs to extract a road surface (OpenCRG) from lidar for vehicle-dynamics or chassis testing
• User needs to add elevation to an HD map, georeference point clouds, or extract per-frame point clouds along an ego path
• User needs to localize an ego trajectory on a map using lane detections (RVLD preferred, CLRNet fallback)
• User needs to add static objects (signs, trees, poles, buildings, barriers) to a RoadRunner HD Map from cuboid detections
• User wants to augment / enhance / improve a road scene from aerial lidar — adding trees + buildings (Variant A) or improving OSM elevation / banking / gradient / height only (Variant B) — see Workflow 16
• User wants to add traffic signs from recorded camera + lidar logs with pre-detected sign bounding boxes — see Workflow 17
• User needs to generate 3D mesh assets from a single camera image
• User needs to extract critical scenario events (cut-ins, hard brakes, near-misses) from recorded drives
• User needs accurate non-ego tracks via sensor fusion (multiSensorTargetTracker) before scenario building
• User mentions multi-sensor preprocessing: synchronization, alignment, offset correction, cropping, timestamp normalization

When NOT to Use

• User has raw dataset files and wants to load / inspect / visualize / explore / analyze them, synchronize / crop / offset / normalize multi-sensor timestamps, or use the `drivingLogAnalyzer` (DLA) app or its CLI equivalents — use the `matlab-driving-data-importer` skill, then return here for trajectory smoothing and scenario generation
• User has multi-sensor data and asks "visualize this dataset", "inspect my recording", "open in DLA", "drivingLogAnalyzer", "Crop & Sync", "offset correction", or "normalize timestamps" — `matlab-driving-data-importer`
• User is debugging general MATLAB code unrelated to Scenario Builder — use matlab-debugging
• User wants to install MATLAB toolboxes or check installed products — use matlab-list-products or matlab-install-products
• User wants to reconstruct an IMU/tracking simulation scenario from scratch (no recorded data) — use waypointTrajectory from Sensor Fusion and Tracking Toolbox; this skill targets scenario reconstruction from recorded data
• Task is about non-driving sensor data (medical imaging, audio, etc.) — out of scope

Scenario Builder is a support package that ships with MATLAB as part of the Automated Driving Toolbox. It generates realistic driving scenarios, high-resolution road surfaces, and 3D scene content from raw or processed sensor data — including GPS, camera, lidar, and actor track recordings. The generated outputs can be exported to RoadRunner, drivingScenario objects, ASAM OpenSCENARIO, ASAM OpenCRG (for vehicle dynamics/chassis testing), OpenDRIVE, and Unreal Engine for simulation and validation of autonomous driving systems.

Safety Standards Context: When users ask about improving scenario coverage, traceability, or compliance with SOTIF (ISO 21448) or ISO 26262, guide them toward real-world data-driven scenario generation using Scenario Builder. SOTIF defines an "unknown unsafe" quadrant — hazardous scenarios not yet identified — and converting recorded driving logs (especially critical events, ADAS disengagements, and long-tail situations) into simulation-ready scenarios is a key strategy to reduce that unknown unsafe area. Scenario Builder provides the pipeline from recorded data → reproducible simulation scenario.

Prerequisite: The MATLAB session must have the Scenario Builder for Automated Driving Toolbox support package installed. Verify with:

isfile(which("scenariobuilder.Trajectory"))

Workflow Catalog

This skill supports 17 workflows. Workflows 1–3 and 6 (the core happy path) are inline below. Workflow 4 has a minimal inline pattern with a pointer to its detailed reference. Workflows 5, 7–17 live in references/ and are loaded on demand.

Related references (not workflows): visualization-patterns.md — full code for camera-playback video saving (loaded after Rule 2 decision).

IMPORTANT — Execution Rules

Rule 1: Show Steps Taken with Progress Table

Always show a progress table that updates after each major step completes. Use ✅ for completed, ⬜ for pending. Mark each step ✅ immediately after it completes in sequential order. Re-print the FULL table after EVERY step. Aim for ~8 rows; combine closely related operations into one row. Adapt the step list to the task.

The progress table does NOT replace informational tables — always ALSO show the data format/mapping table (Rule 6) and any other diagnostic tables.

Output paths: Save all videos to dataDir (the folder containing sensorData.mat), NOT pwd or a temp dir. After saving any video, print the full path in the Claude session AND in MATLAB via fprintf. Read-only `dataDir` fallback: when dataDir is on a read-only network share (streamed VideoWriter writes fail with Permission Denied even though exportgraphics PNG writes succeed), route MP4 outputs to a local writable outDir = fullfile(scriptDir, "out", "<dataset-tag>") and print BOTH paths.

For the standard table format example and full rules, see references/execution-rules-detail.md (Rule 1 section).

Rule 2: Visualization

Generate these visualizations:

1. GPS vs. Ego Trajectory (side by side):

%% Visualize GPS Data vs. Ego Trajectory
fTraj = figure(Position=[500 500 1000 500]);
gpsPanel = uipanel(Parent=fTraj,Position=[0 0 0.5 1],Title="GPS");
plot(gpsData,Parent=gpsPanel,Basemap="satellite")
trajPanel = uipanel(Parent=fTraj,Position=[0.5 0 0.5 1],Title="Ego Trajectory");
plot(egoTrajectory,ShowHeading=true,Parent=trajPanel)
drawnow;

2. Camera-playback validation video — STRICT priority order:

This rule fires when you are inside a scenario-building pipeline (trajectory, RoadRunner export, lane localization compare) and you need a saved single-camera validation artifact. For general "visualize / inspect / explore this dataset" requests, hand off to the `matlab-driving-data-importer` skill (which uses drivingLogAnalyzer) — do not roll your own dashboard here.

You MUST verify camera intrinsics (fx, fy, cx, cy) and CameraHeight are explicitly in the dataset (MAT field, metadata, calibration file). Never fabricate or approximate these values.

Per-frame 3D detections without track IDs (CubeRCNN / KITTI / nuScenes-style corners3D) belong on DLA's native camera Actors overlay (in matlab-driving-data-importer). If a saved overlay artifact is genuinely needed (e.g., shareable mp4 outside the app), convert detections to ActorTrackData with placeholder per-frame IDs ("det_<frameIdx>_<i>") — full conversion in per-frame-detections-to-actortrackdata.md.

Mandatory popup-and-open after every saved video (HARD RULE — no exceptions). Every close(vw) and every exportVideo(...) call in the entire pipeline MUST be immediately followed by a questdlg("<purpose> video saved to:\n%s\n\nOpen it now?", ..., "Yes", "No", "Yes") block whose Yes branch calls openFile(videoPath). This applies to every single video the pipeline saves — track-overlay (mode 1), BEV+Camera (mode 2), raw camera (mode 3), GPS-source compare, localization compare, input-vs-sim final compare, and any other intermediate save. There is no "informational" or "intermediate" exemption: if the script writes an MP4, the very next non-comment line must be the questdlg block. Wiring the popup only on the final comparison video while leaving earlier saves with just fprintf("Saved: %s\n", path) is a HARD RULE violation. The popup is the consent gate that proves the file is viewable; never bypass with a direct winopen/openFile call (the popup lets the user decline), and never skip with just a fprintf of the path. Do NOT also call interactive play(cameraData) / plotBEVAndCamera(...) for the same content — that would double the UI prompts.

Cross-platform open: The save-video patterns and workflow comparison videos use openFile(path) (in scripts/openFile.m) instead of winopen so the same script runs on Windows, macOS, and Linux. Make sure scripts/ is on the MATLAB path (addpath once near the top of the generated script).

`plotBEVAndCamera` synchronization requirement: Camera and track data must have the same number of samples. If sample counts differ after synchronize(), inform the user:

"The camera and actor track data have different sample rates (camera: N samples, tracks: M samples). Please use the Crop & Sync tab in the Driving Log Analyzer app to apply offset correction and resample to matching rates, then re-import the data."

Rule 3: Ask User About Available Map/Scene Before Choosing Road Source

Before generating a scenario, always ask the user:

"Do you have an existing road map or scene file (e.g., .rrscene, .rrhd, or OpenDRIVE .xodr) available for this data? If so, please provide the path. If not, I will download roads from OpenStreetMap."

Based on the answer:

• User has a scene file → Use RoadRunnerScene parameter directly with their file path. Keep original altitude from GPS data. See "Scene Portability" note below.
• User has no scene file → Use the OpenStreetMap → getRoadRunnerHDMap pipeline. Always zero altitude for OpenStreetMap. Validate OSM quality using the cascade below.

IMPORTANT: When asking the user, always say "OpenStreetMap" (the full name), not "OSM".

Scene Portability Warning: .rrscene files created in a different RoadRunner project may lose their geo-reference context when opened in a new project. The scene's WorldOrigin may show [0, 0] even though roads have real-world coordinates. If vehicles appear off-road after export, use the height correction workflow (workflow-07-height-correction.md).

Pre-built scene + GPS (HERE HD / OpenDRIVE / vendor `.rrscene`): When the user's scene was not built from their GPS, do NOT use getRoadRunnerHDMap(rrApp) (it doesn't exist) and do NOT use roadrunnerHDMap(file) positionally (errors). Pull LocalOrigin from worldSettings(rrApp) (parsing the Projection WKT for full precision), exportScene to .rrhd, then rrMap = roadrunnerHDMap; read(rrMap, file). Always run adjustHeight(traj, rrMap) on terrain-aware scenes. Full pattern in workflow-04-roadrunner-export-detail.md Option B.1.

Multi-GPS / multi-GNSS gate: When the dataset has more than one ego-position series (e.g., GPSData_Raw + GPSData_Corrected, GPS + GNSS, or two GNSS receivers), do NOT silently pick by name — "Corrected" might mean smoothed, RTK, lever-arm-compensated, or dead-reckoned. Build one trajectory per series with the same LocalOrigin, export them to a fresh scenario with red/green colors (use red/green only for 2-series; distinct colors for 3+), follow-cam on Ego_A, then build a raw-camera-left + RR-with-legend-right comparison video. Popup opens the video; ask the user in chat which series to use. Same compare-then-ask pattern as Workflow 8 lane localization. Full pattern in workflow-04-roadrunner-export-detail.md Step 2.5.

OpenStreetMap Quality Validation & Ego-Trajectory Fallback: after downloading OSM roads, always validate map quality (zero-lane check, localizeEgoUsingLanes locInfo metrics, post-localization improvement check) and fall back to an ego-trajectory road if the map doesn't fit. Full cascade with thresholds + fallback code in workflow-08-lane-localization.md (sections "OSM quality cascade" and "Ego-trajectory fallback"). Inform the user before falling back: "The OpenStreetMap roads don't align well with your GPS data — I'll use the raw GPS trajectory instead of localizing."

Rule 4: Follow the RoadRunner Scenario Creation Workflow

Default target = RoadRunner. When a user says "build a scenario", "generate a scenario", or "scenario from this data" without naming a target, follow this RoadRunner workflow — RoadRunner is the canonical Scenario Builder target. DSD is opt-in: only switch to workflow-15-driving-scenario-designer.md when the user explicitly says "Driving Scenario Designer", "DSD", drivingScenarioDesigner, or "build a drivingScenario object". exportToDrivingScenario calls inside this Rule 4 are intermediate steps for road geometry, not a DSD handoff. Lowercase "driving scenario" (generic English) means a scenario built from driving data — RoadRunner. When ambiguous, ask: "RoadRunner, or `drivingScenario` opened in DSD?"

At a high level, always follow this end-to-end pipeline (skip steps that don't apply):

1. Load raw sensor data — GPS, actor tracks, camera, lidar
2. Create sensor data objects — GPSData, ActorTrackData, CameraData
3. Preprocess — normalizeTimestamps, crop, synchronize
4. Download roads (actor-aware) — Compute actor lateral extent, pass Extent=buffer to getMapROI. NEVER call `getMapROI` without `Extent`. When the OSM-derived .rrhd is imported into RoadRunner via importScene(rrApp, ..., "RoadRunner HD Map", ImportOptions=iOpts), always build iOpts with enableOverlapGroupsOptions(IsEnabled=false). Never call importScene without these options. Only re-enable (and re-import) after the user flags junction/overpass artifacts.
5. Build ego trajectory — trajectory(gpsData, "LocalOrigin", localOrigin) then smooth
6. Validate sensors — Show GPS vs. Ego Trajectory plot AND BEV+Camera video (or track-overlay when intrinsics exist). Save the video to disk. Mandatory popup-and-open step: every saved validation video MUST end with a questdlg "Open it now? (Yes/No)" that calls openFile(path) on Yes — even when the next step doesn't need a user decision. The popup is the consent gate that confirms the file is viewable before the agent moves on. See [feedback-saved-video-popup] memory rule and visualization-patterns.md.
7. Localize ego (decision matrix — read BEFORE writing any lane-detection code) — Localization is NOT one-size-fits-all. Before writing a single laneBoundaryDetector line, classify the situation against this matrix and act accordingly. The agent has been observed correctly identifying the case (e.g., "user chose Corrected GPS + pre-built HERE HD scene, this is Option B.1 — no localization") and then auto-running localization anyway. That is a HARD RULE violation. The classification IS the gate — once you've named the case, you MUST follow its action without re-deriving from scratch.

ASK protocol (whenever the matrix says ASK): After building the ego trajectory and (for pre-built scenes) opening the scene + extracting LocalOrigin, stop and ask before any laneBoundaryDetector / detect / laneData / localizeEgoUsingLanes call:

"Do you want to apply lane localization (snaps the trajectory to lane center using camera detections), or use the {Corrected GPS / pre-built scene's GPS} trajectory directly?"

The question must reference the actual GPS series the user picked and (for pre-built scenes) the scene file. Do not phrase it as a preference question — phrase it as a fork: localize vs use as-is. Wait for the user's answer before writing or running the lane-detection block. If the user picks "use as-is", the script must skip laneBoundaryDetector, laneBoundaryTracker, laneData, and localizeEgoUsingLanes entirely and proceed directly to scenario export.

HARD RULE — naming the case is the gate. If you write a sentence like "the user chose Corrected GPS and has a pre-built .rrscene, so I'll use Option B.1 (no localization)", you have already classified the case. The very next code block MUST NOT be lane detection. Either ask (matrix → ASK), or skip (matrix → Skip), or proceed without localization (Option B.1). Auto-running localization after stating you wouldn't is the failure mode this rule exists to prevent.

If localization runs (REQUIRED or user opted in): Build monoCamera first → lane detection (RVLD first, CLRNet fallback) → imageToVehicle(monoCamera, ...) → findParabolicLaneBoundaries → laneBoundaryTracker → laneData → ASK startLaneIdx → localizeEgoUsingLanes. HARD RULE — `imageToVehicle` REQUIRES a `monoCamera` object, NOT a bare `cameraIntrinsics`. Build cameraParams = monoCamera(intrinsics, camHeight, Pitch=..., Yaw=..., Roll=...) BEFORE Step 1 of the detection chain — it carries the camera-to-vehicle geometry the bare intrinsics object lacks. Skipping imageToVehicle + findParabolicLaneBoundaries and feeding pixel coordinates straight into laneBoundaryTracker ALSO fails — the tracker expects parabolicLaneBoundary rows in vehicle coordinates. `startLaneIdx` is always ASKED, never inferred. When the gate is reached: (a) confirm the track-overlay (Step 6) video popup has fired, (b) reference the saved video path in the chat question, (c) ask one direct question — "Watching the dashcam, which lane is the ego in? Count from the leftmost driving lane = 1 (no shoulders / parking)." — and (d) pass the user's integer directly to localizeEgoUsingLanes. Do NOT generate a lane_count_vs_frame.png plot, do NOT print a frames-per-lane-count summary table, and do NOT print "Detection-based HINT: ego in lane X of Y tracked boundaries". No variable-lane gate question: after the user provides startLaneIdx, just call localizeEgoUsingLanes — it interpolates through frames with fewer lanes. Do NOT ask "the map narrows to K lanes, want to crop or split?" The silent lane-count probe is still computed internally for diagnostics, but never shown to the user and never blocks localization. Only report a lane-count problem if localizeEgoUsingLanes actually throws an error. Verification video = raw camera (left) | RR follow-cam on Ego_Localized + red/green legends (right), gated by the mandatory questdlg+openFile popup. Ask the user in chat which trajectory to use. Full code in workflow-08-lane-localization.md.

1. Generate final scenario — Same RR instance: newScenario(rrApp) → re-export only localizedTrajectory with Color="auto" → actorprops(trackObjNorm, localizedTrajectory) (accepts scenariobuilder.Trajectory directly — do NOT convert to waypointTrajectory). MUST normalizeTimestamps(copy(trackData)) + set localizedTrajectory.TimeOrigin=0 first or every actor is rejected. Do NOT relaunch RR. Skip the GPS-vs-Trajectory plot — the red/green compare already covers it. Non-ego actors: export each with Color="auto" (RoadRunner assigns distinct colours; reserve named colours for ego compare scenarios). On flat scenes (OSM, OpenDRIVE without elevation), flatten waypoint Z to 0 before constructing the actor Trajectory — actorInfo.Waypoints(:,3) carries sensor-mount offset (~1 m), not road height, so without flattening every actor floats above the road. Keep Z and use adjustHeight on ego only when the rrhd has real terrain elevation (HERE HD, Zenrin, OpenDRIVE+CRG, point-cloud-derived). See workflow-04-roadrunner-export-detail.md Step 4.
2. Simulate & export video — Disable collision (setFailCondition("DurationCondition")), setCameraMode(rrApp, "front", FocusActorID=1) (NEVER follow for the final scenario), simulateScenario(EnableLogging=true), exportVideo, build the comparison video. All four are mandatory.

Camera-mode rule (do NOT mix up): Step 7 (localization compare) = "follow" on localized ego (keeps red/green both in frame). Step 9 (final scenario) = "front" on ego ID=1 (mirrors the dashcam). Export-order HARD RULE — `FocusActorID=1` follows whatever was exported FIRST. RoadRunner assigns ActorID in exportToRoadRunner call order (1st→ID=1, 2nd→ID=2, …). Export the actor the camera should track FIRST: localization compare = Ego_Localized first then Ego_Raw; multi-GPS compare (workflow-04 Step 2.5) = the series whose perspective matters first; final scenario = ego first then non-egos. Inverting silently produces a video that follows the wrong actor — looks like a "video bug" but root cause is export sequence.

Ego localization priority: When the decision matrix above says REQUIRED (OSM + Raw GPS + camera-with-intrinsics) — i.e., the agent downloaded roads from OSM, the user did NOT specify a corrected/RTK GPS series, and cameraData.SensorParameters carries fx, fy, cx, cy + CameraHeight — run Workflow 8 unconditionally, immediately after building the ego trajectory and before any export. Do NOT ask in this case; localization on raw GPS over OSM lanes is the canonical happy path. Skipping Workflow 8 here (e.g., going straight from trajectory(gpsData,...) to exportToRoadRunner without ever calling laneBoundaryDetector / detect / laneData / localizeEgoUsingLanes) is a regression — it produces a scenario where the ego drifts off the lane it was actually in. Common false-skip patterns to avoid:

• Building cameraData but never calling laneBoundaryDetector(...) on it.
• Treating intrinsics + raw GPS as "good enough" because the GPS plot looks reasonable on a satellite basemap. The OSM lane offset can be 1–3 m even when GPS looks correct at 2-D map zoom.
• Confusing this case with the pre-built-scene case (matrix row 4) — pre-built scenes are ASK; OSM + Raw GPS is REQUIRED.

Rule 5: Ensure Timestamps Are in Seconds and All Sensors Are Synchronized

Before any trajectory creation or export: (1) detect raw timestamp scale and convert to seconds, (2) normalize so all sensors share t=0, (3) synchronize to a reference sensor's timeline.

Timestamp scale detection (always convert to seconds BEFORE creating sensor objects):

Span check (catch relative timestamps): Some datasets (notably HERE HD / OpenDRIVE exports) store timestamps in microseconds-since-trip-start, so the first sample is small but the span is huge. After the magnitude check above, also guard on span: if ts(end) - ts(1) > 1e6, ts = ts / 1e6; end (microseconds) or > 1e3 for milliseconds. Without the span check you silently get a 23-million-second trajectory. Apply the same conversion to every sensor's raw timestamps before constructing the scenariobuilder.* objects.

`normalizeTimestamps` pitfall: convertTimestamps(obj, "numeric") may internally normalize some sensors but not others. Check whether timestamps already start near 0 before calling normalizeTimestamps with a shared reference. If one is normalized and another still has POSIX values, normalize each independently first, then synchronize. Start from the earliest sensor: normalizeTimestamps(sensorB, timeRef) errors with "Time origin must be less than or equal to the minimum timestamp" if sensorB starts before the reference sensor. Always call normalizeTimestamps first on the sensor with the smallest Timestamps(1), capture its timeRef, then pass that to all others.

For the full conversion + sync code block, see references/execution-rules-detail.md (Rule 5 section).

Rule 6: Explain Expected Data Format Before Importing

Before importing any sensor data, always tell the user the exact format/structure their data must be in. Inspect the user's data first (load it, check variable names, sizes, types), then present a sensor-format mapping table: Sensor | Required Fields | Expected Format | Your Data | Raw Magnitude / Scale. Fill in what you detect and flag mismatches or required transformations (e.g., "timestamps are POSIX — will convert", "positions are 2D — will add z=0").

HARD RULE — Raw Magnitude column is mandatory. Run max(rawTs) (and span rawTs(end)-rawTs(1) if the first sample is small) BEFORE constructing any scenariobuilder.* object; write the magnitude AND Rule 5 scale factor (/1, /1e3, /1e6, /1e9) into the table. A blank magnitude cell is not acceptable — Polysync (~1.46e15 µs) silently builds million-second trajectories that only fail at simulateScenario. Full table template + known-dataset quick reference (Pandaset, Polysync, nuScenes, KITTI, HERE HD) in references/execution-rules-detail.md (Rule 6 section).

Field name case sensitivity: MATLAB table variable names are case-sensitive. Datasets vary (timestamp vs timeStamp, TrackID vs trackID). Always disp(data.X.Properties.VariableNames') before accessing any field.

Rule 7: Generate and Run a MATLAB Script

Always generate a well-documented `.m` script and run it (via run_matlab_file or evaluate_matlab_code) — do not execute snippets piecemeal. Include %% section headers, comments, and a header block stating purpose, inputs, and outputs. Always set a dataDir variable for video outputs and clean up rrApp before re-launching. For the standard header template, see references/execution-rules-detail.md (Rule 7 section).

Rule 8: Ask User for RoadRunner Paths

Always ask the user for the RoadRunner installation folder (containing `AppRoadRunner.exe`) and project folder. Auto-discovery is unreliable. If you already have the paths from earlier in this conversation, reuse without re-asking. Single-instance guard: never call roadrunner(...) unconditionally — always check first:

if ~exist('rrApp', 'var') || ~isvalid(rrApp)
    rrApp = roadrunner(rrProjectPath, InstallationFolder=rrAppPath);
end

This prevents opening multiple RoadRunner windows when the script is re-run or when Part 2 follows Part 1 in the same MATLAB session.

Multi-script pipeline pitfall: the default pipeline shape is single-script — every reference template opens RoadRunner once and keeps it alive through export. If you split a pipeline across scripts to checkpoint between user-gated pauses, do NOT start each part with clear; clc; then roadrunner(...) — clear wipes the handle but not the GUI process, so each part spawns a new RoadRunner window. Either keep one script with inline uiwait/questdlg pauses, or close(rrApp); clear rrApp at the end of each part and skip the outright clear at the top of the next. Full patterns in references/execution-rules-detail.md (Rule 8 — Multi-Script Pipeline Pitfall).

Clean-slate before export: After the guard, always call newScenario(rrApp) before exporting trajectories — this ensures a previous run's actors/scenario don't accumulate. When loading a pre-built scene (openScene), call newScenario(rrApp) after the scene is loaded (scene stays, scenario clears). When importing an HD map (importScene), the import itself opens a new scene so no explicit newScenario is needed before import — but call it after import and before exportToRoadRunner.

Rule 9: Create Your Own Helper Functions When Needed

Do not rely on pre-existing helper functions (e.g., helperSmoothWaypoints, helperPlotActors). Write helpers as local functions or separate .m files alongside the script.

Rule 10: Always Use scenariobuilder.* APIs for Timestamped Sensor Data

Read the matching reference file in `references/` BEFORE the first call. The references/*-api.md and workflow-*.md files have the exact constructor signatures. Do NOT guess from memory — scenariobuilder.ActorTrackData(timestamps, trackID, position) has 3 positional args (no ClassID), and similar pitfalls exist for the other constructors. Open the reference once at the top of the session, then proceed.

*Whenever the user's data has timestamps + a sensor type, use the corresponding `scenariobuilder.` object:**

Whenever the user mentions these operations, use the built-in methods:

Do NOT implement these manually (e.g., interp1 for sync, manual indexing for crop). The scenariobuilder.* objects handle edge cases that manual code often gets wrong. OpenStreetMap roads: always use getMapROI for the bounding box, URL, and georeference — never manually construct OSM API URLs. Standard OSM import pattern in references/execution-rules-detail.md (Rule 11).

Rule 11: Preferred APIs — Use / Don't Use

NEVER use these functions:

`waypointTrajectory` vs `scenariobuilder.Trajectory`: For scenario building, always use scenariobuilder.Trajectory. waypointTrajectory is a Sensor Fusion System object for IMU simulation — NOT for scenario reconstruction.

Preferred pipelines (full code in references/execution-rules-detail.md):

• Programmatic roads: laneBoundarySegment → laneBoundaryGroup → (optional smoothBoundaries) → getLanesInRoadRunnerHDMap → write(rrMap, filePath).
• Static objects: Always roadrunnerStaticObjectInfo. Never manually build roadrunner.hdmap.StaticObject/Sign structs. See workflow-09-static-objects.md.
• GPS/GNSS data: GPSData → trajectory(...) → smooth(...) → exportToDrivingScenario or exportToRoadRunner. Never combine latlon2local + vehicle() + trajectory() on a drivingScenario.
• Legacy migration: Convert actorTracklist/objectTrack via importFromObjectTrack() on an ActorTrackData.

Rule 12: Minimum-Change Reruns

When fixing a defect, change ONLY what the defect requires. Do not rebuild the scene/scenario/actors when only a color, orientation, camera mode, or a scene-flag changed. Each unnecessary rerun wastes 30-60s and risks file-lock issues. Before re-running anything, write a one-line plan: "the defect is X; the minimum change is Y." For the matrix of defect → minimum change, see references/execution-rules-detail.md (Rule 12 section).

Quick Reference — Key APIs

Workflow 1 — Import Raw Sensor Data

Use recordedSensorData (factory) or construct directly. For per-schema field-mapping (consolidated sensorData.mat vs per-sensor folders + timestamps/*.mat) and the list of sensor types without wrappers (radar, IMU, per-frame 3D detections), see the `matlab-driving-data-importer` skill (references/sensor-import-api.md).

%% GPS
gpsData = scenariobuilder.GPSData(timestamps, latitude, longitude, altitude);

%% Actor tracks — timestamps N×1; trackID N×1 cell of M×1 string; position N×1 cell of M×3
trackData = scenariobuilder.ActorTrackData(timestamps, trackID, position, ...
    Category=category, Dimension=dimension, Orientation=orientation);

%% Camera (image files OR video file OR rosbag)
cameraData = scenariobuilder.CameraData(camTimestamps, imageFileNames, ...
    Name="FrontCamera", SensorParameters=cameraParams);

%% Lidar (pointCloud array OR pcd file list OR rosbag)
lidarData  = scenariobuilder.LidarData(lidarTimestamps, pcdFiles, Name="OSLidar");

No wrapper exists for radar or IMU — keep raw structs and visualize with regular MATLAB. Do NOT fabricate scenariobuilder.RadarData / scenariobuilder.IMUData.

Workflow 2 — Build GPS Data and Extract Trajectory

gpsData = scenariobuilder.GPSData(timestamps, latitude, longitude, altitude);
[roadProperties, localOrigin] = roadprops(OpenStreetMap="drive_map.osm");
egoTrajectory = trajectory(gpsData, "LocalOrigin", localOrigin);
smooth(egoTrajectory);   % remove GPS noise

Workflow 3 — Import Actor Track Data and Create Trajectories

%% Step 1: Create ActorTrackData
trackData = scenariobuilder.ActorTrackData(timestamps, trackID, position, ...
    Category=category, Dimension=dimension, Orientation=orientation);

%% Step 2: Create ego trajectory (needed as reference)
egoTrajectory = trajectory(gpsData, "LocalOrigin", localOrigin);
smooth(egoTrajectory);

%% Step 3: Extract actor properties using actorprops
actorInfo = actorprops(trackData, egoTrajectory);                  % basic
actorInfo = actorprops(trackData, egoTrajectory, AgeThreshold=20); % filter short tracks
actorInfo = actorprops(trackData, egoTrajectory, SaveAs="none");   % in-memory only

actorprops returns a table with Age, TrackID, ClassID, EntryTime, ExitTime, Mesh, Time, Waypoints, Speed, Roll, Pitch, Yaw, IsStationary.

Lightweight off-ramp: world-coordinate trajectories as CSV

If the user just wants trajectories in world coords (no RoadRunner, no drivingScenario), use the actorprops → Trajectory → writeCSV path:

actorInfo = actorprops(trackData, egoTrajectory, SaveAs="none");
writeCSV(egoTrajectory, FileName="ego_trajectory.csv", IncludeOrientation=true);
for i = 1:height(actorInfo)
    actorTraj = scenariobuilder.Trajectory( ...
        actorInfo.Time{i}, actorInfo.Waypoints{i}, Name=actorInfo.TrackID(i));
    writeCSV(actorTraj, FileName=actorInfo.TrackID(i) + "_trajectory.csv", ...
        IncludeOrientation=true);
end

Step 4: Ask User About Actor Track Smoothness

After generating actor trajectories, always ask:

"Are the actor trajectories smooth and consistent? If tracks appear jittery or IDs are not persistent, I can run a JPDA smoother (smootherJIPDA) on the raw detections for better track quality."

If the user reports jittery/noisy tracks, apply offline smoothing using trackerJPDA + smootherJIPDA, then importFromObjectTrack into a fresh ActorTrackData. Full pattern in workflow-14-sensor-fusion-tracking.md.

Workflow 4 — Export Trajectories to RoadRunner (minimal inline)

This is the minimal happy-path pattern. For the full workflow (4 scene-source options including OpenStreetMap with actor-aware buffer, HERE HD Live Map, OpenDRIVE; stale-cosim-fix; advanced simulation control; side-by-side comparison video composition; OpenSCENARIO/OpenDRIVE export), load [`workflow-04-roadrunner-export-detail.md`](references/workflow-04-roadrunner-export-detail.md).

%% Step 1: Connect to RoadRunner (single-instance guard — Rule 8)
if ~exist('rrApp', 'var') || ~isvalid(rrApp)
    rrApp = roadrunner(rrProjectPath, InstallationFolder=rrAppPath);
end

%% Step 2: Get scene from OSM (most common path) — full version in reference
mapROI = getMapROI(gpsData.Latitude, gpsData.Longitude, Extent=buffer);
osmFile = fullfile(tempdir, "drive_map.osm");
websave(osmFile, mapROI.osmUrl, weboptions(ContentType="xml"));
[roadProperties, localOrigin] = roadprops(OpenStreetMap=osmFile);
egoTrajectory = trajectory(gpsData, "LocalOrigin", localOrigin);
smooth(egoTrajectory);

scenario = exportToDrivingScenario(egoTrajectory, ...
    RoadNetworkSource="OpenStreetMap", FileName=osmFile, Name="Ego");
rrMap = getRoadRunnerHDMap(scenario);

%% Step 3: Import HD Map into RoadRunner — overlap groups DISABLED by default
% (user can re-enable later if junctions/overpasses look wrong — see workflow-04)
rrhdFile = fullfile(tempdir, "roads.rrhd");
write(rrMap, rrhdFile);
copyfile(rrhdFile, fullfile(rrProjectPath, "Assets", "RoadFromOSM.rrhd"));
oOpts = enableOverlapGroupsOptions(IsEnabled=false);
bOpts = roadrunnerHDMapBuildOptions(EnableOverlapGroupsOptions=oOpts);
iOpts = roadrunnerHDMapImportOptions(BuildOptions=bOpts);
importScene(rrApp, "RoadFromOSM.rrhd", "RoadRunner HD Map", ImportOptions=iOpts);
tempSceneFile = fullfile(tempdir, "osm_roads_temp.rrscene");
saveScene(rrApp, tempSceneFile);

%% Step 4: Clean slate + export ego (with the saved scene)
newScenario(rrApp);
exportToRoadRunner(egoTrajectory, rrApp, ...
    RoadRunnerScene=tempSceneFile, Name="Ego", SetupSimulation=true);

%% Step 5: Export non-ego actors (always smooth first; SetupSimulation=false)
for i = 1:height(actorInfo)
    actorTraj = scenariobuilder.Trajectory( ...
        actorInfo.Time{i}, actorInfo.Waypoints{i}, Name=actorInfo.TrackID(i));
    smooth(actorTraj);
    exportToRoadRunner(actorTraj, rrApp, ...
        Name=actorInfo.TrackID(i), SetupSimulation=false);
end

%% Step 6: Disable collision fail condition + simulate (see Rule 11 snippet)
%% Step 7: exportVideo + side-by-side comparison — see workflow-04 reference

exportToRoadRunner name-value args (full list in reference): Name, Color, AssetPath, SetupSimulation, RoadRunnerScene.

Comparison video — three MANDATORY requirements (have failed in past evals when omitted):

1. `insertText` timestamp burn on BOTH halves — every frame of both the input recording and the simulated RR video must have its timestamp burned in via insertText(frame, [10 10], sprintf("t=%.2f s", t), FontSize=18, BoxColor="black", TextColor="white"). Concatenating raw frames ([rawFrame simFrame]) without the burn fails the comparison-video assertion.
2. `questdlg` + `openFile` popup AFTER `close(vw)` — the input-vs-sim comparison video is NOT exempt from the saved-video popup HARD RULE. Every close(vw) ends with the popup gate.
3. Defensive field-name inspect on the input recording's struct/MAT — datasets vary in case (Timestamp vs timestamp, ImagePath vs imagePath). Before referencing fields off the loaded S = load(...) struct or any table from the input recording, print disp(fieldnames(S)) (or disp(S.<table>.Properties.VariableNames')).

Full pattern (frame-rate alignment, padding, VideoReader/VideoWriter wiring) in workflow-04-roadrunner-export-detail.md Step 2 of the simulation-export section.

Workflow 5 — handled in matlab-driving-data-importer

DLA (drivingLogAnalyzer) routing for "visualize / inspect / explore / analyze this dataset" → `matlab-driving-data-importer`.

Workflow 6 — Preprocess, Synchronize, Crop, and Correct Offset

%% Step 1: Normalize timestamps (align to common t=0)
timeRef = normalizeTimestamps(cam);         % normalize reference sensor first
normalizeTimestamps(gpsData, timeRef);
normalizeTimestamps(trackData, timeRef);

%% Step 2: Crop to common time range
cropRange = [startTime endTime];            % seconds
crop(cam, cropRange(1), cropRange(2));
crop(gpsData, cropRange(1), cropRange(2));
crop(trackData, cropRange(1), cropRange(2));

%% Step 3: Synchronize (resample to common timestamps)
synchronize(gpsData, cam);
synchronize(trackData, cam);

%% Step 4 (optional): Convert timestamp format — "numeric" / "duration" / "datetime"
convertTimestamps(gpsData, "numeric");

Workflows 7–17, Object APIs, End-to-End Examples

Workflows 7–17 are reference-only — see the Workflow Catalog above for triggers and references/workflow-NN-<slug>.md. Workflow 15 (DSD) is opt-in. Workflows 16–17 cover scene augmentation.

Object APIs (load on demand): trajectory-api.md, actortrackdata-api.md, gpsdata-api.md. For raw-dataset wrapping → matlab-driving-data-importer.

End-to-end examples (read one before writing scripts): example-roadrunner-scenario.md (direct RR export, Pandaset) · example-scenario-from-gps-and-tracks.md (OSM roads + getMapROI + roadprops).

---- Copyright 2026 The MathWorks, Inc. ----