sceneview— agent skill

What SceneView is

SceneView is a declarative 3D and AR SDK. One mental model across every platform:

• Android — SceneView { … } (3D) and ARSceneView { … } (AR) composables.

Filament renderer. Artifacts: io.github.sceneview:sceneview:4.62.0 and io.github.sceneview:arsceneview:4.62.0.

• Apple (iOS / macOS / visionOS) — SceneView { } and ARSceneView { } SwiftUI

views from the sceneview monorepo via Swift Package Manager (tag 4.62.0). RealityKit renderer.

• Compose Multiplatform — SceneViewer(…), one composable from commonMain

(io.github.sceneview:sceneview-compose, unreleased). Viewer subset only — model, orbit camera, key light, environment, tap. No AR, no custom materials, no post-processing. Android delegates to the Filament SceneView { } below; iOS needs a one-time renderer registration; Desktop draws a placeholder. Reach for it only when the ask is genuinely shared-source; for anything platform-specific, use the native API.

• Web — sceneview-web@4 on npm (Filament.js + WebXR).
• Flutter — flutter_sceneview plugin (PlatformView bridge; pub.dev name since #2735, directory flutter/sceneview_flutter/).
• React Native — @sceneview-sdk/react-native@4 (Fabric bridge).
• MCP — sceneview-mcp on npm — gives AI agents direct API access from chat.

Nodes are declared as composables / SwiftUI views inside the parent SceneView's trailing content. No imperative scene.addChild(node).

Authoritative API reference

Always treat `llms.txt` in the repo root as the source of truth. It carries the full SceneView / ARSceneView signatures, every node type, every helper. URL: <https://github.com/sceneview/sceneview/blob/main/llms.txt>

Repo-side samples/android-demo/src/main/java/io/github/sceneview/demo/demos/ contains a working demo for every node type — when in doubt, read the demo, do NOT improvise an API.

When to use this skill

Trigger on any of:

• "Build me a 3D viewer / AR app in Compose / SwiftUI."
• "Load a .glb / .gltf / .usdz / .3mf model in Compose."
• "Open the .3mf that ChatGPT / a slicer gave me, in 3D or in AR."
• "Place a model on a detected AR plane / image / face."
• "Render 3D on the web with Filament.js or WebXR."
• "Bridge a 3D scene to Flutter or React Native."
• "Show the same 3D model from shared commonMain code in a Compose Multiplatform app."
• "Convert a 2.x / 3.x SceneView snippet to 4.x."

Skip for plain ARCore-SDK, Sceneform (deprecated), Unity, Unreal, or RealityKit projects that do NOT use the SceneViewSwift wrapper.

Compose Multiplatform: SceneViewer (viewer subset, unreleased)

sceneview-compose is the only way to write one 3D composable in commonMain. It is not a portable replacement for SceneView { } — it covers the model-viewer case and stops there, on purpose.

Choose it only when the ask is genuinely shared-source. If the app is Android-only, or needs AR, custom materials or post-processing, use the platform-native API instead; this façade will not grow to cover them.

// commonMain — compiles on Android, iOS and Desktop
SceneViewer(
    model = ModelSource.Asset("models/toy_car.glb"),
    modifier = Modifier.fillMaxSize(),
    camera = rememberCameraState(distance = 4f),
    lighting = Lighting(intensity = 100_000f, castShadows = true),
    environment = EnvironmentSource.Default,
    onTap = { hit -> if (hit != null) println("hit at ${hit.position}") },
    onError = { error -> println("load failed: ${error.message}") },
)

Rules an agent must not get wrong:

• `ModelSource` is a sealed interface, not a string: Asset(path), Bytes(byteArray)

or Url(url). Url accepts http/https only and throws IllegalArgumentException on anything else, in commonMain, on every platform.

• `Bytes` and `Url` must be self-contained (a GLB, or glTF with embedded buffers).

Only Asset resolves sibling .bin/texture files. A .gltf with external resources passed as Bytes loads incomplete and silently.

• A failed load has no pixels. The viewport keeps showing the environment, which is

indistinguishable from a load still in progress. Pass onError — without it the only trace is the platform log under the SceneViewer tag. It fires both for an exception (missing asset, HTTP error) and for a malformed model Filament refuses to parse, where SceneViewerError.cause is null.

• Platform status: Android renders (Filament). iOS renders (RealityKit) but the app

must register SceneViewerBridge.factory once at launch — a KMP module cannot depend on a Swift Package. Desktop draws a visible placeholder; it is not wired yet. On an unwired platform this never throws and never shows an empty viewport.

• `usdz` is Apple-only. Every platform accepts glTF and GLB.
• Lighting maps approximately between Filament and RealityKit — a scene tuned on

Android reads slightly differently on iOS. That is a property of the façade.

The minimal correct Android example

Verified against samples/android-demo/.../ModelViewerDemo.kt:

@Composable
fun ModelViewerDemo() {
    val engine = rememberEngine()
    val modelLoader = rememberModelLoader(engine)
    val environmentLoader = rememberEnvironmentLoader(engine)
    val modelInstance = rememberModelInstance(modelLoader, "models/helmet.glb")

    SceneView(
        modifier = Modifier.fillMaxSize(),
        engine = engine,
        modelLoader = modelLoader,
        environmentLoader = environmentLoader,
    ) {
        modelInstance?.let { instance ->
            ModelNode(
                modelInstance = instance,
                scaleToUnits = 0.3f,
            )
        }
    }
}

AR tap-to-place is the same shape with ARSceneView. Verified against samples/android-demo/.../ARPlacementDemo.kt:

@Composable
fun ARPlacementDemo() {
    val engine = rememberEngine()
    val modelLoader = rememberModelLoader(engine)
    val placedAnchors = remember { mutableStateListOf<Anchor>() }
    var latestFrame by remember { mutableStateOf<Frame?>(null) }

    ARSceneView(
        modifier = Modifier.fillMaxSize(),
        engine = engine,
        modelLoader = modelLoader,
        planeRenderer = true,
        sessionConfiguration = { _, config ->
            config.planeFindingMode = Config.PlaneFindingMode.HORIZONTAL_AND_VERTICAL
            config.lightEstimationMode = Config.LightEstimationMode.ENVIRONMENTAL_HDR
        },
        onSessionUpdated = { _, frame -> latestFrame = frame },
        onGestureListener = rememberOnGestureListener(
            onSingleTapConfirmed = { event, node ->
                if (node != null) return@rememberOnGestureListener
                val frame = latestFrame ?: return@rememberOnGestureListener
                val hit = frame.hitTest(event).firstOrNull {
                    it.trackable is Plane && (it.trackable as Plane).isPoseInPolygon(it.hitPose)
                }
                hit?.createAnchor()?.let { placedAnchors.add(it) }
            }
        ),
    ) {
        placedAnchors.forEach { anchor ->
            key(anchor) {
                AnchorNode(anchor = anchor) {
                    rememberModelInstance(modelLoader, "models/helmet.glb")?.let { instance ->
                        ModelNode(modelInstance = instance, scaleToUnits = 0.3f, isEditable = true)
                    }
                }
            }
        }
    }
}

Note: ARSceneView takes a sessionConfiguration: (Session, Config) -> Unit lambda — there is NO rememberARSession() helper, do NOT invent one. AnchorNode takes a com.google.ar.core.Anchor instance; create one via hit.createAnchor() after a hit-test on the latest Frame.

Critical rules (verified — do not break)

1. `rememberModelInstance` returns nullable. First recomposition returns

null while loading. Always guard with ?.let { … } or ?:. Never !!.

1. Filament JNI is main-thread-only. The remember* helpers handle this.

For imperative code use modelLoader.loadModelInstanceAsync (see llms.txt § Threading rules).

1. `LightNode` accepts both top-level params and `apply = { … }` for builder

extras. The canonical form for intensity/color/direction is top-level (verified against LightingDemo.kt):

   LightNode(
       type = LightManager.Type.POINT,
       intensity = 30_000f,
       direction = Direction(-x, -y, -z),
       position = Position(x, y, z),
       color = colorOf(r = 1.0f, g = 0.95f, b = 0.8f),
       apply = { falloff(6f) },   // only Filament-builder extras go here
   )

type is com.google.android.filament.LightManager.Type (DIRECTIONAL, POINT, FOCUSED_SPOT, SPOT, SUN).

1. AR anchors come from ARCore. Build an AnchorNode with a real

com.google.ar.core.Anchor from hit.createAnchor(). There are NO AnchorNode.image() / .face() / .plane() factory functions on Android in v4.2 — use AugmentedImageNode for tracked images and AugmentedFaceNode for face meshes (both in arsceneview package).

1. `SceneView` vs `ARSceneView` ship in different artifacts. Don't mix.

3D-only → io.github.sceneview:sceneview. AR → io.github.sceneview:arsceneview (it transitively includes sceneview).

1. `.3mf` needs no special handling — do not write any. A 3MF (what ChatGPT and

every slicer emit for a printable model) goes through the same rememberModelInstance / loadModel* call as a GLB: ModelLoader detects it by its ZIP magic and converts it to GLB in memory (#3482). Never branch on the file extension or the MIME type to decide — Android reports neither reliably, and a shared .3mf routinely arrives as application/octet-stream with no name at all. If you genuinely need to identify a buffer, ThreeMfLoader.isThreeMf(bytes) in sceneview-core reads the bytes, on every platform. See references/recipes.md § Recipe: open a .3mf print. The conversion also scales the file's declared unit to metres and rotates the printer's Z-up to Y-up, so a 60 mm print is life-size and upright in AR — never add a scale factor of your own to "fix" it.

1. Only glTF/GLB, 3MF (Android) and USDZ (Apple) load. STL, PLY meshes and

OBJ + MTL are open issues (sceneview/sceneview#3486, #3487, #3488), and 3MF is not wired into sceneview-web yet (#3491). Say the format is not supported and name the issue rather than inventing a loader for it.

1. Don't recompose-thrash the loaders. rememberEngine / rememberModelLoader

/ rememberMaterialLoader / rememberEnvironmentLoader belong at the top of the screen-level composable, NOT inside scroll lists or item composables.

Performance / hot paths

A `SceneView` renders on demand by default (4.38.0+). isRendering is gone; frameRatePolicy = FrameRatePolicy.OnDemand() is the default, Continuous() is the old every-vsync behaviour, and maxFps caps either mode. The library tracks what changes the picture, so never generate a hand-rolled dirty flag — but a write made straight into Filament (a MaterialInstance parameter, a light through LightManager, a Skybox / IndirectLight, bone or morph writes) is invisible to it and needs node.requestRender() or rememberRenderInvalidator() after it. A recomposition is not a change, and ARSceneView takes no policy. Details: references/migration.md § Rendering cadence.

Never call a decomposing or allocating getter inside `onFrame` (or any 30–60 Hz loop). Set the whole node.transform = … once instead of writing position / quaternion / scale one at a time (one-at-a-time writes recompose the matrix and drift — issue #2187); use Mat4.copyColumnsInto(scratch) not Mat4.toColumnsFloatArray() for per-frame uniform uploads; prefer the TRS-tuple slerp(startPosition, startQuaternion, startScale, …) when you already hold the components. Reading node.worldPosition / worldQuaternion per frame is fine now — those are cached. Always load via rememberModelInstance / rememberNode (cached + main-thread). Full table: docs/docs/performance.md § Hot Paths & Allocation-Free APIs (audit umbrella #2263).

Toolchain pairing

This skill is most useful paired with the `android-cli` skill:

• ⛔ android run --apks=APK --activity=PKG/.MainActivity — do not use.

Measured three times in this repo (#2796, #2854, #2990) — most recently on CLI 1.0.15498356, where it printed App loaded: and Debuggable: true, then rejected an activity the platform resolves fine, and installed nothing — leaving an older build on the device while a QA run measured it. Install with adb install -r APK and launch with adb shell am start -n PKG/.MainActivity, then confirm the device's lastUpdateTime actually moved (adb shell dumpsys package PKG). An install step that reports success is not evidence the binary on the device is yours.

• android screen capture --annotate -o ui.png + `android screen resolve

--screenshot=ui.png --string="tap #N"` — visual UI testing of a 3D scene.

• android layout --pretty -o ui.json — Compose UI tree dump (the 3D viewport

reports as a single AndroidView, so for in-3D tap targets you still need pointerInput / hit testing).

• android docs search "compose canvas" — underlying Compose APIs.

Haptic feedback

io.github.sceneview.haptic.SceneViewHaptic wraps Android's Vibrator behind seven semantic presets plus low-level escape hatches. Get an instance with rememberHapticFeedback() inside a @Composable; it is a silent no-op (one Log.d, never throws) when the device has no vibrator or the consumer app omits the permission.

import io.github.sceneview.haptic.rememberHapticFeedback

@Composable
fun PlaceAnchorButton(onPlace: () -> Unit) {
    val haptic = rememberHapticFeedback()
    Button(onClick = { haptic.medium(); onPlace() }) { Text("Place") }
}

• Presets: light() medium() heavy() success() warning() error()

selection(). Escape hatches: continuous(intensity, durationMs) and pattern(events). cancel() stops an in-progress vibration — rememberHapticFeedback() calls it automatically on dispose.

• The consumer app MUST opt in by adding

<uses-permission android:name="android.permission.VIBRATE" /> to its manifest — the sceneview library does not auto-merge it.

• The same API ships on iOS (SceneViewSwift.SceneViewHaptic) and Web

(sceneview.haptic.*). See llms.txt § Haptic Feedback.

Resources

• [Cheat sheet](./references/cheatsheet.md) — every public composable, node, and

helper, with their actual signatures pulled from llms.txt.

• [Recipes](./references/recipes.md) — pointers to the working demo in

samples/android-demo/ for each of the 13 canonical patterns. Read the demo file, copy from it. Do not improvise.

• [Migration to 4.x](./references/migration.md) — see also

docs/docs/migration.md for the full rename map.

Workflow guidance

When the user asks for a SceneView feature:

1. Confirm the platform. Android / iOS / Web / Flutter / RN — don't assume.
2. Pick the right entrypoint. SceneView { } for 3D-only,

ARSceneView { } for AR. Mention the matching Gradle artifact.

1. Read the matching demo in samples/android-demo/.../demos/ before

writing code. If you can't find one, fall back to llms.txt. Never invent an API.

1. *Use the `remember` helpers** at the top of the composable. Never call

raw constructors for Engine / ModelLoader.

1. Handle the `null` from `rememberModelInstance` with ?.let { … }.
2. For AR, arsceneview already declares Manifest.permission.CAMERA

and the com.google.ar.core <meta-data> as optional. An AR Optional app declares <uses-feature android:name="android.hardware.camera.ar" android:required="false" /> and <uses-feature android:name="android.hardware.camera" android:required="false" />. An AR Required app overrides the <meta-data> with tools:replace="android:value" (see llms.txt § Setup); without it the manifest merger fails.

1. If the user pastes 2.x / 3.x code, point them at

docs/docs/migration.md and the local references/migration.md.