diff --git a/docs/images/xr_enable_second_viewport.jpg b/docs/images/xr_enable_second_viewport.jpg new file mode 100644 index 000000000..a40bf854d Binary files /dev/null and b/docs/images/xr_enable_second_viewport.jpg differ diff --git a/docs/images/xr_second_viewport_robot_camera.jpg b/docs/images/xr_second_viewport_robot_camera.jpg new file mode 100644 index 000000000..f99bd8f92 Binary files /dev/null and b/docs/images/xr_second_viewport_robot_camera.jpg differ diff --git a/docs/pages/example_workflows/static_apple/step_2_teleoperation.rst b/docs/pages/example_workflows/static_apple/step_2_teleoperation.rst index 402b2ead5..ec9e3fb08 100644 --- a/docs/pages/example_workflows/static_apple/step_2_teleoperation.rst +++ b/docs/pages/example_workflows/static_apple/step_2_teleoperation.rst @@ -99,6 +99,53 @@ Step 2: Start Arena Teleop Arena teleop session with XR running. Stereoscopic view (left) and OpenXR settings in the XR tab (right). +Step 2b: Monitor Recording with a Second Viewport (Optional) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For higher-quality datasets, we recommend a two-person workflow when collecting demonstrations in +Step 4: one person teleoperates from the headset, while a second person watches the host monitor +to confirm each trajectory stays inside the robot's head-camera field of view. Anything that +drifts outside the recording FOV is absent from the saved HDF5 and absent from the policy's view +at training time, so catching it live saves a re-record. + +The Arena application's default viewport shows the *teleoperator's* stereoscopic perspective — +what the headset wearer sees, not what ``record_demos.py`` will store. To watch both side-by-side, +open a second viewport bound to the robot's head camera: + +#. In the running Arena application, open the **Window** menu and toggle on **Viewport 2**. + + .. figure:: ../../../images/xr_enable_second_viewport.jpg + :width: 100% + :alt: Isaac Lab Window menu with the Viewport 2 toggle highlighted. + :align: center + + Enable a second viewport from the **Window** menu. + +#. In the new Viewport 2, click the camera selector in the viewport toolbar and choose the + robot's head-mounted camera (``RobotHeadCam``, under + ``/World/envs/env_0/Robot/head_link``). This is the camera that ``record_demos.py`` writes + to the HDF5 file in Step 4, so any motion that leaves this frame will be absent from the + dataset. + + .. figure:: ../../../images/xr_second_viewport_robot_camera.jpg + :width: 100% + :alt: Two viewports side-by-side: stereoscopic XR view (left) and the robot head-camera view (right). + :align: center + + Dual-viewport layout: the stereoscopic XR view (left) is the teleoperator's perspective, + and the head-camera view (right) is what the dataset captures. The observer keeps every + grasp and placement inside the right viewport and gives the teleoperator live feedback + ("move a touch to your right — your hand is at the edge of frame"). + +.. note:: + + ``RobotHeadCam`` is only spawned when ``--enable_cameras`` is set. The ``record_demos.py`` + command in Step 4 enables it by default, so the camera shows up in the camera selector once + you are recording. The smoke-test ``teleop.py`` command above omits ``--enable_cameras`` for + performance; pass it there too if you want to validate the dual-viewport layout before + entering VR. + + Step 3: Connect from the headset device ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^