Skip to content

Waits for textures to be loaded for Isaac RTX renderer #4956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kellyguo11 merged 11 commits into from
Mar 20, 2026
Merged

Waits for textures to be loaded for Isaac RTX renderer #4956

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
78a42af
Feature: Wait for textures to be loaded.
pbarejko Mar 11, 2026
fc40398
Formatting fix
pbarejko Mar 11, 2026
6c2eac0
Remove redundant try/catch
pbarejko Mar 11, 2026
8704770
Address greptile review comments on streaming subscription
nvsekkin Mar 12, 2026
f43cd49
Pump app.udpate() after waiting for stream
nvsekkin Mar 13, 2026
4e9531e
Add first-frame texture streaming test for Camera and TiledCamera
nvsekkin Mar 13, 2026
2843fcb
Formatting
nvsekkin Mar 13, 2026
4be894d
Add unit tests for RTX streaming wait helpers
nvsekkin Mar 13, 2026
2eb2fee
Merge branch 'develop' into pbarejko/rtx-renderer-load-textures
nvsekkin Mar 13, 2026
bbad8a2
Merge branch 'develop' into pbarejko/rtx-renderer-load-textures
kellyguo11 Mar 13, 2026
c5b671d
Formatting
nvsekkin Mar 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions source/isaaclab/test/deps/isaacsim/check_camera.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,12 +168,6 @@ def main():
initial_joint_pos = view.get_joint_positions()
initial_joint_vel = view.get_joint_velocities()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
world.step(render=True)

# Counter
count = 0
prev_im = None
Expand Down
90 changes: 0 additions & 90 deletions source/isaaclab/test/sensors/test_camera.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,12 +105,6 @@ def test_camera_init(setup_sim_camera):
assert camera._sensor_prims[0].GetPath().pathString == camera_cfg.prim_path
assert isinstance(camera._sensor_prims[0], UsdGeom.Camera)

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()

# Check buffers that exist and have correct shapes
assert camera.data.pos_w.shape == (1, 3)
assert camera.data.quat_w_ros.shape == (1, 4)
Expand Down Expand Up @@ -199,12 +193,6 @@ def test_camera_init_offset(setup_sim_camera):
rtol=1e-5,
)

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()

# check if transform correctly set in output
np.testing.assert_allclose(camera_ros.data.pos_w[0].cpu().numpy(), cam_cfg_offset_ros.offset.pos, rtol=1e-5)
np.testing.assert_allclose(camera_ros.data.quat_w_ros[0].cpu().numpy(), QUAT_ROS, rtol=1e-5)
Expand All @@ -228,11 +216,6 @@ def test_multi_camera_init(setup_sim_camera):
# play sim
sim.reset()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
# Simulate physics
for _ in range(10):
# perform rendering
Expand Down Expand Up @@ -264,11 +247,6 @@ def test_multi_camera_with_different_resolution(setup_sim_camera):
# play sim
sim.reset()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
# perform rendering
sim.step()
# update camera
Expand Down Expand Up @@ -348,12 +326,6 @@ def test_camera_set_world_poses(setup_sim_camera):
# set new pose
camera.set_world_poses(position.clone(), orientation.clone(), convention="world")

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()

# check if transform correctly set in output
torch.testing.assert_close(camera.data.pos_w, position)
torch.testing.assert_close(camera.data.quat_w_world, orientation)
Expand All @@ -376,12 +348,6 @@ def test_camera_set_world_poses_from_view(setup_sim_camera):
# set new pose
camera.set_world_poses_from_view(eyes.clone(), targets.clone())

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()

# check if transform correctly set in output
torch.testing.assert_close(camera.data.pos_w, eyes)
torch.testing.assert_close(camera.data.quat_w_ros, quat_ros_gt)
Expand All @@ -402,12 +368,6 @@ def test_intrinsic_matrix(setup_sim_camera):
# Set matrix into simulator
camera.set_intrinsic_matrices(rs_intrinsic_matrix.clone())

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()

# Simulate physics
for _ in range(10):
# perform rendering
Expand Down Expand Up @@ -460,11 +420,6 @@ def test_depth_clipping(setup_sim_camera):
# Play sim
sim.reset()

# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()

camera_zero.update(dt)
camera_none.update(dt)
camera_max.update(dt)
Expand Down Expand Up @@ -559,11 +514,6 @@ def test_camera_resolution_all_colorize(setup_sim_camera):
# Play sim
sim.reset()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
camera.update(dt)

# expected sizes
Expand Down Expand Up @@ -625,11 +575,6 @@ def test_camera_resolution_no_colorize(setup_sim_camera):

# Play sim
sim.reset()
# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(12):
sim.step()
camera.update(dt)

# expected sizes
Expand Down Expand Up @@ -694,11 +639,6 @@ def test_camera_large_resolution_all_colorize(setup_sim_camera):
# Play sim
sim.reset()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
camera.update(dt)

# expected sizes
Expand Down Expand Up @@ -746,11 +686,6 @@ def test_camera_resolution_rgb_only(setup_sim_camera):
# Play sim
sim.reset()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
camera.update(dt)

# expected sizes
Expand All @@ -773,11 +708,6 @@ def test_camera_resolution_rgba_only(setup_sim_camera):
# Play sim
sim.reset()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
camera.update(dt)

# expected sizes
Expand All @@ -800,11 +730,6 @@ def test_camera_resolution_albedo_only(setup_sim_camera):
# Play sim
sim.reset()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
camera.update(dt)

# expected sizes
Expand All @@ -831,11 +756,6 @@ def test_camera_resolution_simple_shading_only(setup_sim_camera, data_type):
# Play sim
sim.reset()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
camera.update(dt)

# expected sizes
Expand All @@ -858,11 +778,6 @@ def test_camera_resolution_depth_only(setup_sim_camera):
# Play sim
sim.reset()

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
camera.update(dt)

# expected sizes
Expand Down Expand Up @@ -896,11 +811,6 @@ def test_throughput(setup_sim_camera):
targets = torch.tensor([[0.0, 0.0, 0.0]], dtype=torch.float32, device=camera.device)
camera.set_world_poses_from_view(eyes, targets)

# Simulate for a few steps
# note: This is a workaround to ensure that the textures are loaded.
# Check "Known Issues" section in the documentation for more details.
for _ in range(5):
sim.step()
# Simulate physics
for _ in range(5):
# perform rendering
Expand Down
Loading
Loading