[Question] Sending data from Isaac Lab server and consuming from the client side

[sashansamarajeewa]

Question

Hi team,

The isaac-xr-teleop-sample-client-apple client already sends control/button events to the Isaac Lab server, and Isaac Lab can act on those events.

Is there a supported way to send data in the opposite direction (server to client)?

Use case - The Isaac Lab server computes a small amount of text and I want to display it as a panel in the Apple Vision Pro client (SwiftUI overlay).

Is there an example in this repo (or another NVIDIA/Isaac sample) that shows,

1. How the server sends structured messages (e.g., JSON) to the client?
2. How the Swift client receives and consumes those messages (e.g., CloudXR message channels/bidirectional messaging)?

Thanks

Build Info

Describe the versions that you are currently using:

• Isaac Lab Version: 2.3.0
• Isaac Sim Version: 5.1.0

[RandomOakForest]

Thank you for posting this. It seems that this is supported via CloudXR’s bidirectional Message Channel / Opaque Data Channel; you can send JSON (or any bytes) from the Isaac Lab/OpenXR side to the Apple Vision Pro Swift client and render it in a SwiftUI overlay. @rwiltz for vis.

[sashansamarajeewa]

@RandomOakForest Thanks for the suggestion. I read about it in doc. It seems this is a new functionality supported with CloudXR SDK 6+. Currently Isaac Lab uses CloudXR runtime 5.0.1 and apple client uses CloudXRKit 5.0.1.

• Is there a plan to upgrade Isaac Lab’s CloudXR Runtime (and sample clients) to 6.x in the near future?
• Are there known blockers/compatibility constraints with Isaac Lab that currently require staying on 5.0.1? Release notes doc says

CloudXR SDK 6.0 introduces a new architecture that is not backward-compatible with previous versions. Applications built with earlier CloudXR SDK versions require migration to the new APIs.

[sashansamarajeewa]

@RandomOakForest @rwiltz Do you have any update on this?