As it turns out, the Zed camera ecosystem is painfully, unequivocally unportable.
Their [https://github.com/stereolabs/zed-ros2-wrapper package] is particularly bad. You can't "just" add it to your package.xml and call it a day - nope! You've got to do a boatload of setup on each computer you install it on. That's not workable for Autonomous.
A generic solution
I'm thinking of a more generic solution instead. Unfortunately, it only runs on the CPU unless you perform some annoying NVIDIA setup, but we can look into those optimizations later on...
image_publisher
A few instances of these nodes will publish the required data (like in this file):
stereo_image_proc
This package processes camera output into a
Optimizations (hopefully for later)
- We could use
isaac_ros_stereo_image_proc for CUDA acceleration
- Still not very portable :(
- Would need some very hacky
colcon shenanigans to replace it with the non-NVIDIA version if hardware acceleration isn't detected
- Could be possible to do this at runtime? Sounds hacky, impossible to test, and still incompatible with almost all hardware that Autonomous members own lol
- Consider a Rust node making use of
wgpu + compute shaders?
- Would be 9000x more portable; probably eligible to be distributed through
rosdep
- Hilariously, about the same amount of documentation (very little on Rust compute shaders past this intro guide... by Google Fonts? 🤨)
- This'd probably be one of the first (fast) implementations of a hardware-accelerated camera node that can run on anything!
- Would need to take some time to write this
As it turns out, the Zed camera ecosystem is painfully, unequivocally unportable.
Their [
https://github.com/stereolabs/zed-ros2-wrapperpackage] is particularly bad. You can't "just" add it to yourpackage.xmland call it a day - nope! You've got to do a boatload of setup on each computer you install it on. That's not workable for Autonomous.A generic solution
I'm thinking of a more generic solution instead. Unfortunately, it only runs on the CPU unless you perform some annoying NVIDIA setup, but we can look into those optimizations later on...
image_publisherA few instances of these nodes will publish the required data (like in this file):
/zed_2i_left_image_publisher: publishes/sensors/depth_image/left/zed_2i_right_image_publisher: publishes `/sensors/depth_image/rightstereo_image_procThis package processes camera output into a
Optimizations (hopefully for later)
isaac_ros_stereo_image_procfor CUDA accelerationcolconshenanigans to replace it with the non-NVIDIA version if hardware acceleration isn't detectedwgpu+ compute shaders?rosdep