Rover GUI

The deprecated version of the GUI (With flask and vanilla JS) is in ./OldVersion The current version is ./frontend (React) and ./backend (Node)

---

📝 Description

The GUI is divided into 4 resizeable panels. The Control Panel, the Console Panel, the Camera Panel, and the Map Panel.

• Control Panel: Displays data obtained through ROS such as heading and position.
• Console Panel: Connects to the local console and the Jetson's console.
• Camera Panel: Stream videos by drawing frames on canvases. The frames are sent from the backend through websocket. The backend receives frames from the pi through UDP.
• Map Panel: Utilizes pre downloaded chunks. Refer to the README in the frontend/src folder for more information.

👥 Contacts

• Refer to Joseph for any issues with the video streaming
• Refer to Aditya for any issues with the map

---

🔮 Future Improvements

1. More telemetry data through ROS
2. Emergency stop button and launch script
3. Pi Terminals

---

🛠️ To Start the GUI in 3 Easy Steps

The base station has a launch script that does all of this: ./startGUI.sh It also has a launchable app on the side bar that runs the shell file.

📥 Frontend

Located in ./frontend

npm install
npm run start

🗄️ Backend

Located in ./backend

npm install
node index.js

🛰️ To start ROS server

The GUI uses Roslib to connect to Rosbridge, which must be running for ROS to work. Have ros2 humble/jazzy installed, install rosbridge_server package, and run:

source /opt/ros/jazzy/setup.bash
ros2 launch rosbridge_server rosbridge_websocket_launch.xml

💡 Running the server with "delay_between_messages:=0.0" could fix a version error.

🧪 Testing the OpenCV Video streaming

• Install libraries and run streamProducer.py in ./OldVersion/src/
• Change the input port to match video source in index.js if necessary.

---

📺 To start the (Deprecated) Webrtc signalling server for video streaming

1. Change the CameraPanel in the frontend to the gstreamer version.
2. Build gst-plugins-rs and source with:

   export GST_PLUGIN_PATH=[path to gst-plugins-rs]/target/debug:\$GST_PLUGIN_PATH

3. Run the server:

   cargo run --bin gst-webrtc-signalling-server

4. The streamer would run something like this, which would go through the server and then to the GUI:

   gst-launch-1.0 webrtcsink name=ws meta="meta,name=gst-stream" signaller::uri="ws://10.141.81.176:8443" mfvideosrc ! videoconvert ! vp8enc deadline=1 ! queue ! video/x-vp8 ! ws.

⚠️ May have to set port 5000, 9090 inbound to open in firewall for multiple client.

📌 The ros.js file is publishing and then subscribing to the ros websocket to simulate actual input.

---

🔄 To start the Old GUI from ./OldVersion

npm install
pip install -r requirements.txt

Run app.py in wr_hci_hud/src

---

📜 History

Ideas for this node have gone through several iterations. None of the following have been implemented far enough to have a good estimate for the efficacy of the method:

🌐 Option 1: A locally hosted webserver on the base station (probably easiest in Python)

This options live-updates its components to reflect subscriptions over ROS.

• Pros:
  • Most of the GUI heavy-lifting is taken care of by the web browser
  • Lots of options/documentation for the web server itself
  • Could be accessed/used by any other computer on the network (good for debugging)
• Cons:
  • May be hidden complexity to configure layout
  • Transporting images/geospatial data (if rendered) may take up a lot of bandwidth
  • Rendering geospatial data/proprioception might not be easy
  • Camera streaming might not be possible due to bandwidth, may still need extra tool
  • Lack of Internet at competition might limit design or force pre-downloaded dependencies

🖥️ Option 2: Qt/Tk/similar locally hosted application that is ROS-aware

• Pros:
  • Lots of language options
  • Broad customizability
  • Likely easier ROS integration (due to customizability)
  • Easier to make visual rendering more efficient
• Cons:
  • Analysis Paralysis
    • The team does more of the legwork on the GUI magic
    • Likely complex configuration
  • Constrained to platform compiled for, or need to distribute the binary

⌨️ Option 3: Direct console access (current solution)

• Pros:
  • Few/no layers of extra support required between operator and WRover
  • No GUI work
• Cons:
  • Operator must be extremely proficient in ROS/bash/filesystems/networks
    • No assistance/recovery available if something goes wrong
  • Hard to visualize large number of subsystems at once
  • Slow to operate
  • Hard to construct complex diagnostic tools and insights on the fly
  • Little to no visual support outside of rqt

Given past experience at competition, even a small GUI with no input options (a literal HUD) would be massively beneficial compared to reading individaul diagnostics one at a time. This could then be expanded to provide more detailed insights, inputs to the WRover, and specializations for the different competition modes.