Feat: Add Real-time Object Detection with MediaPipe

[kkbin505]

Description

Inspired by MarioFPV, I added a object detection function to OpenIPC. Instead of adding this function to the airunit, I added this funtion to the receiver end (PixelPilot) with Google MediaPipe Tasks Vision:

https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/object_detection

Users can now enable detection overlay to visualize bounding boxes, classification labels, and confidence scores directly on top of the live video stream.

---

Key Changes

1. Build & Dependencies

• libs.versions.toml: Added mediapipe (0.10.29) and gradleDownload (5.5.0) dependencies.
• build.gradle:
  • Applied download tasks (downloadModelFile0, downloadModelFile1) to automatically fetch efficientdet-lite0.tflite and efficientdet-lite2.tflite models during preBuild.
  • Set noCompress 'tflite' to optimize model loading performance.

2. UI & Rendering Setup

• activity_video.xml: Changed the primary renderer mainVideo from SurfaceView to TextureView to support runtime frame grabbing via getBitmap(). Added OverlayView above it.
• VideoPlayer.java: Implemented configureTextureView() to adapter TextureView lifecycle events (such as surface creation and destruction) for native decoding.

3. Detection Core & Integration

• ObjectDetectorHelper.java (New): Helper class wrapping MediaPipe's API. Manages delegate creation (CPU/GPU) and model switching.
• OverlayView.java (New): Handles aspect-ratio scaling and draws labels/bounding boxes onto the canvas.
• VideoActivity.java:
  • Added background single-thread executor execution loop yielding ~10 FPS for detection.
  • Added Object Detection popup submenu to allow toggling state, delegates, and model sizes on the fly.
  • Tied detection state to standard Android activity lifecycle hooks (onResume/onPause) to ensure safe thread termination.

---

Manual Verification

1. Tested on Samsung S22

https://youtu.be/KQsr3Jk9Ab8

[vertexodessa]

@kkbin505 thank you for the PR!
this would significantly increase the binary size (because of the bundled weights), so we need to consider this tradeoff. What is the practical scenario for this feature?

[kkbin505]

this would significantly increase the binary size (because of the bundled weights), so we need to consider this tradeoff. What is the practical scenario for this feature?

That is a valid concern. The current implementation bundles both TFLite models with the app, which increases the APK size.

Would it be preferable to keep the models outside the APK and load them from the app’s storage directory instead?
And use can choose their own model.

The immediate practical scenario is to use the phone’s relatively powerful CPU/GPU for real-time object detection on the receiver side, without adding extra processing load to the air unit.

At this stage, the feature is still experimental. A possible longer-term use case would be to expose detection results to an uplink, like MAVLink, so they could be used for assisted tracking.
I agree that the size and maintenance cost should be weighed against how broadly useful this is.