CrazySim2Real: A Sim2Real Benchmarking Framework for Crazyflie Drones

CrazySim2Real is a comprehensive framework for evaluating control algorithms on Crazyflie quadcopters, providing a standardized testing environment that bridges the gap between simulation and real hardware. This repository contains two main components:

1. A physics-based Crazyflie drone simulator built on NVIDIA Isaac Lab
2. A benchmarking framework for evaluating controller performance in both simulation and real-world environments

Repository Structure

CrazySim2Real/
├── crazyflie_benchmark/     # Benchmarking framework for controller evaluation
│   ├── config/              # Configuration files for hardware and simulator
│   ├── logs/                # Test result logs (both sim and real)
│   ├── test_plans/          # Predefined test plans (step, impulse, sine sweep)
│   └── tests/               # Test implementation modules
├── crazyflie_sim/           # Simulator implementation
│   ├── api/                 # API server for simulator control
│   ├── controllers/         # Controller implementations for simulator (incl. CF2.1 BL port)
│   ├── sim/                 # Simulation manager and physics implementation
│   ├── run.py               # Main simulator entry point
│   ├── xbox_client.py       # Xbox controller teleoperation client
│   └── ros2_vis_client.py   # ROS 2 visualization / monitoring client
├── docker/                  # Docker configurations
│   └── isaaclab/            # Isaac Lab simulator environment
└── scripts/                 # Utility scripts for setup and execution

Key Features

• Unified Testing Interface: Run identical tests on both simulated drones and real hardware
• Standardized Test Protocols: Pre-defined step, impulse, and sine sweep test plans
• Data Collection & Analysis: Automated logging, metrics generation and visualization
• Physics-Based Simulation: High-fidelity drone simulation using NVIDIA Isaac Lab
• Configurable Controllers: Evaluate different control algorithms with minimal code changes
• Safety Monitoring: Built-in safety features to protect hardware during testing
• Docker Integration: Containerized simulation environment for consistent results

Getting Started

Prerequisites

• Python 3.8+ with pip
• Docker and Docker Compose (for simulation)
• Crazyflie hardware and Crazyradio PA (for real hardware tests)
• NVIDIA GPU with latest drivers (for simulation)

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/CrazySim2Real.git
   cd CrazySim2Real

2. Install the required Python dependencies:

   pip install -r crazyflie_benchmark/requirements.txt

3. Set up the Isaac Lab Docker environment (only needed for simulation):

   # First-time setup
   ./scripts/init.sh

Running the Simulator

The simulator runs in a Docker container with NVIDIA Isaac Lab:

# Start the simulator
./scripts/start.sh /workspace/isaaclab/CrazySim2Real/crazyflie_sim/run.py

# Alternative with custom parameters
./scripts/start.sh /workspace/isaaclab/CrazySim2Real/crazyflie_sim/run.py --port 8000

This will start a Docker container with the Isaac Lab environment and launch the Crazyflie simulator inside it. The simulator exposes an HTTP API that the benchmarking framework can connect to.

Interactive Control with Xbox Controller

xbox_client.py provides a teleoperation interface to control a simulated Crazyflie using an Xbox gamepad.

Usage (from repository root, with simulator already running):

cd crazyflie_sim
python xbox_client.py --host localhost --port 8000

You can customize button mappings and sensitivity configuration flags (see xbox_client.py).

ROS 2 Visualization Client

ros2_vis_client.py bridges the simulator with a ROS 2 environment to visualize state and logs in standard ROS tools (e.g., RViz, rqt_plot).

Features:

• Subscribe to simulator state via HTTP/WebSocket API
• Publish drone pose, velocity, and control signals as ROS 2 topics

Usage (ROS 2 environment sourced, simulator running):

cd crazyflie_sim
python ros2_vis_client.py --host localhost --port 8000

You can then visualize the topics in plotjuggler or other ROS 2 tools.

Running Benchmarks

The benchmarking framework can run tests on either the simulator or real hardware:

# List available test plans
cd crazyflie_benchmark
python main.py --list-tests

# Run a benchmark in simulation
python main.py --config config/simulator_config.yaml --plan step_tests.yaml

# Run a benchmark on real hardware
python main.py --config config/hardware_config.yaml --plan step_tests.yaml

Test Plans

Test plans are defined in YAML files in the test_plans/ directory:

1. Step Tests (step_tests.yaml): Evaluates the system's response to step inputs in roll, pitch, and thrust
2. Impulse Tests (impulse_tests.yaml): Measures disturbance rejection and stability
3. Sine Sweep Tests (sine_sweep_tests.yaml): Evaluates frequency response characteristics

Example step test configuration:

name: "Step Response Tests"
description: "Step response tests for roll, pitch, and thrust axes."
tests:
  - type: "step"
    channel: "roll"
    amplitude: 5.0  # degrees
    duration: 1.5   # seconds

  - type: "step"

Analyzing Results

Test results are automatically logged as CSV files in the crazyflie_benchmark/logs/ directory.

Single Log Analysis

Use the provided tool to generate plots and statistics:

# Interactive mode (shows plots in window)
python crazyflie_benchmark/main.py --analyze crazyflie_benchmark/logs/flight_log_YYYYMMDD_HHMMSS.csv

# Save plots to file
python crazyflie_benchmark/main.py --analyze crazyflie_benchmark/logs/flight_log_YYYYMMDD_HHMMSS.csv --save

When using --save, plots will be saved in a new directory named after the log file.

Sim-to-Real Comparison

To compare simulation and real hardware performance:

1. Run the same test plan on both simulator and real hardware.
2. Use the comparison tool:

# Interactive mode
python crazyflie_benchmark/main.py --compare \
  crazyflie_benchmark/logs/sim_log.csv \
  crazyflie_benchmark/logs/real_log.csv \
  --label1 "Simulation" --label2 "Real Hardware"

# Save comparison plots
python crazyflie_benchmark/main.py --compare \
  crazyflie_benchmark/logs/sim_log.csv \
  crazyflie_benchmark/logs/real_log.csv \
  --label1 "Simulation" --label2 "Real Hardware" \
  --save

Configuration

Configuration files specify connection parameters and flight settings:

Hardware Configuration (hardware_config.yaml)

connection_type: "cflib"  # Use real Crazyflie hardware
uri: "radio://0/80/2M/E7E7E7E7E6"  # Radio URI for your Crazyflie
hover_thrust: 40000  # Approximate hover thrust (0-65535 scale)
max_roll_pitch: 15.0  # Safety limit for roll/pitch angle (degrees)

Simulator Configuration (simulator_config.yaml)

connection_type: "simulator"  # Use simulator
sim_host: "localhost"  # Simulator host address
sim_port: 8000  # Simulator port
hover_thrust: 0.6  # Approximate hover thrust (normalized 0-1 scale)
max_roll_pitch: 15.0  # Safety limit for roll/pitch angle (degrees)

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Bitcraze for the Crazyflie platform
• NVIDIA Isaac Lab for the simulation environment