· Yogesh Khedar· Engineering· 4 min read
Simulation Tools for Drones — Fidelity, Photorealism and Rehearsal
SITL, Gazebo, game-engine sims — no tool maxes flight dynamics, sensor realism and photorealism at once. Pick by user, and put rehearsal where operations live.
Ask three drone teams about simulation and you will hear three different tool names, because they are solving three different problems. The landscape makes sense once you see the axes, and we run all three kinds, so here is where each one earns its place.
The landscape
Autopilot SITL and HITL. ArduPilot and PX4 run the real flight-controller software against a simple physics model on a computer, or on the real board with simulated sensors. This is the gold standard for control-stack correctness: the code that flies the aircraft is the code under test. The world is a flat plane and the sensors are perfect, which is the point.
Robotics simulators. Gazebo-class tools add a physics world and sensor models: lidar returns, sonar swaths, cameras with lens models, wind and waves. This is where a rover’s suspension, a boat’s response to a current or a lidar-based navigation stack get tested before the hardware exists. Our USV work runs here: the boat’s steering and speed controllers were tuned in Gazebo against the same procedure the console runs on the real vessel.
Game-engine simulators. Unreal Engine and NVIDIA Isaac Sim bring photorealism. The imagery is good enough to train perception models on and to rehearse a flight over a place that looks like the place. Our photorealistic twin runs in Unreal over real 3D data of a site, with the aircraft flown by the same autopilot SITL, so the camera sees a city while the flight controller flies physics.
The photorealistic twin: a simulated survey over a landfill, flown by ArduPilot SITL, seen through the aircraft’s camera.
No tool maxes all three
Flight dynamics fidelity, sensor simulation quality and environment photorealism are three axes, and every tool picks two at best.
SITL has the real autopilot and nothing else. Gazebo has physics and sensors and a world made of boxes. Unreal has the world and a camera, and its physics is a game’s. Chaining them, autopilot SITL for the flight, Gazebo or Unreal for the world, is how serious setups work, and it is how ours works: the autopilot does not know which world it is flying in.
The Gazebo bench: the surface vessel surveying a quay with its camera footprints drawn.
What each user needs
Developers need dynamics and the real control code: SITL, all day, every commit.
AI teams need sensor realism: thermal signatures, lens distortion, motion blur, the texture of a real roof. The photorealistic twin, with the detections labelled by the simulator for free.
Operators need something else entirely: mission rehearsal. Not perfect physics, but the confidence that this plan, on this site, with this aircraft, does what everyone expects before anything lifts off. And trainees need a fleet to practise on without an aircraft, an airspace slot and an instructor for each of them.
The gap
Rehearsal is an operational activity, and it traditionally happens in none of the tools above, because they live on an engineer’s workstation. The operator does not have Gazebo. The operator has the console.
The console flying a mission over the photorealistic twin: the same fly page, the same plan, a simulated aircraft.
Rehearse where you operate
Our answer is to put the simulated aircraft into the operations platform. A simulated dock and aircraft are devices in the fleet like any other: they appear in the fleet list, take the same plans and missions, stream the same telemetry and video, and leave the same logs. A workspace launches them from the Simulators page when it has the feature; the Unreal and Isaac rigs run on our side for demonstrations, training data and the photorealistic flights.
The result is that a DFR crew rehearses the call on a twin of its own district, a training centre runs a course on a simulated fleet in the console the trainees will use on the job, and a manufacturer tests a new autopilot build against the same missions its customers fly, none of them leaving the platform.
What is built today and what is in development is on the flight simulator page, with the simulated USV in Gazebo and the photorealistic twin shown as they run.