Field Robotics · 2022 · an animated walkthrough

CERBERUS: Autonomous legged and aerial robotic exploration in the Tunnel and Urban Circuits of the DARPA Subterranean Challenge

All 41 authors

Marco Tranzatto, Frank Mascarich, Lukas Bernreiter, Carolina Godinho, Marco Camurri, Shehryar Khattak, Tung Dang, Victor Reijgwart, Johannes Löje, David Wisth, Samuel Zimmermann, Huan Nguyen, Marius Fehr, Lukas Solanka, Russell Buchanan, Marko Bjelonic, Nikhil Khedekar, Mathieu Valceschini, Fabian Jenelten, Mihir Dharmadhikari, Timon Homberger, Paolo De Petris, Lorenz Wellhausen, Mihir Kulkarni, Takahiro Miki, Satchel Hirsch, Markus Montenegro, Christos Papachristos, Fabian Tresoldi, Jan Carius, Giorgio Valsecchi, Joonho Lee, Konrad Meyer, Xiangyu Wu, Juan Nieto, Andy Smith, Marco Hutter, Roland Siegwart, Mark Mueller, Maurice Fallon, Kostas Alexis

The paper in 109 seconds · narrated · sound on
Transcript

A research mine. No GPS, no light. Walking robots map it, flying robots scout ahead, a caged drone takes the stairs, and one person outside runs them all.

Before rescuers enter a mine or a collapsed tunnel, they need to know what’s inside. DARPA’s Subterranean Challenge made this a contest: one supervisor, one hour, twenty hidden objects. Underground there’s no GPS, little light, and radio dies behind rock. A find only counts within 5 m.

Team CERBERUS sent legged robots for endurance, drones for speed, a crash-proof caged drone, and a rover trailing an optical fibre.

Each robot fuses cameras, laser scanners and motion sensors, and passes on only healthy estimates — so one blinded sensor can’t mislead it. One shared planner steers every robot toward unknown space — and always keeps a way home. In the mine, a puddle corrupted the walking robots’ terrain map and stopped them. Next round, a controller learned in simulation felt the ground instead — and walked more than twice as fast. At the base, all maps are stitched into one — cutting one robot’s position error from about a metre to a quarter metre. The robots even carry their own radio network in, dropping relays as they go.

In the mine, a flying scout explored on its own — and then flew itself back home. The result: sixth of eleven teams in a research mine, then fifth of ten in an unfinished nuclear power plant. And a candid list of what broke — from robots stuck out of radio range to tipped-over relays — that guided the team’s next steps.

CERBERUS. Legs, wings, and one shared software stack.

Footage: team CERBERUS videos published by the Autonomous Robots Lab (Kostas Alexis, Tunnel Circuit, Urban Circuit) and the Robotic Systems Lab, ETH Zurich (Urban Beta course), listed in the paper’s appendix. Voice: Kokoro TTS (synthetic). Music and sound effects: synthesized for this video.

The story in plain words

Walking and flying robots, run by one person outside, can search an unknown mine before rescuers go in. Here is how team CERBERUS built such a team and how it did in two DARPA competition rounds.

  1. Why this matters

    First responders entering a collapsed tunnel, a mine or a big basement need to know what is inside: where the passages go, and where people or hazards are. Sending robots first keeps people out of danger. DARPA’s Subterranean Challenge turned this into a competition.

  2. What makes it hard

    Underground there is no GPS, little light, dust and fog, stairs and rubble, and corridors that all look alike. Radio dies behind rock, so robots often work out of contact. The rules allow only one human supervisor, a 60-minute run, and a find counts only if it is reported within 5 m of where it really is.

  3. What people did before

    Earlier systems explored mines with a single ground robot or with small teams of drones, and built accurate multi-sensor maps. Most used one kind of robot and were tested outside competition rules. Other teams in the Challenge also fielded legged robots (Ghost Robotics Vision 60, Boston Dynamics Spot).

  4. What this paper does

    CERBERUS combines walking robots (ANYmal: long battery life, rough ground, drops radio relays) with flying robots (fast scouts, and a caged drone that survives crashes) and a rover that trails a fibre-optic cable. Every robot runs the same core software: a localization method that fuses cameras, LiDAR (a laser scanner) and motion sensors; one exploration planner for legs and wings; and a base-station map that merges what all robots saw.

  5. What they showed

    Tunnel Circuit (research mine, Aug 2019): 5 points, 6th of 11 teams. Urban Circuit (unfinished nuclear plant, Feb 2020): 7 points from 11 correct reports, 5th of 10. The merged map cut a walking robot’s position error from 1.07 m to 0.26 m, well inside the 5 m scoring rule. A new learned walking controller more than doubled walking speed, from 0.2 to 0.45 m/s.

  6. Why it’s a step forward

    It is an honest field report: one shared software stack running on very different robots, tested under competition rules, with a clear list of what broke. Examples are robots stuck out of radio range, relays that tipped over, and a supervisor juggling too many screens. Those lessons shaped the team’s work toward the Final Event.

Words used below
Artifact
an object DARPA hides (e.g. survivor dummy, phone, backpack); 20 per course.
Localization / SLAM
a robot working out where it is while drawing its map.
LiDAR
a spinning laser scanner that measures distances to walls.
Frontier
the edge between mapped and unknown space: where exploring pays off.
Breadcrumb
a small WiFi relay a robot drops to extend the radio network.
Human Supervisor
the single person allowed to direct the robots during a run.
1 / 8
walking robots flying robots shared system artifacts
Scene 1

Read the full section with the paper’s figures ↓

The paper, section by section

Everything the animation skips

Each section matches one scene above. Press “Watch scene” to jump back to its animation; click any figure to enlarge it. Figures are from the paper; the text is a plain-language walkthrough.

Scene 1

The challenge

In short: DARPA asked teams to send robots into unknown underground sites, map them and report hidden objects, with one person in charge and one hour on the clock.

The DARPA Subterranean Challenge (kicked off in September 2018) had a physical “Systems” track and a simulated “Virtual” track. The Systems track visited three kinds of underground places: a mine in the Tunnel Circuit (August 2019), a multi-level man-made structure in the Urban Circuit (February 2020), a planned Cave Circuit that was cancelled, and a Final Event combining all of them (September 2021). This paper covers the first two rounds.

Each course hides 20 artifacts. Survivors, cell phones and backpacks appear everywhere; drills and fire extinguishers in the Tunnel Circuit; CO₂ gas and vents in the Urban Circuit. A team scores one point when it reports the right type of object within 5 m of its surveyed position, in DARPA’s coordinate frame. Every team got two 60-minute scored runs on each of two courses, after 30 minutes of setup by a pit crew of up to nine people. The final score adds up the best run on each course. During a run only the Human Supervisor at the base station may interact with the robots.

Why is this hard? No GPS reaches underground. Mines are dark, dusty and sometimes foggy. Corridors look alike, so it is easy to lose track of where you are. There are stairs, shafts, mud and water. Radio signals fade through rock and around corners, so robots must often decide for themselves.

Paper Fig. 1. The team at work in both rounds: walking robots (one on wheels) in the mine and the power plant, the flying scouts, and the Armadillo rover.
Paper Fig. 2. The concept: one supervisor outside; walking robots cross hard terrain and drop radio relays; drones scout shafts and chimneys; everyone contributes to one map in DARPA’s frame.
Scene 2

The team

In short: legs give endurance and rough-terrain ability, wings give speed and reach, and a tethered rover carries the radio link deeper.

CERBERUS is short for “CollaborativE walking & flying RoBots for autonomous ExploRation in Underground Settings”. The team’s bet: no single robot type suits every underground place, so combine several.

ANYmal B “SubT”: the main explorer

A quadruped with three torque-controlled, compliant joints per leg and 1–2 hours of battery life, the longest of the team. Legs can step over obstacles, crouch under them and move in any direction in narrow spaces. For the Urban Circuit its sensor case held a Velodyne VLP-16 LiDAR, a Robosense BPearl dome LiDAR for the ground right around it, a tracking camera, three FLIR colour cameras, an inertial sensor, a synchronisation board, a CO₂ sensor, and an NVIDIA Jetson AGX Xavier for the neural networks. It could wear point feet, flat feet for grip, or driven wheels for speed. A modified belly plate carries up to four WiFi relays and drops them one by one.

Animated. The relay drop: lower the torso, push one module out, and it switches on (a magnetic switch) and joins the network.
Paper Fig. 3 (right). The Urban Circuit version of ANYmal B “SubT” with its protected sensor case.

Aerial Scouts: fast, and fully on their own

Three scouts (Alpha, Bravo, Charlie) were built on the DJI Matrice 100, each with an Intel NUC computer, a LiDAR, a camera and an inertial sensor. Charlie added a thermal camera. LEDs flash in sync with the camera shutter, so the lights can be bright without wasting power. Flights last 8–10 minutes. After take-off they need no human input and fly home before the battery runs out.

Gagarin is smaller and sits inside a cage, so it survives collisions faster than 2 m/s. It carries a wide-view LiDAR, fisheye cameras and LEDs angled to reduce dust glare, and flies 5–6 minutes. It was built for stairwells and vertical shafts.

Paper Fig. 8. Charlie (conventional quadrotor, left) and the collision-tolerant Gagarin (right).
Real run. Gagarin drops into a stairwell of the unfinished power plant during the Urban Circuit (2× speed); the insets show other camera views and the map it plans in. Footage: Autonomous Robots Lab, team CERBERUS (Urban Circuit video).

Armadillo: the radio carrier

A wheeled rover with a 300 m optical-fibre reel that unrolls as it drives, keeping a wired link to the base. It also carries a directional 5.8 GHz antenna to extend WiFi deep into the course. The supervisor drove it using its camera feeds. It also had cameras and a LiDAR, so it could detect artifacts as a backup.

Paper Fig. 10. Armadillo, a modified Super Mega Bot with a tall sensor head.
Scene 3

Where am I?

In short: each robot tracks its position with cameras, LiDAR and motion sensors together, and checks every hand-off, so one failing sensor doesn’t take the estimate down.

Every sensor has a weak spot underground. Cameras see nothing in darkness or thick dust. A LiDAR, a laser scanner that measures distances to walls, gets confused in long corridors where every slice looks the same. The team’s answer is CompSLAM (complementary multi-modal localization and mapping). It runs as a chain of three steps:

  1. Visual–thermal–inertial odometry. A filter based on ROVIO fuses camera and thermal images with the inertial sensor. It balances how many features it tracks in each image by how informative the image is. If its uncertainty grows too fast, the estimate is marked unhealthy and not passed on.
  2. LiDAR odometry. Matches each new laser scan to the previous one, starting from the camera estimate when that is healthy. It then checks how well each direction is pinned down; any direction that is poorly constrained is replaced with the camera estimate.
  3. LiDAR mapping. Aligns scans to the local map at half the rate, re-checks health, and updates the map.
Animated. In a smooth corridor the laser pins down sideways position but not how far along the robot is. That single weak direction is what gets swapped for the camera estimate.
Paper Fig. 13. The CompSLAM chain: camera–thermal–inertial odometry feeds LiDAR odometry and LiDAR mapping, with a health check at each hand-off.

Good timing matters too. Tiny timestamp errors between a rotating sensor and a camera turn into large position errors for distant points. The walking robots and the rover used a trigger board that synchronises cameras and inertial sensor with the computer. The scouts timestamp images using a pulse the camera sends to the inertial sensor.

The Tunnel Circuit gave a natural comparison. After the runs, the team built true trajectories by aligning their recorded scans to DARPA’s survey map.

Tunnel Circuit, run SR2Sensors usedDrift per 10 mOverall error
Alpha Aerial Scout (~180 m)camera + LiDAR + IMU0.11 m / 0.86°0.48 m / 1.05°
Armadillo rover (~285 m)LiDAR only (technical issue)0.03 m / 0.37°3.39 m / 2.87°

What this means: step-to-step, both were accurate, but the LiDAR-only rover slowly drifted (especially in fast on-the-spot turns) to 3.39 m, still inside the 5 m rule, while the multi-sensor scout stayed under half a metre.

Scene 4

Where next?

In short: one planner (GBPlanner) steers both walking and flying robots toward unknown space, remembers promising places for later, and always keeps a path home.

The planner works on a 3-D grid of 20 cm cubes (voxels), each marked free, occupied or unknown. It adds “no-go” zones called geofences for ground the robot can’t cross. It has two modes.

Local mode. Inside a box around the robot (for example 30 × 30 × 3 m for ANYmal, 20 × 20 × 1 m for Gagarin in the mine), it scatters random points. It links those that can be reached without hitting anything into a graph, and finds shortest paths from the robot to every point. Each point is scored by how many unknown voxels the sensor would see from there. A path’s score adds up these gains, discounted by distance and by how much the path turns away from the current direction of exploration.

Γ(path) = e−ζ·Z(path) · Σj VolumeGain(vj) · e−δ·D(vj)  // Z: how much the path turns away from the exploration direction · D: distance along the path

Global mode. Points with high gain that the robot doesn’t visit become frontiers in a light global graph. When nothing worth exploring is left nearby (“local completion”), the robot picks a frontier. The choice weighs how much exploring time would remain after getting there and back home, how much unknown space it offers, and how far it is. The planner recomputes the path home at every step. When the time budget runs out, the robot follows it.

ΓG(frontier) = [Tleft − time(here→frontier) − time(frontier→home)] · VolumeGain · e−ε·distance
Animated. As exploring time runs down, far frontiers stop fitting in the budget and the choice shifts to nearer ones, until going home is the only option. Numbers are illustrative.

For multi-storey buildings a vertical mode samples densely up and down and rewards reaching a new floor, which pulls the robot into stairwells. In the Urban Circuit the supervisor switched Gagarin into it (with an 8 × 8 × 5.5 m box). Gagarin then flew down a stairwell on its own at 0.24 m/s.

Real run. An Aerial Scout in the Urban Circuit switches to auto-homing and follows the path the planner kept back to its take-off point (4× speed). The coloured points are the LiDAR map; the insets show the floor plan so far and a camera view. Footage: Autonomous Robots Lab, team CERBERUS (Urban Circuit video).
Paper Fig. 11. The planner’s two halves: local exploration around the robot, and global re-positioning to frontiers and homing, with geofences marking ground the robot can’t cross.
Paper Fig. 21. Real planner output in the Urban Alpha course: the Aerial Scout and ANYmal exploring the first floor, and Gagarin descending the stairwell.
Scene 5

Safe footing

In short: stepping on a terrain map failed when the map was wrong, so the team switched to a learned controller that feels the ground and uses the map only to avoid no-go areas.

In the Tunnel Circuit, ANYmal used a model-based controller that optimises footholds and body motion on a terrain map, plus a reflex that catches major slips. The map came from an Intel RealSense depth camera. Puddles and foot slip corrupted it, the controller acted on the wrong terrain, and the robots could not continue their missions.

For the Urban Circuit the team did two things. They swapped the depth camera for a dome LiDAR (Robosense BPearl) after comparing three sensors, and they deployed a neural-network controller trained in simulation with reinforcement learning. That controller uses only a history of joint and inertial readings. From these it infers terrain and pushes without a map. With reliable walking, the robots’ speed was raised from 0.2 to 0.45 m/s, and the walking robots were deployed in all four Urban runs.

Terrain sensorTypeWhat the team found
Intel RealSense D435active stereoless accurate; works in sunlight; missing data on wet ground
Pico Monstartime of flightaccurate, but weak with sunlight or light-absorbing material
Robosense BPearldome LiDAR, 90° viewaccurate and robust, but sparse; chosen for the Urban Circuit

What this means: reflections from water fooled both camera-type depth sensors, which is exactly what corrupted the map in the mine.

The terrain map stayed useful for safety. ANYmal builds a 12 × 12 m elevation map around itself with 4 cm cells. A tiny network (two layers, 120 learnable parameters, trained on about a dozen hand-labelled maps) marks cells it can’t walk on. The planner itself only checks for collisions, so a watchdog projects the robot’s motion 30 cm ahead. If that hits a non-traversable cell, the robot stops, backs up, marks a geofence and asks for a new plan.

Animated. Stop, back up, fence, re-plan. It works, but each stop costs time; the paper names longer-range traversability as future work.
Paper Fig. 6. An elevation map from the Urban Circuit; black cells were judged not walkable.
Scene 6

One shared map

In short: the base station merges every robot’s map into one, in DARPA’s coordinates, which makes reported positions more accurate.

Each robot works on its own and doesn’t need the base to explore. When it has a link, it sends its map in pieces. M3RM (multi-modal, multi-robot mapping, built on the open-source maplab) turns each robot’s data into submaps: a pose graph, visual landmarks and compressed laser scans, cut every 30 s and 4–7 MB each. Transfers resume after a dropped link. The server at the base:

  • anchors every robot to DARPA’s frame from marker tags (AprilTags) on the start gate;
  • links maps where robots saw the same visual landmarks, and aligns nearby laser scans, rejecting links that would cross floors or walls;
  • optimises everything together (up to four submaps in parallel, capped at two minutes per optimisation).
Paper Fig. 14. How three robots’ graphs are joined: two saw the DARPA gate; the third was attached through landmarks it shared with the others.
Urban CircuitDrift per 10 m, onboard → mergedOverall error, onboard → merged
ANYmal Badger · Alpha 20.19 → 0.11 m1.07 → 0.26 m
ANYmal Bear · Beta 20.22 → 0.12 m0.58 → 0.49 m
Alpha Aerial Scout · Alpha 20.07 → 0.13 m0.30 → 0.25 m

What this means: merging lowered every robot’s overall error; all stayed far inside the 5 m scoring limit. The paper notes this matters more as courses get bigger, because onboard error grows with distance travelled.

From a camera box to a reported point

Objects are detected on board with YOLOv3, trained on 1,915 (cell phone) to 3,519 (fire extinguisher) images per class. The box is split into a grid, and rays are cast from each cell into the map. The median hit is taken as the object’s position, and a sphere is drawn around it. Later detections landing in the same sphere update a per-class probability. Only once one class passes a threshold is the find sent to the supervisor, with its image. Phones were also sensed by their Bluetooth signal, and CO₂ by a gas sensor; the supervisor then placed those by hand.

Animated. Rays from the detection box, the median hit, and a confidence that climbs with each repeat detection until the find is confirmed.
Paper Fig. 24. Merged maps from all Urban runs laid over DARPA’s ground truth, with reported, detected and missed artifacts and the dropped relays marked.
Scene 7

Keeping in touch

In short: the robots carry their own network in: an antenna at the base, a fibre-tethered rover, and relays dropped by the walking robots.

All robots talk over a 5.8 GHz WiFi mesh, in which every node forwards traffic for the others. Ground robots and relays are mesh nodes; the drones, too small to carry mesh radios, join as clients of the nearest access point. From the base there were four ways in: a high-gain panel antenna, the fibre-tethered rover with its own antenna, a panel antenna on a pole angled down a stairway by a team member, and a radio in a foam ball thrown down the stairs.

A relay reached about 100 m in a straight tunnel, with up to about 200 Mbps close to the first node. Corners and reflections make signal strength swing by 30–40 % from spot to spot, so the supervisor, not an algorithm, chose where to drop relays, using ping times and the tunnel’s shape.

What happens when a robot loses the link depends on its mode. In supervised mode it retraces its path until the link returns and waits for a decision. In fully autonomous mode it keeps exploring until a time limit set by the supervisor, then plans its way home to upload the map and finds.

On the software side, each robot runs its own ROS master; the Nimbro Network package forwards a hand-picked list of topics over the flaky links. An ANYmal had more than 800 topics and 400 services, far too many to send blindly.

Paper Fig. 16. The network: rover and walking robots plus dropped nodes form a mesh backbone; the rover also has fibre to the base; the drones connect as ordinary WiFi clients.
Paper Fig. 5 (right). ANYmal placing a relay.
Animated. The supervisor’s station. In the mine the supervisor handled four robot computers plus a reporting PC; for the Urban Circuit that became two control PCs plus the reporting PC, and bulky data was sent only on request.
Scene 8

Results

In short: 6th of 11 in the mine and 5th of 10 in the power plant, with the walking robots doing most of the scoring by the second round.

RoundPlaceCorrect reportsPointsRank
Tunnel Circuit · Aug 2019NIOSH research mine, Pittsburgh (room-and-pillar)656th of 11
Urban Circuit · Feb 2020Satsop, an unfinished nuclear power plant, Elma, WA1175th of 10

What this means: a clear improvement between rounds, earned mostly by making the walking robots reliable and the software shared.

Tunnel Circuit

Two ANYmals (Bear and Badger, with point feet, flat feet or wheels), the Armadillo rover and the Alpha and Gagarin scouts took part. The ANYmals ran in every run but scored only one point: corrupted terrain maps stopped them, and the wheeled version slipped in mud with its fixed trotting gait. After three runs had produced just that one point, the team re-fitted the rover overnight, calibrating its cameras and LiDAR for detection and mapping. In the last run the supervisor drove it over the fibre link and it scored four points. Alpha flew about 180 m on its own and came home, but a wrongly set exploration box kept it in areas the ground robots had already seen. Gagarin hit the mine ceiling and flew on.

Real run. An Aerial Scout explores the research mine on its own during the Tunnel Circuit, sped up. The left inset draws its planned exploration path; the right one is a second camera view. Footage: Autonomous Robots Lab, team CERBERUS (Tunnel Circuit video).
Paper Fig. 20. Tunnel Circuit trajectories, estimated (blue) against the truth (green): the multi-sensor scout (left) tracks closely, the LiDAR-only rover (right) drifts.

Urban Circuit

In the Alpha 2 run the rover’s fibre tangled in a wheel early on. Badger explored most of the upper floor on its own, but a state-machine bug meant it neither came back after losing WiFi nor moved on from a dead end. Bear was teleoperated down a first flight of stairs and could go no further. The Alpha scout explored the upper floor and came home, and Gagarin flew down a series of stairwells by itself, even surviving a hit on a metal bar. In the Beta 2 run two narrow doors at the start ruled out the drones. Badger got stuck on an obstacle out of radio range. Bear dropped two relays and scored three points; from about minute 38 a bug kept triggering its homing, and at minute 54 the supervisor switched to manual waypoints.

Paper Fig. 25. Minute-by-minute timelines of the Alpha 2 and Beta 2 runs: which robot was autonomous or teleoperated, and when finds, relay drops and link losses happened.

The drones scored no Urban points. The paper gives three reasons: they started from the entrance and mostly re-covered ground the walking robots had already mapped; their exploration bounds were set at take-off through an unintuitive interface; and they were sent in after the ground robots instead of as early scouts.

Wrap-up

Lessons and limits

In short: the paper is frank that many failures came from corner cases in autonomy, radio hardware and interfaces, not from the core algorithms.

  • Autonomy corner cases. Robots stuck on obstacles out of radio range could not recover. The team planned a behaviour-tree style autonomy and a separate watchdog, and more testing of long missions in simulation.
  • Exploring is not searching. The planner maximises newly mapped volume, but mapping a room with a LiDAR doesn’t mean the cameras saw every object in it. Robots sometimes passed artifacts without detecting them, partly because of limited lighting.
  • Short-range terrain checks. Walking robots discovered no-go areas late and had to stop and re-plan; longer-range traversability or a receding-horizon planner is needed.
  • Walking. The blind learned controller was very resilient; the next step named is a learned controller that also uses perception. The wheeled ANYmal’s mud troubles fed into a new hybrid rolling-and-stepping controller.
  • Radio. The relays were a weak point: sometimes dropped too far from the previous one, or tipped over so their antennas faced the ground. The rover helped a lot but took too much of the supervisor’s attention.
  • Same hardware everywhere. Different cameras, lenses and lights on different robots made it harder to link their maps. The team aimed to unify sensors across robots.
  • Practice like you play. After the Tunnel Circuit the team introduced weekly mission shakeouts and logging in 5-minute chunks, and planned monthly competition-style field tests.

Much of the software was released as open source, including GBPlanner, maplab (with M3RM), voxblox, elevation mapping, traversability estimation, darknet_ros and ROVIO. Simulation models of ANYmal B SubT and the scouts went into DARPA’s virtual competition. The team’s next steps included ANYmal C with the learned controller, new radios, and launching drones from ground robots deeper in the course.