Voice-Commanded Robots for Real Search & Rescue Logistics

In a live rescue mission, nobody has time to become a robot programmer. The question behind this paper is simple: what if first responders could just talk to the robot? Say "deliver the batteries to the drone pilot" — and have an autonomous ground vehicle safely figure out the rest, in the middle of a real emergency exercise.

‹ ›

That's exactly what we demonstrated at the XIX Workshop on Security, Emergencies, and Catastrophes (Málaga, Spain), and what we presented at the 2025 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) in Galway — where the paper was a finalist for the Best Paper Award 🏆. Real-World Deployment of an LLM-Enabled Voice-Commanded UGV for Logistics in SAR Missions.

Talk to the robot

Traditional robotic command interfaces are rigid and cognitively demanding — exactly the wrong properties under the pressure of a live search-and-rescue mission. Our system integrates a large language model-based multi-agent system (LLM-MAS) that manages the movements of uncrewed ground vehicles (UGVs) from natural-language voice commands, ensuring safe, context-aware navigation based on real-time mission data. Figure 1 shows the system architecture and communication workflow.

The LLM-MAS itself, shown in Figure 2, decomposes spoken commands into deterministic tool calls — interpreting spatial references like "five meters west of helmet 3" against the live mission picture.

Validated with real first responders

During the XIX Workshop in Málaga, over 150 first responders participated in six parallel SAR missions, with some teams providing their real-time positional data via equipment-integrated sensors. The LLM-MAS used this data to make autonomous decisions on the ground — such as determining which SAR team the UGV should assist — and then sent the selected target location to an external path planner through a tool call.

The ROS 2-based communication architecture integrates UGVs into existing SAR command hierarchies, with robust methods for message acknowledgment, prioritization, and operational-window synchronization. Figure 3 shows the coordination protocol between the Unified Command Centre (UCC), the Rescue Coordination Point (RCP), and the LLM-MAS.

Figure 4 shows the associated GUI displaying real-time positions in the exercise area: purple and white dots show teams tracked through smart helmets and the UGV (RoverJ8), the purple path traces the UGV's return to the loading point, colored polygons highlight areas of interest (EchoX), and colored circles flag points or persons of interest (RomeoX).

The delivery run

The system successfully executed material transport tasks across three critical service routes that supported the six parallel missions — including delivering fresh batteries to a drone pilot in the field, a job that previously meant a human runner. Figure 5 shows the UGV delivering materials autonomously; Figure 6 walks through the full Mission #1 delivery workflow — (a) requesting batteries, (b) autonomous navigation, (c) pickup — and Table 1 summarizes every operator command with its execution time and outcome. Figure 7 collects footage from the missions and the involved first responders.

a

b

c

a

b

c

Direct feedback from first responders confirmed that the system improved mission efficiency and was praised for its intuitive, responsive performance in coordinated decision-making.

2025 · IEEE SSRRIEEEPDFNAMUR

A. Jarabo-Peñas, J. Bravo-Arrabal, D. Lin-Yang, F. Pastor, R. Ladig, J.J. Fernández-Lozano, A.L. Christensen, A. García-Cerezo