WildDrone Summer School 2025: AI & Drones for Wildlife ConservationWildDrone Summer School 2025: IA y Drones en Conservación de Fauna

In late September 2025, we wrapped up the WildDrone Summer School 2025 (22–26 September) at the University of Konstanz and the Max Planck Institute of Animal Behavior (MPI-AB) in Germany. The intensive five-day programme brought together PhD candidates, postdocs, and leading researchers across ecology, robotic engineering, and computer science to explore how Uncrewed Aerial Systems (UAS), computer vision, and artificial intelligence can advance wildlife conservation. A finales de septiembre de 2025 cerramos la WildDrone Summer School 2025 (22–26 de septiembre) en la University of Konstanz y el Max Planck Institute of Animal Behavior (MPI-AB), Alemania. El programa intensivo de cinco días reunió a doctorandos, postdocs e investigadores de ecología, robótica e informática para explorar cómo los Uncrewed Aerial Systems (UAS), la visión por computador y la IA pueden impulsar la conservación de la fauna.

It was inspiring to witness such strong multidisciplinary engagement and to see how technical innovation in robotics and machine learning is being directly aligned with ecological field needs. Fue inspirador ver un compromiso multidisciplinar tan fuerte y cómo la innovación técnica en robótica y machine learning se alinea directamente con las necesidades ecológicas de campo.

💡 Workshop: Integrating LLMs in Robotics & Drones💡 Workshop: Integrando LLMs en Robótica y Drones

As part of the summer school’s core technical track, I led an interactive, hands-on workshop on “Integrating LLMs in Robotics and Drones” (developed in collaboration with my supervisor Prof. Anders Lyhne Christensen, SDU Drone Center / NAMUR). Como parte del track técnico principal, dirigí un workshop práctico e interactivo sobre “Integrating LLMs in Robotics and Drones” (con mi supervisor Prof. Anders Lyhne Christensen, SDU Drone Center / NAMUR).

The Core MotivationMotivación central

Managing multi-robot teams in complex environments (such as search-and-rescue or wildlife monitoring) is notoriously difficult — particularly for non-expert operators, rangers, or field biologists who don’t have time to write code or pilot multiple drones manually. Large Language Models offer an unprecedented opportunity: what if operators could command autonomous drone fleets just by speaking naturally? Gestionar equipos multi-robot en entornos complejos (búsqueda y rescate, monitorización de fauna) es muy difícil — sobre todo para operadores no expertos, guardas o biólogos que no tienen tiempo de programar ni de pilotar varios drones. Los Large Language Models ofrecen una oportunidad sin precedentes: ¿y si los operadores comandaran flotas de drones autónomos hablando con naturalidad?

“Please elevate the priority of the shoreline search, expand to walking distance of the missing child.” “Eleva la prioridad de la búsqueda en la orilla y amplíala a distancia de paseo del niño desaparecido.”

Our vision is to use LLMs not just as text chatbots, but as intelligent intent parsers that interpret spoken instructions, draft structured mission plans, dynamically adapt user interfaces, and execute closed-loop multi-robot planning via middleware like WildBridge. Nuestra visión: usar LLMs no solo como chatbots, sino como intérpretes de intención que entienden instrucciones habladas, redactan planes de misión, adaptan interfaces y ejecutan planificación multi-robot en bucle cerrado vía middleware como WildBridge.

Foundations, APIs & Tool-Calling AgentsFundamentos, APIs y Tool-Calling Agents

We kicked off by demystifying LLM mechanics: auto-regressive next-word prediction, sampling temperatures (balancing determinism vs. creative exploration), and the fundamental “stochastic parrot” nature of raw text generation. To overcome classic LLM weaknesses — outdated knowledge and lack of mathematical/spatial reasoning — we explored tool-calling agent architectures, where the LLM queries external APIs, geometry calculators, and telemetry feeds to make grounded decisions. Empezamos desmitificando la mecánica de los LLM: predicción autorregresiva, temperaturas de sampling (determinismo vs. exploración) y la naturaleza de “loro estocástico” del texto generado. Para superar sus debilidades clásicas — conocimiento desactualizado y falta de razonamiento matemático/espacial — exploramos arquitecturas de agentes con tool-calling, donde el LLM consulta APIs, calculadoras geométricas y telemetría para decidir con fundamento.

In Interactive Exercise 1 (Google Colab), participants tuned system prompts, adjusted temperature parameters, and built functional tool-calling agents to retrieve real-time external data. En el Interactive Exercise 1 (Google Colab), los participantes ajustaron system prompts y temperaturas y construyeron agentes con tool-calling para recuperar datos en tiempo real.

Drone Safari: Natural-Language UAV GameDrone Safari: Juego UAV natural

To bridge the gap to robotics, we introduced the Drone Safari Game (Figure 1): participants built an AI agent to pilot a drone across a 12×12 grid environment, photograph three distinct wildlife species (Zebra, Elephant, Lion), manage battery/photo limits, and avoid hazards — entirely through conversational commands: Para acercarlo a la robótica, presentamos el Drone Safari Game (la Figura 1): los participantes construyeron un agente de IA que pilota un dron en un grid 12×12, fotografía tres especies (Zebra, Elephant, Lion), gestiona batería/fotos y evita peligros — todo con comandos conversacionales:

• Mapping natural language (“fly forward two tiles and photograph the zebra to your left”) into discrete robotic primitives (move, turn, take_photo).Mapear lenguaje natural (“avanza dos casillas y fotografía la zebra a tu izquierda”) en primitivas robóticas discretas (move, turn, take_photo).
• Proposing multi-step execution challenges so the agent plans and chains complex action sequences from a single high-level command.Proponer retos multi-paso para que el agente planifique y encadene acciones complejas desde un único comando.

From Single Agents to Hierarchical Multi-Agent SystemsDe agentes simples a sistemas multi-agente jerárquicos

We concluded by addressing scalability: why single-agent setups break down in complex missions, and why Multi-Agent Systems (MAS) are essential. Drawing on our real-world deployment in Málaga (where voice-commanded ground and aerial robots assisted first responders during emergency disaster exercises), we walked through our multi-agent architecture (Figure 2): Concluimos con la escalabilidad: por qué un solo agente falla en misiones complejas y por qué los Multi-Agent Systems (MAS) son esenciales. Con nuestro despliegue real en Málaga (robots aéreos y terrestres por voz ayudando a primeros intervinientes en ejercicios de emergencia), repasamos nuestra arquitectura multi-agente (la Figura 2):

• Coordinator Agent: Decomposes complex user commands into structured sub-tasks.Coordinator Agent: descompone comandos complejos en sub-tareas estructuradas.
• Spatial & History Agents: Ground geographic features and retrieve live telemetry context without modifying world state.Spatial & History Agents: anclan features geográficas y leen telemetría en vivo sin modificar el estado del mundo.
• Execution & Safety Gate: Deterministically validates commands before dispatching them to physical drones or rovers.Execution & Safety Gate: valida comandos de forma determinista antes de despacharlos a drones o rovers físicos.

💻 Open-Source Codebase & Interactive Notebooks💻 Codebase open-source y notebooks

All workshop materials, Jupyter notebooks, agent utilities, and the complete game simulator are available open-source on GitHub at alejp1998/wilddrone-llm-workshop. Todos los materiales, notebooks Jupyter, utilidades y el simulador del juego están en open source en GitHub: alejp1998/wilddrone-llm-workshop.

The repository is structured into two guided tracks and modular support libraries:El repositorio tiene dos tracks guiados y librerías de soporte:

• part1-travel-agent.ipynb — LLM Fundamentals & Tool Calling: A beginner-friendly notebook walking through auto-regressive generation, temperature tuning, and building a weather-informed conversational travel assistant using structured tool-calling with litellm and Google Gemini. part1-travel-agent.ipynb — LLM Fundamentals & Tool Calling: Un notebook para principiantes que recorre la generación autorregresiva, el ajuste de temperatura y un asistente de viajes conversacional con tool-calling estructurado (litellm, Google Gemini).
• part2-drone-safari.ipynb — Conversational Drone Agent: The core robotics notebook where participants build an AI agent that maps free-form natural language into discrete robot actions (move, turn, take_picture) with live Jupyter widget rendering. part2-drone-safari.ipynb — Conversational Drone Agent: El notebook central de robótica: los participantes construyen un agente de IA que mapea lenguaje natural libre en acciones discretas (move, turn, take_picture) con widgets de Jupyter en vivo.
• drone_safari_game.py — 2D Safari Game Simulator: A complete grid-based wildlife flight simulator. The drone must navigate a 12×12 reserve to photograph 3 animal species (Zebra 🦓, Elephant 🐘, Oryx 🦌) within a 5-photo budget. Includes realistic mechanics: animals flee if approached too closely (≤ 1 cell), must be framed from exactly 2 cells away, and trees (🌳) act as collision obstacles. drone_safari_game.py — 2D Safari Game Simulator: Un simulador de vuelo de fauna completo basado en grid. El dron navega una reserva de 12×12 para fotografiar 3 especies (Zebra 🦓, Elephant 🐘, Oryx 🦌) con 5 fotos. Mecánicas realistas: los animales huyen si te acercas (≤ 1 celda), deben encuadrarse desde 2 celdas exactas, y los árboles (🌳) son obstáculos.
• llm_agents.py — Agent Utilities: Helper classes for tool parameter schema generation, prompt templates, and conversational execution loops. llm_agents.py — Agent Utilities: Clases auxiliares para esquemas de herramientas, plantillas de prompts y bucles conversacionales.
• colab/ — Google Colab editions: Self-contained versions of both notebooks that download the game scripts automatically and read your API key from Colab secrets — no local setup needed, just click Open in Colab and run. colab/ — Google Colab editions: Versiones autocontenidas de ambos notebooks que descargan los scripts y leen tu API key de los secrets de Colab — sin configuración local: pulsa Open in Colab y ejecuta.
• webgame/ — PixiJS browser game: The playable web edition of Drone Safari deployed to GitHub Pages, with game-core.js mirroring the Python engine rule-for-rule. webgame/ — PixiJS browser game: La edición web jugable de Drone Safari en GitHub Pages, con game-core.js replicando el motor Python regla por regla.

You can clone and run the workshop locally in minutes:Puedes clonar y ejecutar el workshop en minutos:

git clone https://github.com/alejp1998/wilddrone-llm-workshop.git
cd wilddrone-llm-workshop
pip install litellm matplotlib numpy python-dotenv ipywidgets
jupyter notebook part1-travel-agent.ipynb

Prefer not to install anything? The colab/ editions run entirely in Google Colab, and the model is swappable without touching the notebooks — set the LLM_MODEL environment variable (e.g. gemini/gemini-2.5-flash, openai/gpt-4o-mini, or anthropic/claude-sonnet-4.5) to switch providers. The engine and agent utilities ship with 57 deterministic tests — 37 pytest cases covering game rules, movement, photos, sensors, and agent tooling, plus 20 node --test cases verifying the web-game port against the Python engine — all runnable with no API key. ¿Prefieres no instalar nada? Las ediciones colab/ se ejecutan íntegramente en Google Colab y el modelo se cambia sin tocar los notebooks — define la variable LLM_MODEL (p. ej. gemini/gemini-2.5-flash, openai/gpt-4o-mini o anthropic/claude-sonnet-4.5) para cambiar de proveedor. El motor y las utilidades incluyen 57 tests deterministas — 37 casos pytest de reglas del juego, movimiento, fotos, sensores y tooling, más 20 casos node --test que verifican el port web contra el motor Python — todos ejecutables sin API key.

The big idea, in plain words: a large language model is, at its core, a machine that predicts the next word — over and over. Text is chopped into small pieces called tokens, and the model assigns a probability to every possible next token inside its context window: \[ P(w_{t+1} \mid w_1, \dots, w_t) \] What makes this useful for robots is tool calling: the model can only emit text, so the system prompt declares the available functions (move(), turn(), take_picture()), the model responds with a structured call, and your code executes it and feeds the result back. The model becomes the planner, the code becomes the hands — and an agent is the loop that connects them: \[ \text{perceive} \rightarrow \text{decide} \rightarrow \text{act} \rightarrow \text{observe} \rightarrow \cdots \] La gran idea, en palabras simples: un large language model es una máquina que predice la siguiente palabra — una y otra vez. El texto se divide en tokens, y el modelo asigna una probabilidad a cada token posible en su ventana de contexto: \[ P(w_{t+1} \mid w_1, \dots, w_t) \] Lo que lo hace útil para robots es el tool calling: el modelo solo emite texto, así que el system prompt declara las funciones disponibles (move(), turn(), take_picture()), el modelo responde con una llamada estructurada y tu código la ejecuta y devuelve el resultado. El modelo es el planificador, el código las manos — y un agente es el bucle que los conecta: \[ \text{perceive} \rightarrow \text{decide} \rightarrow \text{act} \rightarrow \text{observe} \rightarrow \cdots \]

🎮 Play the Drone Safari Web Game🎮 Juega al Drone Safari Game

The workshop’s game engine now has a polished, browser-playable edition: a PixiJS v8 port that mirrors drone_safari_game.py rule-for-rule — movement, crashes, the “too close” scare mechanic, photo rules, win conditions, all three difficulties, and the sensor summaries — deployed to GitHub Pages. Fly the drone with the arrow keys, snap photos with Space, no setup at all: El motor del juego tiene ahora una edición jugable en el navegador: un port de PixiJS v8 que replica drone_safari_game.py regla por regla — movimiento, choques, la mecánica de asustar por “demasiado cerca”, reglas de foto, victoria, 3 dificultades y resúmenes de sensores — desplegado en GitHub Pages. Pilota con flechas, fotos con Space, sin configuración:

alejp1998.github.io/wilddrone-llm-workshop

• Cockpit UI: live mission HUD (position, facing, moves and turns), safari album checklist, on-screen D-pad, sensors button, and a terminal-style flight log — all wrapped in a system-aware dark/light theme.Cockpit UI: HUD de misión en vivo (posición, orientación, movimientos, giros), checklist del safari, D-pad, sensores y registro de vuelo estilo terminal — con tema claro/oscuro según el sistema.
• Keyboard: ↑↓ move · ←→ turn · Space/Enter photo · R restart · Q quit; a “last key” indicator shows exactly which command was issued.Keyboard: ↑↓ mover · ←→ girar · Space/Enter foto · R reiniciar · Q salir; un indicador de “última tecla” muestra el comando emitido.
• Game-over states: mission-failed (e.g. scaring an animal by getting too close) and victory overlays render as crisp in-canvas cards, with the flight log explaining what went wrong.Game-over states: las pantallas de misión fallida (p. ej., asustar un animal al acercarte) y victoria se ven como tarjetas en el canvas, con el registro explicando qué falló.
• Verified parity: the web core ships its own 20-case node --test suite checking the browser port against the Python engine.Verified parity: el core web trae su suite node --test de 20 casos que verifica el port contra el motor Python.

🚁 Hands-On Demos in the Imaging Hangar🚁 Demos prácticas en el Imaging Hangar

A highlight of the week was working inside the University of Konstanz’s Imaging Hangar — the world-class motion-capture flight arena at the Centre for the Advanced Study of Collective Behaviour (CASCB). Participants ran multi-robot swarm experiments and ground-rover tracking routines under high-speed motion-capture cameras, benchmarking coordinated control and vision tracking in real time. Uno de los momentos de la semana fue trabajar en el Imaging Hangar de la University of Konstanz — el arena de vuelo con motion-capture de clase mundial del Centre for the Advanced Study of Collective Behaviour (CASCB). Los asistentes probaron enjambres multi-robot y seguimiento de rovers terrestres bajo cámaras de motion-capture de alta velocidad, evaluando control y visión en tiempo real.

🌍 World-Class Keynotes & Ecological ResearchKeynotes de élite y ecología

The summer school featured an exceptional lineup of keynote lectures from pioneers at the intersection of AI, robotics, and biology: La summer school contó con keynote lectures excepcionales de pioneros en IA, robótica y biología:

• Prof. Tanya Berger-Wolf (Ohio State University / Director of Translational Data Analytics Institute & Imageomics): Keynote on “AI for Nature: From Science to Impact”, exploring foundational AI for biodiversity monitoring.Prof. Tanya Berger-Wolf (Ohio State University / Director of Translational Data Analytics Institute & Imageomics): keynote “AI for Nature: From Science to Impact”, IA fundacional para monitorear la biodiversidad.
• Prof. Stefano Mintchev (ETH Zürich, Environmental Robotics Lab): “Aerial Robots for Biodiversity Surveys in the Amazon”, sharing insights and bio-inspired robotics from the XPRIZE Rainforest competition.Prof. Stefano Mintchev (ETH Zürich, Environmental Robotics Lab): “Aerial Robots for Biodiversity Surveys in the Amazon”, con ideas y robótica bioinspirada del concurso XPRIZE Rainforest.
• Dr. Aamir Ahmad (MPI for Intelligent Systems & Uni Stuttgart): “Automatic Animal Behavior Inference: How to Rapidly Annotate Behavior on Long Drone Videos?”Dr. Aamir Ahmad (MPI for Intelligent Systems & Uni Stuttgart): “Automatic Animal Behavior Inference: How to Rapidly Annotate Behavior on Long Drone Videos?”
• Dr. Isla Duporge (Princeton University): “Minimising Acoustic Drone Disturbance to Wildlife Using Species Audiograms”.Dr. Isla Duporge (Princeton University): “Minimising Acoustic Drone Disturbance to Wildlife Using Species Audiograms”.
• Prof. Salua Hamaza (TU Delft): “Drones for Nature Conservation — New Paradigms to Extend Environmental Monitoring Missions” (perching and morphing UAS).Prof. Salua Hamaza (TU Delft): “Drones for Nature Conservation — New Paradigms to Extend Environmental Monitoring Missions” (UAS de percha y morfing).
• Dr. Angela Albi (MPI-AB / CASCB): Computer vision tracking with Trex for predator-prey dynamics between reef sharks and fish schools.Dr. Angela Albi (MPI-AB / CASCB): seguimiento con computer vision y Trex para dinámicas de tiburones de arrecife y peces.
• Dr. Blair Costelloe & Dr. Adwait Deshpande (MPI-AB): Scaling up behavioral ecology and collective dynamics in wild African ungulates and Indian wolves.Dr. Blair Costelloe & Dr. Adwait Deshpande (MPI-AB): escalando la ecología conductual y dinámica colectiva en ungulados africanos y lobos indios.
• Prof. Tom Richardson (University of Bristol): Flying drones for nature conservation within the WildDrone project.Prof. Tom Richardson (University of Bristol): volando drones para conservar la naturaleza en WildDrone.

🤝 Posters, Research Pitches & Networking🤝 Posters, pitches y networking

The WildDrone Doctoral Candidates presented their latest results throughout the week in interactive poster sessions and pitch rounds, spanning behavioural ecology, fixed-wing glider aerodynamics, thermal vision analysis, and multi-perspective photogrammetry. Evening networking events fostered lively collaborations across the Max Planck Institute, University of Konstanz, and the wider WildDrone consortium. Los WildDrone Doctoral Candidates presentaron sus últimos resultados en sesiones de posters y rondas de pitch: ecología del comportamiento, aerodinámica de planeadores, visión térmica y fotogrametría multi-perspectiva. Los eventos nocturnos fomentaron colaboraciones entre el Max Planck Institute, la University de Konstanz y el consorcio WildDrone.

Two months earlier, I presented our research on “LLM-Enabled Voice-Command of Multi-Robot Systems” at the Robotics and Software Engineering Meeting (RSE’25) in Odense, sharing the software architecture and lessons learned from live field deployments — slides (PDF). Dos meses antes, presenté nuestra investigación sobre “LLM-Enabled Voice-Command of Multi-Robot Systems” en el Robotics and Software Engineering Meeting (RSE’25) en Odense, compartiendo la arquitectura y las lecciones de los despliegues reales — slides (PDF).

September 2025 · WildDrone Summer School GitHub RepositoryGitHub Repository Play the GameJugar Slides PDFSlides PDF Summer School ProgrammePrograma Summer School WildDrone EU

WildDrone is an MSCA Doctoral Network funded by the European Union’s Horizon Europe research and innovation programme under grant agreement no. 101071224. WildDrone es una MSCA Doctoral Network financiada por el programa Horizon Europe (UE), grant agreement no. 101071224.