← Back to blog

WildDrone Summer School 2025: AI & Drones for Wildlife Conservation

· University of Konstanz & Max Planck Institute of Animal Behavior, Germany

In late September 2025, we wrapped up the WildDrone Summer School 2025 (22–26 September) at the University of Konstanz and the Max Planck Institute of Animal Behavior (MPI-AB) in Germany. The intensive five-day programme brought together PhD candidates, postdocs, and leading researchers across ecology, robotic engineering, and computer science to explore how Uncrewed Aerial Systems (UAS), computer vision, and artificial intelligence can advance wildlife conservation.

It was inspiring to witness such strong multidisciplinary engagement and to see how technical innovation in robotics and machine learning is being directly aligned with ecological field needs.

WildDrone Summer School 2025 full group photo at University of Konstanz Alejandro Jarabo-Peñas giving the tutorial lecture on Integrating LLMs in Robotics and Drones Multi-robot arena demonstration in the Imaging Hangar motion-capture facility Computer vision and animal pose estimation lecture session Professor Anders Lyhne Christensen lecturing on conservation drone datasets

💡 Workshop: Integrating LLMs in Robotics & Drones

As part of the summer school’s core technical track, I led an interactive, hands-on workshop on “Integrating LLMs in Robotics and Drones” (developed in collaboration with my supervisor Prof. Anders Lyhne Christensen, SDU Drone Center / NAMUR).

The Core Motivation

Managing multi-robot teams in complex environments (such as search-and-rescue or wildlife monitoring) is notoriously difficult — particularly for non-expert operators, rangers, or field biologists who don’t have time to write code or pilot multiple drones manually. Large Language Models offer an unprecedented opportunity: what if operators could command autonomous drone fleets just by speaking naturally?

“Please elevate the priority of the shoreline search, expand to walking distance of the missing child.”

Our vision is to use LLMs not just as text chatbots, but as intelligent intent parsers that interpret spoken instructions, draft structured mission plans, dynamically adapt user interfaces, and execute closed-loop multi-robot planning via middleware like WildBridge.

Foundations, APIs & Tool-Calling Agents

We kicked off by demystifying LLM mechanics: auto-regressive next-word prediction, sampling temperatures (balancing determinism vs. creative exploration), and the fundamental “stochastic parrot” nature of raw text generation. To overcome classic LLM weaknesses — outdated knowledge and lack of mathematical/spatial reasoning — we explored tool-calling agent architectures, where the LLM queries external APIs, geometry calculators, and telemetry feeds to make grounded decisions.

In Interactive Exercise 1 (Google Colab), participants tuned system prompts, adjusted temperature parameters, and built functional tool-calling agents to retrieve real-time external data.

Drone Safari: Natural-Language UAV Game

To bridge the gap to robotics, we introduced the Drone Safari Game (Figure 1): participants built an AI agent to pilot a drone across a 12×12 grid environment, photograph three distinct wildlife species (Zebra, Elephant, Lion), manage battery/photo limits, and avoid hazards — entirely through conversational commands:

Drone Safari Game 12x12 grid environment overview from workshop slides
Figure 1 — The Drone Safari Game: participants built LLM agents to navigate a grid, locate three wildlife targets, manage battery budgets, and avoid hazards using natural language.
  • Mapping natural language (“fly forward two tiles and photograph the zebra to your left”) into discrete robotic primitives (move, turn, take_photo).
  • Proposing multi-step execution challenges so the agent plans and chains complex action sequences from a single high-level command.

From Single Agents to Hierarchical Multi-Agent Systems

We concluded by addressing scalability: why single-agent setups break down in complex missions, and why Multi-Agent Systems (MAS) are essential. Drawing on our real-world deployment in Málaga (where voice-commanded ground and aerial robots assisted first responders during emergency disaster exercises), we walked through our multi-agent architecture (Figure 2):

Multi-Agent LLM Coordinator and specialized sub-agents architecture from workshop slides
Figure 2 — Multi-Agent Coordination Pipeline: The Coordinator decomposes natural language commands, passing sub-tasks to Spatial and History agents for grounding, and the Robot agent for deterministic safety execution.
  • Coordinator Agent: Decomposes complex user commands into structured sub-tasks.
  • Spatial & History Agents: Ground geographic features and retrieve live telemetry context without modifying world state.
  • Execution & Safety Gate: Deterministically validates commands before dispatching them to physical drones or rovers.

💻 Open-Source Codebase & Interactive Notebooks

All workshop materials, Jupyter notebooks, agent utilities, and the complete game simulator are available open-source on GitHub at alejp1998/wilddrone-llm-workshop.

The repository is structured into two guided tracks and modular support libraries:

  • part1-travel-agent.ipynb — LLM Fundamentals & Tool Calling: A beginner-friendly notebook walking through auto-regressive generation, temperature tuning, and building a weather-informed conversational travel assistant using structured tool-calling with litellm and Google Gemini.
  • part2-drone-safari.ipynb — Conversational Drone Agent: The core robotics notebook where participants build an AI agent that maps free-form natural language into discrete robot actions (move, turn, take_picture) with live Jupyter widget rendering.
  • drone_safari_game.py — 2D Safari Game Simulator: A complete grid-based wildlife flight simulator. The drone must navigate a 12×12 reserve to photograph 3 animal species (Zebra 🦓, Elephant 🐘, Oryx 🦌) within a 5-photo budget. Includes realistic mechanics: animals flee if approached too closely (≤ 1 cell), must be framed from exactly 2 cells away, and trees (🌳) act as collision obstacles.
  • llm_agents.py — Agent Utilities: Helper classes for tool parameter schema generation, prompt templates, and conversational execution loops.
  • colab/ — Google Colab editions: Self-contained versions of both notebooks that download the game scripts automatically and read your API key from Colab secrets — no local setup needed, just click Open in Colab and run.
  • webgame/ — PixiJS browser game: The playable web edition of Drone Safari deployed to GitHub Pages, with game-core.js mirroring the Python engine rule-for-rule.

You can clone and run the workshop locally in minutes:

git clone https://github.com/alejp1998/wilddrone-llm-workshop.git
cd wilddrone-llm-workshop
pip install litellm matplotlib numpy python-dotenv ipywidgets
jupyter notebook part1-travel-agent.ipynb

Prefer not to install anything? The colab/ editions run entirely in Google Colab, and the model is swappable without touching the notebooks — set the LLM_MODEL environment variable (e.g. gemini/gemini-2.5-flash, openai/gpt-4o-mini, or anthropic/claude-sonnet-4.5) to switch providers. The engine and agent utilities ship with 57 deterministic tests — 37 pytest cases covering game rules, movement, photos, sensors, and agent tooling, plus 20 node --test cases verifying the web-game port against the Python engine — all runnable with no API key.

The big idea, in plain words: a large language model is, at its core, a machine that predicts the next word — over and over. Text is chopped into small pieces called tokens, and the model assigns a probability to every possible next token inside its context window: \[ P(w_{t+1} \mid w_1, \dots, w_t) \] What makes this useful for robots is tool calling: the model can only emit text, so the system prompt declares the available functions (move(), turn(), take_picture()), the model responds with a structured call, and your code executes it and feeds the result back. The model becomes the planner, the code becomes the hands — and an agent is the loop that connects them: \[ \text{perceive} \rightarrow \text{decide} \rightarrow \text{act} \rightarrow \text{observe} \rightarrow \cdots \]

🎮 Play the Drone Safari Web Game

The workshop’s game engine now has a polished, browser-playable edition: a PixiJS v8 port that mirrors drone_safari_game.py rule-for-rule — movement, crashes, the “too close” scare mechanic, photo rules, win conditions, all three difficulties, and the sensor summaries — deployed to GitHub Pages. Fly the drone with the arrow keys, snap photos with Space, no setup at all:

alejp1998.github.io/wilddrone-llm-workshop

Drone Safari PixiJS web game with the 12x12 board, drone facing indicator, wildlife targets, HUD panels, and on-screen controls
  • Cockpit UI: live mission HUD (position, facing, moves and turns), safari album checklist, on-screen D-pad, sensors button, and a terminal-style flight log — all wrapped in a system-aware dark/light theme.
  • Keyboard: ↑↓ move · ←→ turn · Space/Enter photo · R restart · Q quit; a “last key” indicator shows exactly which command was issued.
  • Game-over states: mission-failed (e.g. scaring an animal by getting too close) and victory overlays render as crisp in-canvas cards, with the flight log explaining what went wrong.
  • Verified parity: the web core ships its own 20-case node --test suite checking the browser port against the Python engine.
Drone Safari mission-failed overlay after getting too close to an animal, with the flight log recording the scare event

🚁 Hands-On Demos in the Imaging Hangar

A highlight of the week was working inside the University of Konstanz’s Imaging Hangar — the world-class motion-capture flight arena at the Centre for the Advanced Study of Collective Behaviour (CASCB). Participants ran multi-robot swarm experiments and ground-rover tracking routines under high-speed motion-capture cameras, benchmarking coordinated control and vision tracking in real time.

🌍 World-Class Keynotes & Ecological Research

The summer school featured an exceptional lineup of keynote lectures from pioneers at the intersection of AI, robotics, and biology:

  • Prof. Tanya Berger-Wolf (Ohio State University / Director of Translational Data Analytics Institute & Imageomics): Keynote on “AI for Nature: From Science to Impact”, exploring foundational AI for biodiversity monitoring.
  • Prof. Stefano Mintchev (ETH Zürich, Environmental Robotics Lab): “Aerial Robots for Biodiversity Surveys in the Amazon”, sharing insights and bio-inspired robotics from the XPRIZE Rainforest competition.
  • Dr. Aamir Ahmad (MPI for Intelligent Systems & Uni Stuttgart): “Automatic Animal Behavior Inference: How to Rapidly Annotate Behavior on Long Drone Videos?”
  • Dr. Isla Duporge (Princeton University): “Minimising Acoustic Drone Disturbance to Wildlife Using Species Audiograms”.
  • Prof. Salua Hamaza (TU Delft): “Drones for Nature Conservation — New Paradigms to Extend Environmental Monitoring Missions” (perching and morphing UAS).
  • Dr. Angela Albi (MPI-AB / CASCB): Computer vision tracking with Trex for predator-prey dynamics between reef sharks and fish schools.
  • Dr. Blair Costelloe & Dr. Adwait Deshpande (MPI-AB): Scaling up behavioral ecology and collective dynamics in wild African ungulates and Indian wolves.
  • Prof. Tom Richardson (University of Bristol): Flying drones for nature conservation within the WildDrone project.

🤝 Posters, Research Pitches & Networking

The WildDrone Doctoral Candidates presented their latest results throughout the week in interactive poster sessions and pitch rounds, spanning behavioural ecology, fixed-wing glider aerodynamics, thermal vision analysis, and multi-perspective photogrammetry. Evening networking events fostered lively collaborations across the Max Planck Institute, University of Konstanz, and the wider WildDrone consortium.

Two months earlier, I presented our research on “LLM-Enabled Voice-Command of Multi-Robot Systems” at the Robotics and Software Engineering Meeting (RSE’25) in Odense, sharing the software architecture and lessons learned from live field deployments — slides (PDF).

September 2025 · WildDrone Summer School GitHub Repository Play the Game Slides PDF Summer School Programme WildDrone EU

WildDrone is an MSCA Doctoral Network funded by the European Union’s Horizon Europe research and innovation programme under grant agreement no. 101071224.