Drone Fleet Management for Increased Efficiency 

Explore how advanced fleet management systems enhance the efficiency and collaboration of drone operations, focusing on task allocation, AI integration, and human-robot interaction in challenging environments. 

1. Introduction 

Drone fleet management has emerged as a game-changing technology for industries seeking to enhance efficiency in remote and complex environments, particularly in forestry operations. As drone technology evolves, so do the methods of coordination, navigation, and task allocation that allow for seamless, large-scale operations. Managing a fleet of drones autonomously requires more than just individual drone capabilities - it demands a robust integrated system that can coordinate multiple agents, be it humans or machines, to work together harmoniously. This concept forms the backbone of fleet management systems (FMS), where advanced software platforms, AI-driven task planning, and real-time data analysis come together to streamline operations. 

The primary goal of drone fleet management is to ensure that operations, such as reconnaissance and picking addressed in FEROX, are allocated and executed efficiently with minimal human intervention. Modern FMS frameworks incorporate cutting-edge technologies, like AI, data-driven decision-making, and autonomous navigation to facilitate drone collaboration, even in challenging environments like dense forests. These systems not only enable drones to function in sync with each other, but also provide interfaces for human operators to monitor and intervene when necessary. By leveraging these advancements, drone fleets can improve operational efficiency, reduce human errors, and minimise the need for manual control. 

Within FEROX, drone fleet management plays a critical role in optimising various processes, from under-canopy collaborative mapping for forest inventory to GNSS-guided waypoint navigation for assisting human pickers during berry collection. Furthermore, the integration of AI models that can automatically identify resources, such as berries, combined with a comprehensive FMS, ensures that drone operations align with human needs while maintaining safety and efficiency. This seamless interaction between humans and machines through intuitive user interfaces ensures that the overall operation benefits from the best of both worlds - robotic precision and human flexibility. 

2. FORMIGA: Framework for Optimised Robotic Management, Integration, Guidance, and Automation

The Framework for Optimised Robotic Management, Integration, Guidance, and Automation, or FORMIGA1 for short, is designed to serve as an advanced FMS that orchestrates the coordination of heterogeneous agents, including drones and human operators, for optimised task execution in challenging environments. Built upon a robust Robot Operating System (ROS) architecture, FORMIGA leverages the seamless integration of standardised communication protocols to enhance interoperability between multiple agents (Fig. 1).  

The system ensures that task execution is flexible, scalable, and adaptable, making it suitable for a wide range of applications, from autonomous field operations to human-robot collaboration. With a strong emphasis on automation, FORMIGA incorporates advanced AI technologies, including Large Language Models (LLMs), to facilitate task generation, allocation, and management. Furthermore, the framework’s intuitive user interfaces provide human operators with accessible tools to interact with the system, streamlining collaboration between humans and robots. FORMIGA stands out as a comprehensive system designed to enhance operational efficiency, reduce cognitive overload, and improve task execution in dynamic, real-world conditions.

Fig. 1. FORMIGA General Overview.

2.1 ROS Integration and Standardisation 

The foundation of FORMIGA’s operational efficiency lies in its deep integration with ROS, which facilitates seamless communication and coordination between multiple agents, whether they are humans or robots. By leveraging ROS, FORMIGA ensures that each component of the system can interact through a standardised set of protocols, minimising the complexity typically associated with managing heterogeneous fleets of drones and human operators. This ROS-based architecture not only enhances the modularity of the system, but also allows for the scalability required in dynamic field operations. The standardisation of communication protocols, such as the use of namespaces and action clients, enables FORMIGA to streamline task execution, ensuring that tasks are broken down into manageable units that can be easily assigned and tracked across different agents. This setup is crucial in allowing the system to adapt to both controlled and unstructured environments, ensuring reliable performance even when conditions change unexpectedly. 

In the context of FORMIGA, ROS integration encompasses other critical aspects, including the key communication mechanisms topics, services and actions: 

• Topics: MEx Status Monitoring FORMIGA utilises ROS’s publish-subscribe model to monitor the status of Mobile Executors (MEx), which include both robots and human agents. Each MEx publishes its status to a designated topic, sharing critical information such as battery level, pose, and current task. FORMIGA enriches this data with fleet-level insights to ensure a comprehensive overview of each Mex’s availability and task progression, facilitating real-time decision-making in task allocation and fleet management (Fig. 2). 

• Services: Task Requests and Management ROS services allow for synchronous request-response communication, enabling direct interaction between system components. In FORMIGA, services such as add_task and get_mex_list handle task allocation and status monitoring, while abort_task and unassign_task manage task cancellation and resource reallocation. These services enhance accessibility for users and third-party apps to guide and oversee the robot fleet, integrating seamlessly with FORMIGA’s overall architecture. 

• Actions: Task Decomposition and Execution ROS actions are pivotal for managing long-running tasks in FORMIGA, providing mechanisms for goal management, feedback, and pre-emption. These asynchronous actions enable the system to issue and monitor complex tasks such as navigating to specific locations or handling multiple subtasks. The real-time feedback from action servers allows FORMIGA to dynamically adjust to changing conditions, ensuring efficient task completion and resource optimisation throughout the fleet. 

Fig. 2. MexStatus.msg structure for a Mobile Executor (MEx) in FORMIGA. 

2.2 User Interface

The FORMIGA interface is designed to streamline the management of both human and robotic agents through an intuitive, user-friendly platform (Fig. 3). From the initial setup, users can seamlessly integrate available agents into the system by scanning the network for active agents, such as ground robots, drones, and human participants. FORMIGA automatically detects and categorises the type of each agent, simplifying fleet organisation and ensuring a cohesive human-robot team. Once the agents are added, users can monitor their statuses, available actions, and operational parameters in real time, facilitating continuous task management and system oversight. This ensures a comprehensive overview of fleet activities, including battery levels, task progress, and agent locations, displayed on an OpenStreetMap-based visualisation. 

Moreover, the interface allows users to create, manage, and prioritise tasks within the system. The ability to assign actions to agents and monitor their progress in real-time is augmented by FORMIGA’s dynamic feedback mechanisms, where users can view detailed task information, such as priority levels and task status, and intervene if necessary. This flexibility extends beyond robot tasks, enabling human-mediated interventions when required. FORMIGA’s real-time monitoring and user-centric design ensure a smooth integration of agents and the efficient execution of complex, multi-agent tasks. By offering a seamless interface and powerful management tools, FORMIGA bridges the gap between human and robotic coordination, enhancing the efficiency and performance of human-robot teams. 

Fig. 3. User flow of the FORMIGA interface, illustrating the streamlined process for managing and integrating human-robot teams efficiently.

2.3 LLM-Generated Task Coding  

In FORMIGA, LLMs play a crucial role in automating the development of complex robotic behaviours, significantly reducing the need for manual task coding (Fig. 4). LLMs, such as LLaMA 3, are leveraged to generate Python-based task functions by recognising structured command patterns. This capability enables FORMIGA to dynamically create context-specific tasks, particularly beneficial in real-time human-robot collaboration scenarios. The LLM integration allows for enhanced flexibility and adaptability in managing unforeseen challenges in the field, ensuring that the generated code aligns with the ROS-enabled ecosystem of the framework. 

To achieve this, FORMIGA fine-tunes LLMs by training them on specific agent types, action primitives, and API structures within its ecosystem. By understanding the available robots, humans, and their respective actions, the LLM can generate accurate Python functions that communicate with the ROS action servers. This integration streamlines task management by automatically selecting the appropriate agent, generating executable commands, and handling errors, ultimately enhancing the system's operational efficiency in dynamic environments. 

Fig. 4. LLM-Generated Task Execution with Llama 3. 

3. FORMIGA in FEROX

As stated in other of our published blogs, the FEROX project aims to support human workers in the remote and rugged terrains of Nordic countries by employing robotic technologies to assist in the collection of wild berries and mushrooms. The project emphasises Human-Robot Collaboration (HRC) through the deployment of UAVs to monitor and assist workers during their operations in the field. This collaboration improves worker safety, particularly in remote areas where access to immediate help is limited. The anticipated results of the FEROX project include increased trust in robotic assistance, enhanced berry harvesting efficiency, better quality of harvested produce, optimized picking times, improved worker safety, and reduced worker fatigue. 

FORMIGA plays a central role in FEROX by orchestrating the collaboration between human and robotic agents. Fig. 5 provides an overview of the FEROX use case, showcasing how FORMIGA manages the fleet of drones (Light-Weight Drones, LWDs, for reconnaissance and Heavy-Weight Drones, HWDs, for assistance) and coordinates with human pickers using the PickerApp. This interface allows pickers to request or trigger automated tasks, such as navigation assistance, safety checks, or payload transportation, enhancing the overall efficiency and safety of the berry-picking operations. 

Fig. 5. An overview of the FEROX use case scenario. 1) The fleet manager acts as the watcher, coordinating humans and drones through FORMIGA and confirming triggers raised by robots; 2) Human pickers collect berries using the PickerApp for guidance and to request drone services; 3) Pickers can request tasks via the PickerApp (e.g., WatchdogRequest for guidance back to base) or have tasks automatically triggered (WatchdogSOS); 4) UAVs include LWDs for reconnaissance and HWDs for assisting humans during picking. 

Fig. 5. An overview of the FEROX use case scenario. 1) The fleet manager acts as the watcher, coordinating humans and drones through FORMIGA and confirming triggers raised by robots; 2) Human pickers collect berries using the PickerApp for guidance and to request drone services; 3) Pickers can request tasks via the PickerApp (e.g., WatchdogRequest for guidance back to base) or have tasks automatically triggered (WatchdogSOS); 4) UAVs include LWDs for reconnaissance and HWDs for assisting humans during picking. 

3.1. FEROX Simulator

The FEROX Simulator is a vital component of the project, providing a virtual environment where the interactions between human agents and UAVs can be simulated and optimized before real-world deployment (Fig. 6). Built on the integration of ROS and Unity, the simulator provides a robust platform for replicating real-world scenarios. Unity provides realistic physics and environmental simulations, while ROS offers reliable communication with real robotic systems. The ROS-TCP Connector bridges these two systems, enabling seamless synchronization between the actions of human operators and robotic agents. Through this architecture, FEROX researchers can evaluate different strategies for HRC in controlled, repeatable settings. 

The multi-instance architecture of the FEROX Simulator consists of both robot instances (backend) and human instances (frontend). The robot instance manages UAVs as NavMesh agents, enabling autonomous navigation and task execution based on ROS services. The human instance provides a user-friendly interface where workers (acting as pickers in the simulation) can interact with the UAVs through third-person avatars. This allows for a highly interactive and immersive simulation environment where human operators and robotic systems cooperate to complete tasks such as berry picking, mapping, and navigation. A scoring system further enhances the simulation by rewarding successful operations, fostering a “coopetitive” atmosphere that encourages collaboration while allowing competitive dynamics in task execution. 

Fig. 6. General overview of FEROX agents. 

3.2. Human-Mediated Operations

Human-mediated operations within the FEROX system are critical to optimising the collaboration between humans and drones (Fig. 7). The system splits the berry-picking workflow into two key phases: reconnaissance and picking. During the reconnaissance phase, LWDs autonomously explore and map large regions, using ROS services to carry out tasks like ExploreRegion, which generates data to support berry yield predictions and forestry inventory. Operators can manage this process by segmenting the operational area and assigning specific regions to drones via FORMIGA. The system then selects the closest available LWD to execute the task, ensuring efficient coverage while accounting for drone constraints such as battery life. 

In the picking phase, drones collaborate directly with human pickers to facilitate the collection and transportation of berries. Tasks like PickUp, WatchDogSOS, and WatchDogRequest are executed using FORMIGA’s task management system, allowing pickers to request assistance from drones via the PickerApp. HWDs respond to these requests by collecting harvested berries, providing emergency support, or helping disoriented pickers navigate back to the base station. These operations are synchronized through FORMIGA's backend architecture, ensuring that both human and robotic agents work together seamlessly to complete the berry-picking process efficiently and safely. 

Table 1. Task Descriptions for Human-Mediated Operations. 

3.3. Preliminary Field Experiments

Beyond the many successful simulation experiments reported in the following paper2, the first real-world validation of the FORMIGA system took place during field trials in Rovaniemi, Finland, in September 2024. These trials marked a significant milestone, transitioning FORMIGA from simulated environments to live operations in dynamic, unstructured forestry settings. The system played a pivotal role in coordinating human-drone interactions for berry picking tasks, managing key operations such as PickUp, WatchdogRequest, and WatchdogSOS. While the primary objective of the trials was to evaluate the feasibility of these tasks, the initial results demonstrated FORMIGA’s potential to handle complex real-time operations. However, the team faced several technical challenges, particularly in maintaining efficient inter-robot communication over high-latency cellular networks, which necessitated modifications such as integrating a peer-to-peer VPN and transitioning to a multi-master architecture (see previous blog). These adjustments, although external to FORMIGA’s core system, were essential for its real-world deployment. 

Despite the communication challenges, FORMIGA required minimal changes during its transition from simulation to reality, showcasing the robustness of the system’s design. Drones were able to autonomously execute GNSS-guided tasks, reducing the physical workload on human workers and enhancing safety in the field. For instance, drones efficiently collected berry buckets and guided pickers back to safety. The successful execution of these tasks highlights FORMIGA’s potential to improve operational efficiency and worker safety in labour-intensive, remote environments. While these trials were limited in scale and participants, they provided valuable insights into FORMIGA’s performance, laying the groundwork for more extensive future testing. The feedback gained from these field trials will be instrumental in refining the system and scaling up its deployment across more complex and demanding real-world scenarios. 

Fig. 7. Preliminary real-world experiments executed in Rovaniemi (Finland). Left) Team managing the drones autonomously operating in the forest, acting as the watcher persona; Centre) Two HWD have been deployed to execute a series of tasks, including PickUp, WatchdogRequest and WatchdogSOS; and Right) Smartphone app (PickerApp) used to support berry picking operations, including by requesting specific tasks to FORMIGA, as well as providing feedback to the pickers’ ROS actions. 

4. Conclusion 

In conclusion, FORMIGA represents a cutting-edge framework that significantly enhances the capabilities of HRC, as demonstrated within the FEROX project. By seamlessly integrating ROS-based systems with intuitive user interfaces and advanced AI-driven task management, FORMIGA optimises the interactions between humans and autonomous drones in complex field operations. The comprehensive simulation platform further allows for extensive testing and refinement, ensuring a smooth transition to real-world scenarios. As we continue to explore new avenues, FORMIGA’s potential to improve operational efficiency and safety in challenging environments, such as the remote Nordic landscapes, highlights the transformative role of robotics in agriculture and forestry. 

Keywords 

Drone Fleet Management; Autonomous Drones; Task Allocation; AI Integration; Human-Robot Collaboration; Robotics in Forestry; Efficiency Optimisation; UAV Technology; Fleet Coordination; Remote Operations; AI-Driven Task Planning.