AITL-WING-HITL: Telemanipulation of Autonomous Drones Using Digital Twins of Aerial Traffic Interfaced With WING

Abstract

''By touching an instrument placed in the southern gallery, a miniature Spanish cruiser anchored in the fountain lake on the lower floor, 90 ft away, was blown into the air. There was no connection between the transmitter and the vessel in the lake, reported The New York Times in 1898. Though perceived as magic by many at the time, Nikola Tesla had once again pushed the boundaries of technology, becoming the forefather of modern remotely controlled wireless drones. His invention-an early radio-controlled vessel operated via radio waves-was hailed as the first of a race of robots, mechanical men which will do the laborious work of the human race, and marked a leap far ahead of its time. Since then, human-robot interaction and the telemanipulation of drones have advanced substantially, especially in tandem with the evolution of autonomy. In this context, a scalable, agent-based platform-termed AITL-WING-HITL-was developed for the telemanipulation of Autonomous Unmanned Aerial Vehicles (A-UAVs) from the perspective of a human-multi-robot architecture, leveraging Digital Twins (DTs) of integrated aerial traffic. This system is designed to intervene in scenarios, where an autonomous AI agent-the new driver - encounters conditions too complex or unorthodox for autonomy alone to handle. The platform incorporates a patented, force-sensitive, precision control device known as the WING-an immersive tool tested with participants using measurable parameters to validate the system's effectiveness and identify areas for further improvement. The findings indicate that AITL (Autonomy-In-The-Loop) A-UAV agents and HITL (Human-In-The-Loop) Human Telemanipulator (HTM) agents can co-operate to achieve high-performance, synergistic task execution through a socio-cognitive interaction model embedded in the platform. By addressing current challenges and outlining directions for future exploration, this research not only provides an overview of the evolving landscape of drone telemanipulation but also serves as a roadmap for ongoing and future efforts in this critical field.