Large Language Models on Small Resource-Constrained Systems: Performance Characterization, Analysis and Trade-offs

Generative AI like the Large Language Models (LLMs) has become more available for the general consumer in recent years. Publicly available services, e.g., ChatGPT, perform token generation on networked cloud server hardware, effectively removing the hardware entry cost for end users. However, the reliance on network access for these services, privacy and security risks involved, and sometimes the needs of the application make it necessary to run LLMs locally on edge devices. A significant amount of research has been done on optimization of LLMs and other transformer-based models on non-networked, resource-constrained devices, but they typically target older hardware. Our research intends to provide a 'baseline' characterization of more recent commercially available embedded hardware for LLMs, and to provide a simple utility to facilitate batch testing LLMs on recent Jetson hardware. We focus on the latest line of NVIDIA Jetson devices (Jetson Orin), and a set of publicly available LLMs (Pythia) ranging between 70 million and 1.4 billion parameters. Through detailed experimental evaluation with varying software and hardware parameters, we showcase trade-off spaces and optimization choices. Additionally, we design our testing structure to facilitate further research that involves performing batch LLM testing on Jetson hardware.

Control Barrier Function Based UAV Safety Controller in Autonomous Airborne Tracking and Following Systems

Safe operations of UAVs are of paramount importance for various mission-critical and safety-critical UAV applications. In context of airborne target tracking and following, UAVs need to track a flying target avoiding collision and also closely follow its trajectory. The safety situation becomes critical and more complex when the flying target is non-cooperative and has erratic movements. This paper proposes a method for collision avoidance in an autonomous fast moving dynamic quadrotor UAV tracking and following another target UAV. This is achieved by designing a safety controller that minimally modifies the control input from a trajectory tracking controller and guarantees safety. This method enables pairing our proposed safety controller with already existing flight controllers. Our safety controller uses a control barrier function based quadratic program (CBF-QP) to produce an optimal control input enabling safe operation while also follow the trajectory of the target closely. We implement our solution on AirSim simulator over PX4 flight controller and with numerical results, we validate our approach through several simulation experiments with multiple scenarios and trajectories.

3DS-SLAM: A 3D Object Detection based Semantic SLAM towards Dynamic Indoor Environments

The existence of variable factors within the environment can cause a decline in camera localization accuracy, as it violates the fundamental assumption of a static environment in Simultaneous Localization and Mapping (SLAM) algorithms. Recent semantic SLAM systems towards dynamic environments either rely solely on 2D semantic information, or solely on geometric information, or combine their results in a loosely integrated manner. In this research paper, we introduce 3DS-SLAM, 3D Semantic SLAM, tailored for dynamic scenes with visual 3D object detection. The 3DS-SLAM is a tightly-coupled algorithm resolving both semantic and geometric constraints sequentially. We designed a 3D part-aware hybrid transformer for point cloud-based object detection to identify dynamic objects. Subsequently, we propose a dynamic feature filter based on HDBSCAN clustering to extract objects with significant absolute depth differences. When compared against ORB-SLAM2, 3DS-SLAM exhibits an average improvement of 98.01% across the dynamic sequences of the TUM RGB-D dataset. Furthermore, it surpasses the performance of the other four leading SLAM systems designed for dynamic environments.

Digital Twin in Safety-Critical Robotics Applications: Opportunities and Challenges

Digital Twin technology is being envisioned to be an integral part of the industrial evolution in modern generation. With the rapid advancement in the Internet-of-Things (IoT) technology and increasing trend of automation, integration between the virtual and the physical world is now realizable to produce practical digital twins. However, the existing definitions of digital twin is incomplete and sometimes ambiguous. Herein, we conduct historical review and analyze the modern generic view of digital twin to create its new extended definition. We also review and discuss the existing work in digital twin in safety-critical robotics applications. Especially, the usage of digital twin in industrial applications necessitates autonomous and remote operations due to environmental challenges. However, the uncertainties in the environment may need close monitoring and quick adaptation of the robots which need to be safety-proof and cost effective. We demonstrate a case study on developing a framework for safety-critical robotic arm applications and present the system performance to show its advantages, and discuss the challenges and scopes ahead.

Edge-assisted Collaborative Digital Twin for Safety-Critical Robotics in Industrial IoT

Digital Twin technology is playing a pivotal role in the modern industrial evolution. Especially, with the technological progress in the Internet-of-Things (IoT) and the increasing trend in autonomy, multi-sensor equipped robotics can create practical digital twin, which is particularly useful in the industrial applications for operations, maintenance and safety. Herein, we demonstrate a real-world digital twin of a safety-critical robotics applications with a Franka-Emika-Panda robotic arm. We develop and showcase an edge-assisted collaborative digital twin for dynamic obstacle avoidance which can be useful in real-time adaptation of the robots while operating in the uncertain and dynamic environments in industrial IoT.