Safety Aware Task Planning via Large Language Models in Robotics

The integration of large language models (LLMs) into robotic task planning has unlocked better reasoning capabilities for complex, long-horizon workflows. However, ensuring safety in LLM-driven plans remains a critical challenge, as these models often prioritize task completion over risk mitigation. This paper introduces SAFER (Safety-Aware Framework for Execution in Robotics), a multi-LLM framework designed to embed safety awareness into robotic task planning. SAFER employs a Safety Agent that operates alongside the primary task planner, providing safety feedback. Additionally, we introduce LLM-as-a-Judge, a novel metric leveraging LLMs as evaluators to quantify safety violations within generated task plans. Our framework integrates safety feedback at multiple stages of execution, enabling real-time risk assessment, proactive error correction, and transparent safety evaluation. We also integrate a control framework using Control Barrier Functions (CBFs) to ensure safety guarantees within SAFER's task planning. We evaluated SAFER against state-of-the-art LLM planners on complex long-horizon tasks involving heterogeneous robotic agents, demonstrating its effectiveness in reducing safety violations while maintaining task efficiency. We also verify the task planner and safety planner through actual hardware experiments involving multiple robots and a human.

A Language Vision Model Approach for Automated Tumor Contouring in Radiation Oncology

Background: Lung cancer ranks as the leading cause of cancer-related mortality worldwide. The complexity of tumor delineation, crucial for radiation therapy, requires expertise often unavailable in resource-limited settings. Artificial Intelligence(AI), particularly with advancements in deep learning (DL) and natural language processing (NLP), offers potential solutions yet is challenged by high false positive rates. Purpose: The Oncology Contouring Copilot (OCC) system is developed to leverage oncologist expertise for precise tumor contouring using textual descriptions, aiming to increase the efficiency of oncological workflows by combining the strengths of AI with human oversight. Methods: Our OCC system initially identifies nodule candidates from CT scans. Employing Language Vision Models (LVMs) like GPT-4V, OCC then effectively reduces false positives with clinical descriptive texts, merging textual and visual data to automate tumor delineation, designed to elevate the quality of oncology care by incorporating knowledge from experienced domain experts. Results: Deployments of the OCC system resulted in a significant reduction in the false discovery rate by 35.0%, a 72.4% decrease in false positives per scan, and an F1-score of 0.652 across our dataset for unbiased evaluation. Conclusions: OCC represents a significant advance in oncology care, particularly through the use of the latest LVMs to improve contouring results by (1) streamlining oncology treatment workflows by optimizing tumor delineation, reducing manual processes; (2) offering a scalable and intuitive framework to reduce false positives in radiotherapy planning using LVMs; (3) introducing novel medical language vision prompt techniques to minimize LVMs hallucinations with ablation study, and (4) conducting a comparative analysis of LVMs, highlighting their potential in addressing medical language vision challenges.

BFANet: Revisiting 3D Semantic Segmentation with Boundary Feature Analysis

3D semantic segmentation plays a fundamental and crucial role to understand 3D scenes. While contemporary state-of-the-art techniques predominantly concentrate on elevating the overall performance of 3D semantic segmentation based on general metrics (e.g. mIoU, mAcc, and oAcc), they unfortunately leave the exploration of challenging regions for segmentation mostly neglected. In this paper, we revisit 3D semantic segmentation through a more granular lens, shedding light on subtle complexities that are typically overshadowed by broader performance metrics. Concretely, we have delineated 3D semantic segmentation errors into four comprehensive categories as well as corresponding evaluation metrics tailored to each. Building upon this categorical framework, we introduce an innovative 3D semantic segmentation network called BFANet that incorporates detailed analysis of semantic boundary features. First, we design the boundary-semantic module to decouple point cloud features into semantic and boundary features, and fuse their query queue to enhance semantic features with attention. Second, we introduce a more concise and accelerated boundary pseudo-label calculation algorithm, which is 3.9 times faster than the state-of-the-art, offering compatibility with data augmentation and enabling efficient computation in training. Extensive experiments on benchmark data indicate the superiority of our BFANet model, confirming the significance of emphasizing the four uniquely designed metrics. Code is available at https://github.com/weiguangzhao/BFANet.

Rotatable Antennas for Integrated Sensing and Communications

In this letter, we propose to deploy rotatable antennas (RAs) at the base station (BS) to enhance both communication and sensing (C&S) performances, by exploiting a new spatial degree-of-freedom (DoF) offered by array rotation. Specifically, we formulate a multi-objective optimization problem to simultaneously maximize the sum-rate of multiple communication users and minimize the Cram\'er-Rao bound (CRB) for target angle estimation, by jointly optimizing the transmit beamforming vectors and the array rotation angle at the BS. To solve this problem, we first equivalently decompose it into two subproblems, corresponding to an inner problem for beamforming optimization and an outer problem for array rotation optimization. Although these two subproblems are non-convex, we obtain their high-quality solutions by applying the block coordinate descent (BCD) technique and one-dimensional exhaustive search, respectively. Moreover, we show that for the communication-only case, RAs provide an additional rotation gain to improve communication performance; while for the sensing-only case, the equivalent spatial aperture can be enlarged by RAs for achieving higher sensing accuracy. Finally, numerical results are presented to showcase the performance gains of RAs over fixed-rotation antennas in integrated sensing and communications (ISAC).

Policy Constraint by Only Support Constraint for Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to optimize a policy by using pre-collected datasets, to maximize cumulative rewards. However, offline reinforcement learning suffers challenges due to the distributional shift between the learned and behavior policies, leading to errors when computing Q-values for out-of-distribution (OOD) actions. To mitigate this issue, policy constraint methods aim to constrain the learned policy's distribution with the distribution of the behavior policy or confine action selection within the support of the behavior policy. However, current policy constraint methods tend to exhibit excessive conservatism, hindering the policy from further surpassing the behavior policy's performance. In this work, we present Only Support Constraint (OSC) which is derived from maximizing the total probability of learned policy in the support of behavior policy, to address the conservatism of policy constraint. OSC presents a regularization term that only restricts policies to the support without imposing extra constraints on actions within the support. Additionally, to fully harness the performance of the new policy constraints, OSC utilizes a diffusion model to effectively characterize the support of behavior policies. Experimental evaluations across a variety of offline RL benchmarks demonstrate that OSC significantly enhances performance, alleviating the challenges associated with distributional shifts and mitigating conservatism of policy constraints. Code is available at https://github.com/MoreanP/OSC.

Intrinsic and Extrinsic Factor Disentanglement for Recommendation in Various Context Scenarios

In recommender systems, the patterns of user behaviors (e.g., purchase, click) may vary greatly in different contexts (e.g., time and location). This is because user behavior is jointly determined by two types of factors: intrinsic factors, which reflect consistent user preference, and extrinsic factors, which reflect external incentives that may vary in different contexts. Differentiating between intrinsic and extrinsic factors helps learn user behaviors better. However, existing studies have only considered differentiating them from a single, pre-defined context (e.g., time or location), ignoring the fact that a user's extrinsic factors may be influenced by the interplay of various contexts at the same time. In this paper, we propose the Intrinsic-Extrinsic Disentangled Recommendation (IEDR) model, a generic framework that differentiates intrinsic from extrinsic factors considering various contexts simultaneously, enabling more accurate differentiation of factors and hence the improvement of recommendation accuracy. IEDR contains a context-invariant contrastive learning component to capture intrinsic factors, and a disentanglement component to extract extrinsic factors under the interplay of various contexts. The two components work together to achieve effective factor learning. Extensive experiments on real-world datasets demonstrate IEDR's effectiveness in learning disentangled factors and significantly improving recommendation accuracy by up to 4% in NDCG.

A General Optimization Framework for Tackling Distance Constraints in Movable Antenna-Aided Systems

The recently emerged movable antenna (MA) shows great promise in leveraging spatial degrees of freedom to enhance the performance of wireless systems. However, resource allocation in MA-aided systems faces challenges due to the nonconvex and coupled constraints on antenna positions. This paper systematically reveals the challenges posed by the minimum antenna separation distance constraints. Furthermore, we propose a penalty optimization framework for resource allocation under such new constraints for MA-aided systems. Specifically, the proposed framework separates the non-convex and coupled antenna distance constraints from the movable region constraints by introducing auxiliary variables. Subsequently, the resulting problem is efficiently solved by alternating optimization, where the optimization of the original variables resembles that in conventional resource allocation problem while the optimization with respect to the auxiliary variables is achieved in closedform solutions. To illustrate the effectiveness of the proposed framework, we present three case studies: capacity maximization, latency minimization, and regularized zero-forcing precoding. Simulation results demonstrate that the proposed optimization framework consistently outperforms state-of-the-art schemes.

EPEE: Towards Efficient and Effective Foundation Models in Biomedicine

Foundation models, including language models, e.g., GPT, and vision models, e.g., CLIP, have significantly advanced numerous biomedical tasks. Despite these advancements, the high inference latency and the "overthinking" issues in model inference impair the efficiency and effectiveness of foundation models, thus limiting their application in real-time clinical settings. To address these challenges, we proposed EPEE (Entropy- and Patience-based Early Exiting), a novel hybrid strategy designed to improve the inference efficiency of foundation models. The core idea was to leverage the strengths of entropy-based and patience-based early exiting methods to overcome their respective weaknesses. To evaluate EPEE, we conducted experiments on three core biomedical tasks-classification, relation extraction, and event extraction-using four foundation models (BERT, ALBERT, GPT-2, and ViT) across twelve datasets, including clinical notes and medical images. The results showed that EPEE significantly reduced inference time while maintaining or improving accuracy, demonstrating its adaptability to diverse datasets and tasks. EPEE addressed critical barriers to deploying foundation models in healthcare by balancing efficiency and effectiveness. It potentially provided a practical solution for real-time clinical decision-making with foundation models, supporting reliable and efficient workflows.

Hierarchically Tunable 6DMA for Wireless Communication and Sensing: Modeling and Performance Optimization

This paper proposes a new hierarchically tunable six-dimensional movable antenna (HT-6DMA) architecture for base station (BS) in future wireless networks. The HT-6DMA BS consists of multiple antenna arrays that can flexibly move on a spherical surface, with their three-dimensional (3D) positions and 3D rotations/orientations efficiently characterized in the global spherical coordinate system (SCS) and their individual local SCSs, respectively. As a result, the 6DMA system is hierarchically tunable in the sense that each array's global position and local rotation can be separately adjusted in a sequential manner with the other being fixed, thus greatly reducing their design complexity and improving the achievable performance. In particular, we consider an HT-6DMA BS serving multiple single-antenna users in the uplink communication or sensing potential unmanned aerial vehicles (UAVs)/drones in a given airway area. Specifically, for the communication scenario, we aim to maximize the average sum rate of communication users in the long term by optimizing the positions and rotations of all 6DMA arrays at the BS. While for the airway sensing scenario, we maximize the minimum received sensing signal power along the airway by optimizing the 6DMA arrays' positions and rotations along with the BS's transmit covariance matrix. Despite that the formulated problems are both non-convex, we propose efficient solutions to them by exploiting the hierarchical tunability of positions/rotations of 6DMA arrays in our proposed model. Numerical results show that the proposed HT-6DMA design significantly outperforms not only the traditional BS with fixed-position antennas (FPAs), but also the existing 6DMA scheme. Furthermore, it is unveiled that the performance gains of HT-6DMA mostly come from the arrays' global position adjustments on the spherical surface, rather than their local rotation adjustments.

Movable Antenna Aided Multiuser Communications: Antenna Position Optimization Based on Statistical Channel Information

The movable antenna (MA) technology has attracted great attention recently due to its promising capability in improving wireless channel conditions by flexibly adjusting antenna positions. To reap maximal performance gains of MA systems, existing works mainly focus on MA position optimization to cater to the instantaneous channel state information (CSI). However, the resulting real-time antenna movement may face challenges in practical implementation due to the additional time overhead and energy consumption required, especially in fast time-varying channel scenarios. To address this issue, we propose in this paper a new approach to optimize the MA positions based on the users' statistical CSI over a large timescale. In particular, we propose a general field response based statistical channel model to characterize the random channel variations caused by the local movement of users. Based on this model, a two-timescale optimization problem is formulated to maximize the ergodic sum rate of multiple users, where the precoding matrix and the positions of MAs at the base station (BS) are optimized based on the instantaneous and statistical CSI, respectively. To solve this non-convex optimization problem, a log-barrier penalized gradient ascent algorithm is developed to optimize the MA positions, where two methods are proposed to approximate the ergodic sum rate and its gradients with different complexities. Finally, we present simulation results to evaluate the performance of the proposed design and algorithms based on practical channels generated by ray-tracing. The results verify the performance advantages of MA systems compared to their fixed-position antenna (FPA) counterparts in terms of long-term rate improvement, especially for scenarios with more diverse channel power distributions in the angular domain.