PaSa: An LLM Agent for Comprehensive Academic Paper Search

We introduce PaSa, an advanced Paper Search agent powered by large language models. PaSa can autonomously make a series of decisions, including invoking search tools, reading papers, and selecting relevant references, to ultimately obtain comprehensive and accurate results for complex scholarly queries. We optimize PaSa using reinforcement learning with a synthetic dataset, AutoScholarQuery, which includes 35k fine-grained academic queries and corresponding papers sourced from top-tier AI conference publications. Additionally, we develop RealScholarQuery, a benchmark collecting real-world academic queries to assess PaSa performance in more realistic scenarios. Despite being trained on synthetic data, PaSa significantly outperforms existing baselines on RealScholarQuery, including Google, Google Scholar, Google with GPT-4 for paraphrased queries, chatGPT (search-enabled GPT-4o), GPT-o1, and PaSa-GPT-4o (PaSa implemented by prompting GPT-4o). Notably, PaSa-7B surpasses the best Google-based baseline, Google with GPT-4o, by 37.78% in recall@20 and 39.90% in recall@50. It also exceeds PaSa-GPT-4o by 30.36% in recall and 4.25% in precision. Model, datasets, and code are available at https://github.com/bytedance/pasa.

Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understanding

We introduce Tarsier2, a state-of-the-art large vision-language model (LVLM) designed for generating detailed and accurate video descriptions, while also exhibiting superior general video understanding capabilities. Tarsier2 achieves significant advancements through three key upgrades: (1) Scaling pre-training data from 11M to 40M video-text pairs, enriching both volume and diversity; (2) Performing fine-grained temporal alignment during supervised fine-tuning; (3) Using model-based sampling to automatically construct preference data and applying DPO training for optimization. Extensive experiments show that Tarsier2-7B consistently outperforms leading proprietary models, including GPT-4o and Gemini 1.5 Pro, in detailed video description tasks. On the DREAM-1K benchmark, Tarsier2-7B improves F1 by 2.8\% over GPT-4o and 5.8\% over Gemini-1.5-Pro. In human side-by-side evaluations, Tarsier2-7B shows a +8.6\% performance advantage over GPT-4o and +24.9\% over Gemini-1.5-Pro. Tarsier2-7B also sets new state-of-the-art results across 15 public benchmarks, spanning tasks such as video question-answering, video grounding, hallucination test, and embodied question-answering, demonstrating its versatility as a robust generalist vision-language model.

Privacy Preservation in MIMO-OFDM Localization Systems: A Beamforming Approach

We investigate an uplink MIMO-OFDM localization scenario where a legitimate base station (BS) aims to localize a user equipment (UE) using pilot signals transmitted by the UE, while an unauthorized BS attempts to localize the UE by eavesdropping on these pilots, posing a risk to the UE's location privacy. To enhance legitimate localization performance while protecting the UE's privacy, we formulate an optimization problem regarding the beamformers at the UE, aiming to minimize the Cram\'er-Rao bound (CRB) for legitimate localization while constraining the CRB for unauthorized localization above a threshold. A penalty dual decomposition optimization framework is employed to solve the problem, leading to a novel beamforming approach for location privacy preservation. Numerical results confirm the effectiveness of the proposed approach and demonstrate its superiority over existing benchmarks.

6DMA-Aided Hybrid Beamforming with Joint Antenna Position and Orientation Optimization

This paper studies a sub-connected six-dimensional movable antenna (6DMA)-aided multi-user communication system. In this system, each sub-array is connected to a dedicated radio frequency chain and collectively moves and rotates as a unit within specific local regions. The movement and rotation capabilities of 6DMAs enhance design flexibility, facilitating the capture of spatial variations for improved communication performance. To fully characterize the effect of antenna position and orientation on wireless channels between the base station (BS) and users, we develop a field-response-based 6DMA channel model to account for the antenna radiation pattern and polarization. We then maximize the sum rate of multiple users, by jointly optimizing the digital and unit-modulus analog beamformers given the transmit power budget as well as the positions and orientations of sub-arrays within given movable and rotatable ranges at the BS. Due to the highly coupled variables, the formulated optimization problem is non-convex and thus challenging to solve. We develop a fractional programming-aided alternating optimization framework that integrates the Lagrange multiplier method, manifold optimization, and gradient descent to solve the problem. Numerical results demonstrate that the proposed 6DMA-aided sub-connected structure achieves a substantial sum-rate improvement over various benchmark schemes with less flexibility in antenna movement and can even outperform fully-digital beamforming systems that employ antenna position or orientation adjustments only. The results also highlight the necessity of considering antenna polarization for optimally adjusting antenna orientation.

OpenAI o1 System Card

The o1 model series is trained with large-scale reinforcement learning to reason using chain of thought. These advanced reasoning capabilities provide new avenues for improving the safety and robustness of our models. In particular, our models can reason about our safety policies in context when responding to potentially unsafe prompts, through deliberative alignment. This leads to state-of-the-art performance on certain benchmarks for risks such as generating illicit advice, choosing stereotyped responses, and succumbing to known jailbreaks. Training models to incorporate a chain of thought before answering has the potential to unlock substantial benefits, while also increasing potential risks that stem from heightened intelligence. Our results underscore the need for building robust alignment methods, extensively stress-testing their efficacy, and maintaining meticulous risk management protocols. This report outlines the safety work carried out for the OpenAI o1 and OpenAI o1-mini models, including safety evaluations, external red teaming, and Preparedness Framework evaluations.

EGP3D: Edge-guided Geometric Preserving 3D Point Cloud Super-resolution for RGB-D camera

Point clouds or depth images captured by current RGB-D cameras often suffer from low resolution, rendering them insufficient for applications such as 3D reconstruction and robots. Existing point cloud super-resolution (PCSR) methods are either constrained by geometric artifacts or lack attention to edge details. To address these issues, we propose an edge-guided geometric-preserving 3D point cloud super-resolution (EGP3D) method tailored for RGB-D cameras. Our approach innovatively optimizes the point cloud with an edge constraint on a projected 2D space, thereby ensuring high-quality edge preservation in the 3D PCSR task. To tackle geometric optimization challenges in super-resolution point clouds, particularly preserving edge shapes and smoothness, we introduce a multi-faceted loss function that simultaneously optimizes the Chamfer distance, Hausdorff distance, and gradient smoothness. Existing datasets used for point cloud upsampling are predominantly synthetic and inadequately represent real-world scenarios, neglecting noise and stray light effects. To address the scarcity of realistic RGB-D data for PCSR tasks, we built a dataset that captures real-world noise and stray-light effects, offering a more accurate representation of authentic environments. Validated through simulations and real-world experiments, the proposed method exhibited superior performance in preserving edge clarity and geometric details.

Integrated Positioning and Communication via LEO Satellites: Opportunities and Challenges

Low Earth orbit (LEO) satellites, as a prominent technology in the 6G non-terrestrial network, offer both positioning and communication capabilities. While these two applications have each been extensively studied and have achieved substantial progress in recent years, the potential synergistic benefits of integrating them remain an underexplored yet promising avenue. This article comprehensively analyzes the integrated positioning and communication (IPAC) systems on LEO satellites. By leveraging the distinct characteristics of LEO satellites, we examine how communication systems can enhance positioning accuracy and, conversely, how positioning information can be exploited to improve communication efficiency. In particular, we present two case studies to illustrate the potential of such integration. Finally, several key open research challenges in the LEO-based IPAC systems are discussed.

StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identification

Existing large vision-language models (LVLMs) are largely limited to processing short, seconds-long videos and struggle with generating coherent descriptions for extended video spanning minutes or more. Long video description introduces new challenges, such as plot-level consistency across descriptions. To address these, we figure out audio-visual character identification, matching character names to each dialogue, as a key factor. We propose StoryTeller, a system for generating dense descriptions of long videos, incorporating both low-level visual concepts and high-level plot information. StoryTeller uses a multimodal large language model that integrates visual, audio, and text modalities to perform audio-visual character identification on minute-long video clips. The results are then fed into a LVLM to enhance consistency of video description. We validate our approach on movie description tasks and introduce MovieStory101, a dataset with dense descriptions for three-minute movie clips. To evaluate long video descriptions, we create MovieQA, a large set of multiple-choice questions for the MovieStory101 test set. We assess descriptions by inputting them into GPT-4 to answer these questions, using accuracy as an automatic evaluation metric. Experiments show that StoryTeller outperforms all open and closed-source baselines on MovieQA, achieving 9.5% higher accuracy than the strongest baseline, Gemini-1.5-pro, and demonstrating a +15.56% advantage in human side-by-side evaluations. Additionally, incorporating audio-visual character identification from StoryTeller improves the performance of all video description models, with Gemini-1.5-pro and GPT-4o showing relative improvement of 5.5% and 13.0%, respectively, in accuracy on MovieQA.

Physics-informed Shadowgraph Network: An End-to-end Density Field Reconstruction Method

This study presents a novel approach for quantificationally reconstructing density fields from shadowgraph images using physics-informed neural networks

Physics informed Shadowgraph Density Field Reconstruction

This study presents a novel approach to reconstructing density fields from shadowgraph images using a physics-informed framework. By integrating traditional shadowgraph imaging techniques with physics-informed neural networks (PINNs), we effectively capture refractive index variations within complex flow fields. The proposed method addresses the inherent challenges of shadowgraphy, such as noise and limited spatial resolution, enabling accurate visualization of fluid dynamics. Experimental results demonstrate the feasibility and robustness of our approach, with significant agreement observed between the reconstructed density fields and experimental measurements. This research contributes to the advancement of non-intrusive diagnostic techniques in fluid mechanics and enhances our understanding of flow structures in various applications.