Improving Generative Pre-Training: An In-depth Study of Masked Image Modeling and Denoising Models

In this work, we dive deep into the impact of additive noise in pre-training deep networks. While various methods have attempted to use additive noise inspired by the success of latent denoising diffusion models, when used in combination with masked image modeling, their gains have been marginal when it comes to recognition tasks. We thus investigate why this would be the case, in an attempt to find effective ways to combine the two ideas. Specifically, we find three critical conditions: corruption and restoration must be applied within the encoder, noise must be introduced in the feature space, and an explicit disentanglement between noised and masked tokens is necessary. By implementing these findings, we demonstrate improved pre-training performance for a wide range of recognition tasks, including those that require fine-grained, high-frequency information to solve.

Spectrum Sharing Between Low Earth Orbit Satellite and Terrestrial Networks: A Stochastic Geometry Perspective Analysis

Low Earth orbit (LEO) satellite networks with mega constellations have the potential to provide 5G and beyond services ubiquitously. However, these networks may introduce mutual interference to both satellite and terrestrial networks, particularly when sharing spectrum resources. In this paper, we present a system-level performance analysis to address these interference issues using the tool of stochastic geometry. We model the spatial distributions of satellites, satellite users, terrestrial base stations (BSs), and terrestrial users using independent Poisson point processes on the surfaces of concentric spheres. Under these spatial models, we derive analytical expressions for the ergodic spectral efficiency of uplink (UL) and downlink (DL) satellite networks when they share spectrum with both UL and DL terrestrial networks. These derived ergodic expressions capture comprehensive network parameters, including the densities of satellite and terrestrial networks, the path-loss exponent, and fading. From our analysis, we determine the conditions under which spectrum sharing with UL terrestrial networks is advantageous for both UL and DL satellite networks. Our key finding is that the optimal spectrum sharing configuration among the four possible configurations depends on the density ratio between terrestrial BSs and users, providing a design guideline for spectrum management. Simulation results confirm the accuracy of our derived expressions.

Ergodic Secrecy Rate Analysis for LEO Satellite Downlink Networks

Satellite networks are recognized as an effective solution to ensure seamless connectivity worldwide, catering to a diverse range of applications. However, the broad coverage and broadcasting nature of satellite networks also expose them to security challenges. Despite these challenges, there is a lack of analytical understanding addressing the secrecy performance of these networks. This paper presents a secrecy rate analysis for downlink low Earth orbit (LEO) satellite networks by modeling the spatial distribution of satellites, users, and potential eavesdroppers as homogeneous Poisson point processes on concentric spheres. Specifically, we provide an analytical expression for the ergodic secrecy rate of a typical downlink user in terms of the satellite network parameters, fading parameters, and path-loss exponent. Simulation results show the exactness of the provided expressions and we find that optimal satellite altitude increases with eavesdropper density.

Coverage Analysis of Dynamic Coordinated Beamforming for LEO Satellite Downlink Networks

In this paper, we investigate the coverage performance of downlink satellite networks employing dynamic coordinated beamforming. Our approach involves modeling the spatial arrangement of satellites and users using Poisson point processes situated on concentric spheres. We derive analytical expressions for the coverage probability, which take into account the in-cluster geometry of the coordinated satellite set. These expressions are formulated in terms of various parameters, including the number of antennas per satellite, satellite density, fading characteristics, and path-loss exponent. To offer a more intuitive understanding, we also develop an approximation for the coverage probability. Furthermore, by considering the distribution of normalized distances, we derive the spatially averaged coverage probability, thereby validating the advantages of coordinated beamforming from a spatial average perspective. Our primary finding is that dynamic coordinated beamforming significantly improves coverage compared to the absence of satellite coordination, in direct proportion to the number of antennas on each satellite. Moreover, we observe that the optimal cluster size, which maximizes the ergodic spectral efficiency, increases with higher satellite density, provided that the number of antennas on the satellites is sufficiently large. Our findings are corroborated by simulation results, confirming the accuracy of the derived expressions.

Scalable Link Scheduling with Limited CSI for Dense Device-to-Device Networks

This paper addresses cooperative link scheduling problems for base station (BS) aided device-to-device (D2D) communications using limited channel state information (CSI) at BS. We first derive the analytical form of ergodic sum-spectral efficiency as a function of network parameters, assuming statistical CSI at the BS. However, the optimal link scheduling, which maximizes the ergodic sum-spectral efficiency, becomes computationally infeasible when network density increases. To overcome this challenge, we present a low-complexity link scheduling algorithm that divides the D2D network into sub-networks and identifies the optimal link scheduling strategy per sub-network. Furthermore, we consider the scenario when the statistical CSI is not available to the BS. In such cases, we propose a quasi-optimal scalable link scheduling algorithm that utilizes one-bit feedback information from D2D receivers. The algorithm clusters the links and applies the UCB algorithm per cluster using the collected one-bit feedback information. We highlight that even with reduced scheduling complexity, the proposed algorithm identifies a link scheduling action that ensures optimality within a constant throughput gap. We also demonstrate through simulations that the proposed algorithm achieves higher sum-spectral efficiency than the existing link scheduling algorithms, even without explicit CSI or network parameters knowledge.

CoVA: Exploiting Compressed-Domain Analysis to Accelerate Video Analytics

Modern retrospective analytics systems leverage cascade architecture to mitigate bottleneck for computing deep neural networks (DNNs). However, the existing cascades suffer two limitations: (1) decoding bottleneck is either neglected or circumvented, paying significant compute and storage cost for pre-processing; and (2) the systems are specialized for temporal queries and lack spatial query support. This paper presents CoVA, a novel cascade architecture that splits the cascade computation between compressed domain and pixel domain to address the decoding bottleneck, supporting both temporal and spatial queries. CoVA cascades analysis into three major stages where the first two stages are performed in compressed domain while the last one in pixel domain. First, CoVA detects occurrences of moving objects (called blobs) over a set of compressed frames (called tracks). Then, using the track results, CoVA prudently selects a minimal set of frames to obtain the label information and only decode them to compute the full DNNs, alleviating the decoding bottleneck. Lastly, CoVA associates tracks with labels to produce the final analysis results on which users can process both temporal and spatial queries. Our experiments demonstrate that CoVA offers 4.8x throughput improvement over modern cascade systems, while imposing modest accuracy loss.