Network Pruning Spaces

Network pruning techniques, including weight pruning and filter pruning, reveal that most state-of-the-art neural networks can be accelerated without a significant performance drop. This work focuses on filter pruning which enables accelerated inference with any off-the-shelf deep learning library and hardware. We propose the concept of \emph{network pruning spaces} that parametrize populations of subnetwork architectures. Based on this concept, we explore the structure aspect of subnetworks that result in minimal loss of accuracy in different pruning regimes and arrive at a series of observations by comparing subnetwork distributions. We conjecture through empirical studies that there exists an optimal FLOPs-to-parameter-bucket ratio related to the design of original network in a pruning regime. Statistically, the structure of a winning subnetwork guarantees an approximately optimal ratio in this regime. Upon our conjectures, we further refine the initial pruning space to reduce the cost of searching a good subnetwork architecture. Our experimental results on ImageNet show that the subnetwork we found is superior to those from the state-of-the-art pruning methods under comparable FLOPs.

Take More Positives: A Contrastive Learning Framework for Unsupervised Person Re-Identification

Exploring the relationship between examples without manual annotations is a core problem in the field of unsupervised person re-identification (re-ID). In the unsupervised scenario, no ground truth is provided for bringing instances of the same identity closer and spreading samples of different identities apart. In this paper, we introduce a contrastive learning framework for unsupervised person re-ID, which we call Take More Positives (TMP). In an iterative manner, TMP generates pseudo-labels by clustering samples, and updates itself with such pseudo-labels and the proposed contrastive loss. By considering more positive examples, the framework of TMP outperforms the state-of-the-art methods for unsupervised person re-ID. On the Market-1501 benchmark, TMP achieves 88.3% Rank-1 accuracy and 70.4% mean average precision. Our code will be made publicly available.