Object re-identification method is made up of backbone network, feature aggregation, and loss function. However, most backbone networks lack a special mechanism to handle rich scale variations and mine discriminative feature representations. In this paper, we firstly design a hierarchical similarity graph module (HSGM) to reduce the conflict of backbone and re-identification networks. The designed HSGM builds a rich hierarchical graph to mine the mapping relationships between global-local and local-local. Secondly, we divide the feature map along with the spatial and channel directions in each hierarchical graph. The HSGM applies the spatial features and channel features extracted from different locations as nodes, respectively, and utilizes the similarity scores between nodes to construct spatial and channel similarity graphs. During the learning process of HSGM, we utilize a learnable parameter to re-optimize the importance of each position, as well as evaluate the correlation between different nodes. Thirdly, we develop a novel hierarchical similarity graph network (HSGNet) by embedding the HSGM in the backbone network. Furthermore, HSGM can be easily embedded into backbone networks of any depth to improve object re-identification ability. Finally, extensive experiments on three large-scale object datasets demonstrate that the proposed HSGNet is superior to state-of-the-art object re-identification approaches.