This paper proposes a novel LiDAR-inertial odometry (LIO), named SR-LIO, based on an improved bundle adjustment (BA) framework. The core of our SR-LIO is a novel sweep reconstruction method, which segments and reconstructs raw input sweeps from spinning LiDAR to obtain reconstructed sweeps with higher frequency. Such method can effectively reduce the time interval for each IMU pre-integration, reducing the IMU pre-integration error and enabling the usage of BA based LIO optimization. In order to make all the states during the period of a reconstructed sweep can be equally optimized, we further propose multi-segment joint LIO optimization, which allows the state of each sweep segment to be constrained from both LiDAR and IMU. Experimental results on three public datasets demonstrate that our SR-LIO outperforms all existing state-of-the-art methods on accuracy, and reducing the IMU pre-integration error via the proposed sweep reconstruction is very importance for the success of a BA based LIO framework. The source code of SR-LIO is publicly available for the development of the community.