This paper introduces low-complexity beamforming algorithms for multi-user multiple-input multiple-output (MU-MIMO) systems to minimize inter-user interference and enhance spectral efficiency (SE). A Singular-Vector Beamspace Search (SVBS) algorithm is initially presented, wherein all the singular vectors are assessed to determine the most effective beamforming scheme. We then establish a mathematical proof demonstrating that the total inter-user interference of a MU-MIMO beamforming system can be efficiently calculated from the mutual projections of orthonormal singular vectors. Capitalizing on this, we present an Interference-optimized Singular Vector Beamforming (IOSVB) algorithm for optimal singular vector selection. For further reducing the computational burden, we propose a Dimensionality-reduced IOSVB (DR-IOSVB) algorithm by integrating the principal component analysis (PCA). The numerical results demonstrate the superiority of the SVBS algorithm over the existing algorithms, with the IOSVB offering near-identical SE and the DR-IOSVB balancing the performance and computational efficiency. This work establishes a new benchmark for high-performance and low-complexity beamforming in MU-MIMO wireless communication systems.