Today\'s world of space\'s primary concern is the uncontrolled growth of space debris and its probability of collision with spacecraft, particularly in the low earth orbit (LEO) regions. This paper is aimed to design an optimized micro-propulsion system, Cold Gas Thruster, to de-orbit the PSLV debris from 668km to 250 km height after capturing process. The propulsion system mainly consists of a storage tank, pipes, control valves, and a convergent-divergent nozzle. The paper gives an idea of the design of each component based on a continuous iterative process until the design thrust requirements are met. All the components are designed in the CATIA V5, and the structural analysis is done in the ANSYS tool for each component where our cylinder tank can withstand the high hoop stress generated on its wall of it. And flow analysis is done by using the K-$\epsilon$ turbulence model for the CD nozzle, which provides the required thrust to de-orbit PSLV from a higher orbit to a lower orbit, after which the air drag will be enough to bring back to earth\'s atmosphere and burn it. Hohmann\'s orbit transfer method has been used to de-orbit the PSLV space debris, and it has been simulated by STK tools. And the result shows that our optimized designed thruster generates enough thrust to de-orbit the PSLV debris to a very low orbit.