In robot-assisted minimally invasive surgery (RMIS), inverse kinematics (IK) must satisfy a remote center of motion (RCM) constraint to prevent tissue damage at the incision point. However, most of existing IK methods do not account for the trade-offs between the RCM constraint and other objectives such as joint limits, task performance and manipulability optimization. This paper presents a novel method for manipulability maximization in constrained IK of surgical robots, which optimizes the robot's dexterity while respecting the RCM constraint and joint limits. Our method uses a hierarchical quadratic programming (HQP) framework that solves a series of quadratic programs with different priority levels. We evaluate our method in simulation on a 6D path tracking task for constrained and unconstrained IK scenarios for redundant kinematic chains. Our results show that our method enhances the manipulability index for all cases, with an important increase of more than 100% when a large number of degrees of freedom are available. The average computation time for solving the IK problems was under 1ms, making it suitable for real-time robot control. Our method offers a novel and effective solution to the constrained IK problem in RMIS applications.