Feeding power amplifiers (PAs) with constant envelope (CE) signals is an effective way to reduce the power consumption in massive multiple-input-multiple-output (MIMO) systems. The nonlinear distortion caused by CE signaling must be mitigated by means of signal processing to improve the achievable sum rates. To this purpose, many linear and nonlinear precoding techniques have been developed for the CE MIMO downlink. The vast majority of these CE precoding techniques do not include a power allocation scheme, which is indispensable to achieve adequate performances in the downlink with channel gain imbalances between users. In this paper, we present two algorithms to produce a power allocation scheme for regularized zero-forcing (RZF) precoding in CE MIMO downlink. Both techniques are based on transforming the CE quantized MIMO downlink to an approximately equivalent system of parallel single-input-single-output (SISO) channels. The first technique is proven to solve the sum rate maximization problem in the approximate system optimally, whereas the second technique obtains the local maximum with lower complexity. We also extend another state-of-the-art quantization aware sum rate maximization algorithm with linear precoding to the CE downlink. Numerical results illustrate significant gains for the performance of the RZF precoder when the CE quantization is taken into account in a power allocation. Another key numerical result is that the proposed RZF techniques achieve almost the identical performance so that the one with lower computational complexity is chosen as the main method. Results also show that the proposed RZF precoding schemes perform at least as good as the state-of-the-art method with an advantage that the main RZF method has significantly lower computational complexity than the state-of-the-art.