In reconfigurable intelligent surface (RIS)-assisted symbiotic radio (SR), an RIS is exploited to assist the primary system and to simultaneously operate as a secondary transmitter by modulating its own information over the incident primary signal from the air. Such an operation is called over-the-air modulation. The existing modulation schemes such as on-off keying and binary phase-shift keying suffer from two problems for joint detection of the primary and secondary signals in RIS-assisted SR, i.e., one is the detection ambiguity problem when the direct link is blocked, and the other is the bit error rate (BER) error-floor problem when the direct link is weak. To address the two problems, we propose a novel modulation scheme by dividing the phase-shift matrix into two parts: one is the assistance beamforming matrix for assisting the primary system and the other is the transmission beamforming matrix for delivering the secondary signal. To optimize the assistance and transmission beamforming matrices, we first introduce an assistance factor that describes the performance requirement of the primary system and then formulate a problem to minimize the BER of the secondary system, while guaranteeing the BER requirement of the primary system controlled by the assistance factor. To solve this non-convex problem, we resort to the successive convex approximation technique to obtain a suboptimal solution. Furthermore, to draw more insights, we propose a low-complexity assistance-transmission beamforming structure by borrowing the idea from the classical maximum ratio transmission and zero forcing techniques. Finally, simulation results reveal an interesting tradeoff between the BER performance of the primary and secondary systems by adjusting the assistance factor.