Reconfigurable Intelligent Surface (RIS) is one of the key technologies for the upcoming 6th Generation (6G) communications, which can improve the signal strength at the receivers by adding artificial propagation paths. In the context of Downlink (DL) Multi-User Multiple-Input Multiple-Output (MU-MIMO) communications, designing an appropriate Beamforming (BF) scheme to take full advantage of this reconfigured propagation environment and improve the network capacity is a major challenge. Due to the spatial dimension provided by MIMO systems, independent data streams can be transmitted to multiple users simultaneously on the same radio resources. It is important to note that serving the same subset of users over a period of time may lead to undesired areas where the average Electromagnetic Field Exposure (EMFE) exceeds regulatory limits. To address this challenge, in this paper, we propose a Dual Gradient Descent (Dual-GD)-based Electromagnetic Field (EMF)-aware MU-MIMO BF scheme that aims to optimize the overall capacity under EMFE constraints in RIS-aided 6G cellular networks.