Humans often work closely together and relay a wealth of information through physical interaction. Robots, on the other hand, have not yet been developed to work similarly closely with humans, and to effectively convey information when engaging in physical human-robot interaction (pHRI). This currently limits the potential of physical human-robot collaboration to solve real-world problems. This paper investigates the question of how to establish clear and intuitive robot-to-human communication, while ensuring human comfort during pHRI. We approach this question from the perspective of a leader-follower scenario, in which a full-body humanoid robot leads a slow waltz dance by signaling the next steps to a human partner. This is achieved through the development of a whole-body control framework combining admittance and impedance control, which allows for different communication modalities including haptic, visual, and audio signals. Participant experiments allowed to validate the performance of the controller, and to understand what types of communication work better in terms of effectiveness and comfort during robot-led pHRI.