Remotely controlled mobile robots are important examples of Cyber-Physical Systems (CPSs). Recently, these robots are being deployed in many safety critical applications. Therefore, ensuring their cyber-security is of paramount importance. Different control schemes that have been proposed to secure such systems against sophisticated cyber-attacks require the exchange of secret messages between their smart actuators and the remote controller. Thus, these schemes require pre-shared secret keys, or an established Public Key Infrastructure (PKI) that allows for key agreement. Such cryptographic approaches might not always be suitable for the deployment environments of such remotely mobile robots. To address this problem, in this paper, we consider a control theoretic approach for establishing a secret key between the remotely controlled robot and the networked controller without resorting to traditional cryptographic techniques. Our key agreement scheme leverages a nonlinear unknown input observer and an error correction code mechanism to allow the robot to securely agree on a secret key with its remote controller. To validate the proposed scheme, we implement it using a Khepera-IV differential drive robot and evaluate its efficiency and the additional control cost acquired by it. Our experimental results confirm the effectiveness of the proposed key establishment scheme.