Although soft robots show safer interactions with their environment than traditional robots, soft mechanisms and actuators still have significant potential for damage or degradation particularly during unmodeled contact. This article introduces a feedback strategy for safe soft actuator operation during control of a soft robot. To do so, a supervisory controller monitors actuator state and dynamically saturates control inputs to avoid conditions that could lead to physical damage. We prove that, under certain conditions, the supervisory controller is stable and verifiably safe. We then demonstrate completely onboard operation of the supervisory controller using a soft thermally-actuated robot limb with embedded shape memory alloy (SMA) actuators and sensing. Tests performed with the supervisor verify its theoretical properties and show stabilization of the robot limb's pose in free space. Finally, experiments show that our approach prevents overheating during contact (including environmental constraints and human contact) or when infeasible motions are commanded. This supervisory controller, and its ability to be executed with completely onboard sensing, has the potential to make soft robot actuators reliable enough for practical use.