This paper provides a collision avoidance perspective to maritime autonomy, in the shift towards Maritime Autonomous Surface Ships (MASS). In particular, the paper presents the developments related to the Greenhoper, a Danish autonomous harbour bus. The collision and grounding avoidance scheme, called the Short Horizon Planner (SHP), is described and discussed in detail. Furthermore, the required autonomy stack for facilitating safe and rule-compliant collision avoidance is presented. The inherent difficulties relating to adhering to the COLREGs are outlined, highlighting some of the operational constraints and challenges within the space of autonomous ferries and harbour buses. Finally, collision and grounding avoidance is demonstrated using a simulation of the entire proposed autonomy stack.