Low-cost autonomous Micro Aerial Vehicles (MAVs) have the potential to help humans by simplifying and speeding up complex tasks that require their interaction with the environment such as construction, package delivery, and search and rescue. These systems, composed of single or multiple vehicles, can be endowed with passive connection mechanisms such as rigid links or cables to perform transportation and manipulation tasks. However, they are inherently complex since they are often underactuated, and evolve on nonlinear manifold configuration spaces. In addition, the complexity of systems with cable-suspended load is further increased by the hybrid dynamics depending on the cables' varying tension conditions. In this paper, we present the first aerial transportation and manipulation simulator incorporating different payloads and passive connection mechanisms with full system dynamics as well as planning and control algorithms. Furthermore, it includes a novel model accounting for the transient hybrid dynamics for aerial systems with cable-suspended load to closely mimic real-world systems. The availability of a flexible and intuitive interface further contributes to its usability and versatility. Comparisons between simulations and real-world experiments with different vehicles' configurations show the fidelity of the simulator results with respect to real-world settings and its benefit for rapid prototyping and transitioning of aerial transportation and manipulation systems to real-world deployment.