In recent years, it has been shown that the motion of kinematic chains can be estimated using measurements from inertial sensors placed on segments connected by rotational joints. These methods specifically avoid using magnetometer measurements, which are known to cause issues since the magnetic field at the different sensor locations is typically different. They rely on the assumption that the motion of the kinematic chain is sufficiently rich to assure / yield observability of the relative pose. However, a formal investigation of this crucial requirement has not yet been presented and no specific conditions for observability have so far been given. In this work, we present an observability analysis and show that the relative pose of the body segments is indeed observable under a very mild condition on the motion. We support these results by a simulation study, in which we also show the effect of stationary periods in the data and of the amount of excitation on the accuracy of the estimates. We use experimental data from a human gait experiment to show that the excitation level is sufficient for obtaining accurate estimates even when the subject remains stationary for a period of 47 seconds halfway during the experiment.