Shape decomposition is a fundamental problem in geometry processing where an arbitrary object is regarded as an arrangement of simple primitives or semantic components. The application of 3D shape decomposition in the context of image segmentation, however, is not well-studied. In this paper, we develop a shape decomposition algorithm called cylindrical shape decomposition (CSD) to be applied for the segmentation of tubular structures in large-scale 3D images. CSD starts by partitioning the curve skeleton of a tubular object into maximal-length sub-skeletons, minimizing an orientation objective. Each sub-skeleton corresponds to a semantic component. To determine boundaries between the semantic components, CSD searches for critical points where the object cross-section substantially changes. CSD then cuts the object at critical points and assigns the same label to those object parts which are along the same sub-skeleton, defining a semantic tubular component. CSD further rectify/reconstructs these semantic components using generalized cylinders. We demonstrate the application of CSD in the segmentation of large-scale 3D electron microscopy image datasets of myelinated axons, the decomposition of vascular networks, and synthetic objects. We also compare CSD to other state-of-the-art decomposition techniques in these applications. These experiments indicate that CSD outperforms other decomposition techniques and achieves a promising performance.