Self-evolving networks (SENs) are emerging technologies that dynamically and autonomously adapt and optimize their performance and behaviour based on changing conditions and evolving requirements. With the advent of fifth-generation (5G) wireless technologies and the resurgence of machine learning, SENs are expected to become a critical component of future wireless networks. In particular, integrated vertical heterogeneous network (VHetNet) architectures, which enable dynamic, three-dimensional (3D), and agile topologies, are likely to form a key foundation for SENs. However, the distributed multi-level computational and communication structure and the fully dynamic nature of self-evolving integrated VHetNets (SEI-VHetNets) necessitate the deployment of an enhanced distributed learning and computing mechanism to enable full integration and coordination. To address this need, we propose a novel learning technique, multi-tier hierarchical federated learning (MT-HFL), based on hierarchical federated learning (HFL) that enables full integration and coordination across vertical tiers. Through MT-HFL, SEI-VHetNets can learn and adapt to dynamic network conditions, optimize resource allocation, and enhance user experience in a real-time, scalable, and accurate manner while preserving user privacy. This paper presents the key characteristics and challenges of SEI-VHetNets and discusses how MT-HFL addresses them. We also discuss potential use cases and present a case study demonstrating the advantages of MT-HFL over conventional terrestrial HFL approaches.