The growing need for sustainable energy solutions has driven the integration of digitalized buildings into the power grid, utilizing Internet-of-Things technology to optimize building performance and energy efficiency. However, incorporating IoT point data within deep-learning frameworks for energy management presents a complex challenge, predominantly due to the inherent data heterogeneity. This paper comprehensively analyzes the multifaceted heterogeneity present in real-world building IoT data streams. We meticulously dissect the heterogeneity across multiple dimensions, encompassing ontology, etiology, temporal irregularity, spatial diversity, and their combined effects on the IoT point data distribution. In addition, experiments using state-of-the-art forecasting models are conducted to evaluate their impacts on the performance of deep-learning models for forecasting tasks. By charting the diversity along these dimensions, we illustrate the challenges and delineate pathways for future research to leverage this heterogeneity as a resource rather than a roadblock. This exploration sets the stage for advancing the predictive abilities of deep-learning algorithms and catalyzing the evolution of intelligent energy-efficient buildings.