With mobile, IoT and sensor devices becoming pervasive in our life and recent advances in Edge Computational Intelligence (e.g., Edge AI/ML), it became evident that the traditional methods for training AI/ML models are becoming obsolete, especially with the growing concerns over privacy and security. This work tries to highlight the key challenges that prohibit Edge AI/ML from seeing wide-range adoption in different sectors, especially for large-scale scenarios. Therefore, we focus on the main challenges acting as adoption barriers for the existing methods and propose a design with a drastic shift from the current ill-suited approaches. The new design is envisioned to be model-centric in which the trained models are treated as a commodity driving the exchange dynamics of collaborative learning in decentralized settings. It is expected that this design will provide a decentralized framework for efficient collaborative learning at scale.