The recent successes of deep learning and deep reinforcement learning have firmly established their statuses as state-of-the-art artificial learning techniques. However, longstanding drawbacks of these approaches, such as their poor sample efficiencies and limited generalization capabilities point to a need for re-thinking the way such systems are designed and deployed. In this paper, we emphasize how the use of these learning systems, in conjunction with a specific variation of evolutionary algorithms could lead to the emergence of unique characteristics such as the automated acquisition of a variety of desirable behaviors and useful sets of behavior priors. This could pave the way for learning to occur in a generalized and continual manner, with minimal interactions with the environment. We discuss the anticipated improvements from such neuro-evolutionary frameworks, along with the associated challenges, as well as its potential for application to a number of research areas.