Agent-based Internet of Things (IoT) applications have recently emerged as applications that can involve sensors, wireless devices, machines and software that can exchange data and be accessed remotely. Such applications have been proposed in several domains including health care, smart cities and agriculture. However, despite their increased adoption, deploying these applications in specific settings has been very challenging because of the complex static and dynamic variability of the physical devices such as sensors and actuators, the software application behavior and the environment in which the application is embedded. In this paper, we propose a modeling approach for IoT analytics based on learning embodied agents (i.e. situated agents). The approach involves: (i) a variability model of IoT embodied agents; (ii) feedback evaluative machine learning; and (iii) reconfiguration of a group of agents in accordance with environmental context. The proposed approach advances the state of the art in that it facilitates the development of Agent-based IoT applications by explicitly capturing their complex and dynamic variabilities and supporting their self-configuration based on an context-aware and machine learning-based approach.