EMS-env: A Reinforcement Learning Framework for Residential Energy Efficiency Recommendations

Abstract

Personalized device-level energy consumption recommendations towards energy efficiency can have a notable impact both on electricity bills and on the overall energy supply-demand balance. End-user behavior regarding device activation is usually unknown a priori, thus giving rise to a highly dynamic environment. Hence, Reinforcement Learning (RL) can be utilized for device scheduling and consumption recommendations since it constitutes an Artificial Intelligence (AI) framework that learns a control policy in a dynamic environment through trying actions and observing incurred rewards. However, existing works on energy consumption recommendations do not explicitly take into account human feedback and preferences regarding the issued recommendations, and they train a single RL agent per device, hence missing the human behavior interdependencies in using different devices. In addition, a flexible open-source RL environment model that integrates user behavior in a Markov Decision Process (MDP) model is missing. In this paper, we propose an MDP-driven RL framework for energy efficiency recommendations that jointly learns the user’s behavior for multiple devices. The proposed model is wrapped as an open-source customizable Gymnasium environment, named EMS-env, for multi-device energy efficiency recommendations. EMS-env can simulate different types of consumer behavior profiles based on the MDP model and supports different device types as well as user feedback. Validation experiments demonstrate the framework’s merits and hyperparameters for diverse use cases in terms of user simulation models and RL training policies, resulting in decreased energy costs while maintaining end-user satisfaction.

Odyssefs Diamantopoulos Pantaleon

PhD Student

I am academically interested in creating smart AI agents that are able to perform and adapt in dynamic environments. I am mainly utilizing Reinforcement Learning, Game Theory and Multimodality to achieve my goals.