Abstract: The joint optimization problem of energy management and demand response were studied in order to reduce the long-run cost of electricity users equipped with energy storage unit and smart applications, and to increase their benefits meanwhile. The goals were achieved by controlling both the energy storage unit (charging, discharging, or idle) and the load service (access or delay). Based on the random nature of solar photovoltaic, load demand electricity and electricity price, the joint optimization problem was modeled as infinite-horizon Markov decision process model, and Q-learning algorithm was proposed to find the optimal solution. Simulation results show that the joint control increases the user’s long-term income.

Key words: smart grid, management of energy storage, demand response, MDP, reinforcement learning