Abstract: Aiming at the dynamic stochastic inventory routing problem with periodic fluctuation of demand, a novel simulation optimization approach based on deep reinforcement learning is proposed to achieving periodic steady strategy. Firstly a dynamic combinatorial optimization model is constructed. Then, by deep reinforcement learning and setting heuristic rules, the replenishment nodes set selection and the replenishment batch allocation weights in each period are determined. The simulation experimental results show that the proposed method can improve the average profit of a cycle by about 2.7% and 3.9% in low fluctuating demand case and by about 8.2% and 7.1% in high fluctuating demand case compared with the two solution methods in the existing literature, and the cycle service level can be stabilized within a small fluctuation range under different demand fluctuation environments.

Key words: inventory routing problem, heuristic rules, deep Q-learning, dynamics, periodic steady strategy