A knowledge-data fusion framework accelerates deep reinforcement learning for real-time control of urban drainage systems.

Deep reinforcement learning (DRL) has been applied to real-time control (RTC) of urban drainage systems (UDSs), with impressive performance and efficiency in reducing urban flooding and combined sewer overflows (CSO). However, for complex UDSs, learning from scratch is time-consuming and difficult to converge. In this study we construct a knowledge-data fusion DRL training framework to integrate different types of UDS engineering experience and knowledge into DRL. We first convert knowledge of various types, via simulation, into knowledge-carrying data, which are subsequently used to pre-train a DRL agent using a newly designed offline supervised learning method. The pre-trained model is then fine-tuned using the well-established reinforcement learning procedure, achieving flooding and CSO mitigation comparable to that of fully-trained DRL but with nearly 90 % reduction in the training time, when applied to a benchmark UDS model. The quality of the knowledge used in pre-training influences the DRL performance, making its quality control critical to ensure RTC performance. Meanwhile, integrating diverse types of knowledge can effectively enhance the performance of the framework.