Real-time control of urban drainage system for flood and combined sewer overflow mitigation with a novel recurrent deep reinforcement learning framework.

Urban drainage systems (UDS) are facing increasing challenges such as flooding and combined sewer overflows (CSOs) due to urbanization, aging infrastructure, and intensifying rainfall patterns. Deep reinforcement learning (DRL) has gained attention for enabling more intelligent and adaptive control strategies in UDS management, as it allows agents to learn optimal policies through interaction with dynamic environments. However, conventional DRL methods assume the next state of environment depends solely on the current state and action. In real-time UDS control, the history of environmental states can influence the current decision-making significantly, and besides, monitoring systems deployed in field environments are susceptible to external disturbances, leading to measurement errors and missing information. Therefore, this study proposed a Safe Deep Recurrent Q-Network (DRQN) framework for real-time control of UDS to mitigate flooding and CSOs in scenarios of limited sensor coverage. The framework is implemented in a Storm Water Management Model of a combined sewer system located in Eastern China. DRQN outperforms Deep Q-Network (DQN) and rule-based heuristic control (RBHC) in reducing flooding and CSOs without incorporating critical flow and rainfall monitoring data. The robustness of DRQN control strategy is further validated under projected climate change rainfall scenarios, demonstrating its effectiveness and reliability in extreme rainfall events. This study also uses decision tree models to improve the interpretability of DRQN control logic. The framework offers broad applicability across different drainage systems by allowing the customization of safety constraints and control objectives to align with site-specific operational and regulatory requirements. It provides a robust solution for smart drainage control under conditions with limited sensor deployment and drainage system observability.