Rayleigh-B\'enard convection (RBC) is a recurrent phenomenon in several industrial and geoscience flows and a well-studied system from a fundamental fluid-mechanics viewpoint. However, controlling RBC, for example by modulating the spatial distribution of the bottom-plate heating in the canonical RBC configuration, remains a challenging topic for classical control-theory methods. In the present work, we apply deep reinforcement learning (DRL) for controlling RBC. We show that effective RBC control can be obtained by leveraging invariant multi-agent reinforcement learning (MARL), which takes advantage of the locality and translational invariance inherent to RBC flows inside wide channels. The MARL framework applied to RBC allows for an increase in the number of control segments without encountering the curse of dimensionality that would result from a naive increase in the DRL action-size dimension. This is made possible by the MARL ability for re-using the knowledge generated in different parts of the RBC domain. We show in a case study that MARL DRL is able to discover an advanced control strategy that destabilizes the spontaneous RBC double-cell pattern, changes the topology of RBC by coalescing adjacent convection cells, and actively controls the resulting coalesced cell to bring it to a new stable configuration. This modified flow configuration results in reduced convective heat transfer, which is beneficial in several industrial processes. Therefore, our work both shows the potential of MARL DRL for controlling large RBC systems, as well as demonstrates the possibility for DRL to discover strategies that move the RBC configuration between different topological configurations, yielding desirable heat-transfer characteristics. These results are useful for both gaining further understanding of the intrinsic properties of RBC, as well as for developing industrial applications.
翻译:瑞利-贝纳德对流(RBC)是工业和地球科学中的重复现象,也是从基本流体力学角度来看的研究系统。然而,对于经典控制理论方法而言,例如通过调节典型RBC配置中底板加热的空间分布进行RBC控制仍然是一个具有挑战性的课题。在本研究中,我们应用深度强化学习(DRL)来控制RBC。我们展示了利用不变多智能体强化学习(MARL)来控制RBC可以获得有效的RBC控制,因为该方法充分利用了RBC流体的局部性和平移不变性。应用于RBC的MARL框架允许增加控制区域的数量,而不会遇到因天真地增加DRL动作尺寸维度而导致代价呈指数增加的问题。这是由于MARL具有在RBC领域中不同部分生成的知识被再利用的能力。在一个案例研究中,我们展示了MARL DRL能够发现一个先进的控制策略,该策略破坏了自发的RBC双细胞模式,通过合并相邻的对流细胞改变了RBC的拓扑结构,并积极控制所得到的合并细胞将其带到一个新的稳定配置。这个改变后的流体配置导致了降低的对流传热,这对于几个工业过程来说是有益的。因此,我们的工作既展示了MARL DRL控制大型RBC系统的潜力,也证明了DRL可以发现将RBC配置在不同拓扑结构之间移动的策略,以产生理想的传热特性。这些结果对于进一步了解RBC的内在特性以及开发工业应用都是有用的。