Reinforcement learning for process design

We develop reinforcement learning agents that learn to design chemical processes. The agents represent flowsheets as graphs, interact with process simulators, and iteratively learn which unit operations and design variables lead to economically viable processes.

Challenge

Process synthesis, deciding which unit operations to use, how to connect them, and how to operate them, is traditionally solved with heuristics, engineering experience, or superstructure optimization. Superstructure optimization requires the engineer to specify all alternatives in advance and quickly becomes computationally expensive. As the chemical industry moves towards greener processes, there is a need for design methods that explore large design spaces efficiently and reuse what they have learned.

Our approach

We propose a reinforcement learning algorithm for chemical process design based on an actor-critic logic. Chemical processes are represented as graphs, and graph neural networks inside the agent architecture process the states and make decisions. A hierarchical and hybrid decision-making process generates flowsheets: unit operations are placed iteratively as discrete decisions, and the corresponding design variables are selected as continuous decisions. In an illustrative case study with equilibrium reactions, azeotropic separation, and recycles, the agent learned quickly in discrete, continuous, and hybrid action spaces.

Follow-up work accelerates learning through transfer learning, for example by pre-training agents on fast, low-fidelity simulations before training them with rigorous process simulators, and reviews the state of deep reinforcement learning for process design. In the MSCA project CoAUTHOR, context-aware reinforcement learning combines physical-chemistry information, expert knowledge, and data to design new processes and retrofit existing ones.

Partners & funding

  • NWO Veni grant to Artur M. Schweidtmann on reinforcement learning for chemical process design (announcement, 2023).
  • Marie Skłodowska-Curie Actions (MSCA) Postdoctoral Fellowship, project CoAUTHOR (Ulderico Di Caprio).

Key publications

  • Gao, Q., Yang, H., Theisen, M. F., & Schweidtmann, A. M. (2025). Accelerating process synthesis with reinforcement learning: Transfer learning from multi-fidelity simulations and variational autoencoders. Computers & Chemical Engineering, 201, 109192. doi:10.1016/j.compchemeng.2025.109192
  • Gao, Q., & Schweidtmann, A. M. (2024). Deep reinforcement learning for process design: Review and perspective. Current Opinion in Chemical Engineering, 44, 101012. doi:10.1016/j.coche.2024.101012 · Details
  • Gao, Q., Yang, H., Shanbhag, S. M., & Schweidtmann, A. M. (2023). Transfer learning for process design with reinforcement learning. Computer Aided Chemical Engineering, 52, 2005–2010. doi:10.1016/B978-0-443-15274-0.50319-X · Details
  • Stops, L., Leenhouts, R., Gao, Q., & Schweidtmann, A. M. (2023). Flowsheet generation through hierarchical reinforcement learning and graph neural networks. AIChE Journal, 69(1), e17938. doi:10.1002/aic.17938 · Details
← All projects