The MeLOn toolbox: Machine Learning Models for Optimization

Machine Learning models for Optimization (MeLOn) is toolbox that integrates machine-learning models into optimization problems.

MeLOn provides scripts for the training of various machine-learning models and their C++ implementation which can be used in the open-source solver. The machine-learning module git repository currently contains the following models: Artificial neural networks for regression and Gaussian processes for regression (also known as Kriging). Further models are under current development and will be published soon. MeLOn is a submodule of our global solver MAiNGO. Thus, it will be automatically included as a submodule when you download MAiNGO. Then, the developed machine learning models can be used within MAiNGO.

MeLOn provides scripts for the training of various machine-learning models and their C++ implementation which can be used in the open-source solver The machine-learning module git repository currently contains the following models: Artificial neural networks for regression Gaussian processes for regression (also known as Kriging). Further models are under current develpment and will be published soon. MeLOn is a submodule of our global solver MAiNGO. Thus, it will be automatically included as a submodule when you download MAiNGO. Then, the developed machine learning models can be used within MAiNGO.

Open-source code

The open source code for MAiNGO is accessible here.

Applications

It has been shown that a reduced-space formulation used by us is very profitable when optimizing artificial neural networks and Gaussian processes. It allows for an efficient optimization of mechanistic/data-driven models with artificial neural networks embedded, where the artificial neural networks can, e.g., replace complex thermodynamics or describe experimental data.

The proposed method has been used in various applications:

Applications of deterministic global optimization with artificial neural networks embedded

Hybrid modeling of chemical processes and process optimization [Schweidtmann and Mitsos, 2019]
Rational design of ion separation membranes [Rall et al., 2019, Rall et al., 2020, Rall et al., 2020b]
Optimization of energy processes where accurate thermodynamic is learned by neural networks. Applications to organic Rankine cycle optimization [Schweidtmann et al., 2019, Huster et al., 2019], working fluid selection [Huster et al., 2020]
Using of neural networks as a surrogate with a guaranteed accuracy with application to flash models [Schweidtmann et al., 2019]
Scheduling of a compressed air energy storage system where the efficiency map of compressors and turbines is learned by neural networks [Schäfer et al., 2020]

Applications of deterministic global optimization with Gaussian processes embedded

Chance-constrained programming with Gaussian processes [Schweidtmann et al., 2020]
Bayesian optimization with global optimization of the acquisition function [Schweidtmann et al., 2020]

Key publications

Rall, D., Menne, D., Schweidtmann, A. M., Kamp, J., von Kolzenberg, L., Mitsos, A., & Wessling, M. (2019). Rational design of ion separation membranes. Journal of membrane science, 569, 209-219. https://doi.org/10.1016/j.memsci.2018.10.013
Rall, D., Schweidtmann, A. M., Aumeier, B. M., Kamp, J., Karwe, J., Ostendorf, K., Mitsos, A. & Wessling, M. (2020). Simultaneous rational design of ion separation membranes and processes. Journal of Membrane Science, 117860. https://doi.org/10.1016/j.memsci.2020.117860
Rall, D., Schweidtmann, A. M., Kruse, M., Evdochenko, E., Mitsos, A. & Wessling, M. (2020). Multi-scale membrane process optimization with high-fidelity ion transport models through machine learning. Journal of Membrane Science, In Press. https://doi.org/10.1016/j.memsci.2020.117860
Schweidtmann, A. M., & Mitsos, A. (2019). Deterministic global optimization with artificial neural networks embedded. Journal of Optimization Theory and Applications, 180(3), 925-948. https://doi.org/10.1007/s10957-018-1396-0
Schweidtmann, A. M., Huster, W. R., Lüthje, J. T., & Mitsos, A. (2019). Deterministic global process optimization: Accurate (single-species) properties via artificial neural networks. Computers & Chemical Engineering, 121, 67-74. https://doi.org/10.1016/j.compchemeng.2018.10.007
Schweidtmann, A. M., Bongartz, D., Huster, W. R., & Mitsos, A. (2019). Deterministic Global Process Optimization: Flash Calculations via Artificial Neural Networks. In Computer Aided Chemical Engineering (Vol. 46, pp. 937-942). Elsevier. https://doi.org/10.1016/B978-0-12-818634-3.50157-0
Schweidtmann, A. M., Bongartz, D., Grothe, D., Kerkenhoff, T., Lin, X., Najman, J., & Mitsos, A. (2020). Global optimization of Gaussian processes. Submitted. Preprint available on https://arxiv.org/abs/2005.10902.
Schweidtmann, A. M., Weber, J., Wende, C., Netze, L., & Mitsos, A. (2020). Obey validity domains of data-driven models. Submitted. Preprint available on https://arxiv.org/abs/2010.03405.
Huster, W. R., Schweidtmann, A. M., & Mitsos, A. (2019). Impact of accurate working fluid properties on the globally optimal design of an organic Rankine cycle. In Computer Aided Chemical Engineering (Vol. 47, pp. 427-432). Elsevier.https://doi.org/10.1016/B978-0-12-818597-1.50068-0
Huster, W. R., Schweidtmann, A. M., & Mitsos, A. (2020). Working fluid selection for organic rankine cycles via deterministic global optimization of design and operation. Optimization and Engineering, (Vol. 21, pp. 517-536).https://doi.org/10.1007/s11081-019-09454-1
Huster, W. R., Schweidtmann, A. M., & Mitsos, A. (2020). Globally optimal working fluid mixture composition for geothermal power cycles. Energy, (Vol. 212).https://doi.org/10.1016/j.energy.2020.118731
Huster, W. R., Schweidtmann, A. M., & Mitsos, A. (2020). Deterministic global superstructure-based optimization of an organic Rankine cycle. Computers and Chemical Engineering, (Vol. 141).https://doi.org/10.1016/j.compchemeng.2020.106996
Schäfer, P., Schweidtmann, A. M., Lenz, P. H., Markgraf, H. M., & Mitsos, A. (2020). Wavelet-based grid-adaptation for nonlinear scheduling subject to time-variable electricity prices. Computers & Chemical Engineering, 132, 106598. https://doi.org/10.1016/j.compchemeng.2019.106598