Kolmogorov Arnold Networks (KANs) as surrogate models for global process optimization

Abstract

Surrogate models are widely used to improve the tractability of process optimization. Some commonly used surrogate models are obtained via machine learning such as multi-layer perceptrons (MLPs), Gaussian processes, and decision trees. Recently, a new class of machine learning models named Kolmogorov Arnold Networks (KANs) have been proposed. Broadly, KANs are similar to MLPs, yet they are based on the Kolmogorov representation theorem instead of the universal approximation theorem for the MLPs. Compared to MLPs, it was reported that KANs require significantly fewer parameters to approximate a given input/output relationship. One of the bottlenecks preventing the embedding of MLPs into optimization formulations is that MLPs with a high number of parameters (larger width or depth) are more challenging to globally optimize. We investigate whether the parameter efficiency of KANs relative to MLPs can be translated to computational benefits when embedding them into optimization problems and solving them to global optimality. We propose a Mixed-Integer Nonlinear Programming (MINLP) formulation of a KAN and test its effectiveness using a case study on the optimization of an auto-thermal reforming process. We observe that KANs are a promising alternative as surrogate models particularly for cases with low number of inputs or outputs. For surrogate models with higher inputs or outputs, stronger formulations must be developed to improve global optimization of models with KANs embedded.