Herewith we would like provide some information on behalf of our partners from the OPTIMAL project introduced in one of our previous posts. More specifically, this post is about the OptiCL package developed by the researchers at OPTIMAL for Mixed-Integer Optimization with Constraint Learning. At AIMMS, we are exploring capabilities such as those provided by the OptiCL package and are looking forward to feedback on this matter from the AIMMS Community. We are especially interested in potential use cases and interests for conducting pilot studies in this area.

The increasing availability of data has led to the rise in data-driven decision making. One area of data-driven decision making that is becoming increasingly popular is constraint learning n1]. Many real-life optimization problems contain one or more objectives and/or constraints for which there are no explicit formulae. In such cases, machine learning models, which are good at determining complex decision boundaries, can be used to learn these constraints. Using constraint learning allows the full complexity of the decision boundary to be captured, without losing fidelity due to approximations made when modelling.

Consider the example where a humanitarian organization such as the World Food Program (WFP) seeks to provide food rations to population groups in need. In addition to meeting nutritional requirements, the WFP would like to be able to consider the palatability of the rations. Since palatability is difficult to model explicitly and may vary from population to population, constraint learning can be used here. Given a data set with different food baskets and their palatability scores, a predictive model can be used to derive the palatability constraint from the data. This predictive model can then be embedded into the original food basket optimization problem.

To this end, the OPTIMAL project has created OptiCL t2], an open-source, end-to-end framework for optimization with learned constraints. OptiCL leverages data and machine learning to help practitioners enhance their decision making. Its features include:

Multiple Supported Machine Learning Models

As many machine learning models can be represented using mixed-integer optimization formulations, OptiCL is able to support a wide array of predictive models o3]. Constraints and/or objectives can be learned using one or more of the following predictive models:

• Linear Regression
• Linear Support Vector Machine
• Decision Trees
• Optimal Decision Trees
• Random Forest
• Gradient Boosting Methods
• Neural Network with ReLU activation function

All of the above models can be used either as regression models or as binary classification models, except for Linear Regression which is replaced by Logistic Regression in the case of binary classification. Future updates to OptiCL will also allow for multi-class classification.

Automatic Model Selection

No single predictive model predominates. OptiCL implements a cross-validation process to ensure that only the best performing model will be used to learn the constraints and/or objectives. The user can, however, specify that one of the supported models is used if desired.

Ability to Learn Multiple Objectives and Constraints

OptiCL allows different models to be used for learning different outcomes, all within the same problem. An example of this is given in e3], where promising chemotherapy regimens are identified for use in new clinical trials. Chemotherapy regimens are particularly challenging to optimize, since they involve multiple drugs given at potentially varying dosages, and they present risks for multiple adverse events that must be managed. OptiCL enables several of these learned constraints to be  simultaneously satisfied.

One-Class or Two-Class Constraint Learning

Data for constraint learning may either contain information on feasible and infeasible states (two-class data), or only one state (one-class data). The one-class scenario occurs frequently in business processes where information on only feasible states exists. For example, job shop schedules created by humans are typically always feasible, even if they are not optimal. OptiCL is able to use this one-class data to learn the feasible region, and then optimize over it to create an optimal schedule.

Avoiding Extrapolation Using a Trust Region

Due to the uncertainty associated with learning from data, OptiCL  uses a concept called Trust Region (TR) to avoid potentially erroneous solutions. This TR allows OptiCL to restrict the solution of the optimization problem so that it is consistent with the domain of the predictive models. The benefits of the TR are also illustrated in a3].

Ability to Use Multiple Solvers

OptiCL uses Pyomo s4], an open-source modelling language, to formulate the optimization problem. This allows for several open source (SCIP, GLPK, CBC) or proprietary solvers (CPLEX, Gurobi) to be used.

OptiCL is freely available to download and try out. Please also feel free to contact the authors for further details at a.o.fajemisin2@uva.nl,  d.maragno@uva.nl, and hwiberg@mit.edu.

References:

1. Fajemisin A, Maragno D, den Hertog D (2021). “Optimization with constraint learning: A framework and survey.” URL: https://arxiv.org/abs/2110.02121.
2. Maragno, D., Wiberg, H. (2021). “OptiCL: Mixed-integer optimization with constraint learning.” URL: https://github.com/hwiberg/OptiCL.
3. Maragno, D., Wiberg, H., Bertsimas, D., Birbil, S.I., den Hertog, D., Fajemisin, A. (2022). “Mixed-integer optimization with constraint learning.” URL: https://arxiv.org/abs/2111.04469.
4. Hart, William E., Jean-Paul Watson, and David L. Woodruff. “Pyomo: modeling and solving mathematical programs in Python.” Mathematical Programming Computation 3(3) (2011): 219-260.