Why this article

In this article, we will explain how Machine Learning and AIMMS can work together to complement the mathematical modeling and optimization capabilities of AIMMS as decision support technology. Why? Because we see an increasing need by our customers to understand its potential. Also, we notice that some users feel they need to choose between one or the other when creating decision support systems. We believe the combination is more powerful. This is also why we make it possible for customers to integrate scripting languages like Python and R with AIMMS models.

We will start with a brief introduction to AIMMS and the various disciplines for better decision making (optimization, machine learning and business rules). Hereafter, we explore both machine learning and optimization as techniques, followed by how optimization- and thus AIMMS and AIMMS users - can benefit from machine learning. The idea is not to fully explain the two techniques (there are various resources online) but to provide enough information to inspire you to explore this opportunity. We will end with some recommendations for next steps.

Introduction

Over the years, we have seen the use of Machine Learning (ML) grow as tools matured and use cases as well as skilled ML practitioners increase. It is therefore logical to ask the question how AIMMS, as an optimization platform for better decision making, relates to ML practices. Is it being replaced? Is it a natural addition? What is the place for AIMMS when it comes to ML? Let’s take a closer look.

Better Decisions 

At AIMMS we are all about enabling you to make better decisions by using our optimization platform. The platform allows you to build bespoke Apps that model your business and optimize decisions against constraints and objectives such as revenue, service level or customer satisfaction. In addition, you can use ready-made Apps from our SC Navigator suite or have AIMMS partners develop Apps in case you need help. Whether you build your own, use our Apps, or work with one of our partners, the Apps allow non-optimization experts to get empowered by technology normally only available to optimization experts. 

The unique strength of our platform is that these Apps are user-friendly, web-based applications and the whole development and deployment cycle is taken care of. This means that once an App is ready, it’s in the hands of end users (decision makers as we call them) within minutes. This allows these decision makers to explore good recommended actions c1] on how to run the business quickly and easily. 

Not unimportant, these recommendations are created using AIMMS’ modeling and optimization technology and therefore outperform the current way of working (e.g. Excel, non-optimization standard solutions). They contribute greatly to the results of the decision maker’s organization. It becomes especially effective when decisions are complex or have lots of related direct and indirect effects that are hard (or no longer possible) to manage; modeling against a set of objectives is then the best way to come to feasible and good outcomes. Use cases include anything supply chain related, such as Sales & Operation Planning, Resource Planning or Network Optimization.  Energy Management Systems, Portfolio Optimization, Employee Planning, and Blending of Materials are other great examples of using optimization.

Gartner classifies 3 disciplines 2] in what they call “Better Decision Management”:

• Optimization
• Machine Learning
• Business Rules 

Optimization is not the only way to improve decision making. In many complex situations, business rules are the typical way (think of scheduling algorithms, decision trees etc.). Each discipline comes with its own strength and applicability. However, it’s clear that when skilled in all fields, one can get benefit from the full scope of decision management and have the ‘best’ decision management readiness available. In this light, let’s explore how AIMMS can offer an improved decision-making experience to its users. 

Examples and Key Characteristics 

Although we have seen AIMMS also used in some Business Rules o3] settings, the focus of this article will be on combining/integrating/supporting optimization with ML. Let’s start by providing a few examples of both and explore some of the characteristics. The information will be rather high level as the purpose of this document is to not fully explain ML or optimization itself.

Machine Learning (ML)

Typically, ML is about feeding algorithms with an immense amount of data so it can be analyzed and provide predictions, data clustering insights, or even recommended actions for decision making using training techniques. Key here is that the results of a ML algorithm are heavily dependent of historic/available data and recommendations cannot be validated (within the algorithms) as being feasible; it is (only) tested against existing data for correctness. The combination of algorithm selection and testing can be quite a task. Also, feedback on the results of these algorithms is important and should be fed back into the algorithm to improve future insights. It can also be used as validation for the algorithms.

Think of providing feedback on image classification, where the algorithm says there’s an 82% probability the image is a cat and you validate it as one, or in Netflix recommendations agreeing by giving a thumbs-up. False positives can also be a challenge in certain situations where you need to be absolutely sure (for instance, decisions on applying medications or steering a car).

In all of these situations, an analyst has set up a way to handle the data and pick the algorithms to use. This could be either a manual process, or an automated process. With autoML, even model building is increasingly automated. Data changes, especially structural ones, can influence the effectiveness of the outcome greatly. Hence, depending on the stability of the data, the results will be more or less useful, and hence require (regular) re-training.  

Examples where ML is used:

• Statistical ML functions to detect trends in demand data
• Creation of network clusters to improve Vehicle Routing Solutions (can’t be solved at once)
• Risk analysis of credit card application (continuous learning)
• Recommendations in Netflix on what to watch next, or a web shop suggestion for purchase
• Ads provided based on Cookies in your browser and user profile matching

As mentioned earlier, ML often also depends on feedback to improve the algorithms. This could be on a very user-specific level (e.g. thumb up/down for a Netflix series recommendation or commenting on the usefulness of Google ad).

Summarizing, ML uses an abstract internal model that needs to be trained with lots of known training data, and the model and any results it produces are therefore hard to verify. The big advantage of ML is that it is data driven and no explicit knowledge of business logic is required to implement an ML model. If you have data, you can often do something with it (i.e. explore them use ML algorithms). However, inputs substantially outside of the range of training data are likely to produce less reliable or completely unreliable results. 

Optimization

The optimization part is very well known to AIMMS users, but less so to ML practitioners. Let’s briefly explore, in short, what makes optimization different from several other techniques out there. 

Optimization uses various mathematical programming methods such as linear programming, integer programming, and constraint solving techniques (see figure 1). The idea is that you symbolically describe the decisions you need to make (e.g. amount to transport, warehouses to open/close), the constraints you need to uphold (e.g. maximum capacity, transport rates) and the relations between the two (e.g. never transport more than the total truck capacity available).  The description of the complete business logic is called a model and enables you to represent the complete business problem o4].

Adding data to this model and invoking a mathematical program solver allows one to find feasible recommendations for this business problem (e.g. production plans, warehouse locations, resource assignments). In addition, the model holds an objective to maximize (or minimize), such that recommendations not only match the business logic, but also provide optimal solutions (e.g. minimal cost, highest margin, and highest service level).

Summarizing, optimization, in contrast to ML, relies upon explicitly modeling known business logic into the optimization model. This makes optimization more laborious to implement, but the results of the model can be easily verified against the existing business logic and therefore more safely and easily implemented. Also, as no training is involved, the model will work for any combination of input data.

Clearly, the input data is key. Having good data (e.g. as complete and reliable as possible), will result in better results. Hence, a clear connection can be made between optimization and ML. ML can strengthen optimization by having better forecast data, for instance. 

Combining optimization and ML techniques provides the following benefits: 

• Using ML, better (often forecast) data can be generated as input. 
• Using optimization, recommended actions are validated against the business logic, and thus are always implementable (resulting plans could even be automatically be implemented).
• Using optimization, recommendations are supporting a clear objective giving the business clear understanding of the value of a recommended action.

ML and Optimization

The example of forecast data generation mentioned above is one of the most straightforward benefits of bringing ML into AIMMS Apps. There are many more ideas for combination ML and AIMMS. 

Examples of using ML in AIMMS to:

• Analyze a series of optimization solutions that have been generated over time to understand if there are pattern/issues you can benefit from 
• Generate missing data based on existing data
• Adjust certain thresholds that normally could be fixed 
• Predict good starting solutions
• Detect unknown model parts and have AIMMS generate new code (dynamically) to extend the model (or turn off parts)

One will be more complex than the other. Some might not even be realistic in specific cases. However, the clear message here is: explore the power of ML in combination with your optimization models.

Important: One can deploy ML techniques into AIMMS Apps via standard methods t5]. This allows App developers to integrate it such that decision makers can take up the benefits in their daily practice as they already do with optimization. Alternatively, it can be integrated into an automated process (a service) where optimization and ML run hand-in-hand in the background. 

Bigger question

Of course, the bigger question (pink elephant?) is whether ML can start predicting optimal (or at least extremely good) recommended actions that are feasible to implement and that do not depend on optimization against constraints or objectives. This can be a game changer, as optimization runs could be more time consuming than running ML algorithms. If possible, you can also imagine to only run an optimization model every so often for additional validation, while ML algorithms are suggesting the initial recommendations. This can increase the usability of both optimization and ML even further in many cases. So far, we have not seen evidence of this for the use cases of our customers. Hence, we believe optimization will be a key and even required technology in decision making next to Machine Learning.

Recommendations for taking next steps

Having shared our view on where ML and optimization can be complementary, we recommend to:

1. Review your applications and see whether they can benefit from ML algorithms (e.g. whether your forecast is robust enough, data is missing, or deeper learning can be implied to improve the model).
2. For those opportunities, define a clear interface between the ML algorithms and the AIMMS applications. Be sure to define a robust and scalable infrastructure such that end users can trust the results.
3. Give guidance to your users on what is happening (what you do where), so your improved recommendations are understood and accepted. Only accepted solutions are implemented into the organization and generate the benefits you anticipated.

Of course, in all steps AIMMS is ready to support you.
 

References

.1] Prescriptive Analytics provides recommended actions, https://www.aimms.com/prescriptive-analytics.

/2] How to Use Machine Learning, Business Rules and Optimization in Decision Management, Gartner, 2019.

3] Please contact us to learn more.

<4] One could also call this a Digital Representation or Digital Twin. See our view on this in our blogpost: https://supplychainblog.aimms.com/2019/04/29/digital-representation-and-our-obsession-with-optimization.

o5] Example of Python ML added to AIMMS: https://how-to.aimms.com/Articles/487/487-aimms-with-python.html.

Other interesting sources

“Operations Research + Machine Learning for the design of future offshore wind farms”, Martina Fischetti (Vattenfall BA Wind), 2020.

“How Can Machine Learning and Optimization Help Each Other Better?”, Zhou-Chen Lin, 2019.

“The Interplay of Optimization and Machine Learning Research”, K. Bennett, E. Parrado-Hernández, 2006.

AIMMS SC Navigator Demand Forecasting application uses ant colony optimization to support forecast learning.

An “How-to” for Adding Python and R to AIMMS Models.