A collection of new technologies much improves how you can manage, import and export data in, into and from AIMMS.
This matters because it:
- can improve end-user experience
- reduces the effort to integrate with external data systems
- allows leverage of emerging corporate data repositories
Therefore I invite you to learn about these new technologies and explore their relevance for your job. You can read a bit more in this update.
Without data there is no optimization. Without data AIMMS models cannot do their magic. Without data management tooling AIMMS models will not have data.
For 30+ years we have been developing data management features into our products. First we focused on files and file storage, starting with files on your local computer. Later on centrally in PRO Storage, a first step in enabling multi-user data sharing.
Next we added support for relational databases to manage data for AIMMS models. Many app developers adopted this method. It enabled powerful querying, improved multi-user experiences and improved data exchange with external platforms.
AIMMS data management technology is now entering a 3rd 'wave’ with a return to files, a move to lower-code data exchange features and use of 3rd party Azure technology. In our eyes this 3rd wave is opening an ‘opportunity candy store’ to improve user experience, app developer productivity and data exchange with external systems.
Why are we making this move?
- We noticed customers increasingly consolidating their data in centralized data repositories such as data lakes, often using Parquet files.
- We saw a growing preference for using REST API services for data exchange.
- We realized the majority of AIMMS models work with what-we-call ‘immutable data sets’. A complete set of data values that is loaded/saved as one entity. For that a file is more logical container than a relational database.
- Most of our customers are adopting cloud vendors such as Azure and AWS as their primary IT platforms.
What is the move?
A combination of technologies:
- Data Exchange Library, a new AIMMS library that offers low-code tools for (a) converting model identifiers into files and vice versa, to (b) generating REST Clients and REST Servers for data import and export.
- We are adding an Azure Data Lake Storage service to each cloud account. This enables app developers to build powerful data management solutions, for example storing and retrieving AIMMS model data for access by external systems, possibly with the growing collection of off-the-shelf connectors for Azure Data Lake Storage, including PowerBI, Databricks, Snowflake, SQL Server, etc.
- A growing collection of ready-to-use connectors between AIMMS models and applications such as Snowflake and PowerBI.
- Features to deploy Python scripts inside your AIMMS cloud account, running in an exclusive Docker container with a choice of Python images.
So why am I telling you this?
I am excited to share this '3rd wave’ with you, because of all the opportunities it offers. I believe it will empower you, AIMMS app developer, to achieve more with less effort. I believe it will help you further increase your impact. And ultimately I am confident this will lead to even better decisions, our AIMMS purpose.
And how about CDM?
We have come to see that CDM, Collaborative Data Management, is a very powerful tool for building AIMMS applications that support concurrent data editing by multiple users. And we also realize it is a complex tool that requires considerable learning. So we recommend only to use it for that 'sweet spot'.
Finally
My intention with this update is to make more AIMMS app developers explore and adopt these important innovations, to trigger also their curiosity. I am not the right person to give you detailed explanations. Our documentation and my colleagues in User Support will take care of that.
And I want you to know that these developments are also new for us. We are learning about them everyday, mostly from early adopters among our app developer community. Their feedback and the feedback from our SC Navigator colleagues is flowing back into our development work.
Acknowledgement
I would like to acknowledge Marcel Roelofs, our CTO, for his compelling and credible vision on this topic. Because this 3rd wave of AIMMS data management is mostly his brain-child, like so many AIMMS innovations.