Agile Machine Learning Operations for Technical Leaders in CPG

July 20, 2022
Image courtesy of Diego Delso

In the dynamic realm of Consumer Packaged Goods (CPG), staying ahead of the curve requires a robust framework for swift decision-making and continuous improvement. Enter Agile Machine Learning Operations, a game-changing approach that empowers technical leaders to navigate the complexities of deploying, monitoring, and managing multiple ML models efficiently.

In a previous post we discussed that these days, the speed of change has impacted the Consumer Packaged Goods (CPG) landscape, both in supply chain and in consumer demand. These changes to the market environment have made even the most precise machine learning models obsolete. Furthermore, the data science models show the need for a regional component. That means, for global companies, each product may have many variations of the model, increasing the complexity for global manufacturers with thousands of SKUs globally.

In this post we will speak to the technical decision makers leading the AI/ML and/or data science practice at CPG enterprises. We will go deeper into the different bottlenecks in model operations holding back machine learning from generating consistent value. 

Why Agile Machine Learning Operations?

For data science teams, the issue is not as much about building and deploying a few models but rather deploying, monitoring, and managing many models in production at the same time. There are no tools to iterate quickly when the models need updating. This is really about the last-mile of ML operations. The challenges most current production ML approaches have are:

  • Model deployment takes a very long time: Ad-hoc ML models take time to deploy, making the enterprise less agile to environmental changes. These delays in deployment often render the model ineffective in feeding the business good predictions due to the changing business environment.
  • Model deployment cannot be standardized globally: Most global CPG brands have a central data science Center of Excellence (CoE) and few regional data science teams to deploy the model with regional differences. This leads to non standard deployments and model management in production, further limiting the scalability of data science efforts.
  • No easy way to perform A/B tests for models in a production environment: One cannot assume that the current market conditions match the conditions on which a model was trained. As a result, the best test for a new model’s accuracy is in production, which can cause breakages in the consumer experience.
  • No visibility into when and why a model stops working: When the environment changes because of a sudden spike in supply prices or shifting consumer preferences, the model needs to be changed to be effective. However, once a model goes into production, data scientists lose visibility into the ongoing performance of the models. Most deployment processes and tools do not consider collaboration between data scientists and ML engineers, so when a model starts to drift there is no communication.

The Wallaroo Platform For Agile Last Mile Model Operations

When the Wallaroo team launched the first product in 2017, it was focused on building a high-performance compute engine for the purpose of analyzing large amounts of data via computational algorithms. While the customers could efficiently analyze their data and use it to run ML models at scale, they soon pointed out their next biggest challenge: bringing those models online easily and then understanding how the models were performing to sustainably generate business value. The team quickly discovered that the maximum value of AI/ML doesn’t come from just building precise models but also from the last mile: having ML in production, at scale, driving business results. 

The problem was that most large enterprises followed the DevOps playbook: data scientists build ML models to solve a business problem, while ML engineers deploy them into production using open-source software and containerized model approaches. What they find is that getting each model to production with this approach is like reinventing the wheel every time. Models often have to be painstakingly re-engineered, the deployment software can’t process data fast enough and it’s difficult to measure their ongoing accuracy.

As a result we’ve designed the Wallaroo platform to solve today’s problems:

  • Make deployment easy no matter the environment: While other MLOps deployment platforms require lengthy and costly migrations of the entire life cycle to provide standard deployment processes, Wallaroo fits into any data ecosystem. That is, your data scientists can continue using their favorite tools and frameworks to build their models in any environment, and then use Wallaroo to deploy those models in any cloud, on-prem or at the edge. And with the SDK, API, and dashboard options, deploying a model is as simple as a single line of python (more to come on our autoconvert capabilities to avoid complex model reengineering).
  • Provide powerful testing capabilities so the consumer experience is not broken: Wallaroo has advanced testing frameworks like A/B/n* testing and shadow deployments so data scientists can see how models would perform in an actual production environment. This includes edge ML testing capabilities to run A/B tests across portions of an IoT fleet or do shadow testing for use in manufacturing or logistics.
  • Automate visibility and alerts into ongoing model performance: Agile ML requires immediate insights into when the environment or consumer preferences change. Wallaroo’s model insights and observability capabilities track ongoing model performance via a stream of comprehensive audit logs, and our Model Insights automatically flag when a model is failing. We continue to invest in model explainability to bring transparency to formerly black box algorithms, so the business can understand what’s driving predictions, which is particularly important for CPG brands’ ESG efforts.

CPG brands have turned to data science to drive efficiencies from the factory floor to customers, but in order for data science to be effective it needs to keep up with changes to the environment. With Wallaroo, we have created an agile last mile model operations platform that can keep up with the speed of change in your business. 

If you are interested to see how Wallaroo can help you unlock the potential of ML to get the right SKU to the right customer via the best, most sustainable distribution channel (3rd party or direct to consumer, online or instore), reach out to us at deployML@wallaroo.ai for a specialist. 

About Manish Sinha

Manish is a special advisor to Wallaroo and an award winning CIO with global and multi-industry experience. Winner of the CIO of Year and the Peer CIO awards. Nominated by peer CIOs to the Wall Street Journal CIO Forum in USA. Member of the Senior Leadership Team in IT for Loreal, ANSYS, UBS, Yahoo and Microsoft. Broad range of experience from Infrastructure, Artificial Intelligence (AI), Cloud solutions to Business applications. Recognized by PwC and ANSYS Board for making multi years of information security enhancements in 18 months. Used AI, Master Data Management (MDM) & Enterprise Monitoring to predict incidences and prevent outages. Active in giving back to the IT community through advisory councils (ServiceNow & Microsoft), VC advisory forums, writing publications and speaking at global conferences like Economic Times, and National CIO Review.