What do leading-edge CDOs already in ML production say works best? Download this report to find out.

Lessons from the Leading Edge:

Machine Learning Best Practices and Challenges from Chief Data Officers Already in Production

Executive Summary

For enterprises who are in the next wave of putting ML into production, it is critical to understand what CDOs/CDAOs at leading-edge enterprises are doing to succeed and what is still an obstacle for them.

Discovering How to Get ROI from Machine Learning (ML)

With a $15.7 trillion (U.S.) potential contribution to the global economy by 2030 (PwC), demand for artificial intelligence (AI) has never been higher. But, artificial intelligence is still a project for early adopters in many cases, without the well-documented best practices and wealth of knowledge of a mature technology.

In fact, Gartner reports that 90% of AI initiatives fail to produce substantial return on investment (ROI), with about half never making past the prototype stage.

$15.7 trillion
potential contribution to the global economy by 2030

0%

of AI initiatives fail to produce substantial return on investment (ROI)

But some AI initiatives are in production and generating value for the organizations conducting them. What are these leading-edge firms doing that allows them to succeed when others do not? More importantly, how can others learn from their best practices and the things that did not work for them?

Wallaroo.AI set out to answer these questions.

We commissioned NewtonX, the world’s leading B2B market research company with 100% verified research across 140 industries and 1.1 billion professionals, to reach out to ML leaders at leading-edge firms who have successfully put models in production to find out. This report summarized their findings.

ML in Production Today

There were several clear themes in the research, regarding the value of ML already in production:

The Leading Edge is Already Finding Value in Their Production ML Initiatives

Early adopters do not always realize the promise of new technologies. However, despite all the challenges of being first, 92% of AI team leaders at leading-edge organizations felt they are already finding that their AI initiatives are generating value.

0%

of AI team leaders at leading-edge organizations felt that their AI initiatives are generating value

The value leading-edge enterprises are gaining comes in the areas of personalizing customer experience, fraud detection, optimizing sales and marketing, and improving realtime decision making, among other use cases.

Current Investment and Future Plans Will Fuel Growth, Create Challenges for Others

Gains from current initiatives are at least partly the result of the strong investment made by leading-edge enterprises.

Partly due to the perceived value they feel their AI initiatives are already achieving, leading-edge organizations plan to invest significantly in their AI initiatives in the future (61% will more than double investment). They also intend to scale significantly within the next three years.

0%

of leading-edge organizations plan to investment more than double in their AI initiatives

However, this investment (including cloud cycle costs, staffing costs, etc.) and scale will also impact other enterprises seeking the same human resources, vendors who support the ML ecosystem, and other elements of the AI ecosystem.

Key Challenges

Addressing the ML Skills Gap

Skilled ML experts are a critical factor to the success of leading-edge enterprises, according to CDO survey responses on several questions. In fact, 71% of respondents have more than 100 staffers working on ML. However, acquiring and retaining these ML experts represent some of the greatest challenges these organizations faced.

0%

of respondents have more than 100 staffers working on ML

The Platform is the Challenge

Being at the leading edge means overcoming hurdles in whatever way you can. For enterprises trying to put ML into production and scale to reach ROI, that often means creating in-house ML frameworks that are later found to be suboptimal. In-house development teams then need to rework the software, which impacts staff learning curves, cost, and time-to-value But data shows that these are all challenges that CDO survey respondents are already facing, even before they make any platform revisions. Fifty-one percent cited the effort required to integrate tools as a major challenge in getting ML into production, while another 51% said that the number of discrete tools needed throughout the process was a major challenge.

0%

cited the effort required to integrate tools as a major challenge in getting ML into production

0%

said that the number of discrete tools needed throughout the process was a major challenge

ML Production Best Practices and Challenges by Stage

Where AI teams choose to deploy their models – most often in the cloud – has a major impact on the value they are able to generate for their organizations. However, leadingedge organizations encounter challenges across the entire M production process that stem from the ML production framework they use and the intricacies of working in the cloud.

Takeaways

Next Steps

Your data. Your tools.
Your ecosystem.

No need to replace your data platform or change the popular tools your data scientists use to build models.

Keep up to date with the latest ML production news sign up for the Wallaroo.AI newsletter

Platform
Platform Learn how our unified platform enables ML deployment, serving, observability and optimization
Technology Get a deeper dive into the unique technology behind our ML production platform
Solutions
Solutions See how our unified ML platform supports any model for any use case
Computer Vision (AI) Run even complex models in constrained environments, with hundreds or thousands of endpoints
Resources