What can we learn about the mistakes of the early cloud days and ensure they aren’t repeated in the GenAI Boom?
The year is 2006 and AWS launches Cloud Compute. Early adopters are excited about the ease of use, quick setup, and low commitment. Many are excited, but few could have predicted how much this breakthrough would change the world.
We all know what comes next. Startup after startup is founded on the premise that they can build great products without massive commitments to on-premises servers. This led directly to the tech boom and the creation of many of the successful cloud native companies that dominate the S&P 500 today.
Sounds like an amazing result, right? Of course. But there is also a downside: Predictability. It’s often lacking with the cloud. Unless very strong governance rules are set up, almost anyone can spin up a cloud instance with a credit card. It starts off small but can quickly escalate. The promise of the cloud is innovation and scalability. Too much governance early can thwart that when it is needed the most.
The problem with the inference of cost savings is the data we now have. McKinsey estimates around 30% of cloud spend is simply wasted every year. Boomi’s report indicates that over 75% of companies overspent their cloud budgets in 2023. Innovation is good. Lack of predictability isn’t.
Cloud cost management has attempted to solve this. As the numbers above indicate, there is still work to do. A lot of work.
With the quick rise of Generative AI, the similarities to the beginning of cloud are astounding:
- The ease of spinning up a new LLM allows for great innovation.
- The ease of spinning up a new LLM is leading to runaway costs.
- Forecasting costs from a sandbox to production is extremely difficult.
- Finance has little visibility into forecasted costs or future ROI.
“With wider adoption of Large Language Models (LLMs), many customers, small and large, are experiencing “bill shock”. With Pay-per-Token pricing model, which most cloud service providers have made available, we have also seen a sudden surge in cloud costs which can be addressed through FinOps best practices, specifically for Gen AI applications.”- Srik Rao, SVP at Xoriant
The economic environment today is very different than the late 2000’s-2023. Companies must be smarter with capital while they continue to drive innovation.
If the hope is that we will not be wasting 30% of GenAI costs in 20 years, surely, we can learn some things from the early days of cloud to foster innovation that won’t break the budget.
What we can learn about the past to ensure innovation and optimization in Generative AI:
- Governance and Control:
a. Implement robust governance early to manage costs without stifling innovation.
b. Use automated tools to monitor and control usage and costs effectively. - Visibility and Forecasting:
a. Develop tools for better forecasting of GenAI costs from development to production.
b. Enhance financial visibility to understand ROI and manage budgets efficiently. - Best Practices:
a. Encourage innovation with a focus on cost-efficiency.
b. Promote best practices for developing and deploying GenAI models, balancing innovation and cost management.
Innovation drives business, but the environment isn’t the same as when cloud computing came on the scene. There is a premium on businesses who can innovate with efficiency.
A lot has changed in the last 15-20 years. Let’s not make the same mistakes as we did in the past.
##
ABOUT THE AUTHORS
David Forman is the VP of Sales at DigitalEx. David started in the cloud space in 2016 at Oracle.
David’s focus is to support current and prospective customers at DigitalEx.
David lives in Austin, Texas with his wife and 2 kids. In his spare time, you can catch him on the golf course or trying a new restaurant in Austin.
M S Siddiqui is the Founder and CTO of DigitalEx. He has been in the FinOps space for 7 years and helped develop IBM’s multi-cloud optimization tool. Prior to his work in building FinOps tools, he was a DevOps leader with a focus on driving an optimized cloud practice.
Siddiqui has been an early adopter of GenAI and uses GenAI to power DigitalEx’s platform and bring value to customers.