Rising Latency, Flexibility, and Capacity: The Real Barriers to AI Adoption

Many industries face the challenge of rising compute expenses when considering the adoption of artificial intelligence. However, leading companies in the field are discovering that cost is no longer the primary constraint. The focus has shifted towards addressing tougher challenges such as latency, flexibility, and capacity.

Wonder, a prominent player in the food delivery and takeout industry, exemplifies this shift. While AI only adds a few cents per order for the company, their main concern lies in cloud capacity due to increasing demands. On the other hand, Recursion, a biotech company, has been prioritizing a balance between small and larger-scale training and deployment through on-premises clusters and the cloud, allowing for rapid experimentation.

These real-world experiences shed light on a significant trend in the AI industry. For enterprises operating AI at scale, economics are no longer the key determining factor. The conversation has evolved from how to pay for AI to how quickly it can be deployed and sustained.

Wonder: Capacity Redefined

Wonder leverages AI for various operations, from recommendations to logistics. Despite AI adding a minimal cost per order, the company’s focus has shifted towards managing cloud capacity amidst growing demand. Originally built on the assumption of unlimited capacity, Wonder had to adapt sooner than expected as demand outstripped existing resources.

While the company aims to transition to small, hyper-customized models tailored to individuals, the current cost remains prohibitive. Budgeting for AI operations at Wonder is described as an art rather than a science, with a constant need to balance experimentation with cost efficiency.

The ‘Vindication Moment’ for Recursion

Recursion has taken a hybrid approach to meet its compute needs, utilizing both on-premise clusters and cloud infrastructure. The company’s initial reliance on its setup stemmed from the lack of suitable offerings from cloud providers. By incorporating gaming GPUs and Kubernetes clusters, Recursion has achieved flexibility and longevity in its AI infrastructure.

When deciding between on-premises and cloud deployment, Recursion opts for on-premises setups for big training jobs requiring high parallelism, while cloud usage is reserved for shorter workloads. From a cost perspective, on-premises solutions prove to be significantly cheaper over a five-year period, emphasizing the importance of long-term commitment to AI investments.

Conclusion

As AI adoption continues to gain momentum across industries, the focus has shifted from economic considerations to addressing challenges related to latency, flexibility, and capacity. Companies like Wonder and Recursion are at the forefront of navigating these challenges, highlighting the importance of strategic decision-making and long-term investment in AI infrastructure.