Walmart Adjusts Employee Access to AI Tool Code Puppy to Manage Costs

Walmart has recently implemented restrictions on its employees’ use of the internal AI assistant, Code Puppy, due to higher demands on the LLM supporting the tool than anticipated. Initially, Walmart had encouraged unlimited use of Code Puppy for tasks such as spreadsheet analysis and presentation creation. However, the company has now assigned employees a fixed number of AI tokens to limit usage.

The change in policy is a cost control measure, as LLMs are shifting towards pay-per-use models rather than fixed-price subscriptions. With approximately 2.1 million employees, even small queries and task requests per employee can result in substantial costs for Walmart.

Walmart advises its employees to use AI where it can add value and provides guidance on selecting the appropriate AI tool for specific tasks. Additionally, employees have access to other AI platforms funded by the company.

While Walmart has expanded the use of AI tools within the organization and provided training to employees on AI utilization, the shift to billing per interaction has posed challenges for large enterprises striving to balance productivity gains with associated costs.

One issue lies in measuring productivity based on AI workflows, as tracking the number and complexity of AI tool uses can lead to employees ‘gamifying’ their KPIs, a practice known as ‘token maxxing’. This approach has prompted the emergence of AI leaderboards in companies to recognize top AI software users.

Employing larger, more complex AI models incurs higher costs, as these models require more tokens for processing inputs. Walmart’s guidance to employees to choose models carefully aims to prevent overspending on advanced models for routine tasks.

Multi-agent AI work can also drive unforeseen costs for employers when employees run iterative loops on multiple agents to achieve desired outcomes, resulting in additional expenses to refine and resubmit prompts.

Several AI providers, including Anthropic and OpenAI, have transitioned to per-token billing models for enterprise plans. Microsoft’s decision to charge for GitHub Copilot software aligns with this trend, reflecting a new financial standard for model providers.

By imposing limits on token usage per employee, Walmart seeks to control ongoing costs, promote thoughtful AI tool usage, and establish metrics for evaluating AI return on investment.

(Image source: Pixabay, under licence.)

Interested in AI and big data insights from industry experts? Explore the AI & Big Data Expo happening in Amsterdam, California, and London as part of TechEx alongside other leading technology events. Discover more here.

AI News is brought to you by TechForge Media. Check out upcoming enterprise tech events and webinars here.