AI-Powered Billing System for GitHub Copilot

The Significance of Tokens in Text Analysis

One common way to describe a token is as representing approximately three-quarters of a word. Therefore, when presenting a Large Language Model (LLM) with a text to analyze that consists of 10,000 words, it would translate to about 12,000-13,000 tokens of content. In the context of software development, if Copilot were to examine a codebase containing 10,000 ‘words’ such as expressions, statements, variable names, and functions, using it once in a query would consume 12,000-13,000 tokens from the monthly allocation.

It’s important to note that not only the prompt text and outputs generated by Copilot count towards the token usage, but also the upcoming changes in pricing tiers set to take effect next month. Instead of a set number of queries per month, users will receive ‘AI Credits’ of equivalent value. For instance, a Copilot Pro subscriber at the base tier ($10 per month) will be granted 1,000 credits, with each AI Credit being valued at one US cent as stated by GitHub.

The number of tokens that can be purchased with each credit will vary based on factors such as the model utilized, the mix of input/output, the cache size, and the requested features. Consequently, developers who primarily engage in simple queries may not need to purchase additional tokens in the form of credits each month. On the other hand, complex and extensive queries involving multi-agent interactions with a code base can deplete the AI Credit balance faster. Furthermore, accessing the most advanced frontier models will incur higher costs compared to less powerful ones.

Despite the changes in pricing, GitHub is offering some compensatory benefits to users. Features like code completions, akin to a phone’s auto-complete function, and Next Edit suggestions will continue to be provided free of charge.