Skip to content

Log token budget and usage for Copilot tools #2466

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 29, 2025
Merged

Conversation

minestarks
Copy link
Member

This way we can keep an eye on how various use cases and instructions files affect the token budget for our tools.

While this number doesn't give us true visibility into the internals of Copilot, like the global token budget, it may at least act as a rough indicator of how much context we're eating up.

Some numbers I gathered to help paint a picture:

  • We seem to get about ~43000 tokens in our budget for each tool call. Every message in the conversation history seems to reduce this budget, but not by much - by about 2
  • When we include our Q# instructions file in the context, the token budget seems to decrease about ~500 tokens
  • Claude talks more than GPT 4.1, thereby reducing our budget slightly 🙄
  • A typical short output, like the circuit from a Bell state program) takes up ~200 tokens

@minestarks minestarks added this pull request to the merge queue May 29, 2025
@minestarks
Copy link
Member Author

Thanks @billti for the review

Merged via the queue into main with commit 11426a1 May 29, 2025
18 checks passed
@minestarks minestarks deleted the minestarks/token-budget branch May 29, 2025 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants