-
Notifications
You must be signed in to change notification settings - Fork 6.9k
models_usage shows prompt_tokens and completion_tokens as 0 #6548
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
v0.5.7, cannot reproduce from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient
async def main():
client = OpenAIChatCompletionClient(model="gemini-2.0-flash")
assistant = AssistantAgent(
"assistant",
model_client=client,
system_message="You are a helpful assistant.",
)
result = await assistant.run(task="What is the capital of France?")
print(result)
if __name__ == "__main__":
import asyncio
asyncio.run(main())
|
Hi, below is a sample code using Azure OpenAI but I noticed the same behaviour with Gemini as well. Please let me know what am I missing:
autogen-agentchat version: 0.5.7
{'source': 'user', 'models_usage': None, 'metadata': {}, 'content': 'What is the capital of France?', 'type': 'TextMessage'} |
The problem appears to be limited to streaming responses. Investigating |
The problem here is a missing parameter. As a temporary workaround while I fix the issue, you can add the following parameter on your side: OpenAI/Azure OpenAI:
|
I was encountering this problem with both the streaming and non-streaming calls from agents, when using Azure OpenAI. If I disable the |
## Why are these changes needed? Enables usage statistics for streaming responses by default. There is a similar bug in the AzureAI client. Theoretically adding the parameter ``` model_extras={"stream_options": {"include_usage": True}} ``` should fix the problem, but I'm currently unable to test that workflow ## Related issue number closes #6548 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
What happened?
Describe the bug
I was previously using v0.5.1 and sometime over the last few weeks, I had upgraded to v0.5.6. I was checking the history data and noticed that the
prompt_tokens
andcompletion_tokens
undermodels_usage
are coming as 0.I remember they used to report token count but I can only see zero.
To Reproduce
Expected behavior
Expecting token usage counts
Screenshots
I see output like:
{"source":"assistant","models_usage":{"prompt_tokens":0,"completion_tokens":0},"metadata":{},"content":"Hello! How can I assist you today?","type":"TextMessage"}
Which packages was the bug in?
Python Core (autogen-core)
AutoGen library version.
Python 0.5.6
Other library version.
No response
Model used
gpt-4o-mini, gemini-2.0-flash
Model provider
OpenAI
Other model provider
Gemini
Python version
3.12
.NET version
None
Operating system
None
The text was updated successfully, but these errors were encountered: