get_openai_callback For Streaming Requests Returns 0's for Token Counts

[Thimows]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

with get_openai_callback() as cb:
    async for event in agent_executor.astream_events(
        {
            "input": input,
            "chat_history": history
        },
        version="v1",
    ):

       # Do stuff....

        print(f"Total Tokens: {cb.total_tokens}")

Output:

Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0
Total Tokens: 0

Error Message and Stack Trace (if applicable)

No response

Description

OpenAI supports returning the token count while streaming: https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156

However, this does not seem to work when using Langchain. It only seems to return 0's for agent_executor.astream_events.

This makes it very difficult to get the exact token counts for my agent. Are there any solutions?

System Info

langchain==0.1.16
langchain-community==0.0.38
langchain-core==0.1.52
langchain-experimental==0.0.55
langchain-openai==0.1.7
langchain-text-splitters==0.0.1
langchainhub==0.1.15

Platform: mac
Python version: 3.9.6

[eyurtsev]

This PR will resolve the issue: #21548