Add stream options to Gemini LLM for usage tracking (#4466)

* Add stream options to Gemini LLM for usage tracking

* Update Gemini LLM to disable prompt token calculation

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
This commit is contained in:
Marcello Fitton 2025-10-01 14:00:26 -07:00 committed by GitHub
parent d6f0d305ab
commit 004327264a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -416,9 +416,12 @@ class GeminiLLM {
stream: true,
messages,
temperature: temperature,
stream_options: {
include_usage: true,
},
}),
messages,
true
false
);
return measuredStreamRequest;