Add stream options to Gemini LLM for usage tracking (#4466)
* Add stream options to Gemini LLM for usage tracking * Update Gemini LLM to disable prompt token calculation --------- Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
This commit is contained in:
parent
d6f0d305ab
commit
004327264a
@ -416,9 +416,12 @@ class GeminiLLM {
|
||||
stream: true,
|
||||
messages,
|
||||
temperature: temperature,
|
||||
stream_options: {
|
||||
include_usage: true,
|
||||
},
|
||||
}),
|
||||
messages,
|
||||
true
|
||||
false
|
||||
);
|
||||
|
||||
return measuredStreamRequest;
|
||||
|
||||
Loading…
Reference in New Issue
Block a user