merlyn/server/utils
Timothy Carambat d6f0d305ab
Enable real-time agent tool call streaming for all providers (#4279)
* WIP agentic tool call streaming
- OpenAI
- Anthropic
- Azure OpenAI

* WIP rest of providers EXCLUDES Bedrock and GenericOpenAI

* patch untooled complete/streaming to use chatCallback provider from provider class and not assume OpenAI client struct
example: Ollama

* modify ollama to function with its own overrides
normalize completion/stream outputs across providers/untooled

* dev build

* fix message sanization for anthropic agent streaming

* wip fix anthropic agentic streaming sanitization

* patch gemini, webgenui, generic aibitat providers + disable providers unable to test

* refactor anthropic aibitat provider for empty message and tool call formatting

* Add frontend missing prop check
update Azure for streaming support
update Gemini to streamting support on gemini-* models
generic OpenAI disable streaming
verify localAI support
verify NVIDIA Nim support

* DPAIS, remove temp from call, support streaming'

* remove 0 temp to remove possibility of bad temp error/500s/400s

* Patch condition where model is non-streamable and no tools are present or called resulting in the provider `handleFunctionCallChat` being called - which returns a string.

This would then fail in Untooled.complete since response would be a string and not the expected `response.choices?.[0]?.message`

Modified this line to handle both conditions for stream/non-streaming and tool presence or lack thereof

* Allow generic Openai to be streamable since using untooled it should work fine
honor disabled streaming for provider where that concern may apply for regular chats

* rename function and more gemini-specific function to gemini provider

* add comments for readability
.complete on azure should be non-streaming as this is the sync response

* migrate CometAPI, but disable as we cannot test

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2025-10-01 10:17:18 -07:00
..
agentFlows Implement importing of agent flows from community hub (#3867) 2025-06-05 15:08:58 -07:00
agents Enable real-time agent tool call streaming for all providers (#4279) 2025-10-01 10:17:18 -07:00
AiProviders Patch OpenAI metrics (#4458) 2025-09-30 15:19:34 -07:00
BackgroundWorkers Workspace Chat with documents overhaul (#4261) 2025-08-11 09:26:19 -07:00
boot Fix Typos in MetaGenerator.js Documentation Comments (#3993) 2025-06-13 08:56:40 -07:00
chats New Default System Prompt Variables (User ID, Workspace ID, & Workspace Name) (#4414) 2025-09-29 14:32:56 -07:00
collectorApi Allow user to specify args for chromium process so they dont need SYS_ADMIN on container. (#4397) 2025-09-17 16:31:08 -07:00
comKey [BETA] Live document sync (#1719) 2024-06-21 13:38:50 -07:00
database Prune telemetry events 2025-04-29 13:54:00 -07:00
DocumentManager Enable ability to do full-text query on documents (#758) 2024-02-21 13:15:45 -08:00
EmbeddingEngines Refactor Class Name Logging (#4426) 2025-09-25 15:34:19 -10:00
EmbeddingRerankers/native Patch tokenizer race condition bug (#4091) 2025-07-07 13:25:30 -07:00
EncryptionManager [BETA] Live document sync (#1719) 2024-06-21 13:38:50 -07:00
files patch folder name GET request response (#4395) 2025-09-17 13:05:11 -07:00
helpers Enable custom HTTP response timeout for ollama (#4448) 2025-09-29 12:32:55 -07:00
http Mobile sync support (#4173) 2025-07-31 12:28:03 -07:00
logger patch logger for full logs 2024-07-19 18:35:41 -07:00
MCP Refactor Class Name Logging (#4426) 2025-09-25 15:34:19 -10:00
middleware Flag to disable login UI and endpoints for credentialed auth (#3984) 2025-06-11 12:46:40 -07:00
PasswordRecovery Strengthen field validations on user Updates (#1201) 2024-04-26 16:46:04 -07:00
prisma Fix Typos in Comments and Documentation (#3987) 2025-06-12 13:59:38 -07:00
telemetry Prune telemetry events 2025-04-29 13:54:00 -07:00
TextSplitter Native Embedder model selection (incl: Multilingual support) (#3835) 2025-07-22 10:07:20 -07:00
TextToSpeech Add model field parameter for generic OpenAI TTS (#4171) 2025-07-17 10:28:51 -07:00
vectorDbProviders Sanitize Metadata Before PG Vector Database Insertion (#4434) 2025-09-29 13:49:45 -07:00
vectorStore PGvector vector database support (#3788) 2025-05-09 12:27:11 -07:00