Timothy Carambat
fef5bf06ec
add provider field to chats ( #4959 )
2026-02-02 20:11:18 -08:00
Marcello Fitton
c4f19cec0e
Refactor LLMPerformanceMonitor.measureStream() to Use Options Object Pattern ( #4786 )
...
* Refactor LLMPerformanceMonitor to use options object for measureStream parameters
* Refactor invocations of `measureStream` to use options arguments
* Change invocation of `measureStream` in anthropic provider to use options argument
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2025-12-16 13:10:09 -08:00
Timothy Carambat
664f466e3f
4601 log model on response ( #4781 )
...
* add model tag to chatCompletion
* add modelTag `model` to async streaming
keeps default arguments for prompt token calculation where applied via explict arg
* fix HF default arg
* render all performance metrics as available for backward compatibility
add `timestamp` to both sync/async chat methods
* extract metrics string to function
2025-12-14 14:46:55 -08:00
Timothy Carambat
e80492606a
Automatic Context window detection ( #3817 )
...
* Add context window finder from litellm maintained list
apply to all cloud providers, have client cache for 3 days
* linting
2025-05-14 11:03:19 -07:00
Timothy Carambat
c4f75feb08
Support historical message image inputs/attachments for n+1 queries ( #2919 )
...
* Support historical message image inputs/attachments for n+1 queries
* patch gemini
* OpenRouter vision support cleanup
* xai vision history support
* Mistral logging
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2025-01-16 13:49:06 -08:00
Timothy Carambat
dd7c4675d3
LLM performance metric tracking ( #2825 )
...
* WIP performance metric tracking
* fix: patch UI trying to .toFixed() null metric
Anthropic tracking migraiton
cleanup logs
* Apipie implmentation, not tested
* Cleanup Anthropic notes, Add support for AzureOpenAI tracking
* bedrock token metric tracking
* Cohere support
* feat: improve default stream handler to track for provider who are actually OpenAI compliant in usage reporting
add deepseek support
* feat: Add FireworksAI tracking reporting
fix: improve handler when usage:null is reported (why?)
* Add token reporting for GenericOpenAI
* token reporting for koboldcpp + lmstudio
* lint
* support Groq token tracking
* HF token tracking
* token tracking for togetherai
* LiteLLM token tracking
* linting + Mitral token tracking support
* XAI token metric reporting
* native provider runner
* LocalAI token tracking
* Novita token tracking
* OpenRouter token tracking
* Apipie stream metrics
* textwebgenui token tracking
* perplexity token reporting
* ollama token reporting
* lint
* put back comment
* Rip out LC ollama wrapper and use official library
* patch images with new ollama lib
* improve ollama offline message
* fix image handling in ollama llm provider
* lint
* NVIDIA NIM token tracking
* update openai compatbility responses
* UI/UX show/hide metrics on click for user preference
* update bedrock client
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-12-16 14:31:17 -08:00
Timothy Carambat
446164d7b9
Add Groq vision preview support ( #2511 )
...
Adds support for only the llama3.2 vision models on groq. This comes with many conditionals and nuances to handle as Groqs vision implemention is quite bad right now
2024-10-21 12:37:39 -07:00
Timothy Carambat
99f2c25b1c
Agent Context window + context window refactor. ( #2126 )
...
* Enable agent context windows to be accurate per provider:model
* Refactor model mapping to external file
Add token count to document length instead of char-count
refernce promptWindowLimit from AIProvider in central location
* remove unused imports
2024-08-15 12:13:28 -07:00
timothycarambat
6dc3642661
Patch Groq preview models maxed to 8K tokens due to warning
2024-08-01 09:24:57 -07:00
timothycarambat
7a2ffefdc3
update case stmt for duplicate groq model
2024-07-25 17:39:29 -07:00
Timothy Carambat
61e214aa8c
Add support for Groq /models endpoint ( #1957 )
...
* Add support for Groq /models endpoint
* linting
2024-07-24 08:35:52 -07:00
Timothy Carambat
0b845fbb1c
Deprecate .isSafe moderation ( #1790 )
...
Add type defs to helpers
2024-06-28 15:32:30 -07:00
timothycarambat
9f327d015a
update error handling for OpenAI providers
2024-05-22 09:58:10 -05:00
Timothy Carambat
01cf2fed17
Make native embedder the fallback for all LLMs ( #1427 )
2024-05-16 17:25:05 -07:00
Sean Hatfield
9feaad79cc
[CHORE] Remove sendChat and streamChat in all LLM providers ( #1260 )
...
* remove sendChat and streamChat functions/references in all LLM providers
* remove unused imports
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-05-01 16:52:28 -07:00
Timothy Carambat
547d4859ef
Bump openai package to latest ( #1234 )
...
* Bump `openai` package to latest
Tested all except localai
* bump LocalAI support with latest image
* add deprecation notice
* linting
2024-04-30 12:33:42 -07:00
Sean Hatfield
897e168fd1
[FEAT] Add support for more groq models (Llama 3 and Gemma) ( #1143 )
...
add support for more groq models
2024-04-22 13:14:27 -07:00
Sean Hatfield
0634013788
[FEAT] Groq LLM support ( #865 )
...
* Groq LLM support complete
* update useGetProvidersModels for groq models
* Add definiations
update comments and error log reports
add example envs
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-03-06 14:48:38 -08:00
