michael/merlyn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,801 Commits 1 Branch 0 Tags 51 MiB
0e9dc6572b
Commit Graph

217 Commits

Author SHA1 Message Date
Sean Hatfield
07129e81f8
Add option to disable streaming via env for generic openai provider (#4079) 
* add option to disable streaming via env for generic openai provider

* move env check to streamingEnabled
 2025-07-01 12:47:46 -07:00
Timothy Carambat
4eb951d40e
Fix model map staleness behavior or fallback (#3971) 
* Fix model map staleness behavior or fallback

* patch url

* fix log

* dev build
 2025-06-06 17:39:48 -07:00
Timothy Carambat
a57536b715
Handle invalid response bodies for ContextWindowFinder (#3896) 
Handle invalid response bodies for contextwindowfinder
 2025-05-27 15:40:06 -07:00
timothycarambat
2450e49ac3 hoisting cleanup for format var 2025-05-14 16:25:17 -07:00
timothycarambat
605910b76d forgot files for DPAIS 2025-05-14 15:26:14 -07:00
Timothy Carambat
e80492606a
Automatic Context window detection (#3817) 
* Add context window finder from litellm maintained list
apply to all cloud providers, have client cache for 3 days

* linting
 2025-05-14 11:03:19 -07:00
timothycarambat
492570dfed patch Azure image reading regressions 
resolves #3811
 2025-05-12 11:10:35 -07:00
Danny Steenman
5500fa2bc5
feat: support for iam roles for bedrock client (#2632) 
* feat: implement iam role auth for bedrock

* fix: make client refreshes properly when switching between iam_user and iam_role

* checkout agent flow

* fix aiprovider for bedrock in agent use

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2025-05-06 13:48:15 -07:00
Tristan Stahnke
b64a77f29f
Refactor AWS Bedrock Provider for Multi-modal Support & Correct Token Limits (#3714) 
* Fixed two primary issues discovered while using AWS Bedrock with Anthropic Claude Sonnet models:
- Context Window defaults to 8192 maximum, which isn't correct
- Multimodal stopped working when removing langchain, which was transparently handling image_url to a format sonnet expects.

* Ran `yarn lint`

* Updated .env.example to have aws bedrock examples too

* Refactor for readability
move utils for AWS specific functionality to subfile
add token output max to ENV so setting persits

---------

Co-authored-by: Tristan Stahnke <tristan.stahnke+gpsec@guidepointsecurity.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
 2025-05-06 12:55:24 -07:00
Sean Hatfield
8912d0f0fc
Add option to control KoboldCPP max response tokens (#3746) 
add option to control koboldcpp max response tokens
 2025-05-02 14:12:06 -07:00
Shinya Suzuki
cd900f9e4c
Replace @azure/openai with openai, and update openai to version 4.95.1 (#3691) 
* Replace @azure/openai to OpenAI lib

* Remove @azure/openai dependency and update openai to version 4.95.1

* linting

* update logging
fix translation dictionary error

* remove bad ENV key that DNE
linting
Patch Azure OpenAI
Migrate Azure Agent provider to use OpenAI Schema for tool calling performance

* unset

* migrate azure to use default OAI stream handler

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
 2025-04-29 11:21:39 -07:00
Shinya Suzuki
98c46c04e4
Update Azure AI options and model map with new model configurations (#3660) 
* Update Azure AI options and model map with new model configurations

* linting

---------

Co-authored-by: Shinya Suzuki <shinya.s.825@gmail.com>
Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2025-04-16 09:08:40 -07:00
timothycarambat
1d1fb817b0 linting 2025-04-15 12:51:08 -07:00
Michał Rudziński
be27299897
handling of citations in openRouter provider #3581 (#3620) 
* handling of citations in openRouter provider #3581

* Update pplx enrichToken function comment
Modify OR enrichToken to be generic handler function with optional params
handle _just_ Perplexity in-line citations since no other models support this functionality

* remove console log

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2025-04-15 10:57:09 -07:00
Timothy Carambat
1b59295f89
Refactor Gemini to use OpenAI interface API (#3616) 
* Refactor Gemini to use OpenAI interface API

* add TODO

* handle errors better (gemini)

* remove unused code
 2025-04-07 17:18:31 -07:00
Timothy Carambat
4ac900f645
Gemini model list sync (#3609) 
* Update defaultModels.js

add gemma-3-27b-it to v1BetaModels

* Update defaultModels.js

20250330 model update

* Update defaultModels.js

remove text embedding

* Update name and inputTokenLimit modelMap.js

* Update gemini to load models from both endpoints
dedupe models
decide endpoint based on expieremental status from fetch
add util script for maintainers
reduce cache time on gemini models to 1 day

* remove comment

---------

Co-authored-by: DreamerC <dreamerwolf.tw@gmail.com>
 2025-04-07 13:45:16 -07:00
Timothy Carambat
78c83383d8
Overhaul AWS Bedrock provider (#3537) 
* Patch AWS Bedrock provider for newer models and performance

* patch prompt constructor
 2025-03-25 15:58:16 -07:00
Timothy Carambat
66b4bf2679
Add support for Anthropics /model endpoint (finally) (#3376) 
* Add support for Anthropics /model endpoint (finally)

* dev
 2025-02-28 13:29:43 -08:00
cnJasonZ
2aeb4c2961
Add new model provider PPIO (#3211) 
* feat: add new model provider PPIO

* fix: fix ppio model fetching

* fix: code lint

* reorder LLM
update interface for streaming and chats to use valid keys
linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2025-02-27 10:53:00 -08:00
Skanda Kaashyap
d1354caccb
[FEAT] Add claude-3-7 (#3337) 
* add claude 3-7 sonnet

* made all the changes everywhere

* add 3-7-sonnet-latest model

* lint

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
 2025-02-25 12:52:17 -08:00
timothycarambat
12b43256a0 lint 2025-02-18 20:49:40 -08:00
Sushanth Srivatsa
3fd0fe8fc5
2749 ollama client auth token (#3005) 
* ollama auth token provision

* auth token provision

* ollama auth provision

* ollama auth token

* ollama auth provision

* token input field css fix

* Fix provider handler not using key
sensible fallback to not break existing installs
re-order of input fields
null-check for API key and header optional insert on request
linting

* apply header and auth to agent invocations

* upgrading to ollama 5.10 for passing headers to constructor

* rename Auth systemSetting key to be more descriptive
linting and copy

* remove untracked files + update gitignore

* remove debug

* patch lockfile

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2025-02-18 16:00:17 -08:00
Timothy Carambat
cc3d619061
Add handling to reasoning models for Generic OpenAI connector (#3183) 
* Add handling to resoning models for Generic OpenAI connector
resolves #3177

* linting
 2025-02-12 10:28:44 -08:00
Timothy Carambat
1bfd461719
Patch PPLX streaming for timeouts (#3130) 
Add in-text citations as well for PPLX token streaming
handle timeouts for stream/buffer hanging
 2025-02-05 16:35:22 -08:00
Timothy Carambat
e76baacec4
Add reasoning flag for azure models with "default" fallback (#3128) 2025-02-05 15:22:45 -08:00
timothycarambat
a5ee6121e2 Add patch for o# models on Azure 
connect #3023
Note: depends on user naming the deployment correctly.
 2025-02-05 11:34:03 -08:00
timothycarambat
a4d5b2588f o3 model patch 2025-02-03 14:19:21 -08:00
Timothy Carambat
d1ca16f7f8
Add tokenizer improvments via Singleton class and estimation (#3072) 
* Add tokenizer improvments via Singleton class
linting

* dev build

* Estimation fallback when string exceeds a fixed byte size

* Add notice to tiktoken on backend
 2025-01-30 17:55:03 -08:00
Timothy Carambat
e1af72daa7
parse streams from reasoning_content from DeepSeek API (#3068) 2025-01-30 14:55:20 -08:00
Timothy Carambat
8eac83bb82
Enable include_reasoning in request (#3067) 2025-01-30 14:32:38 -08:00
timothycarambat
6192080635 Update NVIDIA branding 2025-01-28 10:27:31 -08:00
Sean Hatfield
75790e7e90
Remove native LLM option (#3024) 
* remove native llm

* remove node-llama-cpp from dockerfile

* remove unneeded items from dockerfile

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
 2025-01-27 13:42:52 -08:00
Jason
c757c3fb5f
feat: update novita AI logo and default model (#3037) 2025-01-27 08:41:12 -08:00
Timothy Carambat
2ca22abc9c
Add Version to AzureOpenAI (#3023) 2025-01-24 13:41:37 -08:00
Sean Hatfield
48dcb22b25
Dynamic fetching of TogetherAI models (#3017) 
* implement dynamic fetching of togetherai models

* implement caching for togetherai models

* update gitignore for togetherai model caching

* Remove models.json from git tracking

* Remove .cached_at from git tracking

* lint

* revert unneeded change

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
 2025-01-24 11:06:59 -08:00
timothycarambat
273d116586 linting 2025-01-23 16:43:18 -08:00
Sean Hatfield
57f4f46a39
Bump perplexity models (#3014) 
* bump perplexity models

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
 2025-01-23 16:35:38 -08:00
Timothy Carambat
c4f75feb08
Support historical message image inputs/attachments for n+1 queries (#2919) 
* Support historical message image inputs/attachments for n+1 queries

* patch gemini

* OpenRouter vision support cleanup

* xai vision history support

* Mistral logging

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
 2025-01-16 13:49:06 -08:00
Timothy Carambat
21af81085a
Add caching to Gemini /models (#2969) 
rename file typo
 2025-01-13 13:12:03 -08:00
timothycarambat
4b2bb529c9 enable leftover mlock setting 2024-12-28 17:48:24 -08:00
Timothy Carambat
a51de73aaa
update ollama performance mode (#2874) 2024-12-18 11:21:35 -08:00
Timothy Carambat
b082c8e441
Add support for gemini authenticated models endpoint (#2868) 
* Add support for gemini authenticated models endpoint
add customModels entry
add un-authed fallback to default listing
separate models by expiermental status
resolves #2866

* add back improved logic for apiVersion decision making
 2024-12-17 15:20:26 -08:00
Timothy Carambat
dd7c4675d3
LLM performance metric tracking (#2825) 
* WIP performance metric tracking

* fix: patch UI trying to .toFixed() null metric
Anthropic tracking migraiton
cleanup logs

* Apipie implmentation, not tested

* Cleanup Anthropic notes, Add support for AzureOpenAI tracking

* bedrock token metric tracking

* Cohere support

* feat: improve default stream handler to track for provider who are actually OpenAI compliant in usage reporting
add deepseek support

* feat: Add FireworksAI tracking reporting
fix: improve handler when usage:null is reported (why?)

* Add token reporting for GenericOpenAI

* token reporting for koboldcpp + lmstudio

* lint

* support Groq token tracking

* HF token tracking

* token tracking for togetherai

* LiteLLM token tracking

* linting + Mitral token tracking support

* XAI token metric reporting

* native provider runner

* LocalAI token tracking

* Novita token tracking

* OpenRouter token tracking

* Apipie stream metrics

* textwebgenui token tracking

* perplexity token reporting

* ollama token reporting

* lint

* put back comment

* Rip out LC ollama wrapper and use official library

* patch images with new ollama lib

* improve ollama offline message

* fix image handling in ollama llm provider

* lint

* NVIDIA NIM token tracking

* update openai compatbility responses

* UI/UX show/hide metrics on click for user preference

* update bedrock client

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
 2024-12-16 14:31:17 -08:00
wolfganghuse
d145602d5a
Add attachments to GenericOpenAI prompt (#2831) 
* added attachments to genericopenai prompt

* add devnote

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2024-12-16 12:03:51 -08:00
Sean Hatfield
f651ca8628
APIPie LLM provider improvements (#2695) 
* fix apipie streaming/sort by chat models

* lint

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2024-12-13 14:18:02 -08:00
timothycarambat
f8e91e1ffa patch gemini-2.0-key 2024-12-11 16:52:31 -08:00
timothycarambat
69b672b625 add gemini 1206 and gemini-2.0-flash exp models 
connect #2788
 2024-12-11 09:04:29 -08:00
Timothy Carambat
a69997a715
update chat model filters for openai (#2803) 2024-12-11 08:55:10 -08:00
timothycarambat
4b09a06590 persist token window for NIM and not only on model change 2024-12-05 11:57:07 -08:00
Timothy Carambat
b2dd35fe15
Add Support for NVIDIA NIM (#2766) 
* Add Support for NVIDIA NIM

* update README

* linting
 2024-12-05 10:38:23 -08:00
timothycarambat
62be0cd0c5 add gemini-exp-1121 to expiermental set 2024-11-22 09:36:44 -08:00
timothycarambat
246152c024 Add gemini-exp-1121 
resolves #2657
 2024-11-21 11:02:43 -08:00
Sean Hatfield
55fc9cd6b1
TogetherAI Llama 3.2 vision models support (#2666) 
* togetherai llama 3.2 vision models support

* remove console log

* fix listing to reflect what is on the chart

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2024-11-21 10:42:42 -08:00
Timothy Carambat
26e2d8cc3b
Add more expiermental models from Gemini (#2663) 2024-11-20 09:52:33 -08:00
timothycarambat
af16332c41 remove dupe key in ModelMap 2024-11-19 20:20:28 -08:00
Sean Hatfield
e29f054706
Bump TogetherAI models (#2645) 
* bump together ai models

* Run post-bump command

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2024-11-18 13:08:26 -08:00
Sean Hatfield
27b07d46b3
Patch bad models endpoint path in LM Studio embedding engine (#2628) 
* patch bad models endpoint path in lm studio embedding engine

* convert to OpenAI wrapper compatibility

* add URL force parser/validation for LMStudio connections

* remove comment

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2024-11-13 12:34:42 -08:00
timothycarambat
5aa79128f7 bump Anthropic models 2024-11-06 08:14:08 -08:00
Timothy Carambat
80565d79e0
2488 novita ai llm integration (#2582) 
* feat: add new model provider: Novita AI

* feat: finished novita AI

* fix: code lint

* remove unneeded logging

* add back log for novita stream not self closing

* Clarify ENV vars for LLM/embedder seperation for future
Patch ENV check for workspace/agent provider

---------

Co-authored-by: Jason <ggbbddjm@gmail.com>
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
 2024-11-04 11:34:29 -08:00
Timothy Carambat
dd2756b570
add sessionToken validation connection auth for AWSbedrock (#2554) 2024-10-29 16:34:52 -07:00
Timothy Carambat
5bc96bca88
Add Grok/XAI support for LLM & agents (#2517) 
* Add Grok/XAI support for LLM & agents

* forgot files
 2024-10-21 16:32:49 -07:00
Timothy Carambat
446164d7b9
Add Groq vision preview support (#2511) 
Adds support for only the llama3.2 vision models on groq. This comes with many conditionals and nuances to handle as Groqs vision implemention is quite bad right now
 2024-10-21 12:37:39 -07:00
Timothy Carambat
7342839e77
Passthrough agentModel for LMStudio (#2499) 2024-10-18 11:44:48 -07:00
Timothy Carambat
93d7ce6d34
Handle Bedrock models that cannot use system prompts (#2489) 2024-10-16 12:31:04 -07:00
Sean Hatfield
fa528e0cf3
OpenAI o1 model support (#2427) 
* support openai o1 models

* Prevent O1 use for agents
getter for isO1Model;

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2024-10-15 19:42:13 -07:00
Sean Hatfield
6674e5aab8
Support free-form input for workspace model for providers with no /models endpoint (#2397) 
* support generic openai workspace model

* Update UI for free form input for some providers

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
 2024-10-15 15:24:44 -07:00
Timothy Carambat
bce7988683
Integrate Apipie support directly (#2470) 
resolves #2464
resolves #989
Note: Streaming not supported
 2024-10-15 12:36:06 -07:00
a4v2d4
cadc09d71a
[FEAT] Add Llama 3.2 models to Fireworks AI's LLM selection dropdown (#2384) 
Add Llama 3.2 3B and 1B models to Fireworks AI LLM selection
 2024-09-28 15:30:56 -07:00
Sean Hatfield
7390bae6f6
Support DeepSeek (#2377) 
* add deepseek support

* lint

* update deepseek context length

* add deepseek to onboarding

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
 2024-09-26 12:55:12 -07:00
Timothy Carambat
a781345a0d
Enable Mistral Multimodal (#2343) 
* Enable Mistral Multimodal

* remove console
 2024-09-21 16:17:17 -05:00
Timothy Carambat
a30fa9b2ed
1943 add fireworksai support (#2300) 
* Issue #1943: Add support for LLM provider - Fireworks AI

* Update UI selection boxes
Update base AI keys for future embedder support if needed
Add agent capabilites for FireworksAI

* class only return

---------

Co-authored-by: Aaron Van Doren <vandoren96+1@gmail.com>
 2024-09-16 12:10:44 -07:00
Timothy Carambat
906eb70ca1
bump Perplexity models (#2275) 2024-09-12 13:13:47 -07:00
Timothy Carambat
c612239ecb
Add Gemini exp models (#2268) 
Add Gemini  models
resolves #2263
 2024-09-11 13:03:14 -07:00
Timothy Carambat
b4651aff35
Support gpt-4o for Azure deployments (#2182) 2024-08-26 14:35:42 -07:00
timothycarambat
cb7cb2d976 Add 405B to perplexity 2024-08-19 12:26:22 -07:00
Timothy Carambat
99f2c25b1c
Agent Context window + context window refactor. (#2126) 
* Enable agent context windows to be accurate per provider:model

* Refactor model mapping to external file
Add token count to document length instead of char-count
refernce promptWindowLimit from AIProvider in central location

* remove unused imports
 2024-08-15 12:13:28 -07:00
Shahar
4365d69359
Fix TypeError by replacing this.openai.createChatCompletion with the correct function call (#2117) 
fixed new api syntax
 2024-08-14 14:39:48 -07:00
PyKen
a2571024a9
Add prompt window limits for gpt-4o-* models (#2104) 2024-08-13 09:13:36 -07:00
Timothy Carambat
f06ef6180d
add exp model to v1Beta (#2082) 2024-08-09 14:19:49 -07:00
Sean Hatfield
7273c892a1
Ollama performance mode option (#2014) 
* ollama performance mode option

* Change ENV prop
Move perf setting to advanced

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2024-08-02 13:29:17 -07:00
Timothy Carambat
ba8e4e5d3e
handle OpenRouter exceptions on streaming (#2033) 2024-08-02 12:23:39 -07:00
RahSwe
c55ef33fce
Gemini Pro 1.5, API support for 2M context and new experimental model (#2031) 2024-08-02 10:24:31 -07:00
timothycarambat
6dc3642661 Patch Groq preview models maxed to 8K tokens due to warning 2024-08-01 09:24:57 -07:00
timothycarambat
466bf7dc9c Bump Perplexity and Together AI static model list 2024-07-31 10:58:34 -07:00
Timothy Carambat
38fc181238
Add multimodality support (#2001) 
* Add multimodality support

* Add Bedrock, KoboldCpp,LocalAI,and TextWebGenUI multi-modal

* temp dev build

* patch bad import

* noscrolls for windows dnd

* noscrolls for windows dnd

* update README

* update README

* add multimodal check
 2024-07-31 10:47:49 -07:00
Timothy Carambat
5e73dce506
Enable editing of OpenRouter stream timeout for slower connections (#1994) 2024-07-29 11:49:14 -07:00
timothycarambat
296f041564 path perplexity model ids 
closes #1990
 2024-07-28 16:29:18 -07:00
timothycarambat
7a2ffefdc3 update case stmt for duplicate groq model 2024-07-25 17:39:29 -07:00
Timothy Carambat
61e214aa8c
Add support for Groq /models endpoint (#1957) 
* Add support for Groq /models endpoint

* linting
 2024-07-24 08:35:52 -07:00
Timothy Carambat
9366e69d88
Add AWS bedrock support for LLM + agents (#1935) 
add AWS bedrock support for LLM + agents
 2024-07-23 16:35:37 -07:00
Timothy Carambat
76aa2a4fd4
Implement support for selecting basic keep_alive times for Ollama (#1920) 2024-07-22 14:44:47 -07:00
Timothy Carambat
3198718975
Update references to new domain (#1916) 2024-07-22 11:05:34 -07:00
Timothy Carambat
5df6b5f7d9
Bump perplexity models (#1905) 
* Added Supported Models Free Tier - chat_models.txt

Need to fill in correct Parameter Count.

* Bump perplexity model
closes #1901
closes #1900

---------

Co-authored-by: Tim-Hoekstra <135951177+Tim-Hoekstra@users.noreply.github.com>
 2024-07-19 15:11:10 -07:00
Timothy Carambat
0b845fbb1c
Deprecate .isSafe moderation (#1790) 
Add type defs to helpers
 2024-06-28 15:32:30 -07:00
Sean Hatfield
524edd6e69
[FEAT] Add support for Claude Sonnet 3.5 model (#1731) 
add support for claude sonnet 3.5 model
 2024-06-20 10:13:53 -07:00
Sean Hatfield
3f78ef413b
[FEAT] Support for gemini-1.0-pro model and fixes to prompt window limit (#1557) 
support for gemini-1.0-pro model and fixes to prompt window limit
 2024-05-29 08:17:35 +08:00
Timothy Carambat
2f9b785f42
Patch handling of end chunk stream events for OpenAI endpoints (#1487) 
* Patch handling of end chunk stream events for OpenAI endpoints

* update LiteLLM to use generic handler

* update for empty choices
 2024-05-23 10:20:40 -07:00
Sean Hatfield
cc7e7fb3ac
[FEAT] Add support for gemini-1.5-flash-latest model (#1502) 
* add support for gemini-1.5-flash-latest

* update comment in gemini LLM provider
 2024-05-23 09:42:30 -07:00
timothycarambat
9f327d015a update error handling for OpenAI providers 2024-05-22 09:58:10 -05:00
Timothy Carambat
28eba636e9
Allow setting of safety thresholds for Gemini (#1466) 
* Allow setting of safety thresholds for Gemini

* linting
 2024-05-20 13:17:00 -05:00