merlyn/server/__tests__/utils
Timothy Carambat 2c19dd09ed
Native Embedder model selection (incl: Multilingual support) (#3835)
* WIP on embedder selection
TODO: apply splitting and query prefixes (if applicable)

* wip on upsert

* Support base model
support nomic-text-embed-v1
support multilingual-e5-small
Add prefixing for both embedding and query for RAG tasks
Add chunking prefix to all vector dbs to apply prefix when possible
Show dropdown and auto-pull on new selection

* norm translations

* move supported models to constants
handle null seelction or invalid selection on dropdown
update comments

* dev

* patch text splitter maximums for now

* normalize translations

* add tests for splitter functionality

* normalize

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2025-07-22 10:07:20 -07:00
..
agentFlows JSON Parser from agent flow blocks (#3863) 2025-06-05 07:18:00 -07:00
chats Fix multimodal chats via openai compat api (#4135) 2025-07-22 09:57:32 -07:00
safeJSONStringify Handle BigInt in message response (#4110) 2025-07-10 12:33:34 -07:00
SQLConnectors Add option to enable encryption on MSSQL Server db (#4134) 2025-07-15 15:26:38 -07:00
TextSplitter Native Embedder model selection (incl: Multilingual support) (#3835) 2025-07-22 10:07:20 -07:00