* feat: add support for voyage-3-large and voyage-code-3 embedding models
- Add voyage-3-large and voyage-code-3 to VoyageAiOptions dropdown
- Update getMaxEmbeddingLength to support 32k context for new models
- Update .env.example with new model options
* unset env example
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Add support for Google Generative AI (Gemini) embedder
* Add missing example in docker
Fix UI key elements in options
Add Gemini to data handling section
Patch issues with chunk handling during embedding
* remove dupe in env
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* exposes `maxConcurrentChunks` parameter for the generic openai embedder through configuration. This allows setting a batch size for endpoints which don't support the default of 500
* Update new field to new UI
make getting to ensure proper type and format
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* fix indentation of citations
* fix outlines of input fields to be consistent
* remove red from trash icons
* fix margin from input title
* remove unused component and remove borders on provider search inputs
* make back arrow less opacity in onboarding to highlight next arrow
* fix spacing on all provider options (llm, vectordb, embedder
* add ollama automatic url detection in llm and embedder prefrences
* implement auto detection for lmstudio llm and embedder/improve performance of checking common urls
* fix modal not clearing
* fix lmstudio url check
* improve ux for ollama llm provider option
* improve ux for lm studio llm provider option
* improve ux for ollama embedder option
* improve ux for lmstudio embedder option
* ux improvement lmstudio embedder options
* refactor implementation to hook and use native timeout
Swap to promise.any for resolving of available endpoints
* implement useProviderEndpointAutoDiscovery hook for lmstudio and ollama provider options
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* add support for voyageai embedder
* remove unneeded import
* linting
* Add ENV examples
Update how chunks are processed for Voyage
use correct langchain import
Add data handling
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* getChatCompletion working WIP streaming
* WIP
* working streaming WIP abort stream
* implement cohere embedder support
* remove inputType option from cohere embedder
* fix cohere LLM from not aborting stream when canceled by user
* Patch Cohere implemention
* add cohere to onboarding
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* implement new LLM preference UI
* implement new vector db preferences UI to match LLM preferences
* implement new embedding preferences UI to match LLM preferences
* normalize placeholder text for search input
* implement new transcription preferences UI to match LLM preferences
* remove uneeded css
* implement new UI for llm preference onboarding
* implement new UI for embedder preference onboarding
* implement new UI for vector db preference onboarding
* fix placeholder text
* unset onboarding
* move autocomplete field
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* add support for new openai models
* QOL changes/improve logic for adding new openai embedding models
* add example file inputs for Openai embedding ENV selection;
* Fix if stmt conditional
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* move llm, embedder, vectordb items to components folder
* add backdrop blur to search in llm, embedder, vectordb preferences
* implement searchable llm preference in settings
* implement searchable embedder in settings
* remove unused useState from embedder preferences
* implement searchable vector database in settings
* fix save changes button not appearing on change for llm, embedder, and vectordb settings pages
* sort selected items in all settings and put selected item at top of list
* no auto-top for selection
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Implement use of native embedder (all-Mini-L6-v2)
stop showing prisma queries during dev
* Add native embedder as an available embedder selection
* wrap model loader in try/catch
* print progress on download
* Update to progress output for embedder
* move embedder selection options to component
* forgot import
* add Data privacy alert updates for local embedder
