* implement native embedder job queue
* persist embedding progress across renders
* add development worker timeouts
* change to static method
* native reranker
* remove useless return
* lint
* simplify
* make embedding worker timeout value configurable by admin
* add event emission for missing data
* lint
* remove onProgress callback argument
* make rerank to rerankDirect
* persists progress state across app reloads
* remove chunk level progress reporting
* remove unuse dvariable
* make NATIVE_RERANKING_WORKER_TIMEOUT user configurable
* remove dead code
* scope embedding progress per-user and clear stale state on SSE reconnect
* lint
* revert vector databases and embedding engines to call their original methods
* simplify rerank
* simplify progress fetching by removing updateProgressFromApi
* remove duplicate jsdoc
* replace sessionStorage persistence with server-side history replay for embedding progress
* fix old comment
* fix: ignore premature SSE all_complete when embedding hasn't started yet
The SSE connection opens before the embedding API call fires, so the
server sees no buffered history and immediately sends all_complete.
Firefox dispatches this eagerly enough that it closes the EventSource
before real progress events arrive, causing the progress UI to clear
and fall back to the loading spinner. Chrome's EventSource timing
masks the race.
Track slugs where startEmbedding was called but no real progress event
has arrived yet via awaitingProgressRef. Ignore the first all_complete
for those slugs and keep the connection open for the real events.
* reduce duplication with progress emissions
* remove dead code
* refactor: streamline embedding progress handling
Removed unnecessary tracking of slugs for premature all_complete events in the EmbeddingProgressProvider. Updated the server-side logic to avoid sending all_complete when no embedding is in progress, allowing the connection to remain open for real events. Adjusted the embedding initiation flow to ensure the server processes the job before the SSE connection opens, improving the reliability of progress updates.
* fix stale comment
* remove unused function
* fix event emissions for document creation failure
* refactor: move Reranking Worker Idle Timeout input to LanceDBOptions component
Extracted the Reranking Worker Idle Timeout input from GeneralEmbeddingPreference and integrated it into the LanceDBOptions component. This change enhances modularity and maintains a cleaner structure for the settings interface.
* lint
* remove unused hadHistory vars
* refactor workspace directory by hoisting component and converting into functions
* moved EmbeddingProgressProvider to wrap Document Manager Modal
* refactor embed progress SSE connection to use fetchEventSource instead of native EventSource API.
* refactor message handlng into a function and reduce duplication
* refactor: utilize writeResponseChunk for event emissions in document embedding progress SSE
* refactor: explicit in-proc embedding and rerank methods that are called by workers instead of process.send checks
* Abstract EmbeddingProgressBus and Worker Queue into modules
* remove error and toast messages on embed process result
* use safeJsonParse
* add chunk-level progress events with per-document progress bar in UI
* remove unused parameter
* rename all worker timeout references to use ttl | remove ttl updating from UI
* refactor: pass embedding context through job payload instead of global state
* lint
* add graceful shutdown for workers
* apply figma styles
* refactor embedding worker to use bree
* use existing WorkerQueue class as the management layer for jobs
* lint
* revert all reranking worker changes back to master state
Removes the reranking worker queue, rerankViaWorker/rerankInProcess
renames, and NATIVE_RERANKING_WORKER_TTL config so this branch
only contains the embedding worker job queue feature.
* remove breeManaged flag — WorkerQueue always spawns via Bree
* fix prompt embedding bug
* have embedTextInput call embedChunksInProcess
* add message field to `process.send()`
* remove nullish check and error throw
* remove bespoke graceful shutdown logix
* add spawnWorker method and asbtract redudant flows into helper methods
* remove unneeded comment
* remove recomputation of TTL value
* frontend cleanup and refactor
* wip on backend refactor
* backend overhaul
* small lint
* second pass
* add logging, update endpoint
* simple refactor
* add reporting to all embedder providers
* fix styles
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add API key param to Lemonade LLM Provider and Embedding Provider
* add LEMONADE_LLM_API_KEY to .env.example
* add api key to aibitat provider
* fix api key from being sent to frontend
* fix tooltip id
* add null fallback for `apiKey`
* remove console log
* add missing api keys
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add ask to run prompt for tools
* border-none on buttons
* translations
* linting
* i18n (#5263)
* extend approve/deny requests to telegram
* break up handler
* Beta Intelligent Tooling
todo: Agent Skill banner warning when tool # is high or % of content window?
* forgot files
* add UI controls and maxToolCallStack setting
* update docs link
* ISS i18n (#5237)
i18n
* Add automatic chat mode with native tool calling support
Introduces a new automatic chat mode (now the default) that automatically invokes tools when the provider supports native tool calling. Conditionally shows/hides the @agent command based on whether native tooling is available.
- Add supportsNativeToolCalling() to AI providers (OpenAI, Anthropic, Azure always support; others opt-in via ENV)
- Update all locale translations with new mode descriptions
- Enhance translator to preserve Trans component tags
- Remove deprecated ability tags UI
* rebase translations
* WIP on image attachments. Supports initial image attachment + subsequent attachments
* persist images
* Image attachments and updates for providers
* desktop pre-change
* always show command on failure
* add back gemini streaming detection
* move provider native tooling flag to Provider func
* whoops - forgot to delete
* strip "@agent" from prompts to prevent weird replies
* translations for automatic-mode (#5145)
* translations for automatic-mode
* rebase
* translations
* lint
* fix dead translations
* change default for now to chat mode just for rollout
* remove pfp for workspace
* passthrough workspace for showAgentCommand detection and rendering
* Agent API automatic mode support
* ephemeral attachments passthrough
* support reading of pinned documents in agent context
* feat(agents): Add Perplexity Search API as web search provider
Adds Perplexity as a search provider for the agent web-browsing plugin,
using the Perplexity Search API (POST /search) which returns raw ranked
web results — distinct from the existing Perplexity LLM integration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: replace docs.perplexity.ai with console.perplexity.ai
* chore: replace docs.perplexity.ai with console.perplexity.ai
---------
Co-authored-by: kesku <kesku@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
Add ownership validation to prevent users from deleting or embedding
parsed files that don't belong to them. Previously, the delete and
embed endpoints only validated authentication but not resource ownership,
allowing users to delete attached files for users within workspaces they are also a member of.
Changes:
- Delete endpoint now filters by userId and workspaceId
- Embed endpoint validates file belongs to user and workspace (redundant)
- delete() returns false when no matching records found (returns 403)
- Added JSDoc comments for clarity
GHSA-p5rf-8p88-979c
Previously, suspended users could continue using browser extension
endpoints if they had created an API key before suspension. The normal
JWT session path blocked suspended users, but the browser extension
middleware did not.
Changes:
- Add suspension and user existence checks to validBrowserExtensionApiKey
- Delete browser extension API keys when a user is deleted
- Add deleteAllForUser method to BrowserExtensionApiKey model
GHSA-7754-8jcc-2rg3
* fix: Migrate AzureOpenAI model key from OPEN_MODEL_PREF to prevent the naming collision. No effort necessary from current users.
* test: add backwards compat tests for AzureOpenAI model key migration
* patch missing env example file
* linting
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add eslint config to server
* add break statements to switch case
* add support for browser globals and turn off empty catch blocks
* disable lines with useless try/catch wrappers
* format
* fix no-undef errors
* disbale lines violating no-unsafe-finally
* ignore syncStaticLists.mjs
* use proper null check for creatorId instead of unreachable nullish coalescing
* remove unneeded typescript eslint comment
* make no-unused-private-class-members a warning
* disable line for no-empty-objects
* add new lint script
* fix no-unused-vars violations
* make no-unsued-vars an error
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
Introduces a new automatic chat mode (now the default) that automatically invokes tools when the provider supports native tool calling. Conditionally shows/hides the @agent command based on whether native tooling is available.
- Add supportsNativeToolCalling() to AI providers (OpenAI, Anthropic, Azure always support; others opt-in via ENV)
- Update all locale translations with new mode descriptions
- Enhance translator to preserve Trans component tags
- Remove deprecated ability tags UI
* Add the ability to edit existing SQL connections
* Enhance SQL connection management by adding connections prop to DBConnection and SQLConnectionModal components for improved duplicate detection and handling.
* format
* fix: prevent input defocus in SQL connection edit modal
Fixed an issue where typing in input fields would cause the field to lose
focus during editing. The useEffect dependency array was using the entire
existingConnection object, which could change reference on parent re-renders,
triggering unnecessary re-fetches and unmounting form inputs.
Changed the dependency to use the primitive database_id value instead of the
object reference, ensuring the effect only runs when the actual connection
being edited changes.
* fix: prevent duplicate SQL connections from being created
Fixed an issue where saving SQL connections multiple times would create
duplicate entries with auto-generated hash suffixes (e.g., my-db-abc7).
This occurred because the frontend maintained stale action properties on
connections after saves, causing the backend to treat already-saved
connections as new additions.
Backend changes (server/models/systemSettings.js):
- Modified mergeConnections to skip action:add items that already exist
- Reject duplicate updates instead of auto-renaming with UUID suffixes
- Check if original connection exists before applying updates
Frontend changes:
- Added hasChanges prop to SQL connector component
- Automatically refresh connections from backend after successful save
- Ensures local state has clean data without stale action properties
This prevents the creation of confusing duplicate entries and ensures
only the connections the user explicitly created are stored.
* Refactor to use existing system settings endpoint for getting agent SQL connections | Add better documentation
* Simplify handleUpdateConnection handler
* refactor mergeConnections to use map
* remove console log
* fix bug where edit SQL connection modal values werent recomputed after re-opening
* Add loading state for fetching agent SQL connections
* tooltip
* remove unused import
* Put skip conditions in switch statement
* throw error if default switch case is triggered
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add Prisma unique constraint error messaging.
* Create `_identifyErrorAndFormatMessage` private method for identifying the error type and returning the approprioate error message string
* Implement Unix username standard validations on username creation and updating.
* Remove leading underscore permissibility | Replace hardcoded username rules with a centralized USERNAME_REQUIREMENTS_TEXT for better maintainability.
* Add username requirements translations for invite and admin user creation | Replace hardcoded username requirements with localized strings in user modals
* Refactor username requirements localization
* Remove unneeded comment | Move Regex comment to validator fn
* Remove username validation utility function to keep validation responsibilities on the server | Allow onboarding flow multi-user mode username creation step to send pre-validated credentials to server.
* Enhance error handling in system endpoints by returning a JSON response with error details instead of a plain status for internal server errors.
* Update username requirement localization in AccountModal and UserSetup components to use centralized translation key.
* test enforcements
allow users to keep existing usernames without collision
* Normalize Translations (#4861)
* normalize translations
* add translations
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Improve DMR support
- Autodetect models installed
- Grab all models from hub.docker to show available
- UI to handle render,search, install, and management of models
- Support functionality for chat, stream, and agentic calls
* forgot files
* fix loader circle being too large
fix tooltip width command
adjust location of docker installer open for web platform
* adjust imports
* Support Gitee AI(LLM Provider)
* refactor(server): 重构 GiteeAI 模型窗口限制功能,暂时将窗口限制硬编码,计划使用外部 API 数据和缓存
* updates for Gitee AI
* use legacy lookup since gitee does not enable getting token context windows
* add more missing records
* reorder imports
---------
Co-authored-by: 方程 <fangcheng@oschina.cn>
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Add Default System Prompt Management
- Introduced a new route for fetching and updating the default system prompt in the backend.
- Added a new Admin page for managing the default system prompt, including a form for editing and saving changes.
- Updated the SettingsSidebar to include a link to the new Default System Prompt page.
- Implemented fetching of available system prompt variables for use in the prompt editor.
- Enhanced the ChatSettings and ChatPromptSettings components to support the new default system prompt functionality.
This commit lays the groundwork for improved management of system prompts across workspaces.
* Remove validation for system prompt in ChatSettings component
* Add comment for system prompt in workspaces model
* linting, simplify logic for default assumption
* dev build
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* prompt caching for anthropic llm and agent providers
* add UI for control of ENV
simplify implementation
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* feat: add AWS Bedrock API Key option to settings panel
* feat: Bedrock API key auth method
* fix: hide IAM note when using bedrock api key
* move to camcelCase identifier for bedrock api key use
linting
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* auto model context limit detection for ollama llm provider
* auto model context limit detection for lmstudio llm provider
* Patch Ollama to function and sync context windows like Foundry
* normalize how model context windows are cached from endpoint service
todo: move this into global utility class with MODEL_MAP
eager load models on boot to pre-cache them
add performance model improvements into ollama agent as well as apply n_ctx
* remove debug log
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add microsoft foundry local llm and agent providers
* minor change to fix early stop token + overloading of context window
always use user defined window _unless_ it is larger than the models real contenxt window
cache the context windows when we can from the API (0.7.*)+
Unload model forcefully on model change to prevent resource hogging
* add back token preference since some models have very large windows and can crash a machine
normalize cases
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Fix system prompt variable color logic by removing unused variable type from switch statement and adding new types.
* Add workspace id, name and user id as default system prompt variables
* Combine user and workspace variable evaluations into a single if statment, reducing redundant code.
* minor refactor
* add systemPromptVariable expandSystemPromptVariables test cases
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* feat: Implement CometAPI integration for chat completions and model management
- Added CometApiLLM class for handling chat completions using CometAPI.
- Implemented model synchronization and caching mechanisms.
- Introduced streaming support for chat responses with timeout handling.
- Created CometApiProvider class for agent interactions with CometAPI.
- Enhanced error handling and logging throughout the integration.
- Established a structure for managing function calls and completions.
* linting
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* add chroma cloud as new vector db provider
* update docker example env
* extend chroma class to chroma cloud
* update readme
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Added exa-search case to the search provider switch in web-browsing.js
* Added ExaSearchOptions component for API key input
* update
* Patch missing image crashing UI
Fix issue where ENV key did not exist or was saved on click
Update copy for provider
Add Docs for ENV keys for manual placements
update systemssettings for returning key saved to UI
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
