* add API key param to Lemonade LLM Provider and Embedding Provider
* add LEMONADE_LLM_API_KEY to .env.example
* add api key to aibitat provider
* fix api key from being sent to frontend
* fix tooltip id
* add null fallback for `apiKey`
* remove console log
* add missing api keys
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add ask to run prompt for tools
* border-none on buttons
* translations
* linting
* i18n (#5263)
* extend approve/deny requests to telegram
* break up handler
* Add automatic chat mode with native tool calling support
Introduces a new automatic chat mode (now the default) that automatically invokes tools when the provider supports native tool calling. Conditionally shows/hides the @agent command based on whether native tooling is available.
- Add supportsNativeToolCalling() to AI providers (OpenAI, Anthropic, Azure always support; others opt-in via ENV)
- Update all locale translations with new mode descriptions
- Enhance translator to preserve Trans component tags
- Remove deprecated ability tags UI
* rebase translations
* WIP on image attachments. Supports initial image attachment + subsequent attachments
* persist images
* Image attachments and updates for providers
* desktop pre-change
* always show command on failure
* add back gemini streaming detection
* move provider native tooling flag to Provider func
* whoops - forgot to delete
* strip "@agent" from prompts to prevent weird replies
* translations for automatic-mode (#5145)
* translations for automatic-mode
* rebase
* translations
* lint
* fix dead translations
* change default for now to chat mode just for rollout
* remove pfp for workspace
* passthrough workspace for showAgentCommand detection and rendering
* Agent API automatic mode support
* ephemeral attachments passthrough
* support reading of pinned documents in agent context
Add ownership validation to prevent users from deleting or embedding
parsed files that don't belong to them. Previously, the delete and
embed endpoints only validated authentication but not resource ownership,
allowing users to delete attached files for users within workspaces they are also a member of.
Changes:
- Delete endpoint now filters by userId and workspaceId
- Embed endpoint validates file belongs to user and workspace (redundant)
- delete() returns false when no matching records found (returns 403)
- Added JSDoc comments for clarity
GHSA-p5rf-8p88-979c
Previously, suspended users could continue using browser extension
endpoints if they had created an API key before suspension. The normal
JWT session path blocked suspended users, but the browser extension
middleware did not.
Changes:
- Add suspension and user existence checks to validBrowserExtensionApiKey
- Delete browser extension API keys when a user is deleted
- Add deleteAllForUser method to BrowserExtensionApiKey model
GHSA-7754-8jcc-2rg3
* New chat history layout with chat bubbles (#4985)
* new chat history layout, remove message alignment setting
* remove orphaned chat alignment hook and MessageDirection
* remove workspace profile picture setting and fetch
* clean up unnecessary changes
* add light mode colors to chat ui and main page backgrounds
* update chat message and action icon colors for light mode
* update thinking and agent ui, layout, sizing
* update user message uploaded images ui
* update thought, agent containers to use new colors
* add truncatable content with gradient to user chat messages
* fix citations margin
* implement new edit message UI with save and submit actions
* add translations for TruncatableContent subcomponent
* remove unused props
* fix text colors for default mode chats, agent, thoughts container
* Normalize translations for new chat history layout (#5022)
* normalize translations
* update translations with DMR
* lint
* fix mismatched home container colors
* fix: add password character validation to onboarding single-user setup (#5037)
* fix single user mode password bug
* share const
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Native Tool calling (#5071)
* checkpoint
* test MCP and flows
* add native tool call detection back to LMStudio
* add native tool call loops for Ollama
* Add ablity detection to DMR (regex parse)
* bedrock and generic openai with ENV flag
* deepseek native tool calling
* localAI native function
* groq support
* linting, add litellm and OR native tool calling via flag
* fix: resolve Gemini agent 400 error on tool call responses (#5054)
* add gtc__ prefix to tool call names in Gemini agent message formatting
* resolve Gemini agent 400 error on tool call responses
* add comments explaining geminis thought signatures
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* fix: prevent CMD/CTRL+Arrow scroll from overriding textarea cursor movement (#5053)
prevent CMD/CTRL+Arrow scroll from overriding textarea cursor movement
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* linting, assistant speaker spacing and order, copy/edit order
---------
Co-authored-by: Marcello Fitton <106866560+angelplusultra@users.noreply.github.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Implement new citations UI (#5038)
* new chat history layout, remove message alignment setting
* remove orphaned chat alignment hook and MessageDirection
* remove workspace profile picture setting and fetch
* clean up unnecessary changes
* add light mode colors to chat ui and main page backgrounds
* update chat message and action icon colors for light mode
* update thinking and agent ui, layout, sizing
* update user message uploaded images ui
* update thought, agent containers to use new colors
* add truncatable content with gradient to user chat messages
* fix citations margin
* implement new edit message UI with save and submit actions
* add translations for TruncatableContent subcomponent
* remove unused props
* fix text colors for default mode chats, agent, thoughts container
* Normalize translations for new chat history layout (#5022)
* normalize translations
* update translations with DMR
* lint
* fix mismatched home container colors
* implement new citations ui with sources sidebar
* bottom sheet for mobile citations
* convert mobile citations bottom sheet to new modal design
* add score, border separators for mobile citations modal
* push down sources sidebar in password/multiuser mode
* fix animation gap, simplify sources sidebar by splitting state to persist data on animation
* add english translations
* fix spacing from citations sidebar when user has auth
* Normalize translations for new citation UI (#5087)
* normalize translations
* update translations using DMR
* fix pluralize to use i18n native solution
change reset to immediate clear
fix spacing for TTS when showing or not to not have space
* proper pluralize
* hide metrics on mobile, fix last message padding on mobile
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* New prompt input ui/tools menu (#5070)
* wip new prompt input ui/tools menu
* fix colors for prompt input
* redesign workspace llm selector, extract text size + model picker to components
* refactor ToolsMenu component
* fix colors/refactor WorkspaceModelPicker
* fix spacing in ws model picker, change order of tools menu tabs
* fix slash commands showing /reset instead of /exit during active agent session
* refactor ToolsMenu to be much simpler
* cleanup, fix behavior of setupup provider in WorkspaceModelPicker
* simplify AgentSkillsTab toggle logic
* add english translations for new components
* remove legacy slash command/agent popups, add ToolsMenu keyboard nav
* fix spacing of workspace model picker text
* fix SourcesSidebar and TextSizeMenu positioning after merge
* fix keyboard nav in ToolsMenu when clicking on tools button to open
* typo
* only auto pop up tools menu when prompt input is empty with /
* fix z index for tools menu on citation
* fix behavior of / in prompt input
* move global window agent session state to module level variable
* fix prompt input not clearing on /reset
* missing translations
* revert translating slash command
* fix STT auto-submit not working on home page
* Normalize translations for new prompt input/tools menu UI (#5130)
* normalize translations
* update translations using DMR script
* normalize translations
* update translations using DMR script
* remove slash_exit
* fix skills.js import after merge
* fix tooltip z-index rendering behind citations
* patch translation prune script to not remove special cases
* updates to tools input
* factory translations
* use safeJsonParse in clearPromptInputDraft
* normalize translations
* disable agent skill toggles during active agent sessions + show tooltip on disabled
* normalize translations
* handle enter key behavior when tools menu is open
* fix unfocusable modal for slash command edit/new
* fix sending prompt when editing/creating slash commands
* hide/show agent skills in tools menu based on role
* container borders for dark/light mode compliance to designs
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* update how tooltip works for agent menu
* update prompt input to show agent button with CTA in agent panel for user clarify
update agent session start prompt button in input
* translations
* translations + move regex for slash commands to constants
* fix open sidebar ux
* fix tools menu to always open to slash commands, dismiss auto pop up
* fix sidebar open/close button overlapping with ws model picker
---------
Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
Co-authored-by: Marcello Fitton <106866560+angelplusultra@users.noreply.github.com>
* fix: Migrate AzureOpenAI model key from OPEN_MODEL_PREF to prevent the naming collision. No effort necessary from current users.
* test: add backwards compat tests for AzureOpenAI model key migration
* patch missing env example file
* linting
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add eslint config to server
* add break statements to switch case
* add support for browser globals and turn off empty catch blocks
* disable lines with useless try/catch wrappers
* format
* fix no-undef errors
* disbale lines violating no-unsafe-finally
* ignore syncStaticLists.mjs
* use proper null check for creatorId instead of unreachable nullish coalescing
* remove unneeded typescript eslint comment
* make no-unused-private-class-members a warning
* disable line for no-empty-objects
* add new lint script
* fix no-unused-vars violations
* make no-unsued-vars an error
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
Introduces a new automatic chat mode (now the default) that automatically invokes tools when the provider supports native tool calling. Conditionally shows/hides the @agent command based on whether native tooling is available.
- Add supportsNativeToolCalling() to AI providers (OpenAI, Anthropic, Azure always support; others opt-in via ENV)
- Update all locale translations with new mode descriptions
- Enhance translator to preserve Trans component tags
- Remove deprecated ability tags UI
* Migrate all existing deprecated system preferences endpoint services to new service by field | delete old endpoint and service
* format
* destructure settings from response
* nitpick
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add the ability to edit existing SQL connections
* Enhance SQL connection management by adding connections prop to DBConnection and SQLConnectionModal components for improved duplicate detection and handling.
* format
* fix: prevent input defocus in SQL connection edit modal
Fixed an issue where typing in input fields would cause the field to lose
focus during editing. The useEffect dependency array was using the entire
existingConnection object, which could change reference on parent re-renders,
triggering unnecessary re-fetches and unmounting form inputs.
Changed the dependency to use the primitive database_id value instead of the
object reference, ensuring the effect only runs when the actual connection
being edited changes.
* fix: prevent duplicate SQL connections from being created
Fixed an issue where saving SQL connections multiple times would create
duplicate entries with auto-generated hash suffixes (e.g., my-db-abc7).
This occurred because the frontend maintained stale action properties on
connections after saves, causing the backend to treat already-saved
connections as new additions.
Backend changes (server/models/systemSettings.js):
- Modified mergeConnections to skip action:add items that already exist
- Reject duplicate updates instead of auto-renaming with UUID suffixes
- Check if original connection exists before applying updates
Frontend changes:
- Added hasChanges prop to SQL connector component
- Automatically refresh connections from backend after successful save
- Ensures local state has clean data without stale action properties
This prevents the creation of confusing duplicate entries and ensures
only the connections the user explicitly created are stored.
* Refactor to use existing system settings endpoint for getting agent SQL connections | Add better documentation
* Simplify handleUpdateConnection handler
* refactor mergeConnections to use map
* remove console log
* fix bug where edit SQL connection modal values werent recomputed after re-opening
* Add loading state for fetching agent SQL connections
* tooltip
* remove unused import
* Put skip conditions in switch statement
* throw error if default switch case is triggered
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Implement Unix username standard validations on username creation and updating.
* Remove leading underscore permissibility | Replace hardcoded username rules with a centralized USERNAME_REQUIREMENTS_TEXT for better maintainability.
* Add username requirements translations for invite and admin user creation | Replace hardcoded username requirements with localized strings in user modals
* Refactor username requirements localization
* Remove unneeded comment | Move Regex comment to validator fn
* Remove username validation utility function to keep validation responsibilities on the server | Allow onboarding flow multi-user mode username creation step to send pre-validated credentials to server.
* Enhance error handling in system endpoints by returning a JSON response with error details instead of a plain status for internal server errors.
* Update username requirement localization in AccountModal and UserSetup components to use centralized translation key.
* test enforcements
allow users to keep existing usernames without collision
* Normalize Translations (#4861)
* normalize translations
* add translations
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Improve DMR support
- Autodetect models installed
- Grab all models from hub.docker to show available
- UI to handle render,search, install, and management of models
- Support functionality for chat, stream, and agentic calls
* forgot files
* fix loader circle being too large
fix tooltip width command
adjust location of docker installer open for web platform
* adjust imports
* implement cohere agent support
* run yarn lint
* moderize Cohere
add supported langchain method
redo streaming since it was not working
looping of agent calls was not functioning
* change default model to real model tag
add case statement for model tag
* remove debug
* update default
* only whitelist known labels
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add refresh user functionality
* prettier
* add eslint disable comment for exhaustive-deps warning in AuthContext to stop nagging about navigate func
* remove unused imports and fix typo
* handle unsafe parse of undefined for in-session user deleted
* Refactor refreshUser function to handle errors and return structured response. Update AuthProvider to manage user data based on success status.
* Remove console error logging from promise catch in System model for cleaner error handling.
* change status from 404 to 400 and valid to success
* Refactor error handling in AuthProvider's refreshUser logic to remove redundant catch block and streamline user session management on failure.
* prettier
* reorder clauses - return errors
* refactor
account for all user modes
dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add refresh user functionality
* prettier
* add eslint disable comment for exhaustive-deps warning in AuthContext to stop nagging about navigate func
* remove unused imports and fix typo
* handle unsafe parse of undefined for in-session user deleted
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Support Gitee AI(LLM Provider)
* refactor(server): 重构 GiteeAI 模型窗口限制功能,暂时将窗口限制硬编码,计划使用外部 API 数据和缓存
* updates for Gitee AI
* use legacy lookup since gitee does not enable getting token context windows
* add more missing records
* reorder imports
---------
Co-authored-by: 方程 <fangcheng@oschina.cn>
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* feat: add base64 document attachment support to API chat endpoints
* remove parsed documents from api chat process
linting for simplicity
fix logic issues with flow of data through the pipeline
documentiation
* change var scope
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Add Default System Prompt Management
- Introduced a new route for fetching and updating the default system prompt in the backend.
- Added a new Admin page for managing the default system prompt, including a form for editing and saving changes.
- Updated the SettingsSidebar to include a link to the new Default System Prompt page.
- Implemented fetching of available system prompt variables for use in the prompt editor.
- Enhanced the ChatSettings and ChatPromptSettings components to support the new default system prompt functionality.
This commit lays the groundwork for improved management of system prompts across workspaces.
* Remove validation for system prompt in ChatSettings component
* Add comment for system prompt in workspaces model
* linting, simplify logic for default assumption
* dev build
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* paperless ngx data connector
* wip resync paperless ngx
* fix generateChunkSource for resyncing paperless ngx
* lint
* Refactor Paperless-NGX connector
Fix issue with date rendering in tooltip + extended width
Move tooltip details to be column for more space
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add User-Agent header on the requests sent by Generic OpenAI providers.
* Moved getAnythingLLMUserAgent helper fn to server/endpoints/utils.js and changed fallback version string to "unknown"
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Added the ability to pass in metadata to the /document/upload/{folderName} endpoint
* Added the ability to pass in metadata to the /document/upload-link endpoint
* feat: added metadata to document/upload api endpoint
* simplify optional metadata in document dev api endpoints
* lint
* patch handling of metadata in dev api
* Linting, small comments
---------
Co-authored-by: jstawskigmi <jstawski@getmyinterns.org>
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Create parse endpoint in collector (#4212)
* create parse endpoint in collector
* revert cleanup temp util call
* lint
* remove unused cleanupTempDocuments function
* revert slug change
minor change for destinations
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Add parsed files table and parse server endpoints (#4222)
* add workspace_parsed_files table + parse endpoints/models
* remove dev api parse endpoint
* remove unneeded imports
* iterate over all files + remove unneeded update function + update telemetry debounce
* Upload UI/UX context window check + frontend alert (#4230)
* prompt user to embed if exceeds prompt window + handle embed + handle cancel
* add tokenCountEstimate to workspace_parsed_files + optimizations
* use util for path locations + use safeJsonParse
* add modal for user decision on overflow of context window
* lint
* dynamic fetching of provider/model combo + inject parsed documents
* remove unneeded comments
* popup ui for attaching/removing files + warning to embed + wip fetching states on update
* remove prop drilling, fetch files/limits directly in attach files popup
* rework ux of FE + BE optimizations
* fix ux of FE + BE optimizations
* Implement bidirectional sync for parsed file states
linting
small changes and comments
* move parse support to another endpoint file
simplify calls and loading of records
* button borders
* enable default users to upload parsed files but NOT embed
* delete cascade on user/workspace/thread deletion to remove parsedFileRecord
* enable bgworker with "always" jobs and optional document sync jobs
orphan document job: Will find any broken reference files to prevent overpollution of the storage folder. This will run 10s after boot and every 12hr after
* change run timeout for orphan job to 1m to allow settling before spawning a worker
* linting and cleanup pr
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* dev build
* fix tooltip hiding during embedding overflow files
* prevent crash log from ERRNO on parse files
* unused import
* update docs link
* Migrate parsed-files to GET endpoint
patch logic for grabbing models names from utils
better handling for undetermined context windows (null instead of Pos_INIFI)
UI placeholder for null context windows
* patch URL
---------
Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
* WIP on mobile connections
todo: register devices
todo: data sync or connection
* improve connection flow and registration
add streaming from service
TODO: user scoping
* dev build mobile support
* fix path
* handle relative URLs
* handle localhost access in product
* add device de-register
* sync styles
* move UI to be out of the normal path since beta only
* Add user scoping to mobile connection requests
Remigrate DB for user associations
Implement temp token registration to prevent unauthorized device registration requests
cleanup middlewares
