* wip new home page multiuser auth
* implement new login page designs for single and multiuser auth
* fix scroll overflow on mobile
* rounded custom logos
* fix up translation entry to use var
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Implement Unix username standard validations on username creation and updating.
* Remove leading underscore permissibility | Replace hardcoded username rules with a centralized USERNAME_REQUIREMENTS_TEXT for better maintainability.
* Add username requirements translations for invite and admin user creation | Replace hardcoded username requirements with localized strings in user modals
* Refactor username requirements localization
* Remove unneeded comment | Move Regex comment to validator fn
* Remove username validation utility function to keep validation responsibilities on the server | Allow onboarding flow multi-user mode username creation step to send pre-validated credentials to server.
* Enhance error handling in system endpoints by returning a JSON response with error details instead of a plain status for internal server errors.
* Update username requirement localization in AccountModal and UserSetup components to use centralized translation key.
* test enforcements
allow users to keep existing usernames without collision
* Normalize Translations (#4861)
* normalize translations
* add translations
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add keybindings to scroll to top and bottom of chat history
* fix isUserScrolling flag and set scrollToBottom to be instant instead of smoothe
* fix stream scroll
* fix default export by removing unneeded constant
* Replace file-defined `isMac` variable with global util
* extract funcitonality to hooks for clarity
* patch import
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Extract Model Table to component
Add provider icons to header rows and installed models
Light mode supported
Mapping for model name id hints to provider
Update DMR to filter chat models by ability since not available via hub API
* linting + dev
* fix incorrect import
* Improve DMR support
- Autodetect models installed
- Grab all models from hub.docker to show available
- UI to handle render,search, install, and management of models
- Support functionality for chat, stream, and agentic calls
* forgot files
* fix loader circle being too large
fix tooltip width command
adjust location of docker installer open for web platform
* adjust imports
* remove create workspace step for onboarding
* remove unused image
* workspace creation into dedicated useEffect + use translated workspace name
* dev tag
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Refactor non-local LLM Provider, Vector Database, and Embedding Engine privacy information to use their policy URLs instead of descriptions
* Update LLM Provider, Embedding Engine, and Vector Database sections to include privacy policy links
* fix broken links, lint
* Update AstraDB privacy policy URL in onboarding flow
* Refactor AnythingLLM Privacy & Data page to show managed provider privacy policy URLs
* Update Mistral privacy policy URLs in onboarding flow for consistency
* Abstract privacy policies of providers into a reusable component | Refactor Privacy & Data Handling Step of onboarding flow to focus on solely rendering that step | Move provider privacy policy maps into constants.js
* Remove commented-out code for third-party provider privacy policies in Privacy and Data Handling component
* Update privacy policy descriptions for consistency by adding periods at the end of sentences in ProviderPrivacy component and constants.js
* rescope constants for providers
* extract default to external function, add loading state
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add local ESLint configuration and disable rules to allow for errorless state
* Remove unnecessary ESLint disable comments in AuthContext and usePromptInputStorage for cleaner code.
* Update eslint-plugin-react-hooks
* Configure prettier to work with eslint
* Removed trailing commas from eslint config
* Prettier to source code
* add a v2 lint script
* put back eslint-disable comments
* fix eslinter and prettier application
always apply --fix since we --write prettier, otherwise it fails
* precaution dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add model tag to chatCompletion
* add modelTag `model` to async streaming
keeps default arguments for prompt token calculation where applied via explict arg
* fix HF default arg
* render all performance metrics as available for backward compatibility
add `timestamp` to both sync/async chat methods
* extract metrics string to function
* implement cohere agent support
* run yarn lint
* moderize Cohere
add supported langchain method
redo streaming since it was not working
looping of agent calls was not functioning
* change default model to real model tag
add case statement for model tag
* remove debug
* update default
* only whitelist known labels
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Enhance OllamaEmbedder to support authentication by adding an authorization token in headers for client initialization.
* Add optional Auth Token input for Ollama embedding options
* move info elements
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add refresh user functionality
* prettier
* add eslint disable comment for exhaustive-deps warning in AuthContext to stop nagging about navigate func
* remove unused imports and fix typo
* handle unsafe parse of undefined for in-session user deleted
* Refactor refreshUser function to handle errors and return structured response. Update AuthProvider to manage user data based on success status.
* Remove console error logging from promise catch in System model for cleaner error handling.
* change status from 404 to 400 and valid to success
* Refactor error handling in AuthProvider's refreshUser logic to remove redundant catch block and streamline user session management on failure.
* prettier
* reorder clauses - return errors
* refactor
account for all user modes
dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Refactor Azure OpenAI integration to use OpenAI SDK and the v1 API | Enable streaming for Azure Open AI basic inference provider
* Add info tooltip to inform user about 'Model Type' form field
* Add 'model_type_tooltip' key to multiple language translations
* Validate AZURE_OPENAI_ENDPOINT in provider construction
* remove unused import, update error handler, rescope URL utils
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add refresh user functionality
* prettier
* add eslint disable comment for exhaustive-deps warning in AuthContext to stop nagging about navigate func
* remove unused imports and fix typo
* handle unsafe parse of undefined for in-session user deleted
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* fixed empty scroll track
* use overflow-x-none instead of overflow-x-auto for citation list
* keep behaviour of x scroll, but hide scrollbar since it can be overflowing for no reason
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add logic to save prompt input state to local storage and use as initial state on mount
* Synchronize prompt input state with parent component on mount
* lint
* Clear USER_PROMPT_INPUT_VALUE local storage value in all instances of auth clearing
* Remove USER_PROMPT_INPUT_VALUE local storage `removeItem` logic from excessive sources
* Refactor logic to cache prompt input value state by thread | abstract into a custom hook | rename localStorage key variables for clarity.
* Remove console log statement from usePromptInputStorage hook to clean up code.
* Update comments in usePromptInputStorage hook for clarity on localStorage handling
* Implement debounced localStorage updates in usePromptInputStorage hook to improve performance and prevent writing on every keystroke.
* Refactor localStorage handling in usePromptInputStorage hook to utilize safeJsonParse | Remove uneeeded comments
* Remove useEffect cleanup comment
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* docs(i18n): Complete French translations
Complete all missing French translations for AnythingLLM UI
* linting
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Added bypassSSL parameter to constructor and implemented SSL bypass logic in fetchConfluenceData method
* Updated generateChunkSource function to include bypassSSL in the encrypted payload
* Updated the request body to include bypassSSL in the JSON payload sent to the backend
* Updated form submission to include bypassSSL parameter from the checkbox
* Added bypass_ssl: "Bypass SSL Certificate Validation" translation
* passed these parameters to fetchconfluencepage function for proper resync functionality
* allow ignore of SSL cert for Confluence
* add translations
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Support Gitee AI(LLM Provider)
* refactor(server): 重构 GiteeAI 模型窗口限制功能,暂时将窗口限制硬编码,计划使用外部 API 数据和缓存
* updates for Gitee AI
* use legacy lookup since gitee does not enable getting token context windows
* add more missing records
* reorder imports
---------
Co-authored-by: 方程 <fangcheng@oschina.cn>
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* implemented openrouter embedding model support
* ran yarn lint
* data handling entry
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Add Default System Prompt Management
- Introduced a new route for fetching and updating the default system prompt in the backend.
- Added a new Admin page for managing the default system prompt, including a form for editing and saving changes.
- Updated the SettingsSidebar to include a link to the new Default System Prompt page.
- Implemented fetching of available system prompt variables for use in the prompt editor.
- Enhanced the ChatSettings and ChatPromptSettings components to support the new default system prompt functionality.
This commit lays the groundwork for improved management of system prompts across workspaces.
* Remove validation for system prompt in ChatSettings component
* Add comment for system prompt in workspaces model
* linting, simplify logic for default assumption
* dev build
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* prompt caching for anthropic llm and agent providers
* add UI for control of ENV
simplify implementation
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* feat: add AWS Bedrock API Key option to settings panel
* feat: Bedrock API key auth method
* fix: hide IAM note when using bedrock api key
* move to camcelCase identifier for bedrock api key use
linting
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* paperless ngx data connector
* wip resync paperless ngx
* fix generateChunkSource for resyncing paperless ngx
* lint
* Refactor Paperless-NGX connector
Fix issue with date rendering in tooltip + extended width
Move tooltip details to be column for more space
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* feat: add web app manifest and mobile PWA meta tags
* feat: serve dynamic manifest.json with custom branding for pwa
* feat: add ios status bar theming for pwa
* fix: prevent overscroll behavior for mobile
* fix: prevent ios safari auto-zoom on chat input
* fix: remove theme-color meta tags conflicting with ios status bar
* fix: add missing apple-mobile-web-app-capable meta tag for ios pwa
* fix: move catch-all route after manifest endpoint to prevent interception
* feat: add pwa detection helper and conditional styling for standalone mode
* PWA refactor
* undo changes to native CSS
* class fix
* proper response obj
* fix patch for import
* fix manifest errors
---------
Co-authored-by: Christian De Santis <christian.constantino98@gmail.com>
* Refactor DefaultChat component to display a simple welcome message instead of mock chat interface onboarding text
* Add last visited workspace functionality
- Implemented localStorage management for the last visited workspace in DefaultChat and WorkspaceChat components.
- Updated UserButton to clear last visited workspace on logout.
- Refactored workspace navigation to prioritize last visited workspace if available.
* Refactor workspace fetching & last visited workspace logic into one useEffect
* Enhance loading state in DefaultChat component with skeleton UI elements for improved user experience
* minor UI spacing changes
* refactor order for guard clauses for early exit conditions
* move last known to end of getWorkspace loader
* languages for PR #4542 (#4550)
* languages for PR #4542
* Drop welcome message keys from translations
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Reimplement Cohere models
- Redo LLM implementation to grab models from endpoint and pre-filter
- Migrate embedding models to also grab from remote
- Add records for easy context window lookup'
* fix comment
* auto model context limit detection for ollama llm provider
* auto model context limit detection for lmstudio llm provider
* Patch Ollama to function and sync context windows like Foundry
* normalize how model context windows are cached from endpoint service
todo: move this into global utility class with MODEL_MAP
eager load models on boot to pre-cache them
add performance model improvements into ollama agent as well as apply n_ctx
* remove debug log
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add microsoft foundry local llm and agent providers
* minor change to fix early stop token + overloading of context window
always use user defined window _unless_ it is larger than the models real contenxt window
cache the context windows when we can from the API (0.7.*)+
Unload model forcefully on model change to prevent resource hogging
* add back token preference since some models have very large windows and can crash a machine
normalize cases
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* WIP agentic tool call streaming
- OpenAI
- Anthropic
- Azure OpenAI
* WIP rest of providers EXCLUDES Bedrock and GenericOpenAI
* patch untooled complete/streaming to use chatCallback provider from provider class and not assume OpenAI client struct
example: Ollama
* modify ollama to function with its own overrides
normalize completion/stream outputs across providers/untooled
* dev build
* fix message sanization for anthropic agent streaming
* wip fix anthropic agentic streaming sanitization
* patch gemini, webgenui, generic aibitat providers + disable providers unable to test
* refactor anthropic aibitat provider for empty message and tool call formatting
* Add frontend missing prop check
update Azure for streaming support
update Gemini to streamting support on gemini-* models
generic OpenAI disable streaming
verify localAI support
verify NVIDIA Nim support
* DPAIS, remove temp from call, support streaming'
* remove 0 temp to remove possibility of bad temp error/500s/400s
* Patch condition where model is non-streamable and no tools are present or called resulting in the provider `handleFunctionCallChat` being called - which returns a string.
This would then fail in Untooled.complete since response would be a string and not the expected `response.choices?.[0]?.message`
Modified this line to handle both conditions for stream/non-streaming and tool presence or lack thereof
* Allow generic Openai to be streamable since using untooled it should work fine
honor disabled streaming for provider where that concern may apply for regular chats
* rename function and more gemini-specific function to gemini provider
* add comments for readability
.complete on azure should be non-streaming as this is the sync response
* migrate CometAPI, but disable as we cannot test
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
* Fix system prompt variable color logic by removing unused variable type from switch statement and adding new types.
* Add workspace id, name and user id as default system prompt variables
* Combine user and workspace variable evaluations into a single if statment, reducing redundant code.
* minor refactor
* add systemPromptVariable expandSystemPromptVariables test cases
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* fix: resolve Firefox search icon overlapping placeholder text
- Increase input left padding from pl-4 to pl-9 to provide clearance
- Remove redundant placeholder:pl-4 class
- Ensures 24px spacing between search icon and text content
* Update SearchBox component to adjust padding on focus state
---------
Co-authored-by: neha <neha@posthog.com>
Co-authored-by: angelplusultra <macfittondev@gmail.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
Update DPAISOptions component and constants to use new OpenAI endpoint (#4432)
- Changed placeholder in DPAISOptions component to reflect updated endpoint.
- Updated DPAIS_COMMON_URLS in constants to include '/openai' in URLs.
* Enhanced Chat Embed History View
* Robust Markdown Rendering
Improved "Thinking" View
* feat: Improve markdown rendering in chat embed history
* update ui for show/hide thoughts in embed chat history
* refactor -always show thoughts if available
* patch unused imports and use safeJsonParse
* update fallback for loading state to always reset
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
* feat: Implement CometAPI integration for chat completions and model management
- Added CometApiLLM class for handling chat completions using CometAPI.
- Implemented model synchronization and caching mechanisms.
- Introduced streaming support for chat responses with timeout handling.
- Created CometApiProvider class for agent interactions with CometAPI.
- Enhanced error handling and logging throughout the integration.
- Established a structure for managing function calls and completions.
* linting
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Update Security UI to match all other Settings
* Normalizing translations
* Running yarn lint
* Set back all the null values
* fix spacing + lint
* remap name
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* add chroma cloud as new vector db provider
* update docker example env
* extend chroma class to chroma cloud
* update readme
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Update common.js
Improving the Romanian translation and filling in missing elements.
* linting
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Added exa-search case to the search provider switch in web-browsing.js
* Added ExaSearchOptions component for API key input
* update
* Patch missing image crashing UI
Fix issue where ENV key did not exist or was saved on click
Update copy for provider
Add Docs for ENV keys for manual placements
update systemssettings for returning key saved to UI
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Create parse endpoint in collector (#4212)
* create parse endpoint in collector
* revert cleanup temp util call
* lint
* remove unused cleanupTempDocuments function
* revert slug change
minor change for destinations
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Add parsed files table and parse server endpoints (#4222)
* add workspace_parsed_files table + parse endpoints/models
* remove dev api parse endpoint
* remove unneeded imports
* iterate over all files + remove unneeded update function + update telemetry debounce
* Upload UI/UX context window check + frontend alert (#4230)
* prompt user to embed if exceeds prompt window + handle embed + handle cancel
* add tokenCountEstimate to workspace_parsed_files + optimizations
* use util for path locations + use safeJsonParse
* add modal for user decision on overflow of context window
* lint
* dynamic fetching of provider/model combo + inject parsed documents
* remove unneeded comments
* popup ui for attaching/removing files + warning to embed + wip fetching states on update
* remove prop drilling, fetch files/limits directly in attach files popup
* rework ux of FE + BE optimizations
* fix ux of FE + BE optimizations
* Implement bidirectional sync for parsed file states
linting
small changes and comments
* move parse support to another endpoint file
simplify calls and loading of records
* button borders
* enable default users to upload parsed files but NOT embed
* delete cascade on user/workspace/thread deletion to remove parsedFileRecord
* enable bgworker with "always" jobs and optional document sync jobs
orphan document job: Will find any broken reference files to prevent overpollution of the storage folder. This will run 10s after boot and every 12hr after
* change run timeout for orphan job to 1m to allow settling before spawning a worker
* linting and cleanup pr
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* dev build
* fix tooltip hiding during embedding overflow files
* prevent crash log from ERRNO on parse files
* unused import
* update docs link
* Migrate parsed-files to GET endpoint
patch logic for grabbing models names from utils
better handling for undetermined context windows (null instead of Pos_INIFI)
UI placeholder for null context windows
* patch URL
---------
Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
* WIP on mobile connections
todo: register devices
todo: data sync or connection
* improve connection flow and registration
add streaming from service
TODO: user scoping
* dev build mobile support
* fix path
* handle relative URLs
* handle localhost access in product
* add device de-register
* sync styles
* move UI to be out of the normal path since beta only
* Add user scoping to mobile connection requests
Remigrate DB for user associations
Implement temp token registration to prevent unauthorized device registration requests
cleanup middlewares
* WIP on embedder selection
TODO: apply splitting and query prefixes (if applicable)
* wip on upsert
* Support base model
support nomic-text-embed-v1
support multilingual-e5-small
Add prefixing for both embedding and query for RAG tasks
Add chunking prefix to all vector dbs to apply prefix when possible
Show dropdown and auto-pull on new selection
* norm translations
* move supported models to constants
handle null seelction or invalid selection on dropdown
update comments
* dev
* patch text splitter maximums for now
* normalize translations
* add tests for splitter functionality
* normalize
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
* docs(zh-tw, zh-cn): Complete missing Traditional and Simplified Chinese translations
This commit aims to complete all previously 'null' Traditional Chinese and Simplified Chinese translation keys in the project.
**Traditional Chinese (zh-tw):**
- Completed all 'null' descriptions in `/frontend/src/locales/zh_TW/common.js`.
**Simplified Chinese (zh-cn):**
- Completed all 'null' descriptions in `/frontend/src/locales/zh/common.js`.
---
此提交旨在補齊專案中所有原為 'null' 的繁體中文和簡體中文翻譯鍵值。
**繁體中文翻譯 (Traditional Chinese - zh-tw):**
- 完善了 `/frontend/src/locales/zh_TW/common.js` 裡所有 'null' 的說明。
**簡體中文翻譯 (Simplified Chinese - zh-cn):**
- 完善了 `/frontend/src/locales/zh/common.js` 裡所有 'null' 的說明。
* linting
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* configurable message limit for embed widget
* remove console log
* make field optional + add fallback
* rework validation logic
* lint
* remove field specific guard, it cannot be lte 0 like all other fields
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* feat: Add Korean (ko) locale translations
- Add comprehensive Korean translations for UI elements
- Include main page features, workspace settings, and chat components
- Support Korean localization for common user interactions
* updates, lint
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* add hint for chats in query mode
* add and normalize translations
* improve semantic HTML for linking
* use existing util
* linting
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Enable UI/UX for model swapping in chat window
* forgot component
* patch useGetProviders hook to set loading on change of provider
* dev build
* normalize translations
* patch how model default is provided
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
* implement importing of agent flows from community hub
* auto enable flow on import
* remove unused blocks for docker
prevent importing or saving of agent flows that have unsupported blocks for version or platform
* dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* wip: create direct output switch on last block and send response to ui
* lint
* Return flow on direct output enabled
prevent new blocks below direct output block
Update executor/aibitat to handle skipping of handler outputs
* dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* wip create skeleton for new embed chat ui/ux
* update ui for embed chats
* lint
* update sidebar/paths
* remove old embed pages
* patch broken link
* add created timestamp to differentiate embeds
update translation key to lowercase
add created at translation key
* update text colors
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
