* Add Default System Prompt Management
- Introduced a new route for fetching and updating the default system prompt in the backend.
- Added a new Admin page for managing the default system prompt, including a form for editing and saving changes.
- Updated the SettingsSidebar to include a link to the new Default System Prompt page.
- Implemented fetching of available system prompt variables for use in the prompt editor.
- Enhanced the ChatSettings and ChatPromptSettings components to support the new default system prompt functionality.
This commit lays the groundwork for improved management of system prompts across workspaces.
* Remove validation for system prompt in ChatSettings component
* Add comment for system prompt in workspaces model
* linting, simplify logic for default assumption
* dev build
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* feat: add web app manifest and mobile PWA meta tags
* feat: serve dynamic manifest.json with custom branding for pwa
* feat: add ios status bar theming for pwa
* fix: prevent overscroll behavior for mobile
* fix: prevent ios safari auto-zoom on chat input
* fix: remove theme-color meta tags conflicting with ios status bar
* fix: add missing apple-mobile-web-app-capable meta tag for ios pwa
* fix: move catch-all route after manifest endpoint to prevent interception
* feat: add pwa detection helper and conditional styling for standalone mode
* PWA refactor
* undo changes to native CSS
* class fix
* proper response obj
* fix patch for import
* fix manifest errors
---------
Co-authored-by: Christian De Santis <christian.constantino98@gmail.com>
* Migrate gemini agents away from `Untooled`
* disable agents for gemma models as they are not supported for tool calling
* Dev build
resolve#4452 via function name prefix and then stripping within provider
* WIP agentic tool call streaming
- OpenAI
- Anthropic
- Azure OpenAI
* WIP rest of providers EXCLUDES Bedrock and GenericOpenAI
* patch untooled complete/streaming to use chatCallback provider from provider class and not assume OpenAI client struct
example: Ollama
* modify ollama to function with its own overrides
normalize completion/stream outputs across providers/untooled
* dev build
* fix message sanization for anthropic agent streaming
* wip fix anthropic agentic streaming sanitization
* patch gemini, webgenui, generic aibitat providers + disable providers unable to test
* refactor anthropic aibitat provider for empty message and tool call formatting
* Add frontend missing prop check
update Azure for streaming support
update Gemini to streamting support on gemini-* models
generic OpenAI disable streaming
verify localAI support
verify NVIDIA Nim support
* DPAIS, remove temp from call, support streaming'
* remove 0 temp to remove possibility of bad temp error/500s/400s
* Patch condition where model is non-streamable and no tools are present or called resulting in the provider `handleFunctionCallChat` being called - which returns a string.
This would then fail in Untooled.complete since response would be a string and not the expected `response.choices?.[0]?.message`
Modified this line to handle both conditions for stream/non-streaming and tool presence or lack thereof
* Allow generic Openai to be streamable since using untooled it should work fine
honor disabled streaming for provider where that concern may apply for regular chats
* rename function and more gemini-specific function to gemini provider
* add comments for readability
.complete on azure should be non-streaming as this is the sync response
* migrate CometAPI, but disable as we cannot test
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
* Create parse endpoint in collector (#4212)
* create parse endpoint in collector
* revert cleanup temp util call
* lint
* remove unused cleanupTempDocuments function
* revert slug change
minor change for destinations
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Add parsed files table and parse server endpoints (#4222)
* add workspace_parsed_files table + parse endpoints/models
* remove dev api parse endpoint
* remove unneeded imports
* iterate over all files + remove unneeded update function + update telemetry debounce
* Upload UI/UX context window check + frontend alert (#4230)
* prompt user to embed if exceeds prompt window + handle embed + handle cancel
* add tokenCountEstimate to workspace_parsed_files + optimizations
* use util for path locations + use safeJsonParse
* add modal for user decision on overflow of context window
* lint
* dynamic fetching of provider/model combo + inject parsed documents
* remove unneeded comments
* popup ui for attaching/removing files + warning to embed + wip fetching states on update
* remove prop drilling, fetch files/limits directly in attach files popup
* rework ux of FE + BE optimizations
* fix ux of FE + BE optimizations
* Implement bidirectional sync for parsed file states
linting
small changes and comments
* move parse support to another endpoint file
simplify calls and loading of records
* button borders
* enable default users to upload parsed files but NOT embed
* delete cascade on user/workspace/thread deletion to remove parsedFileRecord
* enable bgworker with "always" jobs and optional document sync jobs
orphan document job: Will find any broken reference files to prevent overpollution of the storage folder. This will run 10s after boot and every 12hr after
* change run timeout for orphan job to 1m to allow settling before spawning a worker
* linting and cleanup pr
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* dev build
* fix tooltip hiding during embedding overflow files
* prevent crash log from ERRNO on parse files
* unused import
* update docs link
* Migrate parsed-files to GET endpoint
patch logic for grabbing models names from utils
better handling for undetermined context windows (null instead of Pos_INIFI)
UI placeholder for null context windows
* patch URL
---------
Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
* WIP on mobile connections
todo: register devices
todo: data sync or connection
* improve connection flow and registration
add streaming from service
TODO: user scoping
* dev build mobile support
* fix path
* handle relative URLs
* handle localhost access in product
* add device de-register
* sync styles
* move UI to be out of the normal path since beta only
* Add user scoping to mobile connection requests
Remigrate DB for user associations
Implement temp token registration to prevent unauthorized device registration requests
cleanup middlewares
* WIP on embedder selection
TODO: apply splitting and query prefixes (if applicable)
* wip on upsert
* Support base model
support nomic-text-embed-v1
support multilingual-e5-small
Add prefixing for both embedding and query for RAG tasks
Add chunking prefix to all vector dbs to apply prefix when possible
Show dropdown and auto-pull on new selection
* norm translations
* move supported models to constants
handle null seelction or invalid selection on dropdown
update comments
* dev
* patch text splitter maximums for now
* normalize translations
* add tests for splitter functionality
* normalize
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
* Enable UI/UX for model swapping in chat window
* forgot component
* patch useGetProviders hook to set loading on change of provider
* dev build
* normalize translations
* patch how model default is provided
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
* implement importing of agent flows from community hub
* auto enable flow on import
* remove unused blocks for docker
prevent importing or saving of agent flows that have unsupported blocks for version or platform
* dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* wip: create direct output switch on last block and send response to ui
* lint
* Return flow on direct output enabled
prevent new blocks below direct output block
Update executor/aibitat to handle skipping of handler outputs
* dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add context window finder from litellm maintained list
apply to all cloud providers, have client cache for 3 days
* docker container bootup warning
* update invalid ENV warning
* rebased with current HEAD
* update newline printing
* PGVector support for vector db storage
* forgot files
* comments
* dev build
* Add ENV connection and table schema validations for vector table
add .reset call to drop embedding table when changing the AnythingLLM embedder
update instrutions
Add preCheck error reporting in UpdateENV
add timeout to pg connection
* update setup
* update README
* update doc
* auto STT submit control
* simplified STT auto and TTS autoplay
* add chatId to piper elements
* change defaults
* add chat customization page for enabling/disabling new features + options on account modal
* add translation placeholders
* show keybind command
* dev build
* Account modal ui prevent page reload + show Tooltip
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
* Translate main page landing
* normalize all languages
* Add a few translations, zh,tw,ja,es,de
* fix issues where translation lang change would not effect constants until full reload instead of on update
* dev
* fix noWorkspaceError key in translation usage
* Enable bypass of ip limitations via ENV in collector startup
resolves#3625
connect #3626
* dev build
* bump dockerx build action
* enable runtime setting config of collector requests
* comments and linting for option passing
* unset
* unset
* update docs link
* linting and docs
* wip getting started checklist page
* lint
* add new ui skeleton for new home page/checklist
* make legacy home page appearance setting
* dynamic checklist rendering/close checklist
* make home page functional + update cta buttons in ws settings
* lint
* remove unneeded routes
* wip fixing checklist items (create ws and embed doc broken) + news section updates
* lint
* FINALLY fix all functionality & remove hook to simplify logic
* lint
* hide/show options based on user role/mum enabled
* add hover states and redo checklist ui
* remove welcome checklist page
* add validation to task completion
* polish behavior of checklist/fix roles on checklist items
* lint
* light mode/use tailwind color classes
* remove tutorials link
* Modify how legacy page works
* small UI updates
* remove unused paths
cleanup explore features
* revert save button changes
* conditionally render legacy default chat page when in multiuser
* remove role checks in checklist
* remove role checks in quick links
* remove unused hook
* dark mode new home page checklist ui updates
* Add news logging to repo for record keeping
update new module to pull from CDN
* simplify landing markup
* light mode styles
* remove border in light mode from merge conflict
* Update ignores
* slide up dismiss
* prevent checklist popin
* confetti?
* fix url hash on navigate
* watch for event changes for updating checklist
* useMemo and callback memory optimization
* move handlers to constants via params fwd
* dev
* update github text
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* WIP MCP full compatibility layer
* implement MCP agent function wrapping and invocation methods
* Add `uvx` to docker bin for MCP executions
* dev build
* prune removed data
* Wrap MCP servers to lazy load items to not block the UI
Mobile bug fixes
* arm64 test build
* reset dev builder
* remove unused prop
* chore: rename Github to GitHub
Signed-off-by: Adam Setch <adam.setch@outlook.com>
* chore: rename Github to GitHub
Signed-off-by: Adam Setch <adam.setch@outlook.com>
* Undo some code changes for references
---------
Signed-off-by: Adam Setch <adam.setch@outlook.com>
Co-authored-by: timothycarambat <rambat1010@gmail.com>
* Add tokenizer improvments via Singleton class
linting
* dev build
* Estimation fallback when string exceeds a fixed byte size
* Add notice to tiktoken on backend
