* Refactor non-local LLM Provider, Vector Database, and Embedding Engine privacy information to use their policy URLs instead of descriptions
* Update LLM Provider, Embedding Engine, and Vector Database sections to include privacy policy links
* fix broken links, lint
* Update AstraDB privacy policy URL in onboarding flow
* Refactor AnythingLLM Privacy & Data page to show managed provider privacy policy URLs
* Update Mistral privacy policy URLs in onboarding flow for consistency
* Abstract privacy policies of providers into a reusable component | Refactor Privacy & Data Handling Step of onboarding flow to focus on solely rendering that step | Move provider privacy policy maps into constants.js
* Remove commented-out code for third-party provider privacy policies in Privacy and Data Handling component
* Update privacy policy descriptions for consistency by adding periods at the end of sentences in ProviderPrivacy component and constants.js
* rescope constants for providers
* extract default to external function, add loading state
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* refactor localWhisper to use new custom FFMPEGWrapper class
* stub tests in github actions
* add back wavefile conversion to 16khz 32f to fix docker builds
* use afterEach for cleanup in ffmpeg tests
* remove unused FFMPEG_PATH env check
* use spawnSync for ffmpeg to capture and log output
* lint
* revert removal of try/catch around validateAudioFile for more helpful error msgs
* use readFileSync instead of createReadStream for less overhead
* change import to require for fix-path and stub import in tests
* refactor to singleton to preserve ffmpeg path
dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add local ESLint configuration and disable rules to allow for errorless state
* Remove unnecessary ESLint disable comments in AuthContext and usePromptInputStorage for cleaner code.
* Update eslint-plugin-react-hooks
* Configure prettier to work with eslint
* Removed trailing commas from eslint config
* Prettier to source code
* add a v2 lint script
* put back eslint-disable comments
* fix eslinter and prettier application
always apply --fix since we --write prettier, otherwise it fails
* precaution dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Refactor LLMPerformanceMonitor to use options object for measureStream parameters
* Refactor invocations of `measureStream` to use options arguments
* Change invocation of `measureStream` in anthropic provider to use options argument
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add model tag to chatCompletion
* add modelTag `model` to async streaming
keeps default arguments for prompt token calculation where applied via explict arg
* fix HF default arg
* render all performance metrics as available for backward compatibility
add `timestamp` to both sync/async chat methods
* extract metrics string to function
* implement cohere agent support
* run yarn lint
* moderize Cohere
add supported langchain method
redo streaming since it was not working
looping of agent calls was not functioning
* change default model to real model tag
add case statement for model tag
* remove debug
* update default
* only whitelist known labels
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Enhance OllamaEmbedder to support authentication by adding an authorization token in headers for client initialization.
* Add optional Auth Token input for Ollama embedding options
* move info elements
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add refresh user functionality
* prettier
* add eslint disable comment for exhaustive-deps warning in AuthContext to stop nagging about navigate func
* remove unused imports and fix typo
* handle unsafe parse of undefined for in-session user deleted
* Refactor refreshUser function to handle errors and return structured response. Update AuthProvider to manage user data based on success status.
* Remove console error logging from promise catch in System model for cleaner error handling.
* change status from 404 to 400 and valid to success
* Refactor error handling in AuthProvider's refreshUser logic to remove redundant catch block and streamline user session management on failure.
* prettier
* reorder clauses - return errors
* refactor
account for all user modes
dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* iterate over all pages in paperless-ngx data connector
* add error handling and data validation
* refactor to handle edge cases and null values
* catch edge case to prevent infinite loop
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Refactor Azure OpenAI integration to use OpenAI SDK and the v1 API | Enable streaming for Azure Open AI basic inference provider
* Add info tooltip to inform user about 'Model Type' form field
* Add 'model_type_tooltip' key to multiple language translations
* Validate AZURE_OPENAI_ENDPOINT in provider construction
* remove unused import, update error handler, rescope URL utils
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* test base image bump
* patch frontend compilation due to Vite QEMU exception
* migrate all runners to arm64 to avoid QEMU slowdown
* add back layer cleanup
* remove layer cache fix
* add refresh user functionality
* prettier
* add eslint disable comment for exhaustive-deps warning in AuthContext to stop nagging about navigate func
* remove unused imports and fix typo
* handle unsafe parse of undefined for in-session user deleted
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* fix astra not showing similarity score in citations ui
* fix qdrant not showing similarity score in citations ui
* fix weaviate not showing similarity score in citations ui
* fixed empty scroll track
* use overflow-x-none instead of overflow-x-auto for citation list
* keep behaviour of x scroll, but hide scrollbar since it can be overflowing for no reason
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add logic to save prompt input state to local storage and use as initial state on mount
* Synchronize prompt input state with parent component on mount
* lint
* Clear USER_PROMPT_INPUT_VALUE local storage value in all instances of auth clearing
* Remove USER_PROMPT_INPUT_VALUE local storage `removeItem` logic from excessive sources
* Refactor logic to cache prompt input value state by thread | abstract into a custom hook | rename localStorage key variables for clarity.
* Remove console log statement from usePromptInputStorage hook to clean up code.
* Update comments in usePromptInputStorage hook for clarity on localStorage handling
* Implement debounced localStorage updates in usePromptInputStorage hook to improve performance and prevent writing on every keystroke.
* Refactor localStorage handling in usePromptInputStorage hook to utilize safeJsonParse | Remove uneeeded comments
* Remove useEffect cleanup comment
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
