Commit Graph

694 Commits

Author SHA1 Message Date
Timothy Carambat
753e1e9574
Filter malformed post-processed docs from file picker (#3853)
Filter malformed post-processed files from UI
2025-05-19 13:07:24 -07:00
timothycarambat
7e5e0952c3 remove comment in file util 2025-05-19 12:46:45 -07:00
timothycarambat
cabcc16e49 fix folders accumulating many files 2025-05-16 11:51:03 -07:00
Timothy Carambat
0e1de19408
Respect Max Chunk Len set - Generic OpenAI Embedder connection (#3838)
Respect Max Chunk Len set
resolves #3834
2025-05-16 11:18:24 -07:00
timothycarambat
6c36443657 patch Cached tag in UI not showing from #3833 2025-05-15 21:49:01 -07:00
Timothy Carambat
3bbceb82aa
3416 file picker load times (#3833)
* wip

* implment conditional streaming

* no bench
2025-05-15 13:54:14 -07:00
Sean Hatfield
9371bcc71e
Optional summarization for web-scraping flow block (#3496)
* remove summarization from web-scraping flow block

* add option to enable/disable summarization in web-scraping block

* change layout
summarize default to true (was the normal prior)

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2025-05-15 11:50:09 -07:00
timothycarambat
2450e49ac3 hoisting cleanup for format var 2025-05-14 16:25:17 -07:00
timothycarambat
dbd8afe1d2 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2025-05-14 15:26:20 -07:00
timothycarambat
605910b76d forgot files for DPAIS 2025-05-14 15:26:14 -07:00
Timothy Carambat
378ceaecec
Support Dell Pro AI Studio provider (#3829) 2025-05-14 15:10:48 -07:00
Timothy Carambat
e80492606a
Automatic Context window detection (#3817)
* Add context window finder from litellm maintained list
apply to all cloud providers, have client cache for 3 days

* linting
2025-05-14 11:03:19 -07:00
omahs
946be93f08
Fix typos (#3826)
* Fix typos

* language

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2025-05-14 09:30:28 -07:00
Sean Hatfield
2b274c62b7
Obsidian data connector (#3798)
* add obsidian vault data connector

* lint

* add english translations

* normalize translations

* improve file parser and reader

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2025-05-12 13:45:27 -07:00
timothycarambat
492570dfed patch Azure image reading regressions
resolves #3811
2025-05-12 11:10:35 -07:00
Shixian Sheng
aef455d977
Fixed grammar and typos (#3802)
* Update common.js

* Update README.md

* Update common.js

* Update README.md

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2025-05-12 09:42:50 -07:00
timothycarambat
d99ab52165 add error fix for pgvector 2025-05-09 14:20:52 -07:00
Timothy Carambat
e1b7f5820c
PGvector vector database support (#3788)
* PGVector support for vector db storage

* forgot files

* comments

* dev build

* Add ENV connection and table schema validations for vector table
add .reset call to drop embedding table when changing the AnythingLLM embedder
update instrutions
Add preCheck error reporting in UpdateENV
add timeout to pg connection

* update setup

* update README

* update doc
2025-05-09 12:27:11 -07:00
timothycarambat
f40559b180 fix null entry on new workspace 2025-05-08 08:34:37 -07:00
Sean Hatfield
10e65fc021
Chat prompt history menu (#3770)
* wip system prompt history sidebar ui

* lint

* backend/frontend implementation for prompt history wip

* lint

* rework ui

* add delete menu and delete chat history by id

* lint

* ref for menu button

* reorganize components + light mode styles

* lint

* UI refactor

* backend refactor

* remove unused import

* add border-none to all buttons

* fix spacing on dots 3 icon button

* add window to confirm

* add english translations

* normalize translations

* lin

* patch store logic

* sticky header

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2025-05-08 08:29:46 -07:00
Timothy Carambat
6fc0a6a644
Enable workflow rule for package verification (#3778)
enable workflow rule
2025-05-07 12:51:14 -07:00
Timothy Carambat
f0104fe1a9
Merge commit from fork 2025-05-07 09:37:31 -07:00
Danny Steenman
5500fa2bc5
feat: support for iam roles for bedrock client (#2632)
* feat: implement iam role auth for bedrock

* fix: make client refreshes properly when switching between iam_user and iam_role

* checkout agent flow

* fix aiprovider for bedrock in agent use

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2025-05-06 13:48:15 -07:00
Tristan Stahnke
b64a77f29f
Refactor AWS Bedrock Provider for Multi-modal Support & Correct Token Limits (#3714)
* Fixed two primary issues discovered while using AWS Bedrock with Anthropic Claude Sonnet models:
- Context Window defaults to 8192 maximum, which isn't correct
- Multimodal stopped working when removing langchain, which was transparently handling image_url to a format sonnet expects.

* Ran `yarn lint`

* Updated .env.example to have aws bedrock examples too

* Refactor for readability
move utils for AWS specific functionality to subfile
add token output max to ENV so setting persits

---------

Co-authored-by: Tristan Stahnke <tristan.stahnke+gpsec@guidepointsecurity.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2025-05-06 12:55:24 -07:00
timothycarambat
ae092b4d4a openai filter other niche models 2025-05-05 12:07:48 -07:00
Sean Hatfield
78e6117dd8
Filter image models from OpenAI (#3768)
filter image models from openai llm provider
2025-05-05 12:01:04 -07:00
Sebastián Darío Ramírez
4f23e44756
Add gemini-embedding-exp-03-07 model support (#3767)
* adds gemini-embedding-exp-03-07 support

* unset random changes, make model map

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2025-05-05 09:37:01 -07:00
Sean Hatfield
8912d0f0fc
Add option to control KoboldCPP max response tokens (#3746)
add option to control koboldcpp max response tokens
2025-05-02 14:12:06 -07:00
ALEKSEY KAPUSTYANENKO
e93964b67b
#3707 added sse and streamable stransport support for mcp servers (#3747)
* #3707 added sse and streamable stransport support for mcp servers

* #3707 yarn lint changes

* bump MCP SDK to latest

* option chain transport

* small refactor

---------

Co-authored-by: Aleksey Kapustyanenko <kapustyanenko@mercurydevelopment.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2025-05-02 14:09:25 -07:00
timothycarambat
62627fca7f add preUpdate checks on updateENV 2025-04-29 14:44:23 -07:00
timothycarambat
15acb19a68 Telemetry offset disabled count 2025-04-29 14:10:59 -07:00
timothycarambat
dacd67124a Prune telemetry events
Update telemetry for model tagging
Update readme on telemetry
2025-04-29 13:54:00 -07:00
Timothy Carambat
f9ef5840cb
Remove @google/generative-ai from AnythingLLM (#3740)
* Remove  from AnythingLLM
Migrate to full use of OpenAI Compatible endpoints'
resolves #3618

* lint
2025-04-29 11:39:56 -07:00
Shinya Suzuki
cd900f9e4c
Replace @azure/openai with openai, and update openai to version 4.95.1 (#3691)
* Replace @azure/openai to OpenAI lib

* Remove @azure/openai dependency and update openai to version 4.95.1

* linting

* update logging
fix translation dictionary error

* remove bad ENV key that DNE
linting
Patch Azure OpenAI
Migrate Azure Agent provider to use OpenAI Schema for tool calling performance

* unset

* migrate azure to use default OAI stream handler

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2025-04-29 11:21:39 -07:00
Sean Hatfield
610bdd4673
Allow custom headers in upload-link endpoint (#3695)
* allow custom headers in upload-link endpoint

* override loader.scrape to allow for passing of headers in langchain puppeteer

* lint

* Rename some variables
move positional args to named args
update documentation to reflect arg changes and funciton sigs
validate header object before attempting to end to forward to request

* update header validation for custom headers

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2025-04-22 12:47:12 -07:00
Timothy Carambat
1601eb986c
Enable bypass of ip limitations via ENV in collector processing (#3652)
* Enable bypass of ip limitations via ENV in collector startup
resolves #3625
connect #3626

* dev build

* bump dockerx build action

* enable runtime setting config of collector requests

* comments and linting for option passing

* unset

* unset

* update docs link

* linting and docs
2025-04-21 11:10:41 -07:00
Timothy Carambat
fd4929b4d2
Feature/drupalwiki collector (#3693)
* Implement DrupalWiki collector

* Add attachment downloading and processing functionality (#3)

* linting

* Linting
Add citation image
small refactors
add URL for citation identifier

---------

Co-authored-by: em <eugen.mayer@kontextwork.de>
Co-authored-by: rexjohannes <53578137+rexjohannes@users.noreply.github.com>
Co-authored-by: Eugen Mayer <136934+EugenMayer@users.noreply.github.com>
2025-04-21 09:17:24 -07:00
Timothy Carambat
05ced11650
Add support for /v1/document/upload* endpoints to support auto-add to workspace (#3692) 2025-04-21 08:28:33 -07:00
Timothy Carambat
7afbc6d11b
enable MCP tool use via @agent in API calls (#3683)
* enable MCP tool use via `@agent` in API calls

* relocate import
2025-04-20 14:49:22 -07:00
Timothy Carambat
a76ee21703
Handle env override for PATH and other defaults when env is specific (#3676)
* dev build
auto merge MCP PATH and NODE_PATH

* remove log
2025-04-18 12:44:48 -07:00
timothycarambat
f176a294f5 remove hard-coded token limit check
resolves #3674
2025-04-18 12:05:45 -07:00
Shinya Suzuki
98c46c04e4
Update Azure AI options and model map with new model configurations (#3660)
* Update Azure AI options and model map with new model configurations

* linting

---------

Co-authored-by: Shinya Suzuki <shinya.s.825@gmail.com>
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2025-04-16 09:08:40 -07:00
timothycarambat
1d1fb817b0 linting 2025-04-15 12:51:08 -07:00
Michał Rudziński
be27299897
handling of citations in openRouter provider #3581 (#3620)
* handling of citations in openRouter provider #3581

* Update pplx enrichToken function comment
Modify OR enrichToken to be generic handler function with optional params
handle _just_ Perplexity in-line citations since no other models support this functionality

* remove console log

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2025-04-15 10:57:09 -07:00
timothycarambat
1359151dd6 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2025-04-14 13:54:50 -07:00
Sean Hatfield
15f524ca37
Fix customize logo bug (#3628)
fix customize logo bug on first setup

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2025-04-14 11:51:04 -07:00
timothycarambat
9a14535cf4 update gitignore 2025-04-09 15:40:45 -07:00
Timothy Carambat
1b59295f89
Refactor Gemini to use OpenAI interface API (#3616)
* Refactor Gemini to use OpenAI interface API

* add TODO

* handle errors better (gemini)

* remove unused code
2025-04-07 17:18:31 -07:00
Timothy Carambat
699684e301
Disable Agent retry to prevent looping on crash (#3617) 2025-04-07 16:44:21 -07:00
Timothy Carambat
4ac900f645
Gemini model list sync (#3609)
* Update defaultModels.js

add gemma-3-27b-it to v1BetaModels

* Update defaultModels.js

20250330 model update

* Update defaultModels.js

remove text embedding

* Update name and inputTokenLimit modelMap.js

* Update gemini to load models from both endpoints
dedupe models
decide endpoint based on expieremental status from fetch
add util script for maintainers
reduce cache time on gemini models to 1 day

* remove comment

---------

Co-authored-by: DreamerC <dreamerwolf.tw@gmail.com>
2025-04-07 13:45:16 -07:00