merlyn/collector
Sean Hatfield 610bdd4673
Allow custom headers in upload-link endpoint (#3695)
* allow custom headers in upload-link endpoint

* override loader.scrape to allow for passing of headers in langchain puppeteer

* lint

* Rename some variables
move positional args to named args
update documentation to reflect arg changes and funciton sigs
validate header object before attempting to end to forward to request

* update header validation for custom headers

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2025-04-22 12:47:12 -07:00
..
extensions Feature/drupalwiki collector (#3693) 2025-04-21 09:17:24 -07:00
hotdir Document Processor v2 (#442) 2023-12-14 15:14:56 -08:00
middleware Enable bypass of ip limitations via ENV in collector processing (#3652) 2025-04-21 11:10:41 -07:00
processLink Allow custom headers in upload-link endpoint (#3695) 2025-04-22 12:47:12 -07:00
processRawText Add tokenizer improvments via Singleton class and estimation (#3072) 2025-01-30 17:55:03 -08:00
processSingleFile feat: Add multilingual support for ocr module (#3325) 2025-02-27 12:31:17 -08:00
storage feat: Embed on-instance Whisper model for audio/mp4 transcribing (#449) 2023-12-15 11:20:13 -08:00
utils Enable bypass of ip limitations via ENV in collector processing (#3652) 2025-04-21 11:10:41 -07:00
.env.example devcontainer v1 (#297) 2024-01-08 15:31:06 -08:00
.gitignore Document Processor v2 (#442) 2023-12-14 15:14:56 -08:00
.nvmrc Document Processor v2 (#442) 2023-12-14 15:14:56 -08:00
index.js Allow custom headers in upload-link endpoint (#3695) 2025-04-22 12:47:12 -07:00
nodemon.json Document Processor v2 (#442) 2023-12-14 15:14:56 -08:00
package.json Windows development environment variables support (#3354) 2025-02-27 10:43:31 -08:00
yarn.lock Windows development environment variables support (#3354) 2025-02-27 10:43:31 -08:00