merlyn/.github/workflows/dev-build.yaml
Sean Hatfield 5039045f0c
VectorDB class migration (#4787)
* Migrate Astra to class (#4722)

migrate astra to class

* Migrate LanceDB to class (#4721)

migrate lancedb to class

* Migrate Pinecone to class (#4726)

migrate pinecone to class

* Migrate Zilliz to class (#4729)

migrate zilliz to class

* Migrate Weaviate to class (#4728)

migrate weaviate to class

* Migrate Qdrant to class (#4727)

migrate qdrant to class

* Migrate Milvus to class (#4725)

migrate milvus to class

* Migrate Chroma to class (#4723)

migrate chroma to class

* Migrate Chroma Cloud to class (#4724)

* migrate chroma to class

* migrate chroma cloud to class

* move limits to class field

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

* Migrate PGVector to class (#4730)

* migrate pgvector to class

* patch pgvector test

* convert connectionString, tableName, and validateConnection to static methods

* move instance properties to class fields

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

* Refactor Zilliz Cloud vector DB provider (#4749)

simplify zilliz implementation by using milvus as base class

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

* VectorDatabase base class (#4738)

create generic VectorDatabase base class

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

* Extend VectorDatabase base class to all providers (#4755)

extend VectorDatabase base class to all providers

* patch lancedb import

* breakout name and add generic logger

* dev tag build

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2026-01-13 15:24:42 -08:00

119 lines
4.5 KiB
YAML

name: AnythingLLM Development Docker image (amd64/arm64)
concurrency:
group: build-${{ github.ref }}
cancel-in-progress: true
on:
push:
branches: ['vectordb-class-migration'] # put your current branch to create a build. Core team only.
paths-ignore:
- '**.md'
- 'cloud-deployments/*'
- 'images/**/*'
- '.vscode/**/*'
- '**/.env.example'
- '.github/ISSUE_TEMPLATE/**/*'
- '.devcontainer/**/*'
- 'embed/**/*' # Embed should be published to frontend (yarn build:publish) if any changes are introduced
- 'browser-extension/**/*' # Chrome extension is submodule
- 'server/utils/agents/aibitat/example/**/*' # Do not push new image for local dev testing of new aibitat images.
- 'extras/**/*' # Extra is just for news and other local content.
jobs:
push_dev_build_to_dockerhub:
name: Push development build image to Docker Hub
runs-on: ubuntu-22.04-arm
permissions:
packages: write
contents: read
steps:
- name: Check out the repo
uses: actions/checkout@v4
- name: Check if DockerHub build needed
shell: bash
run: |
# Check if the secret for USERNAME is set (don't even check for the password)
if [[ -z "${{ secrets.DOCKER_USERNAME }}" ]]; then
echo "DockerHub build not needed"
echo "enabled=false" >> $GITHUB_OUTPUT
else
echo "DockerHub build needed"
echo "enabled=true" >> $GITHUB_OUTPUT
fi
id: dockerhub
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: v0.22.0
- name: Log in to Docker Hub
uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
# Only login to the Docker Hub if the repo is mintplex/anythingllm, to allow for forks to build on GHCR
if: steps.dockerhub.outputs.enabled == 'true'
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
with:
images: |
${{ steps.dockerhub.outputs.enabled == 'true' && 'mintplexlabs/anythingllm' || '' }}
tags: |
type=raw,value=dev
- name: Build and push multi-platform Docker image
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/Dockerfile
push: true
sbom: true
provenance: mode=max
platforms: linux/amd64,linux/arm64
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
# For Docker scout there are some intermediary reported CVEs which exists outside
# of execution content or are unreachable by an attacker but exist in image.
# We create VEX files for these so they don't show in scout summary.
- name: Collect known and verified CVE exceptions
id: cve-list
run: |
# Collect CVEs from filenames in vex folder
CVE_NAMES=""
for file in ./docker/vex/*.vex.json; do
[ -e "$file" ] || continue
filename=$(basename "$file")
stripped_filename=${filename%.vex.json}
CVE_NAMES+=" $stripped_filename"
done
echo "CVE_EXCEPTIONS=$CVE_NAMES" >> $GITHUB_OUTPUT
shell: bash
# About VEX attestations https://docs.docker.com/scout/explore/exceptions/
# Justifications https://github.com/openvex/spec/blob/main/OPENVEX-SPEC.md#status-justifications
# Fixed to use v1.15.1 of scout-cli as v1.16.0 install script is broken
# https://github.com/docker/scout-cli
- name: Add VEX attestations
env:
CVE_EXCEPTIONS: ${{ steps.cve-list.outputs.CVE_EXCEPTIONS }}
run: |
echo $CVE_EXCEPTIONS
curl -sSfL https://raw.githubusercontent.com/docker/scout-cli/main/install.sh | sh -s --
for cve in $CVE_EXCEPTIONS; do
for tag in "${{ join(fromJSON(steps.meta.outputs.json).tags, ' ') }}"; do
echo "Attaching VEX exception $cve to $tag"
docker scout attestation add \
--file "./docker/vex/$cve.vex.json" \
--predicate-type https://openvex.dev/ns/v0.2.0 \
$tag
done
done
shell: bash