LOADING…
0%
Complete changelog

Commit

Add CLAP semantic audio search (text -> audio via pgvector)

Commit details

Commit notes

Layer a CLAP (Contrastive Language-Audio Pretraining) model on top of the existing pgvector setup so users can search audio with natural language (e.g. "dark 808s", "crispy hi-hats").

- services/clap-service: new Python FastAPI sidecar running laion/clap-htsat-fused, exposing /embed-text and /embed-audio in a shared 512-dim space (L2-normalized for cosine search). - DB: migration 0024 adds samples.clap_embedding vector(512) + HNSW cosine index (+ scoped partial indexes), guarded on pgvector availability. - Worker: new clap-embedding queue + processor that streams sample audio to the CLAP service and writes the vector; analysis fans out a job on completion. - Search: clap-search service runs the pgvector <=> query with hybrid relational filters (BPM range, key, type, ...), exposed via POST /api/v1/library/clap-search. - Batch backfill script (bun run clap:backfill) for existing samples. - Config, docker-compose service, npm scripts, env example, docs, and a unit test for the CLAP client.

Files changed
24
Lines added
+1,176
Lines removed
−1
This page is a permanent record of commit bedd5376.