LOADING…
0%
Complete changelog

Commit

Fix upload, analysis, and search pipeline issues from review

Commit details

Commit notes

Uploads: - Stop the stale-session auto-cleanup from deleting in-flight direct uploads (and their storage objects): direct sessions never report chunk progress, so they are now only reaped after the presigned URL expires (45 min), in both the route-level and worker-level cleanup. The triplicated route cleanup block is factored into one helper. - Verify the actual uploaded object size at /complete (presigned PUTs don't enforce one): reject oversized files, re-check the storage quota when the real size exceeds the declared size, and record the real size on the sample. - Enforce MAX_AUDIO_FILE_SIZE (500MB) at initiate instead of failing after the user has uploaded the whole file. - Key batch-initiated upload sessions by request index instead of file name: duplicate basenames across pack folders (Kicks/01.wav, Snares/01.wav) previously shared one session and storage key, silently losing a file. - Never persist a session for a request that is about to be rejected.

Duplicate detection: - Store the browser-style quick fingerprint (first 1MB + last 1MB + size) on samples (new quick_hash column, migration 0030) and in the Redis cache; the pre-upload check only ever had this fingerprint, so matching it against the full content hash silently never fired for files over 2MB. - Scope /uploads/check-duplicates to the requesting user (it leaked other users' sample and pack names via content-hash probing) and replace its per-fingerprint queries with one DB query, one pack query, and one Redis MGET. - Let the analysis job fall back to the sample's stored content hash so analysis dedup can actually fire (jobs are enqueued before the hash is computed).

Analysis worker: - Stop wiping folder-derived tags and genre when analysis completes (and stop copying the other sample's tags on the dedup path). - Recover stale CLAP embedding jobs at startup; they were the only queue omitted, so crashed embedding jobs stayed stuck forever.

AI search / CLAP: - Map the query planner's drum-type vocabulary onto the CLAP taxonomy labels it filters ("perc"/"rim" -> percussion, bass -> bass+808, vocal -> vocal+vocal_chop); the mismatch guaranteed zero-result searches. Keep all requested types instead of only the first. - Add a 2.5s timeout + reduced retries to the planner LLM call and cache plans in Redis so search doesn't hang on a slow provider. - Stop caching a rejected facet-embedding promise, which permanently broke AI search after one transient CLAP outage. - Cache the structural pgvector readiness checks (three catalog queries per search) and capture the pgvector version; run the vector query with a raised hnsw.ef_search and iterative scans (pgvector >= 0.8) so scoped searches stop under-returning. - Cache CLAP text embeddings in Redis (they're deterministic), which also removes the double embedding of every AI-search query. - Decode only the model's 10s input window in the CLAP service instead of relying on the HF processor's rand_trunc: embeddings for clips longer than 10s were a random crop and differed on every run.

Library: - Restrict full-text search to user-meaningful text fields; it previously built a per-row tsvector over ~60 columns, matching queries like "true" or "120" against internal state. - Move UMAP computation to a background worker queue (it blocked the API event loop), batch the projection writes, and use a seeded PRNG so layouts are reproducible. - Remove the dead LLM categorization pass (no callers; the worker nulls its outputs).

Frontend: - Fix the similar-samples bar color ramp so higher similarity always reads as better.

Co-Authored-By: Claude Fable 5 <[email redacted]> Claude-Session: [private session redacted]

Files changed
21
Lines added
+949
Lines removed
−496
This page is a permanent record of commit ba7236aa.