LOADING…
0%
Complete changelog

Commit

feat(clap): zero-shot sample type + instrument classification

Commit details

Commit notes

Use CLAP's shared text/audio space to classify samples instead of the heuristic spectral/envelope rules engine. The audio embedding already computed for semantic search is now also compared against a versioned bank of text prompts (taxonomy.py) and softmaxed per group to label each sample's sample type (one-shot/loop/phrase), instrument (kick, 808, vocal, ...) and tonality (tonal vs percussive). No training and no model files required.

CLAP service: - taxonomy.py: versioned prompt bank + instrument->family map - classifier.py: dependency-free cosine/softmax scoring (unit-tested) - clap_model.py: cache label-prompt embeddings, embed_and_classify() - main.py: /classify and /classify-s3 return embedding + classification from a single forward pass; warm label cache on startup - tests/test_classifier.py: 17 tests covering taxonomy + scoring

Backend: - clap-client.ts: typed classifyAudioBytes / classifyAudioFromS3Key - worker.ts: CLAP job now persists clap_embedding + sample_type, instrument, instrument_confidence, classification_source, and fills category when empty - schema + migration 0025: additive sample_type/instrument columns + indexes - clap-search.ts: filter by instrument / sample_type - backfill script: also enqueues rows missing classification

Co-Authored-By: Claude Opus 4.8 <[email redacted]> Claude-Session: [private session redacted]

Files changed
14
Lines added
+931
Lines removed
−28
This page is a permanent record of commit 08903d0b.