Commit
Add AI sonic search over sample embeddings
Commit details
Commit notes
Lets producers search their library with natural language ("dark punchy 808 around 140 BPM in F minor") instead of exact filters. The query is turned into a synthesized 64-dim target embedding and matched against the pgvector audio embeddings already stored per sample.
Backend (services/sonic-search.ts + POST /api/v1/library/ai-search): - parseSonicQuery: uses the existing OpenAI/AI-SDK integration (gpt-5-nano, gated by ENABLE_LLM_ANALYSIS + OPENAI_API_KEY) to extract a structured sonic intent, with a deterministic keyword-lexicon fallback so the feature works with no API key. - synthesizeQueryEmbedding: maps timbral targets (brightness, energy, punchiness, noisiness) onto the known embedding dimensions, encoding only "presence" directions (cosine can't express absence under normalization). - pgvector cosine search (with JS cosine fallback) scoped to the user and refined by exact bpm/key/mode/type/loop filters; relaxes filters or falls back to metadata browsing when needed. - Unit tests for synthesis, key normalization and heuristic parsing.
Frontend: - aiSonicSearch API client + createAiSonicSearchQuery hook + query key. - AI search input above the sample grid (LibraryAiSearch). When active it drives the grid with ranked results and shows an interpretation banner.
[private session redacted]
- Files changed
- 10
- Lines added
- +1,328
- Lines removed
- −7