MIR with AudioMuse-AI-SAE [P]

Hi all,
I recently read this paper:
Julien Guinot, Alain Riou, Elio Quinton, Gyorgy Fazekas. Steering dense music retrieval with open-vocabulary concept discovery.https://arxiv.org/abs/2608.08757

There is multiple model where you can get embedding from Song and Text so that you can search song by text. The challenges is when you get a result that don’t really respect your query, for example you search:

- POP viola with female vocalist

And as a result you usually get POP song with female vocalist because they are usually very common in you library, instead the Viola tend to disappear being uncommon. But the power of MIR isn’t exactly search to specific song even if uncommon ?

Here the idea of the paper is to get the compressed embbeding layer, make it sparse again and be able to identify for some common words exactly which neuron are activated try to avoiding overlapping with other neuron. Then you can improve the value of let’s say the Viola neurons and then bring back to the compressed embbeding space.

This is not mine, is only explanation, what I did is in the past a Distilled version of LAION CLAP called DCLAP that is around 7 million parameters and can efficiently run on CPU:
- https://github.com/NeptuneHub/AudioMuse-AI-DCLAP

The following the paper I also trained a SAE for DCLAP:
- https://github.com/NeptuneHub/AudioMuse-AI-SAE

So if you’re interested you can take a look at let me know. Are all free and opensource.

Finally if you want to see both of them in action I used both the model for AudioMuse-AI, a software dedicated to sonic analysis and automatic playlist creation (with various model):
- https://github.com/NeptuneHub/AudioMuse-AI

If you have any feedback, suggestion, idea, please let me know !