Research topics

Musical Source Separation

Identifying and extracting the individual sound sources that are present in a musical mixture can help content analysis applications such as transcription and classification, since it reduces the complexity of the signals subjected to feature extraction.

Audio Signal Classification

Automatic labeling of audio files allows efficiently searching and organizing large sound databases, such as personal music collections, online databases or sample libraries. I've been working on classification of music genres, sound effects and musical instruments.

Modeling of Musical Timbre

A compact, general and accurate model of the timbral characteristics of musical instruments can be used as a source of a priori knowledge for music content analysis applications such as transcription and instrument classification, for source separation, and for realistic sound transformation and synthesis.

Research projects

10/2008 - 3/2010Quaero, funded by OSEO, France
8/2007 - 9/2008Sample Orchestrator, funded by ANR, France
1/2006 - 7/2007K-Space, funded by the European Union (FP6)
7/2006 - 7/2007VISNET-II, funded by the European Union (FP6)
2/2004 - 12/2005VISNET, funded by the European Union (FP6)
7/2003 - 3/2005MPEG-7-Based Audio Annotation for the Archival of Digital Video, funded by the German Federal Ministry of Economics and Labour

PhD

In September 2008 I completed my PhD thesis, entitled

From Sparse Models to Timbre Learning:
New Methods for Musical Source Separation

In it, I explore several blind and supervised methods for the detection and extraction of instrumental sources from mono and stereo mixtures. For the supervised approaches, I use a novel timbre modeling procedure based on a compact representation of the spectral envelope and its temporal evolution. I also employed those models in classification tasks.

For more details and download of the PDF, please visit the dedicated webpage here.

Research talks

  • Musical Source Separation: Principles and State of the Art
    2nd Int. Workshop on Learning Semantics of Audio Signals (LSAS), IRCAM, Paris, June 21st 2008
    Slides:

    This tutorial offers an introduction to the principles of sound source separation, and to the advantages it offers to Music Information Retrieval applications. A number of general separation concepts, such as mixing models, sparse decompositions and Independent Component Analysis are introduced, followed by a set of techniques aimed specifically at musical mixtures, such as methods relying on Sinusoidal Modeling and on a priori training of instrument models.


  • Supervised Musical Source Separation from Mono and Stereo Mixtures based on Sinusoidal Modeling
    Séminaire Traitement du Signal Audio, TELECOM ParisTech, Paris, June 17th 2008
    Séminaire Recherche-Technologie, IRCAM, Paris, June 30th 2008
    Music Technology Group, Universitat Pompeu Fabra, Barcelona, July 22nd 2008
    Slides:

    Several new methods for the separation of individual musical instruments from instantaneous mixtures are presented. They rely on performing sinusoidal analysis on the mixture and implementing a set of auditory grouping and continuation cues. No harmonicity constraints are assumed, which allows to separate chords played by a single instrument as individual entities, as well as highly inharmonic sounds. Furthermore, no preliminary pitch detection is needed. A priori knowledge is available in the form of a set of timbre models that capture the temporal evolution of the spectral envelope. In the case of stereo mixtures, spatial cues are exploited using kernel density estimation for the detection of the mixing positions, and l1-norm minimization for the resynthesis of the sources. This gives a preliminary separation that is further refined by the abovementioned supervised and sinusoidal methods. Parts of the proposed systems can be additionally be used for other tasks such as musical instrument classification and detection of instruments in polyphonic mixtures.


  • Modeling the Spectral Envelope of Musical Instruments
    Séminaire Recherche-Technologie, IRCAM, Paris, April 12th 2006
    Slides:

    The spectral envelope is one of the most important features contributing to the notion of timbre. Mathematical models of the envelopes constitute an important source of a-priori knowledge for applications such as automatic transcription, classification or source separation, and allow the characterization of timbre perception. In this presentation, the demands of such a generalized representation are analyzed, and several strategies aimed at that goal are compared.