Modern restoration of lost information in digital audio (MERLIN)

Abstract. Locally degraded or lost information is a problem in signal processing, e.g. old audio recordings or lost blocks of time-frequency (TF) coefficients in VoIP. Automatic recovering lost segments in recent years have been collectively referred to as inpainting. However, state-of-the-art methods assume simplistic signal models that fail to capture the characteristic structures of speech and music data. MERLIN provides the development of novel inpainting methods through the combination of adapted TF representations, appropriate signal models, state-of-the-art methods and the consideration of perceptual indicators. Signal models will be learned on the reliable TF information. Input signals will be decomposed into several layers, e.g. transients and harmonics. The inpainting process will be driven by modern algorithms implementing structured sparsity, low-rank approximation or state-of-the-art image processing techniques alongside preexisting procedures in an open-source software toolbox facilitating evaluation via objective and subjective criteria.

Goals. The project goal is to develop novel algorithms for recovering missing audio. The algorithms will be adaptive to the sound character. Moreover, a toolbox, a database of test signals, and a new methodology of objective and subjective evaluation of restoration quality will be the project outcomes.

Keywords: Audio restoration, inpainting, perceptual evaluation

• The Acoustics Research Institute of the Austrian Academy of Sciences, Vienna
  Dr. Nicki Holighaus
  Doz. Dr. Peter Balazs
  Ing. Zdenek Prusa, Ph.D.
  PhD student [tba]
• Signal Processing Laboratory, Faculty of Electrical Engineering and Communication, Brno University of Technology, Brno
  doc. Mgr. Pavel Rajmic, Ph.D.
  doc. RNDr. Vítězslav Veselý, CSc.
  doc. Jiří Schimmel, Ph.D.
  Bc. Pavel Záviška
  Ing. Marie Daňková
  Ondřej Mokrý

Funding