Abstract

Can we make a famous rap singer like Eminem sing whatever our favorite song? Singing style transfer attempts to make this possible, by replacing the vocal of a song from the source singer to the target singer. This paper presents a method that learns from unpaired data for singing style transfer using generative adversarial networks.

system_framework

Figure 1.
System framework for singing style transfer.

Paper

We have uploaded the paper to arXiv, please click the following link to download. arxiv 1807.02254, 2018.

system_framework

Demo (inside test)

Here we show the results of our singing style transfer model (the current best one).
There are total 6 clips, where 3 of them are for male-to-female and the other 3 are for female-to-male.

Noted:

All of the results below are inside test!

Inside Test:

Testing audio clips may be available to the models during training since we extract the spectrograms with overlaps.

  1. Female-to-Male (clip #1 - Input / Output)
  2. Input

    Output

  3. Female-to-Male (clip #2 - Input / Output)
  4. Input

    Output

  5. Female-to-Male (clip #3 - Input / Output)
  6. Input

    Output

  7. Male-to-Female (clip #1 - Input / Output)
  8. Input

    Output

  9. Male-to-Female (clip #2 - Input / Output)
  10. Input

    Output

  11. Male-to-Female (clip #3 - Input / Output)
  12. Input

    Output

Demo (outside test)

Here we transfer 6 vocal clips in DSD100 (the dataset for singing voice separation) to validate our model's capability of generalization.
There are total 6 clips, where 3 of them are for male-to-female and the other 3 are for female-to-male.

Noted:

All of the results below are outside test!
(we use only vocal source tracks to be the input!)

Outside Test:

Testing audio clips are never be seen by the models.

  1. Female-to-Male (clip #1 - Input / Output)
  2. Input

    Output

  3. Female-to-Male (clip #2 - Input / Output)
  4. Input

    Output

  5. Female-to-Male (clip #3 - Input / Output)
  6. Input

    Output

  7. Male-to-Female (clip #1 - Input / Output)
  8. Input

    Output

  9. Male-to-Female (clip #2 - Input / Output)
  10. Input

    Output

  11. Male-to-Female (clip #3 - Input / Output)
  12. Input

    Output