From the rise of artificial intelligence writing platforms like ChatGPT to AI art platforms like DALL-E, the emergence of AI in the past year has made an impact around the world. The newest addition to the list of AI technology is deepfake tunes, which involves the use of AI to generate tunes that sound like there were created by a particular artist, without the original artist ever working on the project.

Beginning of Deepfake Music

According to the dictionary Merriam-Webster, deepfake music is “an image, or a video or audio recording that has been edited using an algorithm to replace the person in the original with someone else (especially a public figure) in a way that makes it look authentic.”

The first computer to sing was the IBM 7094 in 1961, singing the song “Daisy Bell”. The “vocals” of the computer were programmed by John Kelly and Carol Lockbaum with Max Mathews programming the accompaniment. This moment was revolutionary at its time; to have a computer sing had never been done before.

Fast forward to the present-day, any individual has the ability to make a song by replicating an artist’s sound using platforms like the SoftVC VITS Singing Voice Conversion project on GitHub.

How It Works

Deepfake music works by using machine learning algorithms to analyze an artist’s voice, style and musical elements used. Deepfakes rely on artificial neural networks, computer systems that recognize patterns in data. Developing them typically involves feeding hundreds or thousands of sounds and patterns of an artist’s singing into the artificial neural network, “training” it to identify and reconstruct patterns. The information from the AI system uses a large dataset of the artist’s music to learn the patterns and nuances of their voice, and then generates new lyrics and melodies that match their style.

Pros vs. Cons

The biggest advantage of AI music is that it allows the revival of an artist’s sound beyond the end of their career. For example, the late rapper Tupac Shakur has been resurrected in deepfake form with new songs created using his voice and style. Using deepfake technology, users have managed to introduce new songs featuring the lead singer in the classic rock band Queen, Freddie Mercury.

The controversy around deepfake music centers on the impact it could have on the music industry. According to EDM’s reporting, the Recording Industry Association of America (RIAA) states, “To the extent these services, or their partners, are training their AI models using our members’ music, that use is unauthorized and infringes our members’ rights by making unauthorized copies of our members works.” The RIAA report added, “In any event, the files these services disseminate are either unauthorized copies or unauthorized derivative works of our members’ music.”

Many predict that it could lead to a new wave of music piracy, as fans could create their own deepfake songs using an artist’s voice and style. This could have serious financial implications for artists and record labels, who rely on the sale of music to make a living.

Furthermore, one of the things that makes music so special is the personal connection between the artist and the listener. If the music is generated by an AI system, the piece could lack the emotional depth and authenticity that comes from a human artist’s experiences and performance.

In addition to emotional connectedness, there are also concerns about the ethics and legality of deepfake music. AI music can be considered as a form of intellectual property theft, as it uses an artist’s likeness without their permission. There are also concerns that it could be used to create fake songs that could damage an artist’s reputation or even lead to lawsuits.

The Future of Deepfake Music

Despite the controversies around it, deepfake music is here to stay. As AI technology continues to improve, it is likely that we will see more and more deepfake songs being created. However, it is important that we approach this technology with caution and respect for the artists whose voices and styles are being used.

Ultimately, the success of deepfake music will depend on whether it can capture the essence of what makes music so special: the emotional connection between the artist and the listener.