MySong, from Microsoft Research, makes your singing sound a lot better than it really does

Cue Scoble tears. This is one hell of a technology from Microsoft Research that I think will have a huge impact on how amateurs and hobbyists write music. The technology and software prototype is called MySong.

In a nutshell, the software records your singing (preferably in tune) through a microphone, and it systematically generates an instrumental accompaniment for your song. The quality is even comparable with a professional accompanist, not to mention the cost and time involved.

If you don’t believe me, have a look at this quick 5-minute demo with loads of practical examples.

The technology was developed by Ian Simon from University of Washington in collaboration with Dan Morris and Summit Basu from Microsoft Research. They’ve published all of their research and methods in a paper (PDF) of the same name to present at the CHI Conference 2008 in a couple of months in Italy.

Without getting into too much detail, the technology uses the Hidden Markov statistical model which has been ‘trained’ by preprocessing a database of nearly 300 musical lead sheets. The sheets come from a variety of sources, genres and popularity to give a broad set of melody and chord sequence combinations. When it comes to the end user, it uses this abundance of data to find chord sequences which work best for each segment of melody. Presumably that means more ‘training’ can be done to improve the output even more.

The paper also points out “there is not a single correct accompaniment for a particular melody; chord selection will vary among musicians and genres, and a single musician may recognize many appropriate chord sequences for a single melody.” As a result, the software also gives the end user some room for creative adjustments in the form of “jazz factor” and “happy factor”. Both of which have a reasonable effect on the resulting accompaniment as demonstrated in the video.

If that wasn’t enough already, the researchers sought reviews from 30 independent musicians asking them to rate the different accompaniments produced by three different systems from one vocal input. They compared the result of MySong with hand-crafted chords and as well as Band-In-A-Box, the “state-of-the-art” commercially available software which they recognize to be the only automatic accompaniment generation system.

Here is just one example of the twelve they’ve ranked.

Original input
MySong
Band-In-A-Box
Hand-crafted

Like many cool technologies at Microsoft, this remains still a research project. Whilst there’s no written indication of a commercial product as a result of the technology, but I don’t see why they wouldn’t or even just release the software as is.

As for future work, they say “additional development will focus on improving and diversifying the audio generated by MySong; the system is already able to supply chords interactively to a pattern based arrangement tool, which results in compelling audio output. Several study participants indicated that MySong would be of significant value for learning music theory; we are thus excited about exploring educational applications of this technology.”

Note: This blog entry was originally posted on feeds.feedburner.com. View original post