Sunday, June 14, 2009

Music Box: Mapping & Visualizing Music

Music Box was created by Anita Lillie as part of her Thesis for the MIT HyperInstruments group. Created in Java and Processing, Music Box is a software used to organize large databases into visually meaningful maps. It starts off reasonably and you think it is going to be just another app but it soon becomes clear that this is an extremely useful tool. It can be used with any database but Anita has tailored it to music. You can check out her blog on the MusicBox and download her thesis PDF.

I have been looking around lately for more information on the visual representations of sound. I am starting to play around with the Jitter part of Max. It's slow and tedious and for now I can just about make a crappy music visualizer... a far cry away from interactive projected objects that have their own gravity and physical properties... but a start.

HyperInstrument group that Anita is part of was also home to Tristan Jehan. Tristans thesis was based around teaching computers how to listen in order that they may firstly learn the structure of music and then improvise using that learned structure as a base. Music Box above uses similar tactics to analyse acoustic quality of music. Although the desired outcomes are different the methods used to achieve those outcomes are similar. If you have a little time it is well worth watching Tristan's MIT thesis defense where he describes his work with music analysis and computer created music.

About 30 mins in he starts showing all the ways he used to analyse music. Pitch, beat, chord structure, timbre and many more. Many of the methods he used can be achieved in Max using the Msp external objects he created. I can't seem to get access to his site at the minute but you can download the
external for PC from Paul Hills website. It is interesting in his talk to see the techniques he used to interpret and map the results of analysis. Also his models use musical and psychoacoustic principles to help generate even more meaningful results. For example by analyzing the chord structure and the overall beat he was also able to determine the downbeat as being the beat where a new chord was likely to hit everytime.

To find out where this kind of research is going have a
listen to a talk about Echo Nest a project with Tristan Jehan to create a music recommendation tool based on analysis rather than buyer habits or other forms of personal recommendations. It is an interesting talk and offers a possible solution for those talented musicians labouring away on myspace and soundclick who cannot get people to listen to their music because they just do not have access to the promotion machine. Maybe this will make for a fairer playing field.

ED: Anita responded to a question I asked on her blog."Yes, this is very closely tied to Tristan Jehan’s PhD work. Please see the link for more info on what is being visualized; the data come straight from the Echo Nest Analyze API."


Anonymous said...

this is pretentious and arrogant

Anonymous said...

[ed. - arrogant, + ignorant]

AudioLemon said...

Is there anything in particular that offends you? I am surprised at this response. If you are connected to either of these projects I will gladly remove or alter the post. My intention was not to insult or cause any slight.

2BiT said...

Hmmm...yes not particularly constructive feedback there!
Also interested to know what your beef with this project is?

Mind you I've encountered some objections to music analysis from more 'traditional' composers who seem threatened by computerised or automated analysis of their (and others) work. Something along the lines of 'machines cannot interpret or quantify creativity'. Which is ignoring the basic starting point of computerised/algorhythmic analysis at which the author establishes the rules for categorisation, surely as individual an act as composing a melody?

Anyway Mr Anonymous, please elaborate on your rather terse statement and lets get the discussion going!

Aakash said...

This has been a pet project of mine for a few months and I am exploring different methods of analysis for myself (though, it will not be nearly the scale of this program). I am not looking for quite the same goals as MusicBox, but I am interested in if/where this study will go.