Music Visualization for the Deaf and Hearing Impaired

Here is a link that demonstrates my vision for music visualization. MIDI technology enables granular access to note, volume, and controller information, which enables rich displays that supersede current visualization technologies in today's media players.

Please forward this link to any deaf or hearing impaired people you know and send me feedback. I want to eventually upgrade this program to take advantage of Windows Vista graphics capabilities.

http://erangell.com/midivisualizationdemo

Also check out this one on YouTube:

http://www.youtube.com/watch?v=s0yQ8cKAXeU

This project can also become a Music Visualization Toolkit. I am thinking that Java might be a good test platform for it. The holy grail is to be able to recognize pitches and chords from WAV sounds and display them as MIDI sequences would be rendered. Wavelet research may pave the way towards that goal (it has already resulted in major advances in speech recognition).

Here are links to similar products:

MidiJam
This amazing program displays instruments playing midi songs in real time. It generates animation frames in near real-time, so you need a very fast computer to run it.

• MidiJam Demo 1
  
• MidiJam Demo 2
  
• MidiJam Demo 3
  

Animusic

Animusic produces professional computer animations for music on DVD. This company has invested in developing algorithms for generating musical instruments that pre-process midi sequences and generate graphics to look like the instruments are playing the notes, rather than reacting to them. In their videos, you can get a sense of the relative distances between notes. Most other media player plugins use a reactive algorithm which is triggered shortly after the MIDI events. While my program is reactive, I feel it gives a good sense of song structure, relationships between what each instrument is playing, and spatial layout of the channels.

Music Animation Machine by Steve Malinowski

Steve has done fascinating work with music animation. His website contains many valuable links for learning about the theories and technologies. Steve helped me as I was developing my program by giving me the suggestion about adding history trails which stay on the screen and gradually fade away. I would like to continue developing this concept into a Piano Velocity view which would use a vertical line above each piano key to represent how hard the key was struck, and have it gradually thin out analogous to the vibrating string being damped.

Carl Franklin's MIDI Router

Carl recently posted this demo of a program that enables MIDI data to be transmitted between computers over the Internet using a peer-to-peer connection. Some possibile applications of this technology are online music collaboration, online music lessons, and online performance broadcasting. Carl has also posted source code which enables .NET programmers to write applications using this technology.

Carl Franklin's Bubbles demo

Bubbles.zip is a game for toddlers that demonstrates combining colors and MIDI data. It draws filled shapes on the screen and plays notes via MIDI when you move the mouse or tap the keyboard. Source included. Requires .NET Framework 1.1 Uses Carl Franklin's MIDI component for .NET, which is a useful starting point for writing your own applications.

There's plenty more ideas on YouTube for those who want to research this technology.