Muser is a “smart” music visualizer, made by Jon Shamir. It is an experiment to test how maching learning technology can be used to enhance music visualization.
You can see it live here: jonshamir.github.io/muser/.
A neural network - musicnn - predicts the musical genre for each second of a song. The predictions are then used to generate a color scheme:
The final visualization colors are based on the 5 most-fitting genres. The visualization shapes move and react to the audio data, like any classic music visualizer.
The idea for muser and the visualization itself is inspired by Wassily Kandinsky (1866-1944). Generally credited as the pioneer of abstract art, his work is well-known for its musical inspirations. Kandinsky associated specific tones and instruments to shapes and colors, thus “visualizing” a musical composition.
Circles in a Circle by Wassily Kandinsky, 1923
Colors for each genre were chosen according to the Musicmap project so that music genres which are stylistically closer will get similar colors.
The project structure is based on threejs-app.
- Visualization effects
- Add more songs
- Seek bar functionality
Clone, npm install, then:
# start development server
npm run startNow open localhost:9966. You can launch localhost:9966/?gui to open dat.gui.