Audio visualizations using Web Audio API

Have you ever seen some visual effects when playing music with your favourite music player? (Here’s some example photos.)

effects-2

effects-1

effects-3

This is just… great. But how they do this kind of effects? We need to understand how sound is made first. Sound is actually the vibrations of air (and more generally, objects) and the change in air pressure is recorded using a microphone. Everytime when you play a music, the speakers generates the vibrations using the change in air pressures recorded by microphone.

We can make use of these data to make visual effects from sounds. In this tutorial, I’ll teach you how to create a visualization like this:

web-audio-visuals-demo

In this demonstration, we’ll be able to drop audio files in and play the music with visuals. OK! Let’s start!

First, we need to set up the HTML for our visuals. It’ll be pretty simple as we only need a canvas.

Then, it’s time to code some JavaScript!

OK, it’s finished. I’ve prepared a live demo for all you guys. Enjoy! 🙂

GLSL fragment shaders in JavaScript!

Recent I see some very nice visual effects done in GLSL fragment shader here. I love them very much and I started learning GLSL fragment shaders afterwards.

I think that they are great but they must run on GPU. Only some browser vendors provide API to access the GPU (Actually, that API is called WebGL) so these visuals can’t run on some browsers that don’t offer the WebGL API.

In order to overcome these, I tried to port GLSL fragment shaders to javascript and draw the result using HTML5 canvas. The canvas API has a much broader browser support (and a flash fallback is available).

The demos I created are un-optimized and maybe a bit slow. After all, GLSL shaders are supposed to run on a GPU. However, it’s still worth to convert some of the GLSL shaders to achieve some nice effect such as post-processing of photos.

Here’s the demo:

  1. http://jsfiddle.net/licson0729/eBjQ8/
  2. http://jsfiddle.net/licson0729/YJqB9/
  3. http://jsfiddle.net/licson0729/7Qe34/
  4. http://jsfiddle.net/licson0729/T3hb7/
  5. http://jsfiddle.net/licson0729/9QVxA/

The techniques I used is to render the pixels one by one, with the render function be the one in the GLSL fragment shader. Also, we need to change the schematics (vec2, vec3, etc.) to its JavaScript equivalent.

Here’s the format of the code:

Hope you like it and encouraged you to start learning about computer graphics.