Processing images on the web!

Want to add some effects to the images on your own web? Before the era of HTML5, this is only possible with browser-specific extensions (I’m talking about you IE filters & -webkit-filters) and this restricted people to do the processing on the server-side or using Flash.

Now, with the help of new HTML5 APIs called the canvas API, we can draw things like images on a canvas in raster format and do processing. In this post, I’ll talk about the basics of the canvas API and how to process images directly.

The <canvas> tag is supported in:

  1. Chrome
  2. Firefox
  3. IE 9+
  4. Safari
  5. Opera

First, we need to define the canvas that the image is going to be drawn. This can be done by a HTML <canvas> element.

<canvas id="image" width="300" height=150">
    <p>Your browser doesn't support the canvas element.</p>
</canvas>

The content inside the <canvas> element will be ignored by a browser that recognizes the tag so we can put a message to show the user that their browser doesn’t support this feature.

Then, we need to get the canvas using the traditional DOM API.

var canvas = document.getElementById('image');

Here’s the most important point: we’re going to get the canvas context that contains several methods that allows us to draw thing on.

var context = canvas.getContext('2d'); //Get the 2d canvas context

You may have a question: why is it be ‘2d’ ? It’s because the canvas API is designed not just for 2d raster drawing but also for 3d drawings and we are not discussing that here.

Then, load the image with new Image() constructor and draw it on the canvas by using the context.drawImage(image,x,y,w,h) method.

var image = new Image();
image.onload = function(){
    //Set the canvas' size as same as the image
    canvas.width = this.width;
    canvas.height = this.height;
    //Draw the image to the canvas
    context.drawImage(this, //the image object
                      0, //The x-position of the image, 0 means the left
                      0, //The y-position of the image, 0 means the top
                      this.width, //The width of the image
                      this.height //the height of the image
                     );
};
image.src = "<INSERT YOUR IMAGE'S PATH HERE>";

After that, you should see the image is drawn to the canvas. Now, I’m going to show you how to get access to the image data and do processing on it. To access the image data, we use the context.getImageData(x,y,w,h) method.

var pixels = context.getImageData(0,0,canvas.width,canvas.height); //Get the image data
var data = pixels.data;
//data is a large one-dimensional array containing all the pixels' colour values.
//Every pixels contains 4 colour channels: Red, Green, Blue and Alpha values,
//they are contained in the order of [R,G,B,A,R,G,B,A,...]
for(var i = 0; i < data.length; i += 4){
    //Do the processing here
    //The colour values are stored in this order:
    var r = data[i]; //The red component
    var g = data[i+1]; //The green component
    var b = data[i+2]; //The blue component
    var a = data[i+3]; //The alpha component
}
context.putImageData(pixels,0,0); //Put the data back to the canvas

See next page to know how to handle the colour values properly and make some effects!

Turn asynchronous functions to their synchronous counterpart

Recently, asynchronous programming becomes a hit as slow jobs can perform while not blocking other jobs to be done. However, sometimes we want something to be done in a synchronous way like getting a single value from a query (you know callbacks are clumsy). In order to solve the  problem, turning asynchronous functions to synchronous can solve the problem.

To turn asynchronous functions to synchronous, we need to set up a loop checking the result, therefore cause blocking.

Here’s the code: (we use javascript here because it’s straightforward)

function somethingSync(args){
    var ret; //the result-holding variable
    //doing something async here...
    somethingAsync(args,function(result){
        ret = result;
    });
    while(ret === undefined){} //wait for the result until it's available, cause the blocking
    return ret;
}

Create a simple web spider in node.js

I’ve seen many web crawlers written in other languages like PHP, Python, Ruby, etc. However, there’s no one that notices node.js is also good in doing these kind of things. So I utilizes asynchronous network I/O provided by node.js to create a fast, stable web crawler (spider) in node.js and it’s very simple. Here I’ll reveal the logic of a web crawler and allow you to create your own.

A web spider is a long-running program that finds websites and record them. After that we can search the records generated by web spiders to find information we want. It’s the basic unit of a search engine.

First, we need to search for links from a base webpage and record the webpage to the database. Then, follow the links found and repeat the first step to discover more webpages.

Here are some modules that helps us coding a web spider:

  1. request (helps fetching webpages)
  2. cheerio (helps parsing webpages)
  3. mysql (for indexing pages)
  4. async (helper library)

And here’s the code:







Here’s how it works:

First, it crawls the base page specified in config.json first. Then it tries to find all links from the page content and add them to the queue, then the first item on the queue will be executed. This process continues again and again…

Also, we store the queue in a MySQL database for scalability. This script can be spawned across multiple machines.

My own Text-To-Speech (TTS) Service!

Update: Multi-language support added.

I’m now release my own Text-To-Speech Service made with node.js. I think an easy-to-use TTS service is necessary for web applications nowadays. However, creating your own one isn’t easy. So I created one based on eSpeak and hosted an API for you to use easily.

The API is fairly simple. Just issue a HTTP GET or HTTP POST request to the address below (https protocol also supported):

http://licson-node.hp.af.cm/tts

And it’ll return a wave file representing the result. Here’s some parameters you need to provide with the request.

  1. text – The text to read. (required)
  2. speed – The speed of reading. (optional) Default: 180 (Range 10-300)
  3. pitch – The pitch of reading. (optional) Default: 50 (Range 0-100)
  4. wordgap – The addtional time delay between each word. (optional) Default: 0
  5. lang – The language of the text. (optional) Default: en/en  See all language options below.

Here’s an example of using the API through HTTP GET requests:

http://licson-node.hp.af.cm/tts?text=Hi!+I+can't+wait+to+meet+you.&speed=185&lang=en/en-us

Supported languages:

  1. ca (Catalan)
  2. cs (Czech)
  3. de (German)
  4. el (Greek)
  5. en/en (English)
  6. en/en-n (English, regional)
  7. en/en-rp (English, regional)
  8. en/en-sc (English, Scottish)
  9. en/en-us (English, US)
  10. en/en-wm (English, regional)
  11. eo (Esperanto)
  12. es (Spanish)
  13. es-la (Spanish, Latin America)
  14. fi (Finnish)
  15. fr (French)
  16. hu (Hungarian)
  17. it (Italian)
  18. kn (Kannada)
  19. la (Latin)
  20. lv (Latvian)
  21. nl (Dutch)
  22. pl (Polish)
  23. pt (Portuguese, Brazil)
  24. pt-pt (Portuguese, European)
  25. ro (Romanian)
  26. sk (Slovak)
  27. sv (Swedish)
  28. tr (Turkish)

These parameters can be sent as query parameters or form body when using HTTP POST. Beware that long passages should be splitted in parts to avoid long processing times. If you need to render a long passage, send the text with HTTP POST because query parameters have a length limit of 2,000 characters.

Here’s an example of the speech: http://ow.ly/kHtxK. If you like, please help me do this survey in order to improve the service.

Multiplayer Piano – a nice website for playing music together

http://www.multiplayerpiano.com/

This website is for people to play songs together. I first discovered this in Chrome Experiments and it looked attractive. I found it good so I’m going to share it.

The user interface of this website is pretty straightforward. It has a piano with all 88 keys and a simple chat interface. You can click on the keys to play, or you can use your keyboard. It also has a cool feature that allows you to plug in your own MIDI keyboard and play the notes directly and that suits me very much as I have a MIDI keyboard.

When using the MIDI keyboard you may get problems with those delayed notes. To solve the problem you just need to minimize the window and it’ll get better soon.

The chat interface are fairly simple: you just need to type in the words and press enter to send. The only thing that hurts is the colour of the text makes checking typing errors difficult.

Overall, it’s a really good website, so definitely worth to be used, before or while looking for best piano keyboard to practice at home.

GLSL fragment shaders in JavaScript!

Recent I see some very nice visual effects done in GLSL fragment shader here. I love them very much and I started learning GLSL fragment shaders afterwards.

I think that they are great but they must run on GPU. Only some browser vendors provide API to access the GPU (Actually, that API is called WebGL) so these visuals can’t run on some browsers that don’t offer the WebGL API.

In order to overcome these, I tried to port GLSL fragment shaders to javascript and draw the result using HTML5 canvas. The canvas API has a much broader browser support (and a flash fallback is available).

The demos I created are un-optimized and maybe a bit slow. After all, GLSL shaders are supposed to run on a GPU. However, it’s still worth to convert some of the GLSL shaders to achieve some nice effect such as post-processing of photos.

Here’s the demo:

  1. http://jsfiddle.net/licson0729/eBjQ8/
  2. http://jsfiddle.net/licson0729/YJqB9/
  3. http://jsfiddle.net/licson0729/7Qe34/
  4. http://jsfiddle.net/licson0729/T3hb7/
  5. http://jsfiddle.net/licson0729/9QVxA/

The techniques I used is to render the pixels one by one, with the render function be the one in the GLSL fragment shader. Also, we need to change the schematics (vec2, vec3, etc.) to its JavaScript equivalent.

Here’s the format of the code:

//The requestAnimFrame fallback for better and smoother animation
window.requestAnimFrame = (function () {
    return window.requestAnimationFrame || window.webkitRequestAnimationFrame || window.mozRequestAnimationFrame || window.oRequestAnimationFrame || window.msRequestAnimationFrame || function (callback) {
        window.setTimeout(callback, 1000 / 60);
    };
})();

//Prepare our canvas
var canvas = document.querySelector('#render');
var w = window.innerWidth;
var h = window.innerHeight;
canvas.width = w;
canvas.height = h;
var ctx = canvas.getContext('2d');

var time = Date.now();
var buffer = ctx.createImageData(w, h);//The back buffer we used to paint the result into the canvas

//The main rencer function
function render(time, fragcoord) {
    /* put the GLSL fragment shader's JavaScript equivalent here. */
    return [0,0,0,0]; //the final colour value
};

function animate() {
    var delta = (Date.now() - time) / 1000;
    buffer = ctx.createImageData(w, h);
    ctx.clearRect(0, 0, w, h);
    for (var x = 0; x < w; x++) {
        for (var y = 0; y < h; y++) {
            var ret = render(delta, [x, y]);
            var i = (y * buffer.width + x) * 4;
            buffer.data[i] = ret[0] * 255;
            buffer.data[i + 1] = ret[1] * 255;
            buffer.data[i + 2] = ret[2] * 255;
            buffer.data[i + 3] = ret[3] * 255;
        }
    }
    ctx.putImageData(buffer, 0, 0);
    requestAnimFrame(animate);
};

window.onresize = function () {
    w = window.innerWidth;
    h = window.innerHeight;
    canvas.width = w;
    canvas.height = h;
};

animate();

Hope you like it and encouraged you to start learning about computer graphics.