Today we are going to explore what I think is one of the coolest features of the canvas tag. The ability to extract and manipulate pixel data. This is an amazing feature to canvas that will help developers create all sorts of cool applications from image manipulations to code driven artwork.
I think this graphic sums it up…

Preview

I wanted to make a special graphic for this post so I cracked open Blender and came up with this HTML5 rocks 3D scene. The idea was to have a very distinct difference between the range of colors so that I could detect when I was over a orange pixel versus a grey pixel. For a surprise hover over any orangish pixel in the scene. See it in action.

Loading an image onto the Canvas

In order to get the pixels from our image we will first need to load it onto the canvas. This process is pretty simple. First we need to create some canvi : ) (Is that a word?)

<div id="canvas-container">
		<canvas id="rocks-canvas" width="960" height="540">Sorry no support for canvas!</canvas>
</div>

Next, we will need to load our image and shove it into the “rocks-canvas” element. Upon load we will fire an init function. Init will setup references to our canvas and canvas context. The main magic is line 11. We can take our loaded rocksImg and use the drawImage function to draw it directly to the canvas context. If you recall we used this drawImage function in our canvas spritesheet example as well.

// Load the HTML5 rocks image.
var rocksImg = new Image();
rocksImg.onload = function() {
	  init();	
};
rocksImg.src = 'img/rocks.png';

function init(){
	canvas = document.getElementById('rocks-canvas');
	context = canvas.getContext('2d');
	context.drawImage(rocksImg,0,0);

	canvas.addEventListener('mousemove', onMouseMove, false);
}

Mouse Movin’

Once we have our image drawn we need to sample pixels based on the mouse movement. As you can see from our previous code snippet the last line is setting up the onMouseMove event so we should be ready to go. FYI, in order to keep it simple we are going to condense down the code from the working example. The full source is at the bottom if you want all the bells and whistles.

Ok so we first get ev.offsetX and ev.offsetY which will give us our mouse position. Next, we call getImageData from our canvas context and pass in the x and y coordinates followed by a width and height which in our case we want a 1px x 1px sample size. Then we call the data attribute. This will get our image data into a byteArray. The pixel data is stored in this byteArray in RGBA format. This means that our resulting pixels values in the byteArray are pixel[0] = RED, pixel[1] = GREEN, pixel[2] = BLUE, pixel[3] = ALPHA. RGB values range from 0 to 255 so each of these values returned from the byteArray will be within those ranges. The last thing we can do is check each of the RGB values for a range of colors and we could trigger something on the page if the user is on a certain color. (Example Lines 13 – 19)

function onMouseMove (ev) {
		var row,column,pixel,pixelString;
		
		column = ev.offsetX;
		row = ev.offsetY;
			
		pixel = context.getImageData(column,row,1,1).data;
		pixelString = pixel[0] + "," + pixel[1] + "," + pixel[2];
		
		//Set color swatch to our rgb value for our color swatch div element.
		$colorSwatch.css('background-color', 'rgb(' + pixelString + ')');
		
		if(pixel[0] >= 190 && pixel[0] <= 255 && 
		   pixel[1] >= 60 && pixel[1] <= 120 &&
		   pixel[2] >= 30 && pixel[2] <= 80) {
		   	// Orange pixel range.  Let's do something cool.
		} else {
			// Not orange pixel range.
		}
}

Tip: If you want to see what the value will look like just pull up the color picker in photoshop.

Conclusion

Accessing canvas pixel data opens up many possibilites for cool and interesting projects. As always I would be interested in any cool applications of these techniques. Feel free to post a comment or link.

Source links.
Code Source
HTML5 Blend file. If you want to change the colors in the 3D file... : )

Last post I showed how to animate a sprite sheet using a jquery plugin spritely. The technique essentially moved the css background-position property to show the animation. Today we are going to show a technique that will be one step better. We are going to do the same animation using the HTML 5 canvas tag and no jquery at all! Note: If you need info on how to create a sprite sheet visit the last post for more details…

Canvas

So what is canvas? Think of canvas just like the name says. It’s a big drawing canvas that we can draw lines, shapes, images, and videos on. However remember everything is bitamp based. It is very similar to if you have used the BitmapData class within Flash. This basically means once we draw to the canvas we cannot get object information back since it is now just a drawing full of pixels. However, for our purposes this is fine since we will be clearing the canvas and redrawing each time we update our robin animation.

Let’s get it started!

Unfortunately, your browser does not support the canvas tag.

First we need to add the canvas tag. The h1 message is an alternative the user will see if their browser does not support canvas. Cough. IE < 9. Fail... Anyway there is a special function we can call on canvas called drawImage. The syntax is as follows.

drawImage(image, sx, sy, sWidth, sHeight, dx, dy, dWidth, dHeight)

Let’s break down what this function does. It takes in an image, x pos, y pos, height and width and draws it to the canvas. By changing the x and y position we can move our image around on the canvas. By setting the height and width to one “frame” within the image we can make it look like an animation. Let’s see a diagram.

Changing the x and y position properties of the drawImage function we can move the window region that we want to animate. Let’s take a look at the whole draw function.

this.intAnimation = 1000 / this.FPS;

CanvasSprite.prototype.draw = function(time) {
	var frame = Math.round(time / this.intAnimation) % this.TOTAL_FRAMES;
	var column_frame = frame % this.TOTAL_COLUMNS;
  			
  	var sx = this.sw * column_frame;
  	var sy = this.sh * Math.floor(frame / this.TOTAL_COLUMNS);
  			
	var context = this.canvas.getContext('2d');
	//drawImage(image, sx, sy, sWidth, sHeight, dx, dy, dWidth, dHeight)
	context.clearRect(0,0,this.width,this.height);
  	context.drawImage(this.spritesheet,
     		sx, sy, this.width, this.height,
      		0, 0, this.width, this.height);
}

Ok let’s highlight some of the features of the draw method. You can see we are doing a clearRect() each time right before the draw image. Think of it like an etch-a-sketch. Canvas would keep the past drawings if we did not clear it each time. The rest of this method is calculating frames based on the time elapsed and the FPS to position x and y and draw. Neato! One other small note. We are using the requestAnimationFrame shim to call this draw function and NOT setInterval(). If you need more info on why go to Paul Irish’s blog post.

Let’s ACTUALLY get it started…

To start drawing to the canvas we need to call the draw method repeatedly. To do this we will create another method called tick. This method will call draw and then call the requestAnimFrame shim which will in turn call tick again when the browser is ready. The result is a solid animation loop.

CanvasSprite.prototype.tick = function(time) {
  	this.draw(time);
  	requestAnimFrame(this.tick.bind(this), this.canvas);
};

Finally here is the result.

The benefits of this method may not seem immediately obvious. However, using the other method we are updating the css background property continuously which means we are touching the dom over and over. By using the drawImage method with canvas we can have an animation that will preform much better when used to create games or multiple animations.