PRANDIVM‧GRATIS‧NON‧EST

Kaleidoscope updated for Firefox

Julien — Tue, 26 Jan 2010 20:39:00 +0100

The video kaleidoscope now works in Firefox (at least from version 3.5+.) In my previous post, I mentioned some cross-browser issues; let’s have another look at them.

First, media format issues: since Firefox and Safari do not support the same formats, I needed at least a “default” video in Ogg format, which I found on TinyVid. Then, instead of simply specifying an src attribute on the video element, we skip that attribute and add two child source nodes for the video element, one for an H.264-encoded movie, the other for an Ogg Theora-encoded movie. Normally, we would want to provide the same movie in different formats, but until I provide my own movies for this demo, we'll just use interesting videos that we find online.

Second, I noted how I could not get the canplaythrough event from the video element in Firefox, even though a playable video was provided. The problem lies in the fact that Safari buffers the video by default, even when the autobuffer attribute is not present, whereas Firefox doesn’t. After adding this attribute, things work the same in both browsers.

Get the new Firefox 3.6 and see for yourself!

Youtube HTML5 Video Player

Julien — Thu, 21 Jan 2010 11:51:00 +0100

Youtube now has an open beta for their HTML5 video player. Because they use H.264 it really only works in Chrome and Safari and not all features are available (not to mention not all videos are playable, in which case they fall back to Flash.) Well worth a look anyway.

A video kaleidoscope

Julien — Wed, 20 Jan 2010 16:48:00 +0100

Another short demo before we move on to more complex applications. In this one I wanted to do something with the HTML5 video element; I had probably seen something about kaleidoscopes recently so I thought about doing a video kaleidoscope. Something that would like the screenshot below... That should be easy!

Actually, it wasn’t too bad. Although I suspect that it could be done more concisely in SVG (something for a next episode, I guess) this implementation based on video and canvas, with an additional help of CSS 2D Transforms, is really simple conceptually. And it will exercise your processor quite a bit in the process...

Sampling video

As the screenshot above shows, our kaleidoscope is a circle rotating on its own or following the cursor, divided in six “slices”, where each slice contains the same picture, rotating on its own, and flipped. A first approach was to have six video elements playing the same content at once, but it proved clearly impractical to synchronize, and without looking too much into it, masking and overlaying different videos seemed pretty tricky.

Fortunately, we can easily sample the video by drawing the current frame in a canvas element (with the drawImage() method of a 2D drawing context, and using a video element as the first argument.) We can draw our kaleidoscope in a canvas element, with a single video playing in the background. The video is playing behind the canvas so that we don’t see it but can still hear it and of course sample it. And although we could draw all six slices in the same canvas, I have chosen to draw the same picture in six different canvases and let CSS do the rest. We can describe the full process in four steps.

Set up a video element, and wait for it to start (see below.)
Set up six canvas elements. For each canvas, we’ll have an initial clipping mask which will produce the shape of the slice. This needs to be done only once as it operates on the drawing context of the canvas. The tricky part here is to understand how to use the arcTo() drawing primitive, and honestly I’m not quite sure that I do but it seems to work out well enough.
At a given rate (something like 15-30 frames per seconds, which is sufficient for video), draw the current video image in every canvas. We set a new context before drawing where we apply a rotation incrementally so that the image in each slice appears to rotate on its own.
Set the transform style property (or rather -webkit-transform property) of each canvas element to rotate by increments 60 degrees, and flip around the X-axis for every other canvas (so that the symmetry is respected.) Since the whole kaleidoscope rotates, we add the current rotation angle of the kaleidoscope to every slice to achieve global rotation.

Cross-browser issues

The current demo only supports Safari. I haven’t had much success playing audio or video in Chrome, so maybe some day it will work. Firefox should work, but needs a few fixes:

The main issue is that Safari and Firefox do not support the same media formats. This means that a fallback OGG video is necessary for Firefox.
Media elements generate an awful lot of events, and because they can be quite large it means that we need to wait for enough data to be available before we can start playback. It may take a while even knowing what the size of the video is, which is necessary to set up the canvases, drawing contexts, &c. In the current implementation I wait for the canplaythrough event before starting playback, which means that the whole video should be playable. Unfortunately I wasn’t able to catch that event in Firefox. I will write a bit more on this topic later and hopefully fix this demo to work in Firefox.
As in the poetry magnets demo, we need to add support for the -moz-transform style property for CSS transforms.

That’s all for now; on the next episode we’ll see more canvas animation action.

Update: these issues have now been addressed.

うわー

Julien — Mon, 18 Jan 2010 18:13:00 +0100

Sorry about all this text, next entry should be the same — here's a picture of a tomato in the meantime.

Fridge magnets with CSS transformations

Julien — Mon, 18 Jan 2010 17:18:00 +0100

I put together this demo quickly last summer to experiment with CSS 2D Transforms as they were being implemented in Webkit and Firefox. (We’ll see about 3D transforms soon, BTW.) Since I have always liked the idea of poetry magnets, this seemed like an ideal little project to get started with those. Here is what it looks like now, as seen by Safari on Mac OS 10.6:

Scattering the magnets

The implementation is very simple: a magnet is a just a div element on the ~~document~~ fridge door. We just need a little bit of style such as a border, a soft shadow, and some colors, rounded corners of course, and here we have our first magnet:

anathema

At first, I wanted to do as much as possible with transformations. So I just threw in a bunch of magnets in a parent div and set a CSS transform property with a random translation and rotation to scatter the magnets everywhere. Which was fine... except that if for some reason the original position of a magnet changed (for instance when resizing the browser window) then the translated position would also change in a sometimes unpredictable way. It was also difficult to keep track of the magnets when letting the user move them around, so I decided to stick with the old-school approach of absolute positioning for the magnets, and using the transform property for rotations only.

In the end, magnets share the same magnet class, with some additional classes for custom styles. The position (top, left, and z-index properties) and rotation (transform property with a rotate(θ deg) value) are set programmatically through Javascript so that the user can drag them around the fridge door surface. Dragging can either change the position or the rotation of the magnet; we catch the mousedown events on magnets and decide to change the position if the pointer close to the center of the magnet, or change the rotation if the pointer was closer to the corners.

A note about browser compatibility: since transforms are pretty new, both Webkit and Firefox implement the property with their own names, which means that we have to set both -webkit-transform and -moz-transform for every element to work across both browsers and their derivative, such as Safari and Chrome for Webkit. (Opera 10.50 implements their own -o-transform attribute, so we throw that into the mix as well; and the less said about IE, the better.) Fortunately, they all seem to agree on the syntax and semantics of the values. So let’s see our above magnet again, this time translated and rotated, using the translate CSS transform now because I do want to translate it relative to its normal position:

anathema

Finally, I know that I should say something about z-index, which was a bit tricky to make work correctly across different browsers, but I forget what the trick was. Also I’m still not sure that I understand how z-index actually works (or most of CSS, for that matter.)

Generating magnets

Now we need to generate sets of words to have interesting magnets to play with. I started by downloading Shakespeare’s sonnets and Walt Whitman’s Leaves of Grass from Project Gutenberg, fed these to a very naive tokenizer written in Perl and extracted all words that had more occurrences than a given threshold (I forget what the threshold was set to, but something quite low.) I threw in Perl’s keywords and operators and a bunch of Unicode symbols for good measure, did a little bit of editing, and generated hundreds of poetry magnets with little effort.

In the future I will try and add different languages; the problem here is that the very naive approach taken here works well for English, but shows its limit very quickly when moving to other languages. There are two main issues: first, we need a real morphological analyzer for other languages, and not such a quick and dirty, regular-expression based tokenizer. This is not so bad because we can find good quality, open-source tools for this job. The second issue is a bit trickier as we need to worry about conjugation, plural, gender, agreement, &c. so this part requires some experimentation.

Another source of words to play with is not famous poets, but anonymous ones: the users themselves. Since we don’t deal with actual magnets but virtual ones, we can make them editable. This is as simple as setting the contentEditable property of the magnet’s div element to true when a blank magnet is created, and setting it back to false when done (i.e., when the return key is pressed.) The only issue left was to deal with positioning of a magnet that can change size, because transforms are relative to the center of the magnets. So every time a key is pressed, while the magnet is editable, we check for its new size and change to position so that its center does not move. This is where getBoundingClientRect() comes in very handy, by the way. (And this needs to be fixed for Opera, actually.)

`localStorage` to the rescue

Another nifty new feature that I wanted to experiment with was the new storage features of HTML5, i.e., Web Storage. At first I tried the client-side database API, which worked well in Webkit but not in other browsers (even Chrome, although based on Webkit, didn’t implement any storage API at that point) and in retrospect seems like a misguided effort to specify SQLite as part of HTML5. (Nothing against SQLite, which is a great piece of software.)

So I switched to the window.localStorage interface where I simply stored some JSON strings with the position of every magnet. It seems to be consistently implemented all-around (it took some time for Chrome to catch up though) and allowed the user’s fridge door to become persistent. As soon as a magnet is moved or rotated, everything gets saved and is ready to be restored by the time the next session starts.

The last piece of the puzzle was to implement a little server back-end to allow users to share their poetry. Because the browser’s local storage is not accessible to the outside world, we need to save the data on some shared resource to make it accessible to others. By associating a unique identifier to every saved session, they can now be easily shared by URLs, such as the one in the screenshot above.

Dragging again

Another new HTML5 feature that I experimented on during development was HTML5 drag and drop. In the end it was not a very good fit for my use case and I ended up dropping it, if you will, although it will make a come back in a future post about a different application.

Implementing dragging of the magnets was not too bad, although handling rotation added a bit of complexity (see above.) Dropping was another matter, because I wanted to be able to drop magnets over the trash can as a way to discard them. I ended up using the elementFromPoint method to check whether my drop-off point was the trash can element; but in order for that to work, the magnet that was being dragged had to be momentarily hidden, by setting its visibility style property to hidden, so that it wouldn’t interfere with elementFromPoint. I even threw in some sounds for the trash can, but I’ll write in more details about the audio element very soon.

That’s all for now; next time we’ll see some more usages of CSS 2D transforms but this time we’ll throw in some video as well.

It’s 2010!

Julien — Sat, 16 Jan 2010 18:30:00 +0100

So it’s 2010 and I’m treating myself to a first-year consulting anniversary with a blog to give some more technical details about what I’m working on while I'm working on it. I’ve found a lot of useful, and way more really stupid information on technical blogs and I hope to give some back in the future. The first posts will be about the HTML5/Javascript demos that can be seen in my secret lab. Enjoy!