Acko.net

My JS1K Demo - The Making Of

2010-08-06T00:00:00-07:00

My JS1K Demo - The Making Of

If you haven't seen it yet, check out the JS1K demo contest. The goal is to do something neat in 1 kilobyte of JavaScript code.

I couldn't resist making one myself, so I pulled out my bag of tricks from my Winamp music visualization days and started coding. I'm really happy with how it turned out. And no, it won't work in Internet Explorer 8 or less.

Edit: OH SNAP! I just rewrote the demo to include volumetric light beams and still fit in 1K:

Original Version

Improved Version

Now, whenever size is an issue, the best way to make a small program is to generate all data on the fly, i.e. procedurally. This saves valuable storage space. While this might seem like a black art, often it just comes down to clever use of (high school) math. And as is often the case, the best tricks are also the simplest, as they use the least amount of code.

To illustrate this, I'm going to break down my demo and show you all the major pieces and shortcuts used. Unlike the actual 1K demo, the code snippets here will feature legible spacing and descriptive variable names.

Initialization

JS1K's rules give you a Canvas tag to work with, so the first piece of code initializes it and makes it fill the window.

From then on, it just renders frames of the demo. There are four major parts to this:

Animating the wires
Rotating and projecting the wires into the camera view
Coloring the wires
Animating the camera

All of this is done 30 times per second, using a normal setInterval timer:

setInterval(function () { ... }, 33);

Drawing Wires

The most obvious trick is that everything in the demo is drawn using only a single primitive: a line segment of varying color and stroke width. This allows the whole drawing process to be streamlined into two tight, nested loops. Each inner iteration draws a new line segment from where the previous one ended, while the outer iteration loops over the different wires.

The lines are blended additively, using the built-in 'lighten' mode, which means they can be drawn in any order. This avoids having to manually sort them back-to-front.

To simplify the perspective transformations, I use a coordinate system that places the point (0, 0) in the center of the canvas and ranges from -1 to 1 in both coordinates. This is a compact and convenient way of dealing with varying window sizes, without using up a lot of code:

with (graphics) { ratio = width / height; globalCompositeOperation = 'lighter'; scale(width / 2 / ratio, height / 2); translate(ratio, 1); lineWidthFactor = 45 / height; ...

I also add a correction ratio for non-square windows and calculate a reference line width lineWidthFactor for later.

Then there's the two nested for loops: one iterating over the wires, and one iterating over the individual points along each wire. In pseudo-code they look like:

For (12 wires => wireIndex) { Begin new wire For (45 points along each wire => pointIndex) { Calculate path of point on a sphere: (x,y,z) Extrude outwards in swooshes: (x,y,z) Translate and Rotate into camera view: (x,y,z) Project to 2D: (x,y) Calculate color, width and luminance of this line: (r,g,b) (w,l) If (this point is in front of the camera) { If (the last point was visible) { Draw line segment from last point to (x,y) } } else { Mark this point as invisible } Mark beginning of new line segment at (x,y) } }

Mathbending

To generate the wires, I start with a formula which generates a sinuous path on a sphere, using latitude/longitude. This controls the tip of each wire and looks like:

offset = time - pointIndex * 0.03 - wireIndex * 3; longitude = cos(offset + sin(offset * 0.31)) * 2 + sin(offset * 0.83) * 3 + offset * 0.02; latitude = sin(offset * 0.7) - cos(3 + offset * 0.23) * 3;

This is classic procedural coding at its best: take a time-based offset and plug it into a random mish-mash of easily available functions like cosine and sine. Tweak it until it 'does the right thing'. It's a very cheap way of creating interesting, organic looking patterns.

This is more art than science, and mostly just takes practice. Any time spent with a graphical calculator will definitely pay off, as you will know better which mathematical ingredients result in which shapes or patterns. Also, there are a couple of things you can do to maximize the appeal of these formulas.

First, always include some non-linear combinations of operators, e.g. nesting the sin() inside the cos() call. Combined, they are more interesting than when one is merely overlaid on the other. In this case, it turns regular oscillations into time-varying frequencies.

Second, always scale different wave periods using prime numbers. Because primes have no factors in common, this ensures that it takes a very long time before there is a perfect repetition of all the individual periods. Mathematically, the least common multiple of the chosen periods is huge (414253 units ~ 4.8 hours). Plotting the longitude/latitude for offset = 0..600 you get:

The graph looks like a random tangled curve, with no apparent structure, which makes for motions that never seem to repeat. If however, you reduce each constant to only a single significant digit (e.g. 0.31 -> 0.3, 0.83 -> 0.8), then suddenly repetition becomes apparent:

This is because the least common multiple has dropped to 84 units ~ 3.5 seconds. Note that both formulas have the same code complexity, but radically different results. This is why all procedural coding involves some degree of creative fine tuning.

Extrusion

Given the formula for the tip of each wire, I can generate the rest of the wire by sweeping its tail behind it, delayed in time. This is why pointIndex appears as a negative in the formula for offset above. At the same time, I move the points outwards to create long tails.

I also need to convert from lat/long to regular 3D XYZ, which is done using the spherical coordinate transform:

Source: Wikipedia

distance = f.sqrt(pointIndex+.2); x = cos(longitude) * cos(latitude) * distance; y = sin(longitude) * cos(latitude) * distance; z = sin(latitude) * distance;

You might notice that rather than making distance a straight up function of the length pointIndex along the wire, I applied a square root. This is another one of those procedural tricks that seems arbitrary, but actually serves an important visual purpose. This is what the square root looks like (solid curve):

The dotted curve is the square root's derivative, i.e. it indicates the slope of the solid curve. Because the slope goes down with increasing distance, this trick has the effect of slowing down the outward motion of the wires the further they get. In practice, this means the wires are more tense in the middle, and more slack on the outside. It adds just enough faux-physics to make the effect visually appealing.

Rotation and Projection

Once I have absolute 3D coordinates for a point on a wire, I have to render it from the camera's point of view. This is done by moving the origin to the camera's position (X,Y,Z), and applying two rotations: one around the vertical (yaw) and one around the horizontal (pitch). It's like spinning on a wheely chair, while tilting your head up/down.

x -= X; y -= Y; z -= Z; x2 = x * cos(yaw) + z * sin(yaw); y2 = y; z2 = z * cos(yaw) - x * sin(yaw); x3 = x2; y3 = y2 * cos(pitch) + z2 * sin(pitch); z3 = z2 * cos(pitch) - y2 * sin(pitch);

The camera-relative coordinates are projected in perspective by dividing by Z — the further away an object, the smaller it is. Lines with negative Z are behind the camera and shouldn't be drawn. The width of the line is also scaled proportional to distance, and the first line segment of each wire is drawn thicker, so it looks like a plug of some kind:

plug = !pointIndex; lineWidth = lineWidthFactor * (2 + plug) / z3; x = x3 / z3; y = y3 / z3; lineTo(x, y); if (z3 > 0.1) { if (lastPointVisible) { stroke(); } else { lastPointVisible = true; } } else { lastPointVisible = false; } beginPath(); moveTo(x, y);

Coloring

Each line segment also needs an appropriate coloring. Again, I used some trial and error to find a simple formula that works well. It uses a sine wave to rotate overall luminance in and out of the (Red, Green, Blue) channels in a deliberately skewed fashion, and shifts the R component slowly over time. This results in a nice varied palette that isn't overly saturated.

pulse = max(0, sin(time * 6 - pointIndex / 8) - 0.95) * 70; luminance = round(45 - pointIndex) * (1 + plug + pulse); strokeStyle='rgb(' + round(luminance * (sin(plug + wireIndex + time * 0.15) + 1)) + ',' + round(luminance * (plug + sin(wireIndex - 1) + 1)) + ',' + round(luminance * (plug + sin(wireIndex - 1.3) + 1)) + ')';

Here, pulse causes bright pulses to run across the wires. I start with a regular sine wave over the length of the wire, but truncate off everything but the last 5% of each crest to turn it into a sparse pulse train:

Camera Motion

With the main visual in place, almost all my code budget is gone, leaving very little room for the camera. I need a simple way to create consistent motion of the camera's X, Y and Z coordinates. So, I use a neat low-tech trick: repeated interpolation. It looks like this:

sample += (target - sample) * fraction

target is set to a random value. Then, every frame, sample is moved a certain fraction towards it (e.g. 0.1). This turns sample into a smoothed version of target. Technically, this is a one-pole low-pass filter.

This works even better when you apply it twice in a row, with an intermediate value being interpolated as well:

intermediate += (target - intermediate) * fraction sample += (intermediate - sample) * fraction

A sample run with target being changed at random might look like this:

You can see that with each interpolation pass, more discontinuities get filtered out. First, jumps are turned into kinks. Then, those are smoothed out into nice bumps.

In my demo, this principle is applied separately to the camera's X, Y and Z positions. Every ~2.5 seconds a new target position is chosen:

if (frames++ > 70) { Xt = random() * 18 - 9; Yt = random() * 18 - 9; Zt = random() * 18 - 9; frames = 0; } function interpolate(a,b) { return a + (b-a) * 0.04; } Xi = interpolate(Xi, Xt); Yi = interpolate(Yi, Yt); Zi = interpolate(Zi, Zt); X = interpolate(X, Xi); Y = interpolate(Y, Yi); Z = interpolate(Z, Zi);

The resulting path is completely smooth and feels quite dynamic.

Camera Rotation

The final piece is orienting the camera properly. The simplest solution would be to point the camera straight at the center of the object, by calculating the appropriate pitch and yaw directly off the camera's position (X,Y,Z):

yaw = atan2(Z, -X); pitch = atan2(Y, sqrt(X * X + Z * Z));

However, this gives the demo a very static, artificial appearance. What's better is making the camera point in the right direction, but with just enough freedom to pan around a bit.

Unfortunately, the 1K limit is unforgiving, and I don't have any space to waste on more 'magic' formulas or interpolations. So instead, I cheat by replacing the formulas above with:

yaw = atan2(Z, -X * 2); pitch = atan2(Y * 2, sqrt(X * X + Z * Z));

By multiplying X and Y by 2 strategically, the formula is 'wrong', but the error is limited to about 45 degrees and varies smoothly. Essentially, I gave the camera a lazy eye, and got the perfect dynamic motion with only 4 bytes extra!

Addendum

After seeing the other demos in the contest, I wasn't so sure about my entry, so I started working on a version 2. The main difference is the addition of glowy light beams around the object.

As you might suspect, I'm cheating massively here: rather than do physically correct light scattering calculations, I'm just using a 2D effect. Thankfully it comes out looking great.

Essentially, I take the rendered image, and process it in a second Canvas that is hidden. This new image is then layered on the original.

I take the image and repeatedly blend it with a zoomed out copy of itself. With every pass the number of copies doubles, and the zoom factor is squared every time. After 3 passes, the image has been smeared out into an 8 = 2³ 'tap' radial blur. I lock the zooming to the center of the 3D object. This makes the beams look like they're part of the 3D world rather than drawn on later.

For additional speed, the beam image is processed at half the resolution. As a side effect, the scaling down acts like a slight blur filter for the beams.

Unfortunately, this effect was not very compact, as it required a lot of drawing mode changes and context switches. I had no room for it in the source code.

So, I had to squeeze out some more room in the original. First, I simplified the various formulas to the bare minimum required for interesting visuals. I replaced the camera code with a much simpler one, and started aggressively shaving off every single byte I could find. Then I got creative, and ended up recreating the secondary canvas every frame just to avoid switching back its state to the default.

Eventually, after a lot of bit twiddling, a version came out that was 1024 bytes long. I had to do a lot of unholy things to get it to fit, but I think the end result is worth it ;).

Closing Thoughts

I've long been a fan of the demo scene, and fondly remember Second Reality in 1993 as my introduction to the genre. Since then, I've always looked at math as a tool to be mastered and wielded rather than subject matter to be absorbed.

With this blog post, I hope to inspire you to take the plunge and see where some simple formulas can take you.

Making Worlds 4 - The Devil's in the Details

2009-12-25T00:00:00-08:00

Making Worlds 4 - The Devil's in the Details

Last time I'd reached a pretty neat milestone: being able to render a somewhat realistic rocky surface from space. The next step is to add more detail, so it still looks good up close.

Adding detail is, at its core, quite straightforward. I need to increase the resolution of the surface textures, and further subdivide the geometry. Unfortunately I can't just crank both up, because the resulting data is too big to fit in graphics memory. Getting around this will require several changes.

Strategy

Until now, the level-of-detail selection code has only been there to decide which portions of the planet should be drawn on screen. But the geometry and textures to choose from are all prepared up front, at various scales, before the first frame is started. The surface is generated as one high-res planet-wide map, using typical cube map rendering:

This map is then divided into a quad-tree structure of surface tiles. It allows me to adaptively draw the surface at several pre-defined levels of detail, in chunks of various sizes.

Source

This strategy won't suffice, because each new level of detail doubles the work up-front, resulting in exponentially increasing time and memory cost. Instead, I need to write an adaptive system to generate and represent the surface on the fly. This process is driven by the Level-of-Detail algorithm deciding if it needs more detail in a certain area. Unlike before, it will no longer be able to make snap decisions and instant transitions between pre-loaded data: it will need to wait several frames before higher detail data is available.

Uncontrolled growth of increasingly detailed tiles is not acceptable either: I only wish to maintain tiles useful for rendering views from the current camera position. So if a specific detailed portion of the planet is no longer being used—because the camera has moved away from it—it will be discarded to make room for other data.

Generating Individual Tiles

The first step is to be able to generate small portions of the surface on demand. Thankfully, I don't need to change all that much. Until now, I've been generating the cube map one cube face at a time, using a virtual camera at the middle of the cube. To generate only a portion of the surface, I have to narrow the virtual camera's viewing cone and skew it towards a specific point, like so:

This is easy using a mathematical trick called homogeneous coordinates, which are commonly used in 3D engines. This turns 2D and 3D vectors into respectively 3D and 4D. Through this dimensional redundancy, we can then represent most geometrical transforms as a 4x4 matrix multiplication. This covers all transforms that translate, scale, rotate, shear and project, in any combination. The right sequence (i.e. multiplication) of transforms will map regular 3D space onto the skewed camera viewing cone.

Given the usual centered-axis projection matrix, the off-axis projection matrix is found by multiplying with a scale and translate matrix in so-called "screen space", i.e. at the very end. The thing with homogeneous coordinates is that it seems like absolute crazy talk until you get it. I can only recommend you read a good introduction to the concept.

With this in place, I can generate a zoomed height map tile anywhere on the surface. As long as the underlying brushes are detailed enough, I get arbitrarily detailed height textures for the surface. The normal map requires a bit more work however.

Normals and Edges

As I described in my last entry, normals are generated by comparing neighbouring samples in the height map. At the edges of the height map texture, there are no neighbouring samples to use. This wasn't an issue before, because the height map was a seamless planet-wide cube map, and samples were fetched automatically from adjacent cube faces. In an adaptive system however, the map resolution varies across the surface, and there's no guarantee that those neighbouring tiles will be available at the desired resolution.

The easy way out is to make sure the process of generating any single tile is entirely self-sufficient. To do this, I expand each tile with a 1 pixel border when generating it. Each such tile is a perfectly dilated version of its footprint and overlaps with its neighbours in the border area:

This way all the pixels in the undilated area have easily accessible neighbour pixels to sample from. This border is only used during tile generation, and cropped out at the end. Luckily I did something similar when I played with dilated cube maps before, so I already had the technique down. When done correctly, the tiles match up seamlessly without any additional correction.

Adaptive Tree

Now I need to change the data structure holding the mesh. To make it adaptive, I've rewritten it in terms of real-time 'split' and 'merge' operations.

Just like before, the Level-of-Detail algorithm traverses the tree to determine which tiles to render. But if the detail available is not sufficient, the algorithm can decide that a certain tile in the tree needs a more detailed surface texture, or that its geometry should be split up further. Starting with only a single root tile for each cube face, the algorithm divides up the planet surface recursively, quickly converging to a stable configuration around the camera.

As the camera moves around, new tiles are generated, increasing memory usage. To counter this steady stream of new data, the code identifies tiles that fall into disuse and merges them back into their parent. The overall effect is that the tree grows and shrinks depending on the camera position and angle.

Queuing and scheduling

To do all this real-time, I need to queue up the various operations that modify the tree, such as 'split', 'merge' and 'generate new tile'. They need to be executed in between rendering regular frames on screen. Whenever the renderer decides a certain tile is not detailed enough, a request is placed in a job queue to address this.

While continuing to render regular frames, these requests need to be processed. This is harder than it sounds, because both planet rendering and planet generation have to share the GPU, preferably without causing major stutters in rendering speed.

The solution is to spread this process over enough visible frames so that the overal rendering speed is not significantly affected. For example, if a new surface texture is requested, several passes are made. First the height map is rendered, the next frame the normal map is derived from it, then the height/normal maps are analyzed and put into the tree, after which they will finally appear on screen:

I took some inspiration from id Tech 5, the next engine coming from technology powerhouse id Software. They describe a queued job system that covers any frame-to-frame computation in a game engine (from texture management to collision detection), and which schedules tasks intelligently.

Do the Google Earth

With all the above in place, the engine can now progressively increase the detail of the planet across several orders of magnitude. Here's a video that highlights it:

And some shots that show off the detail:

W00t, certainly one of the niftiest things I've built.

Engine tweaks

Along with the architecture changes, I implemented some engine tweaks, noted here for completeness.

In previous comments, Erlend suggested using displacement mapping, so I gave it a shot. Before, the mesh for every tile was calculated on the CPU once, then copied into GPU memory. However, this mesh data was redundant, because it was derived literally from the height map data. Instead I changed it so that now, the transformation of mesh points onto the sphere surface happens real-time on the GPU in a per-vertex program.

This saves memory and pre-calculation time, but increases the rendering load. I'll have to see whether this technique is sustainable, but overall, it seems to be performing just fine. As a side effect, the terrain height map can be changed real-time with very low cost.

Technical hurdles

I spent some time tweaking the engine to run faster, but there's still plenty of work and some technical hurdles to cover.

One involves the Ogre Scene Manager, which is the code object that manages the location of objects in space. In my case, I have to deal with both the 'real world' in space as well as the 'virtual world' of brushes that generate the planet's surface. I chose to use two independent scene managers to represent this, as it seemed like a natural choice. However, it turns out this is unsupported by Ogre and causes random crashes and edge cases. Argh. It looks like I'll have to refactor my code to fix this.

Another major hurdle involves the planet surface itself. Currently I'm still just using a single distored-crater-brush to create it, and the lack of variation is showing.

Finally, surfaces are being generated using 16-bit floating point height values, and their accuracy is not sufficient beyond a couple levels of zooming. This results in ugly bands of flat terrain. To fix this I'll need to increase the surface accuracy.

Future steps

With the basic planet surface covered, I can now start looking at color, atmosphere and clouds. I have plenty of reading and experimentation to do. Thankfully the web is keeping me supplied with a steady stream of awesome papers... nVidia's GPU Gems series has proven to be a gold mine, for example.

Random factoid: what game developers call a "cube map", cartographers call a "cubic gnomonic grid". It turns out that knowing the right terminology is important when you're looking for reference material...

Code

The code is available on GitHub.

References

Great ideas are best discovered when standing on the shoulders of giants:

id Tech 5 Challenges, From Texture Virtualization to Massive Parallelization, J.M.P. van Waveren, id Software
GPU Gems, nVidia

Making Worlds - Intermission

2009-11-07T00:00:00-08:00

Making Worlds - Intermission

Today at BazCamp YVR I gave a short presentation and demo of my "Making Worlds" project, as well as an overview of procedural content generation in general.

The slides are available for download.

Making Worlds 3 - That's no Moon...

2009-11-05T00:00:00-08:00

Making Worlds 3 - That's no Moon...

It's been over two months since the last installment in this series. Oops. Unfortunately, while trying to get to the next stage of this project, I ran into some walls. My main problem is that I'm not just creating worlds, but also learning to work with the Ogre engine and modern graphics hardware in particular.

This presents some interesting challenges: between my own code and the pixels on the screen, there are no less than three levels of indirection. First, there's Ogre, a complex piece of C++ code that provides me with high-level graphics tools (i.e. objects in space). Ogre talks to OpenGL, which abstracts away low-level graphics operations (i.e. commands necessary to draw a single frame). The OpenGL calls are handed off to the graphics driver, which translates them into operations on the actual hardware (processing vertices and pixels in GPU memory). Given this long dependency chain, it's no surprise that when something goes wrong, it can be hard to pinpoint exactly where the problem lies. In my case, an oversight and misunderstanding of an Ogre feature lead to several days of wasted time and a lot of frustration that made me put aside the project for a while.

With that said, back to the planets...

Normal mapping

Last time, I ended with a bumpy surface, carved by applying brushes to the surface. The geometry was there, but the surface was still just solid white. To make it more visually interesting, I'm going to apply light shading.

The most basic information you need for shading a surface is the surface normal. This is the vector that points straight away from the surface at a particular point. For flat surfaces, the normal is the same everywhere. For curved surfaces, the normal varies continuously across the surface. Typical materials reflect the most light when the surface normal points straight at the light source. By comparing the surface normal with the direction of incoming light (using the vector dot product), you can get a good measure of how bright the surface should be under illumination:

Lighting a surface using its normals.

To use normals for lighting, I have two options. The first is to do this on a geometry basis, assigning a normal to every triangle in the planet mesh. This is straightforward, but ties the quality of the shading to the level of detail in the geometry. A second, better way is to use a normal map. You stretch an image over the surface, as you would for applying textures, but instead of color, each pixel in the image represents a normal vector in 3D. Each pixel's channels (red, green, blue) are used to describe the vector's X, Y and Z values. When lighting the surface, the normal for a particular point is found by looking it up in the normal map.

The benefit of this approach is that you can stretch a high resolution normal map over low resolution geometry, often with almost no visual difference.

Lighting a low-resolution surface using high-resolution normals.

Here's the technique applied to a real model:

(Source - Creative Commons Share-alike Attribution)

Normal mapping helps keep performance up and memory usage down.

Finding Normals

So how do you generate such a normal map, or even a single normal at a single point? There are many ways, but the basic principle is usually the same. First you calculate two different vectors which are tangent to the surface at the point in question. Then you use the cross product to find a vector perpendicular to the two. This third vector is unique and will be the surface normal.

For triangles, you can pick any two triangle edges as vectors. In my case, the surface is described by a heightmap on a sphere, which makes things a bit trickier and requires some math.

I asked my friend Djun Kim, Ph.D. and teacher of mathematics at UBC for help and he recommended Calculus on Manifolds by Michael Spivak. This deceptively small and thin book covers all the basics of calculus in a dense and compact way, and quickly became my new favorite reading material.

Differential Geometry

In this section, I'll describe the formulas needed to calculate the normals of a spherical heightmap. Unlike what I've written before, this will dive shamelessly into specifics and not eschew math. The reason I'm writing it down is because I couldn't find a complete reference online. If math scares you, this section might not be for you. Scroll down until you reach the crater, or take a detour by reading A Mathematician's Lament by Paul Lockhart, which will enlighten you.

First, we're going to derive normals for a regular flat terrain heightmap. To start, we need to define the terrain surface. Starting with a 2D heightmap, i.e. a function f(u,v) of two coordinates that returns a height value, we can create a 3 dimensional surface g:

We can use this formal description to find tangent and normal vectors. A vector is tangent when its direction matches the slope of the surface in a particular direction. Differential math tells us that slope is found by taking the derivative. For our function of 2 variables, that means we can find tangent vectors along curves of constant v or constant u. These curves are the thin grid lines in the diagram. Actually, we can find tangents along any line, in any direction. But along u and v lines, the other variable acts like a constant, which simplifies things.

To do this, we take partial derivatives with respect to u (with v constant) and with respect to v (with u constant). The set of all partial derivatives is called the Jacobian matrix J, whose rows form the tangent vectors t_u and t_v, indicated in red and purple:

The cross product of t_u and t_v gives us n, the surface normal.

When applied to a discrete heightmap, the function f(u,v) is a 2D array map[u][v], and the partial derivatives at the end have to be replaced with something else. We can use finite differences to approximate the slope of the surface by differencing neighbouring samples:

This result and the formula for n are usually provided as-is in terrain mapping guides, without going through the full process of finding tangents first. However, it's important to use the Jacobian matrix formulation once you switch to spherical terrain.

To make a sphere, we add an additional function k which warps the flat terrain into a spherical shell. Each shell is the result of warping a single face of the cubemap and covers exactly 1/6th of the sphere. In what follows, We'll only consider a single face and its shell.

We designate the intermediate pre-warp coordinates (s,t,h), and the final post-warp coordinates as (x,y,z):

The principle behind the spherical mapping is this: first we take the vector (s, t, 1), which lies in the base plane of the flat terrain. We normalize this vector by dividing it by its length w, which has the effect of projecting it onto the sphere: (s/w, t/w, 1/w) will be at unit distance from (0, 0, 0). Then we multiply the resulting vector by the terrain height h to create the terrain on the sphere's surface, relative to its center: (h·s/w, h·t/w, h/w)

Just like with the function g(u,v) and J(u,v), we can find the Jacobian matrix J(s,t,h) of k(s,t,h). Because there are 3 input values for the function k, there are 3 tangents, along curves of varying s (with constant t and h), varying t (constant s and h) and varying h (constant s and t). The three tangents are named t_s, t_t, t_h.

PS: If your skills at derivation are a bit rusty, remember that Wolfram Alpha can do it for you.

How does this help? The three vectors describe a local frame of reference at each point in space. Near the edges of the grid, they get more skewed and angular. We use these vectors to transform the flat frame of reference into the right shape, so we can construct a new 90 degree angle here.

In mathematical terms, we multiply the 'flat' partial derivatives by the Jacobian matrix. This is similar to the chain rule for regular derivatives, only for multiple variables.

That is, to find the partial derivatives (i.e. tangent vectors) of the final spherical terrain with respect to the original terrain coordinates u and v, we can take the flat terrain's tangents t_u and t_v and multiply them by J(s,t,h). Once we have the two post-warp tangents, we take their cross product, and find the normal of the spherical terrain:

It's imporant to note that this is not the same as simply multiplying the flat terrain normal with J(s,t,h). J(s,t,h)'s rows do not form a set of perpendicular vectors (it is not an orthogonal matrix), which means it does not preserve angles between vectors when you multiply by it. In other words, J(s,t,h) * n, with n the flat terrain normal, would not be perpendicular to the spherical terrain. This is why it's important to return to the basic calculus underneath, so we can get the correct, complete formula.

Thus ends the magical math adventure. If you read it all the way through, cheers!

No Wait, It is a Moon.

With the normal map in place, I can now render the planet's surface and get a realistic idea of what it looks like. To show this off, I tweaked the brush system a bit: instead of using the literal brush image (e.g. a smooth, round crater), the brush is distorted with fractal noise. It makes every application of the brush subtly different from the next, and saves me from manually drawing e.g. a hundred different craters.

Here's a side by side comparison of the original brush and a distorted version.

Currently I've only implemented one type of distortion, which lends a rocky appearance to the surface. With that in place, my engine can now generate somewhat realistic looking moon surfaces. Here's the demo:

References

The techniques I used were pioneered by people smarter and older than me, I'm just building my own little digital machine with them.

Creating Spherical Worlds, Maxis/Electronic Arts. (source).
Calculus on Manifolds, Michael Spivak.

Making Worlds 2 - Scaling Heights

2009-08-31T00:00:00-07:00

Making Worlds 2 - Scaling Heights

Last time, I had a working, smooth sphere mesh. The next step is to create terrain.

Scale

Though my goal is to render at a huge range of scales, I'm going to focus on views from space first. That strongly limits how much detail I need to store and render. Aside from being a good initial sandbox in terms of content generation, it also means I can comfortably keep using my current code, which doesn't do any sophisticated memory or resource management yet. I'd much rather work on getting something interesting up first rather than work on invisible infrastructure.

That said, this is not necessarily a limitation. The interesting thing about procedural content is that every generator you build can be combined with many others, including a copy of itself. In the case of terrain, there are definite fractal properties, like self-similarity at different levels of scale. This means that once I've generated the lowest resolution terrain, I can generate smaller scale variations and combine them with the larger levels for more detail. This can be repeated indefinitely and is only limited by the amount of memory available.

Perlin Noise is a celebrated classic procedural algorithm,
often used as a fractal generator.

Height

To build terrain, I need to create heightmaps for all 6 cube faces. Shamelessly stealing more ideas from Spore, I'm doing this on the GPU instead of the CPU, for speed. The GPU normally processes colored pixels, but there's no reason why you can't bind a heightmap's contents as a grayscale (one channel) image and 'draw' into it. As long I build my terrain using simple, repeated drawing operations, this will run incredibly fast.

In this case, I'm stamping various brushes onto the sphere's surface to create bumps and pits. Each brush is a regular PNG image which is projected onto the surface around a particular point. The luminance of the brush's pixels determines whether to raise or lower terrain and by how much.

Three example brushes from Spore. (source)

However, while the brushes need to appear seamless on the final sphere, the drawing area consists only of the straight, square set of cube map faces. It might seem tricky to make this work so that the terrain appears undistorted on the curved sphere grid, but in fact, this distortion is neatly compensated for by good old perspective. All I need to do is set up a virtual scene in 3D, where the brushes are actual shapes hovering around the origin and facing the center. Then, I place a camera in the middle and take a snapshot both ways along each of the main X, Y and Z directions with a perfect 90 degree field of view. The resulting 6 images can then be tiled to form a distortion-free cube map.

Rendering two different cube map faces. The red area is the camera's viewing cone/pyramid, which extends out to infinity.

To get started I built a very simple prototype, using Ogre's scene manager facilities. I'm starting with just a simple, smooth crater/dent brush. I generate all 6 faces in sequence on the GPU, pull the images back to the CPU to create the actual mesh, and push the resulting chunks of geometry into GPU memory. This is only done once at the beginning, although the possibility is there to implement live updates as well.

Here's a demo showing a planet and the brushes that created it, hovering over the surface. I haven't implemented any shading yet, so I have to toggle back and forth to wireframe mode so you can see the dents made by the brushes:

The cubemap for this 'planet' looks like this when flattened. You can see that I haven't actually implemented real terrain carving, because brushes cause sharp edges when they overlap:

The narrow dent on the left gets distorted and angular where it crosses the cube edge. This is a normal consequence of the cubemapping, as it looks perfectly normal when mapped onto the sphere in the video.

Engine Tweaks

The demo above also incorporates a couple of engine improvements. With a real heightmap in place, I can implement real level-of-detail selection. That means the resolution of any terrain tile is decided based on how much detail would be lost if a simpler tile was used. The flatter a tile, the less detail is necessary. This ensures complex geometry is used only on those sections that really need it. This is great for visual fidelity, but causes a lot of geometry to pop up if sharp ridges are present in the terrain. In this case, my rendering engine was happily trying to push 700k triangles through the GPU per frame. While even my laptop GPU can actually do that at pretty smooth frame rates nowadays, some optimizations are in order to give me some breathing room.

The culprit was that I wasn't really doing any early removal of geometry that was hidden or otherwise out of frame. To fix that, I now do visibility checks together with the level-of-detail selection. First it checks if a chunk is over the horizon or not before considering it for selection. This is easy to calculate and eliminates a lot of unnecessary drawing, especially when looking straight down. If that first visibility check passes, I perform a tighter check using the camera's viewing cone. With these two measures in place, I'm only averaging about 50,000-100,000 triangles visible per frame, with room for more optimization. These optimizations only remove geometry that's already off screen, so there is no visual difference.

Cubemap Seams and Dilation

When rendering into cube maps, each side is rendered independently. In theory each face should match perfectly with adjacent ones due to the way they've been created. In practice however, slight mismatches can occur due to rounding errors at the edges, creating seams. This can be fixed by explicitly copying one pixel-wide edges from one face into the adjacent ones, until they all match up.

The next big step is to start shading the surface, but in order to do that I need to be able to run filters on the cube map. Specifically, I need to be able to compare neighbouring height samples anywhere on the surface. In the straight forward cubemap scenario this is non-trivial, because neighbouring samples at the edges need to be fetched from different cube faces at different orientations in space.

I decided to implement something I call 'dilated cubemaps'. I've never really heard this described formally, though I doubt it's never been thought of before:

Instead of every face neatly matching with the next, I dilate the cube faces so they stick through eachother. At the same time, I use a larger texture size to compensate, and I adjust the field-of-view of the rendering camera to match. If done right, the resulting cubemap is a pixel-perfect expanded version of the undilated map.

The dilated cubemap provides reliable neighbouring samples for all samples in the original cube map up to a distance as wide as the new border. Unlike regular cubemap wrapping, the dilated regions are distorted to conform to the current face's grid. This matches the real change in grid direction that occurs on the final sphere mesh and lets you sample exactly across cube map edges.

I played with the cubemap dilation because I was thinking of some complicated filters to run that require regular grids (like CFD). But in retrospect, I probably don't need the exact spacing of sample points at the edges for this, so regular undilated cubemapping will probably do. Still, it's good to have around, and certainly was an interesting exercise in pixel-exact rendering.

What Next?

With basic heightmap generation in place, I can now start putting in some 'tech artist' time to play with various brushes and drawing behaviour. Lighting and shading is another big one and should provide a massive improvement to the visuals.

Right now I've taken a week between postings, though it remains to be seen whether I can maintain that. Creating these blog entries is turning into a pretty time consuming endeavour, especially as I get into territory where I have to make my own diagrams and illustrations.

References

The techniques I used were pioneered by people smarter and older than me, I'm just building my own little digital machine with them.

Creating Spherical Worlds, Maxis/Electronic Arts. (source).
Ken Perlin, who invented a lot of this stuff.

Making Worlds 1 - Of Spheres and Cubes

2009-08-23T00:00:00-07:00

Making Worlds 1 - Of Spheres and Cubes

Let's start making some planets! Now, while this started as a random idea kind of project, it was clear from the start that I'd actually need to do a lot of homework for this. Before I could get anywhere, I needed to define exactly what I was aiming for.

The first step in this was to shop around for some inspirational art and reference pictures. While there is plenty space art to be found online, in this case, nothing can substitute for the real thing. So I focused my search on real pictures, both of landscapes (terran or otherwise) as well as from space. I found classy shots like these:

Hopefully I'll be able to render something similar in a while. At the same time, I eagerly devoured any paper I could find on rendering techniques from the past decade, some intended for real-time rendering, some old enough to be real-time today. Out of all this, I quickly settled on my goals:

Represent spherical or lumpy heavenly bodies from asteroids to suns.
With realistic looking topography and features.
Viewable across all scales from surface to space.
At flight-simulator levels of detail.
Rendered with convincing atmosphere, water, clouds, haze.

For most of these points, I found one or more papers describing a useful technique I could use or adapt. At the same time, there are still plenty of unknowns I'll need to figure out along the way, not to mention significant amounts of fudging and experimentation.

The Spherical Grid

To get started I needed to build some geometry, and to do that I needed to figure out what geometry I should use. After reviewing some options, I quickly settled on a regular spherical displacement map (AKA a heightmap). That is, starting with a smooth sphere, move every surface point up or down, perpendicular to the surface, to create terrain on the surface.

If these vertical displacements are very small compared to the sphere radius, this can represent the surface of a typical planet (like Earth) at the levels of detail I'm looking for. If the displacements are of the same order as the sphere radius, you can deform it into very irregular potato-like shapes. The only thing heightmaps can't do is caves, tunnels, overhang and other kinds of holes, which is fine for now.

The big question is, how should the spherical surface be divided up and represented? With a sphere, this is not an easy question, because there is no single obvious way to divide a spherical surface into regular sections or grids. Various techniques exist, each with their own benefits and specific use cases, and I spent quite some time looking into them. Here's a comparison between four different tesselations:

Source

Note that the tesselation labeled ECP is just the regular geographic latitude-longitude grid.

The main features I was looking for were speed and simplicity, so I settled on the 'quadcube'. This is where you start with a cube whose faces have been divided into regular grids, and project every surface point out from the middle to an enclosing sphere. This results in a perfectly smooth sphere, built out of 6 identical round shells with curved edges. This arrangement is better known as the 'cube map' and often used for storing arbitrary 360 degree panorama views.

Here's a cube and its spherical projection:

The projected cube edges are indicated in red. Note that the resulting sphere is perfectly smooth and round, even though the grid has a bulgy appearance.

Cube maps are great, because they are very easy to calculate and do not require complicated trigonometry. In reverse, mapping arbitrary spherical points back onto the cube is even simpler and in fact natively supported by GPUs as a texture mapping feature.

This is important, because I'll be generating the surface terrain and texture dynamically and will need to index and access each surface 'pixel' efficiently. Using a cube map, I simply identify the corresponding face, and then index it using x/y coordinates on the face's grid.

The downside of cube maps is that the distance and area between points varies along the grid, which makes it harder to perform certain operations on a surface equally. However, these area distortions are much smaller than e.g. a lat-long grid, where the grid spacing actually approaches zero near the poles. Even more, the distortions made by a cube map are the exact opposite of those you get with a regular perspective projection. This makes it easy to render into cube maps, which will be useful for texture generation.

Level of Detail

There's another reason I picked the cube map approach, and that has to do with the level of detail requirements. My goal is to make a planet that can be viewed from the ground, the air as well as from space. It would be incredibly slow to always render everything at maximum detail, so I need to adaptively add and remove detail as the viewer gets closer to the surface.

However, increasing the level of detail uniformly across the entire sphere is not enough, because I only want to render detail where the viewer will see it. To a viewer on the ground, most of the planet is hidden by the horizon, and the engine should be able to effectively cut away the unseen pieces, so no wasteful processing takes place.

It is here that I get a huge benefit from the cube map layout of the sphere, because it lets me apply the well-researched realm of grid-based flat terrain rendering with only minor adjustments. Specifically, I am using a 'chunked LOD' approach. Every face of the cube map becomes a quadtree, with each level splitting four ways to form the next level with more detail:

Source

The chunks for the various levels of detail are all loaded into GPU memory, ready to be accessed at any time. When the terrain has to be rendered, the engine walks down the quad-tree, determines the appropriate level-of-detail for each section, and outputs the list of chunks to be rendered for a particular frame. Then, the GPU does its work, blasting through each chunk at a blistering pace, leaving the CPU to do other things.

Because all the data is already in memory, changing the level of detail just means rendering a different set of chunks. Each chunk has the same geometrical complexity, and performance is directly proportional to how many are rendered on screen. More detail means more chunks, but that usually also means you can cut away pieces of the terrain that are far away.

The chunked approach is also very easy to work with, because there is no data dependency between the different chunks. Each chunk has a copy of its own vertex data, which means individual chunks can be paged in and out of GPU memory at will. This is important for keeping memory usage down while still being able to scale to massive sizes.

Putting It All Together

At this point, I have all the pieces in place to render an adaptive sphere mesh. This is what it looks like (sorry, the video capture is a bit jerky):

The detail increases as the camera gets closer to the sphere and shifts around the surface as it moves.

Far from being a little coding experiment, it actually took me quite some time to get to this point, because I was learning OGRE, sharpening my C++ skills, as well as researching the techniques to use.

The next step is to look at generating heightmaps and textures for the surface.

References

The techniques I used were pioneered by people smarter and older than me, I'm just building my own little digital machine with them.

Creating Spherical Worlds, Maxis/Electronic Arts. (source).
Rendering Massive Terrains using Chunked Level of Detail Control, Thatcher Ulrich. (source)

Making Worlds: Introduction

2009-08-22T00:00:00-07:00

Making Worlds: Introduction

For the past year or so I've been reacquainting myself with an old friend: C++.

More specifically, I've been exploring graphics programming again, this time with the luxurious flexibility of the modern GPU at my fingertips. To get me started, I shopped around for an open source engine to play with. After trying Irrlicht and finding its promises to be a bit lacking, Ogre turned out to be a really good choice. Though its architecture is a bit intimidating at first, it is all the more sound. More importantly, it seems to have a relatively healthy open-source community around it.

So with Ogre as my weapon of choice, I've started a new project: Making Worlds. More specifically, I want to procedurally generate a 3D planet, viewable from outer space as well as the ground (at flight-sim levels of detail), which can be rendered real-time on recent graphics hardware.

Why? Because I really like procedural content generation. It's an odd discipline where anything goes, and techniques from across mathematics, engineering and physics are applied. Then, you add a good dose of creativity and artistic sense, and perhaps mix in some real-world data too, until you find something that looks right.

Plus, far from being an exercise in pointlessness, procedural content is gaining in popularity, especially for video games.

So, in the style of Shamus Young's excellent Procedural city series, I'm going to start blogging about Making Planets. Unlike him however, I'm not going to adhere to a strict schedule.

Here's a teaser for the first installment.

CSS Sub-pixel Background Misalignments

2008-11-18T00:00:00-08:00

CSS Sub-pixel Background Misalignments

Update: and now, IE8 adds even more odd behavior to the mix!

A while ago, John Resig pointed out some issues with sub-pixel positioning in CSS. The problem he used is one of percentage-sized columns inside a container, where the resulting column widths don't round evenly to whole pixels or don't sum to the correct total. His conclusion is that browsers each have their own way of dealing with the problem.

I've recently been bumping into a related issue however, that shows the situation is even worse: rounding is inconsistent even inside a single browser.

Take the following scenario: a fixed width element that is horizontally centered in a viewport using margin-left: auto; margin-right: auto;. The viewport has a horizontally centered background image, having background-position: 50% 0. This is an extremely common page structure.

You'd logically expect the background image and the element to line up, and move as one when the viewport is resized. However, this is not the case. Depending on the viewport width, the background can be offset one pixel to the left or right. This obviously wreaks havoc on many designs. I decided to investigate this more closely and the results are not pretty.

My test case consists of the basic structure described above, repeated in a bunch of mini-viewports. Each background image contains a black box of a certain size, and is overlaid with a grey element that covers this box exactly. If the two pieces align, there should be no black peeking through on the sides, and each box should be fully gray.

For full coverage, I vary the following parameters:

The width of the viewport
The odd/even size of the box/element
Background image is bigger/smaller than the viewport
Background image is padded evenly/unevenly around the box (1px difference). (*)

I tested this in IE6, IE7, Safari 3.1.2, Firefox 3.0.4 and Opera 9.6.2.

The result is quite baffling: not a single browser out there rounds background image positions the same as element positions, resulting in misalignments. The tell-tale black lines show up in every browser:

IE6 and IE7:

Safari 3.1 and Opera 9.6:

Firefox 3.0:

What's worse is there isn't a single case (across viewport sizes) that is handled consistently between all the browsers. So this CSS technique should in fact be considered broken.

Of course this brings up the question: is it really a browser bug or just an implementation quirk? I would argue that at least in the case where the image's width parity matches the element's, you'd expect perfectly matching rounding (i.e. for the first four rows of test cases). The other test cases are more ambiguous, and all you could hope for is consistent behaviour in each browser individually.

I wonder why this hasn't been brought up more though. A quick sampling of designers around me shows that they have all encountered this bug, but don't really know a fix and just tweak the design or layout structure to mask the effect.

If you really do need to align a background image properly, there is an ugly work-around: place your background image on an additional fixed-width element layered behind the center column. Center the background's element using margins rather than the background image itself, and clip it off at the sides using overflow: hidden on an additional wrapper. This causes the background's position to be rounded the same way as the column on top.

(*) Note that there is a choice whether to pad more on the left or on the right. I chose the left. This means that the last 4 rows of test cases are inherently ambiguous: a browser that misaligns all of these in the same fashion is in fact being consistent, just in the opposite direction.

Projective Texturing with Canvas

2008-11-11T00:00:00-08:00

Projective Texturing with Canvas

The Canvas tag's popularity is slowly increasing around the web. I've seen big sites use it for image rotation, graph plotting, reflection effects and much more.

However, Canvas is still limited to 2D: its drawing operations can only do typical vector graphics with so-called affine transformations, i.e. scaling, rotating, skewing and translation. Though there have been some efforts to try and add a 3D context to Canvas, these efforts are still experimental and only available for a minority of browsers through plug-ins.

So when my colleague Ross asked me if we could build a Cover Flow-like widget with JavaScript, my initial reaction was no... but really, that's just a cop out. All you need are textured rectangles drawn in a convincing perspective: a much simpler scenario than full blown 3D.

Perspective views are described by so-called projective transforms, which Canvas2D does not support. However, it does support arbitrary clipping masks as well as affine transforms of both entire and partial images. These can be used to do a fake projective transform: you cut up your textured surface into a bunch of smaller patches (which are almost-affine) and render each with a normal affine transform. Of course you need to place the patches just right, so as to cover any possible gaps. As long as the divisions are small enough, this looks convincingly 3D.

So some hacking later, I have a working projective transform renderer in JavaScript. The algorithm uses adaptive subdivision to maintain quality and can be tuned for detail or performance. At its core it's really just a lot of linear algebra, though I did have to add a bunch of tweaks to make it look seamless due to annoying aliasing effects.

Unfortunately Safari seems to be the only browser that can render it at an acceptable speed, so this technique is just a curiosity for now. The current code was mostly written for readability rather than performance though, so it's possible it could be optimized to a more useful state. Feel free to browse the code.

A real 3D Canvas in the browser would obviously rock, but you can still do some nifty things if you know the right tricks...

Abusing jQuery.animate for fun and profit (and bacon)

2008-09-22T00:00:00-07:00

Abusing jQuery.animate for fun and profit (and bacon)

The days of static UIs that only have jarring transitions between pages are pretty much over. With frameworks like CoreAnimation or jQuery, it's easy to add useful animations to applications and webpages. In the case of jQuery, you can easily animate any CSS property, and you get free work-arounds for browser bugs to boot. You can run multiple animations (of arbitrary duration) at the same time, queue animations and even animate complex properties like colors or clipping rectangles.

But what if you want to go beyond mere CSS? You might have a custom widget that is drawn using , whose contents are controlled by internal variables; maybe you're using 3D transformations to scale and position images on a page, and simple 2D tweening just doesn't cut it.

In that case, it would seem you are out of luck: jQuery's .animate() method can only be applied to a collection of DOM elements, and relies heavily on the browser's own semantics for processing CSS values and their units. However thanks to JavaScript's flexibility and jQuery's architecture, we can work around this, and re-use jQuery's excellent animation core for our own nefarious purposes.

Hackity hack hack

First, we need an object to store all the variables we wish to animate. We use an anonymous

outside of the main document, so that jQuery's DOM calls still work on it. We simply add our own properties to it:

var vars = $.extend($('

')[0], {

  foo: 1,

  bar: 2,



  customAnimate: true,

  updated: true

});

In this case, our properties are foo and bar. We also set customAnimate and updated to identify this object (see below).

Next we need to override jQuery's default step function, which gets called for every step of an animation, and applies new values to an element's CSS properties.

// jQuery.fx.step._default _default: function(fx) { fx.elem.style[fx.prop] = fx.now + fx.unit; }

We can replace it using the following snippet:

var $_fx_step_default = $.fx.step._default; $.fx.step._default = function (fx) { if (!fx.elem.customAnimate) return $_fx_step_default(fx); fx.elem[fx.prop] = fx.now; fx.elem.updated = true; };

With the new step function, jQuery will check for the presence of a customAnimate property on any element it is animating. If present, it will assign the (unit-less) value to element.property rather than element.style.property and mark the element by setting element.updated to true.

Now we're ready to animate, using the normal $.animate syntax:

$(vars).animate({ foo: 5, bar: 10 }, { duration: 1000 });

The values vars.foo and vars.bar will now smoothly change over time. You can use any of jQuery's animation abilities as usual.

Now what about that updated variable? Well to actually use these animated values, you will need some kind of timer or step callback to read them back and draw them on the page. If you're using , you need to redraw your entire widget for every change, but you don't want to be wasting CPU time by constantly refreshing it. Furthermore, if you're running multiple animations at the same time, you'll want to aggregate all your property changes into a single redraw per frame. This is easy with the updated property and a simple timer:

setInterval(function () { if (!vars.updated) return; vars.updated = false; drawWidget(); }, 30);

Now your widget will only refresh itself when its values are changed by the animation step function we defined earlier, and very few CPU cycles are wasted. As a plus, you can render updates as fast or as slow as you want, without affecting the duration of your animations.

Demo

I whipped up a quick demo which renders a cloud of bacon using . All the motion in the demo is created through $.animate(), with a bunch of animations running at once.

This demo will not work in Internet Explorer, and has only been tested in Firefox 3 and Safari 3.

This is a rather esoteric example, but there are plenty of useful ways to apply this technique. I've used it to implement smooth, beautiful, usable widgets. You can combine multiple motion and opacity animations triggered by clicks and hovers without issues.

Final notes

While this technique works great, there is one big caveat. You should avoid animating any property that exists in CSS ('float', 'display', 'opacity', ...) because these have unexpected side effects depending on the browser.

There are also a couple of weaknesses:

jQuery does not support continued easing. That is, when you override an animation that is already in progress, the variable being animated will instantly stop and restart from its current position. The rate of change is not continuous between the two animations.
Animating angles is tricky. E.g. when animating from 350˚ to 0˚, you want it to animate across 10˚ and not the long way around. This requires manual correction.

And obviously, it would be cleaner if jQuery's animation core was refactored to separate out the CSS-specific code instead...

Poster design for Interfacultair Theaterfestival 2008

2008-02-28T00:00:00-08:00

Poster design for Interfacultair Theaterfestival 2008

The design is meant to look like the cover of a board game box and accompanies the web site's design.

Drupal's Designer Future

2007-03-01T00:00:00-08:00

Drupal's Designer Future

In the past months I've been doing a lot more graphical design, and it's caused me to think about how it relates to Drupal. This prompted me to write a rather long blog piece with some insights and a call to action. If you are interested in the future of Drupal, please read on.

The trigger was that I noticed that I'm getting less and less motivated to do graphics work for Drupal. It's not that I don't like design... I loved designing and building that LeuvenSpeelt.be site last month for example. But when it comes to Drupal graphics, the personal reward that I feel from doing it doesn't seem proportional to the effort I put in. This includes designing little banners for the Drupal.org spotlight, doing a t-shirt, making ad buttons, doing the association theme and more.

The most recent big example was the Garland theme. When Stefan Nagtegaal showed a work-in-progress version of his 'Themetastic' theme (as it was then called) in September, I was instantly charmed and knew that this was our new default theme in the making and said so clearly.

Many others were not convinced though and hammered on details, even though the basic design for the theme was rock solid. Some were not convinced of the theme's potential, or simply didn't see that we needed a theme that was graphically smashing rather than a good base to develop on.

At that point, I essentially said "screw the community, this is going to be our default theme" and started refining the theme so it was perfect for core. This took several weeks.

Until then, the rest of the community put its eggs in the wrong baskets and got a lot of useless design-by-committee done. These designs, which were in my opinion mediocre at best, were being pushed for inclusion. This may sound a bit harsh, but I honestly believe that if the most popular candidate theme, Deliciously Zen, had become the new default core theme, we'd have been ridiculed for still not 'getting' design after 6 years and Drupal 5 would not have been such a big release. Just like 4.7, most people would not stick to Drupal long enough to discover how good it is.

Now, when the Garland theme was finally done, everyone suddenly changed their opinion and congratulated the community on its excellent work. I have to admit this hit a nerve, especially after I'd been spending countless days and nights the two weeks before fixing annoying IE rendering bugs, redoing the CSS layout and adding a whole new layer of Glitz und Glanz to Drupal core.

Only three people did serious work on what became the Garland theme: Stefan Nagtegaal did the original design from scratch and worked with Adrian Rossouw to come up with a proof-of-concept of the recolorable theme. I wrote the color picker, improved the theme and coded what became the color.module based on Adrian's stuff.

Only a handful of people helped with testing of the theme during its development and only after the main theme was finished — most of the bugs were in the recoloring mechanism. How can such a vital piece of Drupal 5 have only have 3 serious contributors, when the whole release had almost 500 people submitting patches?

To me, this shows that we have a problem in the Drupal community, or rather a knowledge void. Not enough Drupal people are savvy enough about theming and design to help out with even small tasks (like a banner) or even give quality tips and feedback on other work. The result is that theming and design receives little attention. Most contributed themes and sites could look a lot better, if they just themed it some more. And getting patches into core that give the defaults a little more oomph is tough, as they are often considered to be useless embellishments.

Still, ever since Drupal started, there has been the recurring cry of doing more to attract great designers to the platform. The overall effects of this have been minimal. However, something similar did happen before.

Before Drupal 4.0 was released, the focus was mainly on features and Drupal was a highly experimental project. After a while, as more people started using it, many users complained that Drupal was too hard or confusing to use. Because of this, 4.0 was the first of many releases that contained significant usability improvements, in this case in the administration area. Many small and large usability features were added. With the menu system and tabs having been added to core by Drupal 4.5, even contributed modules started using the same UI concepts as core. Drupal's UI ended up much more consistent across configurations and it became easier to learn and document. Now with Drupal 5.0, we have undoubtedly produced the most usable release yet.

How did this happen? Over the years, the idea has popped up many times to bring usability experts on board to do a review, and the hope has lived that a usability expert or two will magically pop up in the community and solve all our problems (sound familiar?). Neither has happened so far.

What did happen is that usability became a big priority for the project, and as a result, many people started educating themselves about it. The community quickly identified those in its ranks knowledgable about usability and listened to their advice. Soon, big UI gurus were being quoted on the mailing lists and "-1 isn't usable" became a valid reason to dismiss a feature. Sure, this process took time, but it definitely happened. Plus, the combined usability knowledge and effort of the community, though individually not at expert level, had a much larger effect in the long run than any single expert could have.

The same needs to happen for design. For years now, the Drupal community has been hoping for a group of prodigy designers to magically appear and design a set of jaw dropping themes and UI. They have not shown up. Talking and maintaining a high quality of design across Drupal still often feels like swimming upstream, because most community members don't care much for design unless it is delivered in front of their noses on a silver platter. For many, design is still something to only be enjoyed, not something to be created.

Now, I really want to see this change. For one thing, the shortage of design talent means Drupal is generally perceived to be ugly. It's quite demotivating, because we put a lot of time into it. Unfortunately people illogically, but consistently, assume a relation between how something looks and how good it is built. With Drupal 5 we've done a lot to improve this, but we could still do a lot better. Drupal is no OS X (yet).

For another, when only a handful people are always doing the same jobs, the passion tends to slip out and the challenge becomes a chore. I honestly have no ideas left for a spotlight banner at the moment. That's why the scalability banner is so mind-numbingly boring, though I made plenty of cool ones before.

This is also why I'm holding that OSCMS talk about design this month: I want more people to realize that if your site and/or module is ugly, people aren't going to like it or use it. It's as simple as that. If you mess up something as basic as text formatting, your message simply doesn't get through (hello MySpace users). The only way to change that is to put in the effort to make things look clean and nice. Nice products and nice sites tend to cause happy, dedicated and long-term users.

The community not only needs to realize this, but also needs to teach itself the knowledge and skill to do something about it. Drupal has infinite potential, but it only goes where the community takes it. If the majority remains allergic to design and graphics, very little will change and only at a glacial pace.

Children of Men Fake Media

2007-02-28T00:00:00-08:00

Children of Men Fake Media

I got linked this video, which contains all the fake media created for the movie 'Children of Men' (see my earlier post).

Aside from sci-fi geek fun, I loved watching them to analyse the graphical designs they used. One of the subjects I'll be talking about in my OSCMS talk about design is branding and style. If you're going to attend, here's a great opportunity to do your homework.

Having an eye for graphical design is as important as creative skill, but luckily you can train on this. Each of these ads or clips has a different look tailored towards the product and its audience. Look at the graphical elements, such as images, colors, typography and animation and try to figure out why it's appropriate and effective. There's also some public signage in there which has a style of its own.

If you have some time, a good trick is to take a particular design, look at it for a couple minutes, then try to reproduce it in a graphical program like Photoshop or Illustrator. When you're done, compare your version with the original, and try to figure out what you did different and whether this makes it better or worse. Look for qualities like readability, alignment, typography, contrast and aesthetics. The ones in the movie are probably a bit too graphical each, but you can do this for logos or web sites too.

The clips can be viewed in QuickTime and were done by London-based design studio Foreign Office.

Tip: you can slowly move forwards or backwards in a QuickTime video by scrolling up/down.

DrupalCon - Theme Development presentation

2005-03-23T00:00:00-08:00

DrupalCon - Theme Development presentation

At FOSDEM I gave a presentation about Theme Development for the Drupal Conference in the Drupal developer's room. You can take a look at the slides. The topics I covered are:

Overview of the Drupal theme system
Making the FriendsElectric theme
Clean, semantic XHTML/CSS
Some examples of sexy Drupal sites

The presentation was filmed, the video is available through BitTorrent. For info on how to play the Ogg Theora files, check these instructions.

The other presentations are linked on drupal.org.

Read on for some useful links related to Drupal theming. Drupal-related

Drupal theme developer's guide
Drupal Theme Garden (theme showcase)

Some useful CSS links