Coloured light in Proun

(At Ronimo Games, the Dutch game developer where I am lead programmer, we are currently looking for a coder with experience on the 360 and/or PS3. Interested? Have a look here for more information!)

Last weak I talked a bit about the technical side of lighting in Proun. This week, let's have a look at the more artistic side: the strange light colours I chose for each level.

A very common lighting setup among 3D artists is to let the sky emit a soft blue light, and the sun a bright yellow. This of course makes sense, since these are the colours that the sun and the sky have in the real world.

The fun of having these different colours for the skylight and the sun, is that the sun's shadows become a subtle blue. Brightly lit spots, on the other hand, may become slightly yellow. This fits what my highschool art teacher used to tell me: never paint a shadow with black, always add colour. (Of course, I didn't really understand her at the time...) Since I like this setup, I used it for Proun's first track.



For the second track, I wanted to use a totally different colour palette. Now I am not very knowledgeable in colour theory, but I did notice that yellow and blue are exactly opposite colours. So again I chose opposite colours and after a lot of experimenting with the colour scheme that I had in mind, I came up with a cyan/blue sun and an orange skylight. That is one really weird lighting setup, but I do think it looks good:



The third track is probably the weirdest. A greenish/blueish sun is combined with a pinkish skylight. I think this mainly proves that as long as two colours are chosen that work well together, the colours can be as crazy as you like. So no need to go for the standard lighting setups any more!



So, as these images show, the bright colours of Proun's objects are combined with much more subtle colour effects from the lights. Having just the bright colours worked perfectly well for many works by Kandinsky and Mondriaan, but I think in Proun's case, the subtle colours and shades that the lights add are what really make the graphics shine.

Lighting in Proun

Proun is getting a lot of very positive responses to how it looks. Most people seem to think that the depth of field blur is the core reason for this, but the game still looks pretty good when that is turned off. What really makes the biggest difference, is the careful lighting.

Since most of Proun's environments are static, the lighting is calculated beforehand and stored in lightmaps. Calculating lighting beforehand is called baking. Baking should not be confused with cooking, which stands for saving memory directly to disc (this is used to speed-up loading times, because you don't have to parse cooked data). Interestingly, I have heard rumours that the terms baking and cooking are also sometimes used for certain algorithms that are used in kitchens, but I don't know anything about that.

Lightmaps take up a lot of space (one third of Proun's total size), but they have some big benefits. Because lighting is calculated beforehand on the developer's computer, it doesn't matter if it takes a lot of time. This means that much more complex lighting can be used than when it is calculated in real-time. In Proun, I use this a lot. I think calculating all the lightmaps for just the second track cost my laptop about 30 hours!



The further a shadow is from the object that casts it, the blurrier is gets, especially if the light source is large. Lights that cast shadows that get blurrier with distance are usually called area lights. In practice, it is hardly ever possible to calculate this in real-time, so most games have either fully sharp shadows (as if the light source is infinitely small), or a fixed blur that ignores distance.





(On a sidenote: as soon as Proun is finished, I am going to implement a technique that I have come up with that makes it possible to render real-time area light shadows in games with a Diablo-like perspective.)

In the real world, a lot of light does not come directly from the sun: part of the light is scattered by the atmosphere and comes from all directions. Such light from all directions is called skylight. Skylight makes shadows less deep and add a lot of subtle shadow effects to the world.



Objects reflect light. So if light falls on a red object, then this object will reflect a little bit of red light onto the objects around it. This is called global illumination. This is a very subtle effect, but because of Proun's bright colours it is clearly visible in a couple of places.



All of these things are only possible because in Proun, lights don't move and almost all objects are static. I could not have used lightmaps otherwise. The levels are also small enough to store high quality lightmaps of all objects.

The things I discussed here today are pretty technical. In a future blogpost I will talk a bit about the artistic side of Proun's lighting: crazy light colours.

Sun rays

Writing shaders is one of the things that I enjoy most in making games. Kind of surprising, really, because generally I dislike doing low-level coding and optimisation. Shaders are among the worst in terms of dirty hacks and hardware specific code. So why do I enjoy writing them, then? I think shaders attract me because of this: they are so limited, that science and optics become totally useless. The way real light works is so much more complex than what you can do in a shader, that you need to just make something up that kind of looks similar, although it is often totally unrealistic. Physically, most shaders are utter nonsense. So instead of being all about knowledge and math, shaders centre on creativity and experimenting. Now that's fun! (At least, to a special kind of people, also known as Joosts.)

One of the nicest examples of creativity in shaders is how light shafts are rendered in games. Light shafts are also known as god rays, volume light, sun rays or Jacob's ladders. They are what you see when a light shines through misty air. The microscopic particles in the air are lit and become visible.



Rendering volume light realistically takes a lot of computation time. You need to trace rays through air, take into account the density of the mist, and for a large number of points in the air you need to calculate whether the particles there are illuminated by your light. This includes doing shadow calculations for each of these points in the air. This is way too complex to do well in real-time.

So, some clever programmer somewhere (not me) came up with a trick that looks great and is a whole lot simpler to do. The idea is really simple: you just walk over the image from the pixel towards the sun, and look how much light there is in the image.



Click here for the original article about this technique, which is also the source of these images.

Things like volume, fog density and shadow are completely ignored. There is hardly any relation between this technique and how real volume light works, but somehow it looks very similar and really convincing. Games like Far Cry 2 and Crysis use it to create beautiful lighting effects through the leaves.

There are some limitations to this technique, though. Most importantly, it only works if the light source itself is actually in view. For example, using this technique, it is impossible to render the effect of a flashlight that is pointing away from the viewer.

Rendering light shafts this way not only looks awesome; it is also one of the most beautiful, simple and ingenious shader techniques I know of. Since I like this effect so much, I couldn't help myself but add it to Proun's third track. The track's graphics are far from final, but the light shafts are there. The little video at the beginning of this post shows how dynamic the effect looks in motion.

Stereoscopic 3D

Stereoscopic 3D has been all the hype of late. You know, with those black shutter glasses and expensive televisions, or on the new Nintendo 3DS. Although I usually totally ignore these kinds of hypes (I don't have an iPhone, for example), I somehow already ended up making two games for 3D: Swords & Soldiers for the Playstation 3 and Proun for the PC. The latter will be playable in 3D in a few weeks at the STRP festival in Eindhoven.

I won't bother you with the technical aspects and the weird camera math of stereoscopic 3D, since I think the implications of stereoscopic 3D for graphics are a lot more interesting and not as straightforward as one might think.

The first thing to realize is that you should really consider the monitor to be a window into another world. Things may come out from this world, through the monitor towards the player, but if this happens, then these objects should not be cut-off by the edges of the screen. If an object is behind the screen in the 3D space, then the viewer's brain easily accepts that it is partially behind the monitor's edge. However, if the object is in front of the screen, yet is cut-off by the edge of the screen, then that is really weird to watch: how can an object be hidden behind something that is further away? This is called a window violation. A design implication of this is that in a first person shooter, the gun cannot be in front of the monitor and thus cannot be really close to the player.



However, it is a lot of fun to have objects fly towards you. This looks so real, that as a player, you will want to duck to evade that coconut that is being thrown at you. So it is a good idea to put some of these effects in the game. Unfortunately, both Proun and Swords & Soldiers lack such effects, because I didn't have the time to add additional special effects just for 3D.

For pure realism, one could argue that the distance between the left and right eye in the game world should be the same as on a real person. However, as a programmer, you have control over this. Moving the virtual eyes further away from each other essentially makes the depth effect stronger. The stronger, the more spectacular, but making it too strong causes headaches.



A difficulty of 3D is that your eyes always focus on the screen, even though your brain thinks you are seeing objects that are behind the screen or in front of it. Because of this difference between your brain and your eyes, switching from where you are looking takes longer for stereo 3D than it does in the real world. Even worse: doing it a lot causes headaches, especially with certain people who are sensitive to this (like myself).



This is especially important to think about for the GUI. For example, in Swords & Soldiers, the healthbars are deliberately placed at the same distance as the soldiers. I would have liked to also position the map and the player's gold at this same distance, but because we also have foreground graphics, I had to place the interface in front of the foreground graphics.



A very cool effect in Swords & Soldiers is that because all the objects are themselves 2D, making the entire scene 3D means that all objects look like cardboard cut-outs. In a normal game that wouldn't be cool, but it works really well with the cartoony art style of Swords & Soldiers and looks awesome!

Finally, stereoscopic 3D means that certain cheap tricks cannot be used any more. Many games create rain by simple placing a rain texture on the screen, instead of having actual rain in the 3D world. With stereoscopic 3D, this trick becomes very obvious, so you need to somehow do better rain.

So, even though it is pretty easy to get 3D to work at all, it definitely takes some extra design work to make a game that works really well with stereoscopic 3D!

Designing against motion sickness in Proun

Last week Proun was shown on Kotaku and Gametrailers. Most reactions were extremely positive (awesome, woohoo! ^-^ ), but there were also quite a few remarks about people fearing motion sickness when playing the game. I totally agree that this is a serious issue with this kind of game, so I constantly took it into account with a lot of design decisions. I tested Proun on large flocks of people at different occasions and the nice result of my work against nausea is that it is pretty rare for people who actually play Proun to get motion sickness.

The emphasis on the word play in that previous sentence is important: watching someone play the game or, even worse, watching a video montage of it, is actually a lot more nauseating then playing yourself. The reason for this is simple: the camera only rotates when you steer, so when you play yourself, you see the camera movement coming. This really makes a huge difference, so playing the game yourself is a lot better than watching that trailer.

Preventing motion sickness was an important argument for a lot of design decisions in Proun, especially concerning the camera. A great help is that I have designed and coded quite a few camera systems in other games already: De Blob, Swords & Soldiers, Ronimo's secret new game and the cancelled 3D game that we were developing at Ronimo before we made Swords & Soldiers. Those camera systems weren't necessarily great, but at least I learned a lot while programming them!

In general, an important cause for motion sickness is camera smoothing. Without smoothing, camera movement may look very blocky and ugly. However, smoothing also means that the camera does not directly follows the player's movements, since it must add a little bit of delay to smooth things out. I experimented with smoothing a lot for Ronimo's cancelled 3D game, and it turned out that certain types of extreme smoothing cause almost instant nausea, but that even the tiniest bit already has this effect to a small extent. With a lot of smoothing, the camera may never stand still. So for Proun, I chose not to have any camera smoothing at all. I did experiments with it, but all the smoothing schemes I applied caused too much nausea for Proun's own good.

The reason motion sickness is so important for Proun specifically, is that there is no up or down. The camera constantly rotates the world around and there is no horizon to focus your eyes on. Even the cable whooshes and curves by real fast. So I chose to position the player's vehicle at the centre of the screen and to always keep it there, without zooming or looking around it. This way the ball replaces the horizon as the stable factor on the screen.

Proun may still result in motion sickness a bit quicker than other games, but because of the counter-measures I took in the camera design, I think this is solved well enough. Playtesting shows that most people can play this crazy rotating game as well as any other 3D game. :)

Dirty collision detection trickery in Proun

Some people seem to read Pr0n instead of Proun. I think this may be due to the dirtiness of some of the coding tricks I used in Proun.

Or maybe it is because these words are written so similarly. No, really, I don't think so.

Anyway, programmers who worked with me might agree that I have pretty radical ideas on what good programming is, especially in comparison to how large console games are usually made. I almost always choose simplicity and understandability over performance, over memory use and often even over amount of work.

Most coding interns need to get used to the idea that sometimes when they consider something done, but there is still a way of structuring it in a better way, I actually make them go back and change their already working code, just to make the code itself better. Since the games at Ronimo are around 100.000 lines of code, keeping the code clean is very important. No way will we find our bugs if such an amount of code is also a mess!

An extreme example of choosing simplicity over performance can be found in the collision detection in Proun.

Collision detection in games is usually done per frame. So 60 times per second, all objects are moved a bit and then the game checks whether they collide. This means that the time between two frames is completely ignored!

This becomes a problem when fast moving objects (e.g. bullets) pass through thin obstacles: in one frame they are on one side of the obstacle, and in the next frame they already passed it.



Common solutions to this problem are to cast a ray between the position at the previous frame and the current position. An alternative is to do sub-steps just for the collision detection and only when moving really fast.



Both of these solutions share that they would need additional work in the collision detection of vehicles in Proun. When doing the solution with subframes, the collision detection would run at a different framerate from the rest of the game, which increases the complexity of the code. Not dramatically, but it is more complex nevertheless.

For Proun, I implemented a simpler solution: all gameplay in Proun runs at a fixed 3000 frames per second. Three thousand. Three thousand! The graphics still runs at 60fps, of course (or less if your videocard is too slow).

The result is that even when you race extremely fast (grab a turbo on Impossible difficulty to do that), you still don't move too fast to pass through objects without collision detection detecting this.

Any reasonable programmer would say this is idiotic. So much performance wasted! I even update all obstacle animations at this framerate. Unnecessary! However, Proun's performance is almost entirely in the graphics. The gameplay is very simple and can be processed really quickly, so why would I bother implementing something more efficient and more complex? Doing a lot of gameplay updates per frame is really simple to implement and everything just works. I don't need to add any math for the time-in-between or anything like that, I don't need to think about having different framerates for collision detection and for other gameplay aspects. Any other solution would have made the collision detection code more complex than it is now.

I tested the actual performance impact of this and it turns out that this only impacts the framerate in a relevant way when a really slow processor is combined with a very fast videocard. No one actually has such a combination, so I don't consider that relevant.

So. Simple! Clear!

(And in fact not dirty at all, so maybe the words Pr0n and Proun do look pretty similar...)

Camera mapping tips and tricks

While experimenting with camera mapping, I learned a lot about how to use this technique, so I figured I'd share some of what I've learned here!

1. The Camera map modifier

This is the cheap and simple hint: 3D Studio MAX has a modifier specifically for doing camera mapping and it is called, surprisingly, the Camera map modifier.

2. Do not model object edges

The edges of the object should come from the image itself, not from your model. The reason for this is that polygons have straight, hard edges, while drawings have more blurry and zigzagging edges. You cannot really create a blurry, brushed edge using polygons. Using the image's transparency for the edges also makes modelling a lot easier, since you don't have to model the objects as precisely. The image below shows that you can even just stretch the polygons as far as you like.



I guess with sharp photographs of buildings it may work to model object's edges, but with the kind of art I have been working with, doing object edges exclusively through image transparency looks way better.

3. Split into many layers

To use transparency, you will need to split the image into a lot of layers. This greatly depends on the type of scene you are doing, but in general, the more layers you have, the easier it is to make your 3D scene. For the Evil Pope I used three layers (head, body, thorns), while for Captain August I used a whopping 28 layers! Splitting the original image into all those layers and drawing the parts behind layers was a lot of work, but was also absolutely necessary to make it look good.



4. Put a curve on everything

If you map an entire character to a flat plane, then it will look like a cardboard cut-out as soon as the camera starts moving. Putting the arms and such in separate layers is important, but it also works wonders to simply bend objects to roughly mimic the shape of the real character. Don't overdo this, though, for if you look too far from the side at an object, it will squash the lines of the original drawing too much, decreasing the 2D effect.



5. Animating after the mapping

Once an object has been camera mapped, you can animate it any way you like. For the little video of August, I rigged the bodies and arms of the characters with bones to be able to rotate and bend their arms and heads. This works surprisingly well, without breaking the illusion of a 2D drawing. For the Evil Pope, I was even able to change the character's expression without using any new drawings. This was done by morphing the mesh of the face into different expressions. The only things that I animated with extra textures are blinking and closing eyes. Everything else is just morphing, bending and moving the 3D models.

Last words on camera mapping for the moment: I'm thinking of trying camera mapping in a small real-time 3D game prototype to create a totally unique and awesome graphical style. I'm looking for a concept artist to design that with, so if you think you can draw a lively outdoor environment in a rough, colourful impressionistic style, maybe a little bit like this or this or this or this or this, then please do contact me at joost@ronimo-games.com!

In the meanwhile, I've headed back to working on Proun for a while! :)