Sunday, 18 October 2020

Screen Space Reflections in Blightbound

An important focus during development of our new game Blightbound (currently in Early Access on Steam) is that we want to combine 2D character animation with high quality 3D rendering. Things like lighting, normal maps, depth of field blur, soft particles, fog and real-time shadows are used to make it all gel together and feel different from standard 2D graphics. One such effect I implemented into our engine is SSR: Screen Space Reflections. Today I’d like to explain what SSR is and what fun details I encountered while implementing it into our engine.

A compilation of places with reflections in rain puddles in Blightbound.

Reflections in games can be implemented in many ways. The most obvious way to implement reflections is through raytracing, but until recently GPUs couldn’t do this in any reasonable way, and even now that GPU raytracing exists, too few people have a computer that supports it to make it a feasible technique. This will change in the coming years, especially with the launch of Xbox Series X and Playstation 5, but for Blightbound that’s too late since we want it to look good on currently common GPUs.

So we need something else. The most commonly used techniques for implementing reflections in games are cubemaps, planar reflections and Screen Space Reflections (SSR). We mostly wanted to use reflections for puddles and such, so it’s important to us that characters standing in those puddles are actually reflected in real-time. That means that static cubemaps aren’t an option. A pity, since static cubemaps are by far the cheapest way of doing reflections. The alternative is dynamic reflections through cubemaps or planar reflections, using render-textures. These techniques are great, but require rendering large portions of the scene again to render-textures. I guessed that the additional rendercalls and fillrate would cost too much performance in our case. I decided to go for Screen Space Reflections (SSR) instead.

The basic idea behind SSR is that since you’re already rendering the scene normally, you might as well try to find what’s being reflected by looking it up in the image you already have.


SSR has one huge drawback though: it can only reflect things that are on-screen. So you can’t use a mirror to look around a corner. Nor can you reflect the sky while looking down at the ground. When you don't focus on the reflections this is rarely a problem, but once you look for it you can see some really weird artefacts in reflections due to this. For example, have a look at the reflections of the trees in this video of Far Cry 5.



So we’re looking up our reflections in the image of the scene we already have, but how do we even do that? The idea is that we cast a ray into the world, and then look up points along the ray in the texture. For this we need not just the image, but also the depth buffer. By checking the depth buffer, we can look whether the point on our ray is in front of the scene, or behind it. If one point on the ray is in front of whatever is in the image and the next is behind it, then apparently this is where the ray crossed the scene, so we use the colour at that spot. It’s a bit difficult to explain this in words, so have a look at this scheme instead:


Since SSR can cast rays in any direction, it’s very well suited for reflections on curved surfaces and accurately handles normal maps. We basically get those features for free without any extra effort.

A demo in the Blightbound engine of SSR, with a scrolling normal map for waves.

At its core SSR is a pretty simple technique. However, it’s also full of weird limitations. The challange of implementing SSR comes from working around those. Also, there are some pretty cool ways to extend the possibilities of SSR, so let’s have a look at all the added SSR trickery I implemented for Blightbound.

First off there’s the issue of transparent objects. Since the world of Blightbound is covered in a corrupting mist (the “blight”), our artists put lots of foggy layers and particles into the world to suggest local fog. For the reflections to have the right colours it’s pretty important that these fog layers are included in the reflections. But there are also fog layers in front of the puddles, so we can't render the reflections last either.

The solution I chose is simple: reflections use the previous frame instead of the current frame. This way we always look up what's being reflected in a complete image, including all transparent objects. The downside of this is of course that our reflection isn't completely correct anymore: the camera might have moved since the previous frame, and characters might be in a different pose. However, in practice the difference is so small that this isn't actually noticeable while playing the game.


Transparent objects pose another problem for SSR: they're not in the depth buffer so we can't locate them correctly. Since Blightbound has a lot of 2D animation, lots of objects are partially transparent. However, many objects can use alpha test. For example, the pixels of a character's texture are either fully transparent of not transparent at all. By rendering such objects with alpha test, they can write to the depth buffer without problems.

This doesn't solve the problem for objects with true transparency, like smoke, fog and many other special effects. This is something that I don't think can be reasonably solved with SSR, and indeed we haven't in Blightbound. If you look closely, you can see that some objects aren't reflected at all because of this. However, in most cases this isn't noticeable because if there's an opaque object closely behind it, then we'll see the special effects through that. While quite nonsensical, in practice this works so well that it seems as if most explosions are actually reflected correctly.

Transparent objects like this fire aren't in the depth buffer and thus can't be found for reflections. However, if another object is close behind, like the character on the left, then the transparent object is reflected through that. The result is that it seems as if many special effects are properly reflected.

Having perfectly sharp reflections looks artificial and fake on many surfaces. Reflections in the real world are often blurry, even more so the further the reflected object is from the surface. To get reflections that accurately blur with distance I've applied a simple trick: the rays get a slight random offset applied to their direction. This way objects close remain sharp and objects get blurrier with distance. Artists can tweak the amount of blur per object.

However, this approach produces noise, since we only cast one ray per pixel. We could do more, but that would be really heavy on performance. Instead, to reduce the noise a bit, when far enough away I also sample from a low-resolution blurry version of the scene render-texture. This is a bit redundant but helps reduce the noise. Finally, by default in Blightbound we do supersampling anti-aliasing (SSAA) on the screen as a whole. This results in more than one ray being shot per screen-pixel. Only on older GPUs that can't handle SSAA is this turned off.


Another issue is precision. For really precise reflections, we would need to take a lot of samples along the ray. For performance reasons that's not doable though, so instead we make small jumps. This however produces a weird type of jaggy artefacts. This can be improved upon in many different ways. For example, if we would render the reflections at a lower resolution, we would be able to take a lot more samples per ray at the same performance cost. However, with how I implemented SSR into our rendering pipeline that would have been quite cumbersome, so I went for a different approach which works well for our specific situation:
  • More samples close to the ray's origin, so that close reflections are more precise.
  • Once the intersection has been found, I take a few more samples around it through binary search, to find the exact reflection point more precisely. (The image below is without binary search.)
  • The reflection fades into the fog with distance. This way the ray never needs to go further than a few meters. This fits the world of Blightbound, which is basically always foggy.
The result of these combined is that we get pretty precise reflections. This costs us 37 samples along a distance of at most 3.2 meters (so we never reflect anything that's further away than that from the reflecting surface).

Any remaining imperfects become entirely unnoticeable since blur and normal maps are often used in Blightbound, masking any artefacts even further.



A challenge when implementing SSR is what to do with a ray that passes behind an object. Since our basic hit-check is simply whether the previous sample was in front of an object and the next one is behind an object, a ray that should pass behind an object is instead detected as hitting that object. That's unintended and the result is that objects are smeared out beyond their edges, producing pretty ugly and weird artefacts. To solve this we ignore a hit if the ray dives too far behind an object in one step. This reduces the smearing considerably, but the ray might not hit anything else after, resulting in a ray that doesn't find an object to reflect. In Blightbound we can solve this quite elegantly by simply using the level's fog colour for such failed rays.

By default, SSR produces these kinds of vertical smears. Assuming objects have a limited thickness reduces this problem greatly, but adds other artefacts, like holes in reflections of tilted objects.

This brings us to an important issue in any SSR implementation: what to do with rays that don't hit anything? A ray might fly off the screen without having found an object, or it might even fly towards the camera. Since we can only reflect what we see, SSR can't reflect the backs of objects and definitely can't reflect anything behind the camera. A common solution is to have a static reflection cubemap as a fallback. This cubemap needs to be generated for each area so that the reflection makes sense somewhat, even if it isn't very precise. However, since the world of Blightbound is so foggy I didn't need to implement anything like that and can just fall back to the fog colour of the area, as set by an artist.

The final topic I would like to discuss regarding SSR is graphics quality settings. On PC users expect to be able to tweak graphics quality and since SSR eats quite a lot of performance, it makes sense to offer an option to turn it off. However, what to do with the reflecting objects when there's no SSR? I think the most elegant solution is to switch to static cubemaps when SSR is turned off. Static cubemaps cost very little performance and you at least get some kind of reflection, even if not an accurate and dynamic one.

However, due to a lack of time that's not what I did in Blightbound. It turned out that just leaving out the reflections altogether looks quite okay in all the spots where we use reflections. The puddles simply become dark and that's it.


For reference, here's the shader code of my SSR implementation. This code can't be copied into an engine directly, since it's dependent on the right render-textures and matrices and such from the engine. However, for reference when implementing your own version of SSR I expect this might be useful.

SSR is a fun technique to implement. The basics are pretty easy, and real-time reflections are a very cool effect to see. As I've shown in this post, the real trick in SSR is how to work around all the weird limitations inherent in this technique. I'm really happy with how slick the reflections ended up looking in Blightbound and I really enjoyed implementing SSR into our engine.

Monday, 7 September 2020

The games that inspired me most

A few years ago I was asked by a videogame magazine what games inspired me most. I had a lot of fun compiling a short list for them. Today I've updated that list. These aren't necessarily my favourite games (although they're close): Doom II isn't on this list for example. I'm going to focus not on why I had fun with a game, but why it inspired me, how it taught me something new. They all have some unique aspects that changed how I look at games and game development.

Command & Conquer

As a kid I was so much a fan of Command & Conquer that I even rebuilt some of its units with Lego. I started playing around with 3D computer graphics when I was only 13 years old and C&C was my big inspiration. I wanted to make 3D as awesome as in the cut-scenes for this game. Especially the video where a flame tank drives into a town and torches everything is still vividly in my mind. Many of the game designs that I wrote down on paper at this age were for RTS games inspired by C&C. 



Star Control II

This is easily my favourite game of all time. Great music, crazy humour, fun combat and a rich universe with deep backstories and tons of personality. What is most special though is that this is one of the few games I know where information and exploration are intertwined so strongly. Everything is there at the start of the game, but the universe is way too big to find things without knowing the coordinates of interesting planets. Gathering information and even buying it is a key feature here. In combination with the lack of invisible walls this makes exploration extremely rewarding and quite different from most open world games.


An interesting aside is that in the newly released Star Control: Origins (which I really like, by the way) some players complain about the universe being so empty that grinding through planets is boring. The problem here is that modern players have been taught that it's useful to randomly wander around the world, because most open world games these days make sure there are interesting things everywhere. That's fun of course, but that means that in a game like Fallout information isn't very important: by randomly walking around you'll likely find everything (although that does cost a lot of time). Star Control's exploration is totally different because the universe is so much bigger. I think the problem in Star Control: Origins is purely expectations: it isn't clearly communicated to players that this game is different.

Another aspect of Star Control II that has greatly influenced me is the music. Each alien race you encounter has a theme song that captures the personality of that race. Music in many games (as well as movies) mostly serves to set the mood and influence the viewers emotions. But I like music that takes the foreground and really adds to the personality of the game. Inspired by the alien themes from Star Control II, I pushed for having character theme songs in our game Awesomenauts as well. This resulted in Sonic Picnic making an amazingly diverse soundtrack for Awesomenauts, adding tons of personality.

Bioshock

In most games the story and the gameplay are told intertwined but are not truly related. Not in Bioshock: here the gameplay and the story revolve around a common theme. Both add meaningful insights to the concepts of freedom and choice that are explored. Bioshock is not just a marvellously fun game; it also made me ponder the philosophical themes it explores. Especially the twist halfway the game felt incredibly smart and exciting. I felt like I had personally fallen victim to exactly what had happened to the character in-game. One of the few games with true philosophical depth.


World of Goo

World of Goo is the ultimate example of how much creativity is needed to extend a simple and original concept to a full game. It seems to me like many of the participants of gamejams think that coming up with something original is the highest goal in game design. But much more difficult is to develop that original concept from a gimmick into a full game. World of Goo started out as a gimmicky mini-game and ended up as a glorious game. It adds enormous diversity and depth without adding a lot of features or complexity. This is true mastery of game design.


Deus Ex

The original Deus Ex is a great game in general and the combination of RPG + FPS is pure gold for my taste in games. However, the most unique thing in this game to me is the ending. Usually when games have different endings, they are very black-and-white. There are the good ending and the evil ending. Or maybe there were extra challenges and then there are a good ending and a super good ending. Not in Deus Ex: here the three endings all have up- and downsides. It's not clear which would be the best for humanity and the options are clever enough that I spent considerable time pondering each. Deus Ex has similarly deep philosophical thoughts as Bioshock, but while Bioshock is about the game manipulating the player, Deus Ex is about the player manipulating the world.


Dear Esther

A core aspect of game design to me is that a game should be fun without the graphics and the polish. If a game is already fun with just boxes, then it will be even better when visuals and sound are added. Dear Esther is a fantastic example of a game that proves me wrong. In fact, there's hardly any gameplay. But the storytelling, the music and the visuals all add up to an incredible experience. Dear Esther is also interesting in that it suggests a lot and explains very little. I played this game with my wife and we were constantly discussing what it all meant. The game continuously pushed our thoughts into different directions with new snippets of information.


Railroad Tycoon

I am a big fan of trains and always want to see more of them in games. I fondly remember Railroad Tycoon, Transport Tycoon and Sid Meier's Railroads. As a teen I wrote tons of pages of notes on an RTS that resolved around building railroads to transport resources. Trains could be armed to defend them and gather the resources needed to build an army. I doubt that game will ever happen, but anything with trains makes me all giddy with excitement. In fact, most of the game ideas I come up with today still have room for trains somewhere. I hope that in a few years I will finally have made at least one game that actually features trains!


Ori And The Blind Forest & Rayman Origins


I'm grouping these two games together because they have done a similar thing: they have both raised the bar for 2D art in games. 2D art is sometimes seen as lesser than 3D art, but Ori and Rayman have shown that it's possible for a 2D game to have the appeal of a big AAA production. At Ronimo we have studied the animation, special effects and level art in these two games in detail to see what techniques we could apply to our own games.


Braid


Of all the games in this list, Braid has probably influenced me most, and on several levels. For starters, Braid introduced us to the concept of self-publishing and indie development. Xbox Live Arcade had been around for a little while already but Braid, World of Goo and Castle Crashers really showed us the potential of indie and self publishing. When we started Ronimo we thought we needed to make disk based full price games. This was however not a path we were succeeding at so when we realised that we could do what these games were doing, we adjusted our plans and made a smaller game on our own instead. This became our first commercial release: Swords & Soldiers on Nintendo Wii.

Braid also influenced me on a more personal level. My interpretation of Braid's story is that it's not about an actual princess, but about the danger of looking for perfection. The main character has a girlfriend, but he breaks up with her because she isn't perfect. So he looks for the perfect woman, his 'princess'. But he doesn't find her because no one is perfect. The result is that he's alone and doesn't have his actually quite wonderful girlfriend anymore. In the epilogue the game also mentions other examples of the dangers of obsessing over perfection, like a scientist looking for the perfect science but creating the nuclear bomb as part of the Manhattan Project.


I don't know whether the story was truly meant like this, but this interpretation has greatly influenced me in my view on both creativity and life. Perfection is an impossible goal so striving for it only hampers you. Perfectionism causes you to spend too much time looking for an even better game concept to make, or endlessly iterating your game instead of releasing it and making another, or breaking up with your partner instead of making it work. Braid has truly changed my mindset in what to strive for in life and work.

So, that's it: the games that inspired me most! What games changed your view on games or the world?

Sunday, 9 August 2020

Three nasty netcode bugs we fixed around Blightbound's launch

In the weeks before and after the Early Access launch of our new game Blightbound we've been fixing a lot of bugs, including 3 very stubborn netcode bugs. They weren't any fun while we were looking for them for days, but now that we've found and fixed these they're actually quite intriguing. So today I'd like to share 3 of the more interesting bugs I've seen in the past few years.
While these bugs mostly say something about issues in our code, I think they're also nice examples of how complex low level netcode is, and how easy it is to make a mistake that's overlooked for years, until a specific circumstance makes it come to the surface. All three of these bugs were also in the code of our previous game Awesomenauts, but none of them actually occurred there due to the different circumstances of an Awesomenauts match compared to a Blightbound party.

The one thing all three of these bugs have in common is that they were very hard to reproduce. While they happened often enough to be game-breaking, they rarely happened in testing. This made them very hard to find. For the first two we found a way to reproduce them after a lot of experimenting, while for the final one we never got it ourselves until we had figured out the solution and knew exactly how to trigger it. Finding that one without being able to see it was some solid detective work by my colleague Maarten.

Endless loading screens number 1


This bug occasionally caused clients to get stuck in the loading screen forever. After adding additional logging we saw that the client never received the HostReady message. This is a message that the host sends to the clients to let them know that loading is done and they should get into the game. The client was forever waiting for that HostReady message. Upon further investigation it turned out that the host did actually send the message, but the client received it and then threw it away as irrelevant. Why was that? 

To answer that question, I need to give some background information. An issue with the internet is that it's highly unreliable. Packets can be lost altogether and they can also come in out-of-order or come in very late. One odd situation this can cause, is that a message from a previous level can come in once you're already in the next level, especially if loading times are very low. Applying a message from the previous level to the next level can cause all kinds of weird situations, so we need to recognise such an outdated message and throw it away. We do this with a simple level ID that tells us which level the message belongs to. If we get a message with the wrong level ID, we discard it.

That's simple enough, but we saw that both client and host were in the loading screen for the next level. Why then would the client discard the HostReady message as being from the wrong level? The reason turned out to be in our packet ordering system. Some messages we want to process in the order in which they are sent. So if one message goes missing, we store the ones after it and don't process them until the previous missing one is received and handled.

The bug turned out to be in that specific piece of code: the stored messages didn't store their level ID. Instead, when they were released, we used the level ID for the message that had just come in. This went wrong when a message from the previous level went missing and came in a second or two too late. By that time the HostReady message had already been received, but had been stored since it needed to be processed in order. Since the level ID from the previous room was used for all out-of-order messages, the HostReady message was processed as if it came from the previous room. Thus it was discarded.

I imagine reading this might be mightily confusing, so hopefully this scheme helps explain what went wrong:

Once we had figured this out, the fix was simple enough: store the level ID with each message that is stored for later processing, so that we know the correct level ID  when we get to it.

Endless loading screens number 2

The second loading screen bug we had around that time had a similar cause, but in a different way. When a client is done loading, it tells the host so in a ClientReady message. The host needs to wait for that. Similar to the bug I described above, the ClientReady message was received but discarded because it was from the wrong level. The cause however was a different one.

Sometimes the client starts loading before the host, for whatever reason. Since loading times between levels are very quick in Blightbound, the client can be done loading before the host starts loading. Once the client is done loading, it sends the ClientReady message. However, if the host isn't loading yet and is still in the previous room, then it discards the ClientReady message because it's from the wrong room. A moment later the host starts loading and when it's done loading, it starts waiting for the ClientReady message. Which never comes because it was already received (and discarded).

Here too the solution was pretty straightforward. We first changed the level ID to be incremental (instead of random) so that we can see in the level ID whether an incorrect message is from the next or the previous room. If it's from the previous room, we still discard the message. But if the message is from the next room, we store the message until we have also progressed to that room, and then process it.

The big desync stink


The final netcode bug I'd like to discuss today is rather painful, because unlike the others it still existed at launch. In fact, we didn't even know about it until a day after release. Once we knew it existed it still took us days to find it. In the end this one was fixed in update 0.1.2, six days after launch.

So, what was the bug? What players reported to us was that randomly occasionally the connection would be lost. In itself this is something that can happen: the internet is a highly unreliable place. However, it happened way too often, but never when we tried to reproduce it, not even with simulated extremely bad connections. Also, we have code that recognises connection problems, but this bug didn't trigger that code, so the game thought it was still connected and kept playing, with very odd results.

After a few days a pattern started to emerge: many players reported that this always happened once they had notoriety 3 or 4, when a legendary item was attainable. We didn't think that the bug was actually linked to notoriety or legendary items, because these are quite unrelated to netcode. Instead, we suspected something else: maybe the bug happened after playing together for a certain amount of time.

With this hypothesis in hand we started looking for anything where time matters. Maybe a memory leak that grew over time? Or, more likely: maybe some counter that looped incorrectly? As I described in the above issues, netcode can have many types of counters. I already described the level ID counter and the counter for knowing whether a message is received in-order. There are others as well.

The thing with data that is sent over the netwerk is that it needs to be very efficient. We don't want to send any bytes more than necessary. So whenever we send a counter, we use as few bits as possible. Why send a 32 bit number when 16 bits will do? However, when using smaller numbers they will overflow more quickly. For example, a 16 bit number can store values up to 65,535. Once we go beyond that, it loops back to 0.

Often looping back to 0 isn't a problem: the last time we received the lower packets is so long ago that we won't have to expect any overlap with the last time we used number 0. However, we do need to handle the looping numbers correctly and not blindly throw away number 0 because it's lower than number 65,535.

And this indeed turned out to be the problem. One of the counters we used was a 16 bit number and after about one hour it looped back. In that particular spot we had forgotten to include code to handle that situation. The result is that from there on the game received all messages but discarded them without applying them. Like in the cases above, the solution was simple enough to implement once we knew what the problem was.

One question remains: why wasn't this a problem in Awesomenauts? For starters, Awesomenauts matches are shorter and the connection is reset after a match. So the counter never reaches 65,535. In Blightbound however the connection is kept up in between dungeon runs, so if players keep playing together the number will go out of bounds. But there's more: if the bug would have happened in Awesomenauts, the game would have thought it was a disconnect and would have reconnected, resetting the counter in the process. In Blightbound we determine connection problems in a different way, causing the connection to not be reset in this particular case.


A lesson relearned


Netcode being hard is not new to us and getting weird bugs is inherent to coding such complex systems. However, there is one step we did for Awesomenauts that we didn't do here: auto testing. Since Blightbound uses a lot of netcode from Awesomenauts and Awesomenauts has been tested endlessly, we thought we didn't need to do extensive auto testing for this same code in Blightbound. This was a mistake: the circumstances in Blightbound turn out to be different enough that they cause hidden bugs to surface and wreak havoc.

We're currently running autotests for Blightbound to further stabilise the game. This already resulted in another netcode fix in update 0.1.2 and several obscure crash fixes that will be in the next major update.

To conclude I'd like to share this older but still very cool video of auto testing in Awesomenauts. It shows four computers each randomly bashing buttons and joining and leaving matches. The crazy movement of the Nauts is fun to watch. For more details on how we implemented auto testing in Awesomenauts, have a look at this blogpost.




Tuesday, 4 August 2020

Blightbound OUT NOW! Early Access on Steam!

Last Wednesday our new game Blightbound launched on Steam in Early Access! This is a super exciting moment as this is our first new IP to release in 8 years. We've been developing Blightbound for nearly 3 years now and it's really special to finally have the game live and see the community play it! Here's our launch trailer:



We have big plans for this game in Early Access. As the roadmap shows we have a ton of things in development to improve and expand the game. Since this is an Early Access release, there are plenty of fixes and improvements to be made to the game and the first two hotfixes have already released.

Through many years of expanding Awesomenauts we've gained a lot of experience running a live game and responding to the community and we plan on applying all of those lessons here. Robin and I did a talk on Games As A Service a few years back at Steam Dev Days and it's fun to be able to get back to that.

For example, during the first period of Awesomenauts live-ops we didn't really brand our patches and didn't distinguish between fixes and content drops. Here we're doing that right away: hotfixes happen in between and don't contain new content, while major updates get a name and branding and a strong dosis of new stuff to keep players interested and bring players back who had seen it all already. Updates will even come with their own accompanying short stories, written by Roderick Leeuwenhart. The launch story, named "Forging Ahead", is already live on our website and more will follow in the coming period.

Now that the game is out there, I'm also able to write more about it. In the coming period I hope to find the time to talk about how things were built, since there are tons of interesting topics to discuss! We've even already encountered our first set of blog-worthy networking bugs, so I think I'll start there in my next blogpost.

Sunday, 14 June 2020

Blightbound gameplay revealed!

Yesterday during the PC Gaming Show the gameplay for our new game Blightbound was revealed! We've been working on this game for nearly 3 years now, so it's really cool to finally show it to the world.



Blightbound is a multiplayer dungeon crawler that tasks three heroes to venture down from their mountain refuge to face the abominations of the Blight - a mysterious and corrupting fog that enshrouds the land.

I've personally been having a lot of fun with the tech for the characters. Since Awesomenauts characters are sprites they can't be big and they can't be really high framerate: that would just use too much texture memory. We wanted to fix these problems for Blightbound so I've implemented a new system that does allow us to do screen-filling bosses at high framerate. I've also implemented tech for doing real-time lighting on 2D characters with real volume and depth, instead of just lighting them as flat cut-outs as would be the obvious way to light 2D elements in a 3D world.

And that's just a few of the things I've personally worked on. Ronimo is currently 29 people so there are tons of interesting things the rest of the team has been making for Blightbound! This is by far the biggest game we've made so far and I'm really happy with how it's all turning out.

Last month we also released a story trailer:



There's a lot of interesting development stuff so I hope I can find the time to write a series of blogposts on specific aspects of Blightbound development in the future.

Tuesday, 9 June 2020

5 tips for reducing stress as a game company owner

Being a company owner can be a stressful occupation. Your decisions can make the company successful or run it into the ground. There's plenty one can worry about: the game market is constantly changing, so what's the best strategy to follow? How to handle when a co-owner wants something different than you? Mistakes can cost you both your livelihood and your dreams: after all, most people who start a game company do so because they have a passion for doing their own thing. Oh, and if you have employees, you're responsible for their fate as well!

While such topics can indeed cause a lot of stress, I have been relatively relaxed under all of that for the past 13 years. So I was wondering: do I have any mental tricks that help me cope with all of it? Turns out I do! Here are 5 things that I do that help me experience less stress. I hope sharing these can help make entrepreneurship a slightly more chill experience for others as well.

Before I continue I should mention that there are many other factors that determine how stressful it is to run a business. For example, I've always made sure to live slightly cheaper than I could afford so that I could build up savings for cases of economic hardship. This post is not about such financial choices. This post is also not about how a successful game launch or signing a good publishing deal will alleviate stress. Nor is it about raging communities review-bombing your game over a single removed feature or seeing company finances being only a few months from bankruptcy. At Ronimo we've experienced all of these things and much more over the past 13 years and while the things I'll discuss in this post have definitely reduced the stress a lot for me personally, it's been an emotional rollercoaster nevertheless! This post however is about coping with the day-to-day, constant pressure of running a business.

Also, one more side note: I wrote most of this post half a year ago and held back on posting it because one of my colleagues got a burn-out just before I got around to publishing this article. This post isn't about that colleague at all, but it seemed rather insensitive to post anything about stress right after that so I shelved this one. Now that that's a little bit less recent I feel I can post this. In a way it's even more reason to post this, since it shows just how stressful things can be!

1. Figure out why the worst-case is still nice


To me the most important element of being relaxed when I'm afraid things might go wrong, is to figure out why the worst case would actually still be pretty nice. This way regardless of whether something is a success or not, I can always be happy about it. One of the easiest examples for this is that if your game fails to sell well, at least you can be proud of finishing and releasing your own game. That's pretty amazing by itself!

The point where this made the biggest difference for me, was when we started Ronimo. Something I wondered about back then was: what if we would work on a game for a year or two but then can't find a publisher so it would end there? No game released, no income for two years, company gone. That's pretty bad, right? Well, actually, to me back then even that scenario wasn't a nightmare. Making games was (and is) my dream job and making my own games with my friends is one of the best things in the world. In the worst case I would still have done the coolest thing for a year or two. That's pretty neat even if it hadn't been a success!

Another example of this way of thinking is more recent. The owner of another studio told me they were really stressed out over the idea that if their company would go bankrupt, their employees would loose their jobs. That's a huge responsibility and indeed a horrible thing if it happens! However, even here I would argue that the worst case still has a silver lining. If you had never started your company, those people wouldn't have had that job in the first place. Even if it ends, you as the studio owner have provided them with an awesome job for years. You'll have given them a chance to gather a lot of experience that will go on their resume and with which they can get their next job. This may sound crude and even though it stinks if it ends, being able to have done that even for a few years is already an amazing thing to have achieved!

2. Better to make a wrong decision than no decision


An easy trap to fall into when being responsible for big decisions is to analyse them into infinity. "Maybe if we gather some more information, we can make a better choice." "Maybe if we discuss it a bit longer, we'll all agree on the next decision." While it's certainly true that important decisions need to be researched and discussed thoroughly, it's also important to act. Try things. Experiment. Don't get stuck in decision paralysis.

As Asimov wrote in his Foundation series (which just so happen to be among my favourite books):

"To succeed, planning alone is insufficient. One must improvise as well."

Often it's better to try something and see whether it works, than to endlessly think about what the best choice is. In game design this is an obvious rule that most people know: prototype, experiment, test, iterate. What you may not realise is that this same mindset can be applied to business as well. The key here is to not just act, but also constantly evaluate whether what you're doing works. If it doesn't, then change it. Don't be afraid to make mistakes and don't be afraid to admit that you've made a mistake. It's far better to fix it afterwards, than to pretend it didn't happen.

For example, we released many updates to Awesomenauts and often we would debate how to communicate about them. What would be the best marketing strategy for an update? However, far better than that was to simply try things and if it didn't work, try something different with the next update.

Another example is a lot more mundane. At Ronimo our daily lunch with the entire team is a coveted tradition (pro-Corona, that is). However, now that we're growing it becomes impractical to let the entire team lunch at the exact same time. So it was suggested that we should spread the lunch period so that people don't all lunch at the exact same time anymore. Some of the owners of the company (including me) really disliked this idea since it seemed to go pretty strongly against Ronimo's very DNA. However, instead of debating this forever, it was decided to just give it a try and evaluate afterwards. Now that we've tried this, it turns out this not only solves the overcrowded lunch, but also solves the issue that the same cliques formed daily at the lunch table. Now that people are moving in and out of the lunch, where people sit is much less static. Even I now think the spread lunch period is an improvement! If we had tried to make a final decision on this topic before trying it, then I doubt it would have happened at all.

3. Ask lots of advice


Whatever you're doing, someone is bound to have tried something similar before, especially when it comes to the business side of running a game studio. Hearing their experiences is an invaluable source of information for making better decisions yourself.

You may think the games industry is pretty closed off, with all those NDAs (Non-Disclosure Agreements), press embargoes and secretiveness around new games. However, towards fellow devs many game studios are a lot more open. I've often emailed other studios, asking them how they approached something, and often I got really in-depth answers, sometimes even including offers for Skype calls. Of course this is easiest if you know someone at the studio or know someone who can introduce you, but in some cases I've even cold-emailed someone I didn't know at all and still got an insightful answer. And if you think that only goes for indie developers, think again: I know some people at AAA studios and even they were happy to give us advice on topics we struggled with ourselves.

Just keep in mind: be respectful of other people's time and don't go prying just out of curiosity. If you have a real question, ask it and learn from their experiences.

You may wonder how to do something back for people who share their knowledge. Often that's difficult because in many cases the person who gives advice is more experienced than you are or might just not be in need of your own knowledge. In my opinion that's fine. Instead of doing something for them, pay it forward: help someone else when you can and ask nothing in return.

4. Don't feel responsible for everything


A common pitfall when running your own company is that you feel responsible for everything. After all, it's your own company, or at least it partially is. While this may work at first, it's also a great way to get a burn-out, especially as the company grows. Once you employ a dozen people, it's impossible to check all the details of everything they do. Trying to do so costs too much time and distracts you from your own work. You'll simply have to trust them to do their work well and only check on some parts of it.

My own approach to this is that when a new programmer joins our team, during the initial period I review all their code. Then, as they get more experienced with our way of working and thus the amount of feedback I give decreases, there comes a point where I rarely check their code anymore at all. The key in my experience is that while that first period might be time-consuming and maybe even frustrating for both parties, it sets clear expectations as to what I expect in terms of code quality, working method and coding style.

Letting go can be really hard. The boss of another studio recently had a nice way of explaining why you need to do so anyway. He said something along these lines: "Previously I did everything myself. Now I employ dozens of people and don't have the time to code much myself anymore. This was frustrating, until I realised that all those people are working for me. Previously I could spent months making one thing. Now I tell all those people what to work on, and a few months later many things are done. Much more than I could ever do on my own. In a sense, while I don't do those things myself anymore, I've gotten more productive than ever."

5. Accept that things won't always go your way


Most people who start a game company do so with a few co-founders: often there are 2 to 4 founders. In our case we went a little over-the-top and started the company with no less than 7 founders, all with equal ownership rights over Ronimo Games. I've often been asked how we managed to make this work: seven captains on a ship is a whole lot and they're bound to all steer the boat in a different direction. That's bound to produce a lot of friction.

The reason this has worked for us for over a dozen years now, is that having such a big group of founders meant that we immediately ran into disagreements but needed to make decisions anyway. So we decided that if we don't agree on something, we'll just decide by voting. In practice that means that a lot of times we make decisions that I personally don't agree with. Realising this early on also made me accept this early on. Once you've accepted that many decisions will not be exactly what you want them to be, it becomes a lot easier to cope with this.

I expect smaller groups of founders will often have this problem to a lesser degree: it's easier for 3 people to share the same opinion than for 7. However, that also means that when you don't agree, a group of 3 founders might also be more likely to get into a fight over it, resulting in lots of frustration and stress. So I think that even for smaller groups of founders, it's crucial to accept early on that even though it's your own company, it will not always go according to your own plan.

Note that this mindset will also help when dealing with employees. To make talented people shine, you have to give them some room to express their own creativity and ingenuity. This also means occasionally letting something happen even though you disagree with it. As a boss you may technically have the right to force your own decisions on your employees, but sometimes it's better not to.

Regardless of your mindset, there's bound to be stress involved when starting and running your own (game) company. However, the things I've listed in this post have greatly helped me cope with it all and have made me relaxed most of the time. What are your tricks to reduce stress and stay chill?

Sunday, 24 May 2020

Then the Halls Were Empty... and I Turned It On!

I've finished a new song! It's an instrumental that tells a little story. This composition is the centrepiece for my album: here the gate that it's all about is opened for the first time, into who-knows-where. After this the real drama starts: the next song is about how the entire facility needs to be destroyed in order to stop creatures from flooding in!



The first 2 minutes of this song actually started as the soundtrack for a cancelled Ronimo game, all the way back in 2012. I wrote a few short pieces for that and this one made it in. I started by building a rhythm from workshop sound effects, like a tape-measure sliding back in, and then composed the rest from there. While the atmosphere worked really well with our game prototype, the rhythmic sound effects turned out to be problematic: the game also had real sound effects and that didn't combine well. So I simply removed the rhythmic part from my song and then it sounded good in the game. I think this is a beautiful example of how creativity and iteration work: the things you make serve a purpose in the process, even those that get scrapped in the end.

When that game was cancelled, this track went into the fridge. Nevertheless it remained one of my favourite tracks and I came back to listen to it quite often. I wanted to do something with it, but didn't know what. Back then it was only 1:50 minutes long and that felt too short to be 'finished'.

Then I heard Home By The Sea by Genesis. I'm a huge fan of Genesis so it's not rare for me to listen to their music, but this particular time I suddenly realised that I could try doing the same structure for my own song. Home By The Sea is basically two songs in one: it starts as a pop song but halfway it quiets down and then at 5:07 a big beat sets in and it becomes an amazing instrumental. I wanted a similar transformation in my own song! So I tried emulating that vibe, but I failed and it doesn't sound anything like the original. In this case however it did capture something else: I really liked the resulting sound! To me that's another great example of how creativity can work: take an inspiration and then morph it into something completely different to make it your own.

I continued from there, turning it into a 6:25 minutes long piece. That actually makes it my longest composition since I was in a prog-rock band over 15 years ago.

For the title I drew some more inspiration from Genesis. The original title of my song was "And Then The Halls Were Empty", but the more active second part didn't fit that quiet title anymore, so I needed to add something. Genesis has a wonderful pair of songs called "Unquiet Slumbers for the Sleepers..." and "...In That Quiet Earth". I figured I could do something similar with the title of my song, hence the ellipsis (...) in my title. (Note that the second half of the title also sounds a whole lot like another Genesis song, but that's actually a coincidence that I didn't notice until I started writing this blogpost.)

The inspiration for the final bit (starting at 3:44) comes from the song Prelude 1 by Floex and Tom Hodge. Here the result is closer to the inspiration, but I feel my composition is different enough that it's become its own thing. I actually met Floex (Tomáš Dvořák) at a conference a few years ago and was fan-boying to the max, since his soundtrack for Machinarium is one of my favourite game soundtracks. Especially the second half of The Glasshouse With Butterfly is mind-blowingly beautiful. Turns out Tomáš Dvořák is also a very friendly guy to talk to. :)

Since I've started the tradition of creating cello sheet music for all of my songs, I've done so for this one as well. From the perspective of the cello this isn't my most interesting composition, but if you'd like to play the cello parts, you can find sheet music and an MP3 to play along to (without the cello) on music.joostvandongen.com. This website also has sheet music for all my other compositions.

This song actually completes the line-up for my album, which will be called "The Ageless Gate - A Cello Tale". All 13 songs are now complete, making for a grand total of 47:03 minutes of music. The only steps left now are mixing/mastering and making things like a cover and booklet, since I want to make a proper CD out of this one (as well as put it on streaming services like Spotify, of course).