Showing posts with label QA. Show all posts
Showing posts with label QA. Show all posts

Sunday, 6 January 2019

An overview of many ways of doing a beta

Giving players access to the beta of a new game or new content before it's released is a great way to get feedback and find bugs, allowing you to add that extra bit of polish, balance and quality before the official full release. There are many different ways to give players access to a beta. Which to choose? In this article I'd like to give a comprehensive list of options in today's market and discuss the differences.

Traditionally bugs in games are found by QA testing companies. However, hiring a QA company to exhaustively test a complex game is very expensive. Many smaller companies don't have the budget to hire QA at all, or can only get a limited amount of QA and can't let QA cover every aspect of the game, let alone doing so repeatedly for every update. However, even if you do have the budget for large amounts of QA testing, that won't give good feedback on whether a new feature is actually fun or balanced. That requires real players, experiencing the content in the wild. So whether you can afford paid QA or not, a beta might still be a good idea.

There are many aspects to doing a beta. Should everyone get access, or only a limited number of players? Is the beta for a new game that hasn't released yet, or for new content for an existing game? Is the beta also intended to gather additional development funds, or only for testing purposes?

Another interesting topic is what to actually put in a beta. Should it be all the content, or only a portion so as not to spoil the main release too much? (There's an interesting bit about that in this talk about Diablo 3.) How early should we do a beta? Although these are important questions, to limit the scope of this post I'm going to ignore the content of the beta: today I'm focusing exclusively on how the beta is delivered to customers.

Over the years at Ronimo we've done a bunch of different approaches to betas. With Awesomenauts and the recently released Swords & Soldiers 2 Shawarmageddon we tried betas before release and for new content, through DLC and through beta branches, limited paid betas and open betas, and more. That means a large portion of this post is based on our own experiences, but since I want this list to be as comprehensive as possible I'll also discuss approaches that we haven't tried ourselves.



Since consoles offer very few possibilities for betas and since Steam is the biggest and most complete platform on PC, this post mostly lists options in Steam. Some of these will probably be possible in similar ways on competing platforms like GoG or Itch.io. If I missed anything that's fundamentally different on other platforms or if I missed some approaches altogether, then please let me know below in the comments so that I can add them.

Steam beta branches

For updates to an already released game

This is the most common way to do a beta on Steam. When uploading a build you can select in which branch it should go live. This makes it possible to release a build under a 'beta' branch only. Users can then simply right click the game in Steam and select the branch they want, after which Steam will download it and replace the main game with the version from the branch.



If you want to limit access to the beta you can set a password for the branch. This works fine, but it's a single password for all users, so if you share this password with players there's a good chance that some might share it further with others. For Awesomenauts we got lucky with our community: no players posted the passwords for closed betas publicly online. Undoubtedly some players did share a password with a few friends privately, but that never caused any problems.

Branches are also great for internal testing purposes. When we want to test a build internally or want to provide a build to QA we also use Steam branches and simply share the password only internally or with the QA company.

Beta through (free) DLC

For updates

A downside of Steam beta branches is that when you switch to or from a branch, Steam downloads and updates the game to this version. In other words: switching takes time and bandwidth and it's not possible for users to have both the beta and the main game on their computer simultaneously. If updates are hundreds of megabytes or even bigger then this gets cumbersome for users. In cases where an Awesomenauts beta didn't get a lot of feedback we often saw players mention that they didn't like to wait for the download.

Our solution was to not use beta branches anymore and instead put the entire beta build in a DLC. Users can then enable the DLC to get the beta. This allows them to have both the main game and the beta on their computer simultaneously. On Steam it's possible to set a DLC to being disabled by default, so users can deliberately choose to get betas or not by enabling the DLC in the Steam interface.

That the beta is a DLC doesn't mean it needs to be paid: it's possible to do free DLC on Steam. Nevertheless, the option to make the beta paid is useful in some cases. For example, backers of the Starstorm Kickstarter campaign were initially the only players who got Awesomenauts beta access. We could have handled this by making the DLC unlisted in the Steam store and sending keys for it to Kickstarter backers. I guess it's even possible to make a beta a paid DLC directly on Steam, although I imagine this might rub some players the wrong way.

Giving out keys for the main game before launch

For new games

This is the easiest way to do a beta before the release of a game. Just put the build live before the store opens and give keys to the players you want to have access to the beta.

A big question with this approach is what to do once the game actually releases. Do those users keep the game, or do you revoke those keys? If this was an open beta then you'll probably want to revoke the keys, but in case of a closed beta with a small group you may also choose to consider the game a gift for those who did beta testing.

If you choose to revoke the keys, be sure to do so a few days before launch. I've heard of cases where users couldn't buy the game on launch because the revocation of the keys had been done too shortly before launch: Steam apparently hadn't processed that entirely yet. I have no idea how much time is needed for that to not go wrong, but revoking the keys a few days before launch seems safe enough I guess.

Beta as a separate app

Both for new games and for updates

An issue with giving users keys to the main game and then revoking those before launch is that if they had the game wishlisted, then the wishlisting will be gone after this. A solution for this is to do the beta in a separate app entirely. This way it's an entirely separate game with its own settings, achievements, leaderboards and AppID. This version of the game is not listed in the store, so it only exists for those users who activated the game with a beta key.



An additional option when doing your pre-launch beta through a separate app is that it can continue being used after the game has launched and can then be used to do betas for updates.

Note that some developers have reported that Steam didn't allow them to do this. We've applied this method in Autumn 2018 ourselves and it wasn't a problem then. Apparently Steam's rules for whether this is allowed or not are not entirely clear.

Steam Early Access

For new games

Early Access allows you to sell a game that's not actually finished yet. This way development of the game can be done in a very public and interactive manner, getting constant player feedback while still adding core systems to the game. Another benefit is that Early Access games are generally paid, so this can help generate additional funding before the actual launch of the game.

Common wisdom seems to be that the launch into Early Access should be considered the main launch of the game. In many cases the final launch of a game is completely ignored by press and players alike, unless the game became a success during Early Access already. This means that a game should be really strong before going into Early Access: it might have missing features and bugs but if it's not super fun yet, then you likely won't get a second chance when the game releases out of Early Access.

An interesting aspect of Early Access is that it functions as a strong excuse towards players for bugs, balance issues and a lack of content. Reviews by both players and press for an Early Access game will often mention things like "It's buggy but that's okay since it's still in Early Access." The equivalent of that for a normally released game is "Don't buy this buggy mess."

For some users Early Access has left a sour taste because some games never launched out of it or didn't deliver on the promised features. Nevertheless, Early Access remains a strong category with many successful games.

Xbox Game Preview

For new games

While Xbox Game Preview is roughly the same as Steam Early Access, I'm listing it as a separate option because it's the only form of beta or early access that's currently available on consoles (as far as I know), making it quite a unique thing.

I don't know what Microsoft's policy around Game Preview is exactly, but I expect this option is not open to just everyone, so if you want to go this route you probably need to talk to Microsoft. I'm guessing that for the right projects, Microsoft might even have some budget to help get them into Game Preview. One thing to keep in mind is that competing platforms might be less interested in featuring your game on launch if it has already been on Game Preview for a while, just as they are generally less interested in featuring a game that launched on another console first.

Soft-launch on a smaller platform

For new games

As I mentioned above, the Early Access launch version of a game needs to be pretty strong. That's kind of counter to the goal of Early Access: getting feedback in an early stage. To work around this problem some developers choose to release their games on a smaller platform like Itch.io first, and then come to Steam (with or without Early Access) once the game is strong enough.



One might expect this strategy doesn't work: by the time the game gets into Steam Early Access, it's been available elsewhere for a while so it's old news. However, I've heard from some devs that if a PC game is not on Steam, to a lot of people it doesn't exist. So even if it's been on another store for a while, the moment it gets to Steam is apparently considered the 'real' launch. (This logic of course doesn't apply to juggernauts like Fortnite, Minecraft and League of Legends.)

Regional soft launch

For new games and for updates

This is a common approach in the world of free to play mobile games: launch in a specific country, improve the game until it makes enough money per user, and only then launch worldwide. I haven't heard of any PC games using this approach, but undoubtedly it has been done. I expect the challenge here would be that hardcore gamers are much more informed and internationally connected than casual free-to-play mobile gamers. If your game is to sell through word-of-mouth then it's going to be weird if the word on Reddit and Discord ends at a single nation's border. Still, I imagine this approach might work in some cases, especially for single player games.

So, that's it! These are all the relevant ways I know of doing a beta in today's market. Did I miss any? Let me know below in the comments so that I may add them! Which approach do you prefer?

Sunday, 2 September 2018

The challenge of finding enough playtesters

I recently watched Jan Willem Nijman's great talk on The Art of Screenshake and noticed an interesting YouTube comment to that video by someone named SquidCaps. SquidCaps mentions that for an unknown developer, it's really hard to find playtesters who actually test regularly. I agree that this gets a lot easier the more well-known you are, but I think it might also help to try a different approach to playtesting. So I figured it would be interesting to discuss how we approach playtesting at Ronimo and how to find playtesters.

Let's start by having a look at SquidCaps' comment. It's quite long so I've selected only the parts that I'm going to reply to:


Heh, "let people play your game and iterate".. quite a bit easier to say when you have plenty of gametesters already in the office.. For solo, unknown developer, having just ONE game tester that suffers 10 minutes per week is too much.. What you need is full game, with menus, high scores save games etc, basically a beta before you get any testers and then you get plenty of them.. When it's still in alpha and the biggest, most revolutionary stuff is happening, then you are alone. Something that developers who are already successful almost always forget; you get plenty of free resources at one point. [...]

The truth is that your mates play for a while but without actual team behind you, you can't support them enough, interest dies down and soon, you see that no one downloaded latest build.. [...] I've been on half a dozen playtesting groups with hundreds of members and they all fail.. The sad truth is that no one cares until you are almost done. big teams have people who they employ that can do playtesting and can even afford to pay for playtesting.. [...]"

Playtesting versus QA

Before actually getting into SquidCaps' comment, I'd like to start with making a clear distinction between playtesting and QA (Quality Assurance, not to be confused with Q&A which means Questions & Answers). QA is all about finding bugs, while playtesting is all about making the game more fun, more understandable and finding the right difficulty curve and balance.

QA is usually an extremely boring and repetitive task. For example, in our game Awesomenauts we've had cases where we hired people to systematically test whether all upgrades in the game function correctly. Awesomenauts has around 1200 different upgrades, so you can imagine that systematically testing them all is not a fun job. Another example of something that's done in QA, is trying to walk into walls to see whether collision is correct everywhere. This usually involves running into all walls in a game and is an incredibly monotonous thing to do. Since QA is so little fun, it's usually not done by volunteers or friends, but instead by professional testers who get paid to do this. Often QA is outsourced to specialised companies in low-wages countries. An alternative is of course that the devs do such testing themselves.

The goal of playtesting on the other hand is to figure out whether the game is fun to play, whether players understand how to play the game and whether the difficulty and balance are right. This is a completely different way of approaching testing. Someone who systematically goes through all 1200 upgrades in Awesomenauts is never going to have fun, so QA's way of testing is not going to tell you anything about how fun individual upgrades are to play with. To find out whether something is fun, you need to let testers play in a more natural, unforced way. You can then just observe how they play and ask them afterwards which things they liked and which they didn't.



Now that we have a clear distinction between QA and playtesting, let's get back to SquidCaps' comment. SquidCaps talks about getting people to play new versions of the game every week. For most purposes of playtesting I think you shouldn't just let the same people play every week. Of course it's helpful, since you can ask them questions like "Is the game more fun now than it was last week?", but you won't get a wide diversity of opinions from a small group. Also, these playtesters play with a preconceived notion of the game based on previous playtests, so their opinions will not represent what fresh players will experience.

For these reasons for Swords & Soldiers 2, spread out during development we invited around 120 people into our office so that we could observe them playing. Every few weeks half a dozen people would come in on a Saturday, play for a few hours and share their opinions. We made sure these were different people every time, so that we would get fresh, unbiased opinions.

For figuring out whether the mechanics and tutorial are clear you also need fresh players every time. Anyone who has already played the game can't truly judge whether the tutorial explains it well, since they already know how it works beforehand. Same for difficulty: after practice you get better, so you can't judge anymore whether it's playable for someone who's new to the game. This is also why the devs themselves are horrible playtesters: they know exactly how the game is intended and are usually really good at it, so they can't judge difficulty and clarity.



When to playtest

This is where I totally agree with SquidCaps: playtesting should start early on: "When it's still in alpha and the biggest, most revolutionary stuff is happening." I regularly encounter developers who think you need complete and bug-free gameplay to do your first playtesting, but there's no reason to wait that long. As soon as you've got something playable, you can let others play it.

Sure, in an early stage you probably need to sit next to the tester to explain a lot of stuff, but you can still learn a ton about which parts of the game are fun and which are not. It might even happen that the playtesters don't enjoy the core but really enjoy some other aspects of the game and this might lead you to change the direction of the game altogether. Or maybe they share some great ideas that you didn't think of yet. If you're wondering "Should I wait for the game to be more complete before testing?", the answer is almost always "Nope, just test anyway and see what you can learn!"

Finding playtesters

This is SquidCaps' main point: it's really difficult to find playtesters if you're not successful. I'd like to add to this that it's even difficult to find playtesters when you are successful: at Ronimo we've sold millions of copies of our games, but finding playtesters is still sometimes hard and takes time.

I think the best playtesting happens in person. That way you can see what the other person is doing and ask questions or help out if needed, especially when testing an early version of a game that doesn't contain a tutorial yet and might be full of bugs. That's why SquidCaps' approach of sharing builds online and joining playtesting groups is not something we'd consider until very late in development. We usually only share builds online for players to test in case of betas, like we do with Awesomenauts updates.

Instead, we mostly search in our local communities. For example, I occasionally talk at game dev schools here in the Netherlands. Whenever I do that, I end my talk with asking people to sign up for playtesting. Usually around 20% of the students in the audience do so, so we now have a significant number of email addresses of potential playtesters we can invite. Unfortunately in practice few of them actually respond, but it's still enough to get dozens of playtesters for a game, usually spread out over a bunch of Saturday afternoons.

When our mailing list runs dry, we look further. We've asked around at local game development schools, where students are often eager to visit a real game dev company and play a secret, unannounced game. We've also asked on social media, hoping enough gamers from the Netherlands follow us to actually get some visitors.

In total these usually get us enough playtesters. However, you can go further than that when needed. Here are some examples:
  • I know a game company where the devs took an iPad version of their game, went to Gamescom and let people who were standing in line for some big triple-A game play their game. Queuing people don't have anything better to do so it's rather easy to get them to play for as much as fifteen minutes or more.
  • Similarly, you can go to a random public space with a laptop and just ask people to play. If you're a student this could be the hall of your own school, or it could be a station, or anywhere really. This requires getting over some awkwardness when approaching strangers, but really, this is very doable.
  • If you've got some budget you can also get a booth at a game event like Gamescom or PAX. Booths are not just nice for marketing but also for playtesting. Lots of different people will play your game and you'll get the most honest feedback in the world: the moment they don't like it anymore, they'll just walk away. Painful, but very useful information.
  • You can organise local playtesting events together with other devs. Here in the Netherlands Adriaan de Jongh organises PlayDev.Club, which is simply a gathering of local devs who play early versions of each other's games and share feedback. If there's nothing like that in your community, you can organise something like that yourself, as long as you're in an area that has enough game devs.



None of these approaches will easily get us people who play our game for a huge number of hours. However, if our game isn't that fun yet then maybe that shouldn't be our focus and the focus should be on making that first hour super awesome. Once the core game is a lot of fun, it becomes easier to find playtesters who are willing to come back and invest more hours to get to the advanced tactics and pro skills.

While all of these ideas are easier if you're a more well-known developer and are better connected, with enough effort an unknown dev can also find playtesters in such ways. Before we released our first game Swords & Soldiers we also managed to find playtesters, despite being an unknown studio at the time. Finding playtesters takes initiative, creativity and effort, but in most cases it's doable even for those of us who aren't well-known.

These were some of the things we do at Ronimo and that I've heard from other developers. How do you find playtesters? What are your challenges and have you got any clever tricks?

Saturday, 28 June 2014

Finding bugs through autotesting

Some bugs and issues can only be found by playing the game for ages in a single play session, or by triggering lots of random situations. As a small studio we don't have the resources to hire a ton of people to do such tests, but luckily there is a good alternative: a fun method is to hack automated controls into the game and let the game test itself. We have used this method in both Swords & Soldiers and Awesomenauts and found a bunch of issues this way.

Autotests are quite easy to build. The core idea is to let the game press random buttons automatically and leave it running for many hours. This is very simple to hack into the game. However, such a simplistic approach is also pretty ineffective: randomly pressing buttons might mean it takes ages to simply get from the menu to actual gameplay, let alone ever actually finishing any levels. It gets better if you make it a little bit smarter: increase the likelihood of pressing certain buttons, or even just automatically press the right button in certain menus to get through them quickly. With some simple modifications you can make sure the autotesting touches upon many different parts of the game.



This kind of autotesting has very specific limited purposes. There are a lot of issues you will never find this way, like if animations are not playing, texts are not displaying, or characters are glitching through walls. The autotester does not care and keeps pressing random buttons. Basically anything that needs to be seen and interpreted is difficult to find with autotesting, unless you already know what you are looking for and can add a breakpoint in the right position beforehand.

Nevertheless there are important categories of issues that can be found very well through autotesting: crashes, soft-crashes and memory leaks. A soft-crash is a situation where the game does not actually crash, but the user cannot make anything happen any more. This happens for example if the game is waiting for a certain event but the event is never actually triggered. Memory leaks are when the game forgets to clean up memory after usage, causing the amount of memory the game uses to keep rising until it crashes. Especially subtle memory leaks can take many hours before they crash and are thus often never found during normal development and playtesting.

Another category of issues that can be found very well through autotesting is networking bugs. This one is very important for Awesomenauts, which has a complex matchmaking system and features like host migration that are hard to thoroughly test. Our autotesting automatically quits and joins matches all the time, potentially triggering all kinds of timing issues in the networking. If you leave enough computers randomly joining and quitting for long enough, almost any combination of timings is likely to happen at some point.

Recently we needed this in Awesomenauts. After we launched patch 2.5 a couple of users had reported a rare crash. We couldn't reproduce the crash, but did hear that in at least one case the connection was very laggy. Patch 2.5 added Skree, a character that uses several new gameplay features (most notably chain lightning and spawnable collision blocks). This made it likely that the crash was somewhere in Skree's netcode.



We tried reproducing the crash by playing with Skree for hours and triggering all kinds of situations by hand. To experiment with different bad network situations we used the great little tool Clumsy. However, we couldn't reproduce the crash.

I really wanted to find this issue, so I reinvigorated Awesomenauts' autotesting system. We had not used that in a while, so it was not fully functional anymore and lacked some features. After some work it functioned again. I made the autotester enter and leave matches every couple of minutes. Since I didn't know whether Skree was really the issue I made the game choose him more often, but not always. I also made the autotester select a random loadout for every match and made it immediately cheat to buy all upgrades. The autotester is not likely to accidentally buy upgrades otherwise, so I needed this to have upgrades tested as well.

I ran this test on around ten computers during the soccer match Netherlands-Australia. While we beat the Australians our computers were beating this bug. Using Clumsy I gave some of those computers really high artificial packet loss, out-of-order and packet duplication.

Watching the computer press random buttons is surprisingly captivating, especially as it might leave the match at any moment. Simple things like a computer being stuck next to a low wall become exciting events: will it manage to press jump before quitting the match?

Here is a video showing a capture of four different computers running our autotest. The audio is from the bottom-left view. Note how the autotester sometimes randomly goes back to the menu, and can even randomly trigger a win (autotesters are not tactical enough to destroy the base otherwise):



And indeed, after a couple of hours already three computers had crashed! Since I had enabled full crash dumps in Windows, I could load up the debugger and see exactly what the code was doing when it was crashing.

The bug turned out to be quite nice: it required a very specific situation in combination with network packets going out of order in a specific way. When Skree dies just after he has started a chain lightning attack, the game first sends a chain lighting packet and then a character destroy packet. If these go out of order because of a really bad internet connection, the character destroy packet can arrive first. In this case the Skree has already been destroyed when his lightning packet is received. Chain lightning always happens between two characters, so the game needs both Skree and his target to create a chain lightning.

Of course we know that this kind of thing can happen when sending messages over the internet, so our code actually did check whether Skree and his target still existed. However, due to a typo it also created the chain lightning if only one of the two characters existed, instead of if they both existed. This caused the crash. Crashes are often just little typos, and in this case accidentally typing || ("OR") instead of && ("AND") caused this crash.

Once we knew where the bug was, it was really easy to fix it (the fix went live in hotfix 2.5.2). Thus the trick was not in fixing the code, but in reproducing the issue. This is a common situation in game programming and autotesting is a great tool to help reproduce and find certain types of issues.

Friday, 1 February 2013

The craziest bugs, part 2

Last week I started the top 7 of my favourite bugs with the numbers 7 to 4. The top 3 is where it gets really crazy. Sit back and enjoy to see how amazingly stupid game development can sometimes be!

Click here for numbers 7 to 4

3. Hidden functionality to turn off bugs

On one of the platforms Awesomenauts launched on, we had a lot of trouble getting the internet connection between players to remain stable. After a while the ping would always start slowly increasing, until in the end it got too high and the game disconnected. Sometimes this took a couple of minutes to start, sometimes half an hour, but in the end this always happened.

We contacted the support team for the platform-specific networking library that we were using, and their answer was that we used too much bandwidth and sent too many packages. So we spent a lot of time optimising, and we managed to half the number of packages and half the bandwidth. However, the problem remained, and they again told us we used too much bandwidth. Again we halved the bandwidth and the number of packages, but the problem remained. At this point we were well below what they said was the ideal bandwidth usage and we couldn't optimise much more, so we were getting pretty desperate and contacted them again.

And then it happened...

Their answer was that their bandwidth throttling code was quite buggy, so there was a hidden enum that we could use to turn that code off. We used that and... the problem was instantly fixed! So they knew they had a bug, they even had an option to turn that bug off, but they didn't tell us for two months! I spent all that time doing extra bandwidth optimisations and it wasn't even necessary! Blargh!

Of course, using less bandwidth is always an important improvement for a multiplayer game, but we were already enormously behind on schedule at that point and this took a lot of time for a small indie studio...



2. Lying videocards

This has been a personal gripe of mine for years. Some videocards simply lie about their specs. Your game asks the videocard what it can do, and it will proudly brag about features it doesn't actually have!

I have not yet used any of the newer videocard features like geometry shaders, hull shaders and compute shaders, so I have not encountered any lying videocards recently, but I would be surprised if this doesn't still happen when you try to use state-of-the-art videocard features. It sure did a lot around the appearance of shader models 2 and 3. (Note that I haven't used the new shader types because I think they are uninteresting, but that's a long story that I will not go into today. Instead, I will just leave it at the short and controversial statement that they are irrelevant.)

I had to work around such lying videocards in both Proun and De Blob. What happens is that I make special versions of my shaders for different shader models, so that older videocards can still run the game, but with less special effects. The Ogre-engine has a very elegant system to handle this, and thus Proun features proper materials for shader models 1, 2.0, 2.x and 3.0.

The core of this solution, however, is that you ask the videocard which shader models it supports, and then pick the highest allowed for the best quality. This works very well, unless the videocard lies. It might claim to support 3.0, but doesn't really work with it. In the worst case, the videocard doesn't even give a compile error when being fed a 3.0 shader and simply outputs black pixels!

Several older Ati videocards turned out to do this in a horrible way. The solution ended up to hardcode the names of such videocards and feed them different shaders based on their name. If you look in the Proun folder structure, you can see folders with the same materials, but for different videocards, with beautiful names like "NotX8orX9". That last one contains materials that cannot be used on Ati X8** and X9** cards. I even have another set of materials for Ati X1*** and X2*** cards, because they lie in a different way...



1. "If I buy the game, it isn't the demo any more"

Yes, you read that title correctly. This is by far the most hilarious bug report I have ever seen, and easily claims the number 1 spot in this list, despite not even being a real bug. It was reported to us through the bug database though, so it qualifies for this list!

A professional QA testing company that was testing one of our games for us, at some point reported to us that if they bought the full game from the demo, then when they came back to the game, it wasn't the demo any more. It was instead... the full game!

Oh really?

That happens to be the point of buying the game, now isn't it?

When I replied in the bug database that either I didn't understand what they meant, or this bug report was a slight mistake from the tester, a producer quickly removed the bug from the database, so I never received an actual answer to that.

I think the reason they reported this, is that that particular shop (outside the game, not made by us) concluded the buying process with a question like "Do you want to go back to the game?" Whether you chose "Okay" or "Cancel", the shop always brought you back to the game, and apparently the tester concluded from the fact that there were two options that one of them ought to bring you back to the demo, even though the game had just been bought. This is some pretty broken reasoning, but I can imagine where it came from.

(As crazy as this bug report may be, though, I would like to emphasize that this is the only silly report this testing company wrote to us. The rest of the reports made perfect sense, so this one mistake really shouldn't be held against them! That doesn't make it any less hilarious, though...)

That's it, folks! The 7 weirdest bugs I have encountered! Come back next week when I will discussion the Awesomenauts animation pipeline or visual effects performance (I haven't really made up my mind yet which it is going to be...)