Let's start by having a look at SquidCaps' comment. It's quite long so I've selected only the parts that I'm going to reply to:
| Heh, "let people play your game and iterate".. quite a bit easier to say when you have plenty of gametesters already in the office.. For solo, unknown developer, having just ONE game tester that suffers 10 minutes per week is too much.. What you need is full game, with menus, high scores save games etc, basically a beta before you get any testers and then you get plenty of them.. When it's still in alpha and the biggest, most revolutionary stuff is happening, then you are alone. Something that developers who are already successful almost always forget; you get plenty of free resources at one point. [...] The truth is that your mates play for a while but without actual team behind you, you can't support them enough, interest dies down and soon, you see that no one downloaded latest build.. [...] I've been on half a dozen playtesting groups with hundreds of members and they all fail.. The sad truth is that no one cares until you are almost done. big teams have people who they employ that can do playtesting and can even afford to pay for playtesting.. [...]" |
Playtesting versus QA
Before actually getting into SquidCaps' comment, I'd like to start with making a clear distinction between playtesting and QA (Quality Assurance, not to be confused with Q&A which means Questions & Answers). QA is all about finding bugs, while playtesting is all about making the game more fun, more understandable and finding the right difficulty curve and balance.
QA is usually an extremely boring and repetitive task. For example, in our game Awesomenauts we've had cases where we hired people to systematically test whether all upgrades in the game function correctly. Awesomenauts has around 1200 different upgrades, so you can imagine that systematically testing them all is not a fun job. Another example of something that's done in QA, is trying to walk into walls to see whether collision is correct everywhere. This usually involves running into all walls in a game and is an incredibly monotonous thing to do. Since QA is so little fun, it's usually not done by volunteers or friends, but instead by professional testers who get paid to do this. Often QA is outsourced to specialised companies in low-wages countries. An alternative is of course that the devs do such testing themselves.
The goal of playtesting on the other hand is to figure out whether the game is fun to play, whether players understand how to play the game and whether the difficulty and balance are right. This is a completely different way of approaching testing. Someone who systematically goes through all 1200 upgrades in Awesomenauts is never going to have fun, so QA's way of testing is not going to tell you anything about how fun individual upgrades are to play with. To find out whether something is fun, you need to let testers play in a more natural, unforced way. You can then just observe how they play and ask them afterwards which things they liked and which they didn't.
Now that we have a clear distinction between QA and playtesting, let's get back to SquidCaps' comment. SquidCaps talks about getting people to play new versions of the game every week. For most purposes of playtesting I think you shouldn't just let the same people play every week. Of course it's helpful, since you can ask them questions like "Is the game more fun now than it was last week?", but you won't get a wide diversity of opinions from a small group. Also, these playtesters play with a preconceived notion of the game based on previous playtests, so their opinions will not represent what fresh players will experience.
For these reasons for Swords & Soldiers 2, spread out during development we invited around 120 people into our office so that we could observe them playing. Every few weeks half a dozen people would come in on a Saturday, play for a few hours and share their opinions. We made sure these were different people every time, so that we would get fresh, unbiased opinions.
For figuring out whether the mechanics and tutorial are clear you also need fresh players every time. Anyone who has already played the game can't truly judge whether the tutorial explains it well, since they already know how it works beforehand. Same for difficulty: after practice you get better, so you can't judge anymore whether it's playable for someone who's new to the game. This is also why the devs themselves are horrible playtesters: they know exactly how the game is intended and are usually really good at it, so they can't judge difficulty and clarity.
When to playtest
This is where I totally agree with SquidCaps: playtesting should start early on: "When it's still in alpha and the biggest, most revolutionary stuff is happening." I regularly encounter developers who think you need complete and bug-free gameplay to do your first playtesting, but there's no reason to wait that long. As soon as you've got something playable, you can let others play it.
Sure, in an early stage you probably need to sit next to the tester to explain a lot of stuff, but you can still learn a ton about which parts of the game are fun and which are not. It might even happen that the playtesters don't enjoy the core but really enjoy some other aspects of the game and this might lead you to change the direction of the game altogether. Or maybe they share some great ideas that you didn't think of yet. If you're wondering "Should I wait for the game to be more complete before testing?", the answer is almost always "Nope, just test anyway and see what you can learn!"
Finding playtesters
This is SquidCaps' main point: it's really difficult to find playtesters if you're not successful. I'd like to add to this that it's even difficult to find playtesters when you are successful: at Ronimo we've sold millions of copies of our games, but finding playtesters is still sometimes hard and takes time.
I think the best playtesting happens in person. That way you can see what the other person is doing and ask questions or help out if needed, especially when testing an early version of a game that doesn't contain a tutorial yet and might be full of bugs. That's why SquidCaps' approach of sharing builds online and joining playtesting groups is not something we'd consider until very late in development. We usually only share builds online for players to test in case of betas, like we do with Awesomenauts updates.
Instead, we mostly search in our local communities. For example, I occasionally talk at game dev schools here in the Netherlands. Whenever I do that, I end my talk with asking people to sign up for playtesting. Usually around 20% of the students in the audience do so, so we now have a significant number of email addresses of potential playtesters we can invite. Unfortunately in practice few of them actually respond, but it's still enough to get dozens of playtesters for a game, usually spread out over a bunch of Saturday afternoons.
When our mailing list runs dry, we look further. We've asked around at local game development schools, where students are often eager to visit a real game dev company and play a secret, unannounced game. We've also asked on social media, hoping enough gamers from the Netherlands follow us to actually get some visitors.
In total these usually get us enough playtesters. However, you can go further than that when needed. Here are some examples:
- I know a game company where the devs took an iPad version of their game, went to Gamescom and let people who were standing in line for some big triple-A game play their game. Queuing people don't have anything better to do so it's rather easy to get them to play for as much as fifteen minutes or more.
- Similarly, you can go to a random public space with a laptop and just ask people to play. If you're a student this could be the hall of your own school, or it could be a station, or anywhere really. This requires getting over some awkwardness when approaching strangers, but really, this is very doable.
- If you've got some budget you can also get a booth at a game event like Gamescom or PAX. Booths are not just nice for marketing but also for playtesting. Lots of different people will play your game and you'll get the most honest feedback in the world: the moment they don't like it anymore, they'll just walk away. Painful, but very useful information.
- You can organise local playtesting events together with other devs. Here in the Netherlands Adriaan de Jongh organises PlayDev.Club, which is simply a gathering of local devs who play early versions of each other's games and share feedback. If there's nothing like that in your community, you can organise something like that yourself, as long as you're in an area that has enough game devs.
None of these approaches will easily get us people who play our game for a huge number of hours. However, if our game isn't that fun yet then maybe that shouldn't be our focus and the focus should be on making that first hour super awesome. Once the core game is a lot of fun, it becomes easier to find playtesters who are willing to come back and invest more hours to get to the advanced tactics and pro skills.
While all of these ideas are easier if you're a more well-known developer and are better connected, with enough effort an unknown dev can also find playtesters in such ways. Before we released our first game Swords & Soldiers we also managed to find playtesters, despite being an unknown studio at the time. Finding playtesters takes initiative, creativity and effort, but in most cases it's doable even for those of us who aren't well-known.
These were some of the things we do at Ronimo and that I've heard from other developers. How do you find playtesters? What are your challenges and have you got any clever tricks?
No comments:
Post a Comment