The psychology of matchmaking

Matchmaking is a touchy subject and this has previously made me somewhat hesitant to write about it in an open and frank manner. Today's topic especially so, since some players might interpret this post as one big excuse for any faults in the Awesomenauts matchmaking. However, the psychology of matchmaking is a very important topic when designing a matchmaking system, so today I'm going to discuss it anyway. For science! :)

While we were designing the Galactron matchmaking systems I did quite a lot of research into how the biggest multiplayer games approach their matchmaking. The devs themselves often don't say all that much about it, but there's plenty of comments and analysis by the players of those games. The one thing they all have in common, is that whatever game you look for, you'll always find tons of complaints from users claiming the matchmaking for that particular game sucks.



My impression is that no matter how big the budget and how clever the programmers, a significant part of the community will always think they did a bad job regarding matchmaking. Partially this might be because many games indeed have bad or mediocre matchmaking, but there's also a psychological factor: I think even a theoretical 'perfect' implementation will meet a lot of negativity from the community. Today I'd like to explore some of the causes for that.

The first and most obvious reason is that matchmaking is often a scapegoat. Lost of match? Must be because of the crappy matchmaking. My teammates suck? Must be the crappy matchmaking. Got disconnected? Definitely not a problem in my own internet connection, must be the crappy matchmaking. Undoubtedly in many cases the matchmaking is indeed part of the problem, but these issues will exist even with 'perfect' matchmaking. Sometimes you're just not playing well. Sometimes a teammate has a bad day and plays much worse than they normally do. Sometimes your own internet connection dropped out. No matchmaker can solve these issues.

There's a strong psychological factor at play here: for many people their human nature is to look for causes outside themselves. I think this is a coping mechanism: if you're not to blame, then you don't have to feel bad about yourself either.

Of course there is such a thing as better or worse matchmaking. That players use matchmaking as a scapegoat shouldn't be used as an excuse to not try to make better matchmaking. But it sure makes it difficult to asses the quality of your matchmaking systems. Whether your matchmaker is doing well or not, there will practically always be a lot of complaints. The more players you have, the more complaints.

For this reason it's critical to collect a lot of metrics about how your matchmaking is objectively doing. We gather overall metrics, like average ping and match duration and such, but we also store information about individual matches. This way when a player complains we can look up their match and check what happened exactly. This allows us to analyse whether specific complaints are caused by problems in the matchmaker, are something that the matchmaker can't fix (like a beginner and a pro being in a premade together) or whether the user is using matchmaking as a scapegoat for something else.



Another problem for matchmaking is that player's don't have a single, predictable skill level. The matchmaker matches a player based on their average skill, but how well they actually play varies from match to match. One match they might do really well, and then the next they might do badly. For example, maybe the player gets overconfident and makes bad decisions in the next match because of that. Or maybe the player is just out of luck and misses a couple of shots by a hair that they would normally hit. Or maybe the player got home drunk from a party and decided to play a match in the middle of the night, playing far below their normal skill level. These are things that a matchmaker can't predict. This will often make it seem like the matchmaker didn't match players of similar skill. Sometimes this might result in a teammate who would normally be as good as you but happens to play like a bag of potatoes during this one match in which they're in your team.

While this problem is not truly solvable by matchmaking, there are some things developers can do to improve on it. For example, in Heroes Of The Storm you select your hero before you go into matchmaking. This allows the matchmaker to take into account that you might be better at some heroes than at others. If it detects that you're playing a hero that you haven't played in a long while, then maybe it should matchmake you below your skill level. I have no idea whether Heroes Of The Storm actually does this, but it's certainly a possibility. This would allow detecting some of the cases in which a player is normally really good, but is currently trying something new that they haven't practised yet.



However, this particular trick comes at a heavy cost, which is why we decided not to put it into Awesomenauts: if players select their hero before matchmaking happens then matchmaking is severely limited in who can play with whom, which damages other matchmaking criteria. (I've previously discussed this in my blogpost about why you need huge player numbers for good matchmaking.)

A very different kind of psychological aspect is that players are often bad at estimating their own skill level. The following example of this is something that I have no doubt will be recognised by many people who play multiplayer games. A while ago I played a match in which a teammate was constantly complaining about how badly I was playing and how I was causing us to lose the match. However, looking at the scoreboard I could see that he was constantly dying and was by far doing worst of all players in the match. Apparently that player didn't realise that he was much less good at the game than he thought he was.

There's more to this than anecdotal evidence however. The developers of League of Legends have described that on average, players rate their own skill 150 points higher than their real MatchMaking Rating. (That part of the post has been removed in the meanwhile, but you can still find it here on Wayback Machine.) As they also mention there, psychology actually has a term for this: it's called the Dunning-Kruger effect. A League of Legends player analysed a poll about this here and gives an excellent explanation of how it works:

"According to the Dunning-Kruger-Effect people overestimate themselves more the more unskilled they are. This isn’t caused by arrogance or stupidity, but by the fact that the ability to judge a certain skill and actually being good at that skill require the same knowledge. For example if I have never heard of wave management in LoL I am unable to notice that I lack this skill, because how would I notice something if I don’t even know it exists? I would also not notice this skill in other people, which is why I would overestimate my own skill if I compared myself to others. This it what causes the Dunning-Kruger-Effect." - Humpelstilzche

As some readers suggested, all of these psychological factors invite an interesting thought: would it help to give players more control? After all, if you have no control you blame the system, while if you do have more choice, you also have yourself to blame and maybe accept the results more. Giving players choice over matchmaking is in many ways old-fashioned and reminds me of the early days of online multiplayer, where you selected your match yourself from a lobby browser. The common view these days seems to be that players expect a smooth and automated experience. They don't want to be bothered and just want to click PLAY and get a fun match. Or at least, most devs seem to assume that that's what players want.

For Awesomenauts I've actually been curious for a while what would happen if we removed the automated matchmaking and instead relied entirely on opening and selecting lobbies. The modding scene would surely thrive a lot more that way, but how would it affect player experience and player counts? It would be a risky change and also too big a change to just try though so I doubt we'll ever get to know the answer to that question. Also, I'm not sure whether our current lobby browser provides a smooth enough experience for making it that important.

I do think it's interesting to explore this further though. I wonder what would be the result if matchmaking were from the beginning designed around being a combination of automation, communication and player control.

Before ending this blogpost I should mention one more psychological aspect of matchmaking: the developer's side. As a developer it can be really frustrating to get negative comments on something you've spent a lot of time on. Understanding why can help in coping with this frustration. Just like matchmaking can be a scapegoat for players after losing a match, the psychology of matchmaking can be a scapegoat for developers after getting negative feedback from players.

None of the psychological factors discussed in this post are an excuse to just claim that the matchmaking in a game is good despite players complaining. However, for a developer it's really important to realise that these factors do exist. Understanding the psychology of matchmaking allows you to build better matchmaking and helps to interpret player comments. I have no doubt that I've only scratched the surface of this topic, so I'm quite curious: what other psychological aspects influence how matchmaking is experienced by players?

The Awesomenauts matchmaking algorithm

Matchmaking is a big topic with lots of challenges, but at its core is a very simple question: who should play with whom? A couple of years after releasing Awesomenauts we rebuilt our entire matchmaking systems, releasing the new systems in the Galactron update. Today I'd like to discuss how Galactron chooses who you get to play with. While the question is simple enough, the answer turns out to be pretty complex.



Many different factors influence what's a good match. Should players of similar skill play together? Should players from the same region or language player together? Should we take premades into account? Ping? Should we avoid matching players who just played together already? Should we use matchmaking to let griefers play against each other and keep them away from normal folks?

If the answer to all of these questions is 'Yes, let's take that into account!', then you'd better have a lot of players! I've previously written about how many players you need for that and the short answer is: tens of thousands of simultaneous players, all the time. That's not realistic except for the very biggest hits, so you'll usually need to do with fewer players and make the best out of your matchmaking.

For Awesomenauts we chose to let the matchmaker gather a lot of players and then once every couple of minutes match them all at once. This way the matchmaker has as many players as possible to look for the best possible combinations.



We'd like to match more than 100 people at the same time so that we have plenty of choice. More than that would be even better: our research shows that our algorithm keeps getting better up to as many as 300 people per round, because smaller distant regions (like Australia) don't have enough players for proper matchmaking until that many. However, the more players we wait for, the worse the player experience gets because the waiting times become too long.

Scoring the quality of a match

Okay, so we have 100+ players that all need to be put in matches, how do we decide what's the best match-up? For example, if we can choose between making a match where the players have equal skill, or one where the ping is low, which do we prefer? And which do we prefer if the difference is more subtle, like we can get slightly more equal skill or slightly better ping? Where do we draw the line? And how do we balance this against wanting to let premades play against each other? Etc. etc.

What we need here is some kind of metric so that we can compare match-ups. Somehow all of those matching criteria need to culminate in one score for the entire group. Using that, the computer can look for the best match-up: the one that gives us the highest score.

To calculate a total score we start by calculating the matchmaking score for each match. We do this be taking a weighted average of a bunch of factors:

• Equally skilled teams. This is the most obvious factor in matchmaking: both teams should be equally good. Turning this requirement into a number is simple: we take the average skill per team and look at the difference between those. The bigger the difference, the lower our score. Note that this does require some kind of skill system in your game, like ELO or Trueskill. This also means that beginning players are difficult to match, because you don't know their actual skill until they've played a bit.
• Equally skilled players. Even if the two teams are perfectly balanced, the match might not be. For example, if both teams contain two pros and one beginner, then the average skills of the teams are the same, but the match won't be fun. So we use a separate score that ignores teams and simply looks at the skill differences between all players. In a 6 player match there are 15 combinations of players and we simply average all of those. The bigger the difference, the lower the score.
• Premades. Ideally a group of 3 friends who coordinate together should play against a similar group and not against three individual players who don't know each other. Assigning a score to this is simple: if both teams have the same situation, then we have a 100% score. If there's a small difference (like for example a 3 player premade versus a 2 player premade + a solo player) then we get a 60% score. For a big difference (a 3 player premade versus 3 solo players) the score is 0%.
• Ping with enemies. For each player we check the ping with all 3 players in the enemy team. The higher the ping, the lower the score. There are 9 combinations of players this way and we simply average those 9 scores to get the total ping score for this match.
• Opponent variation. This is a subtle one that we added a while after launching Galactron. Since our matchmaker basically looks for players of similar skill with a good connection to each other, it tends to repeatedly put the same players against each other. We expected this to be rare enough that it would be fun when it would happen, but in practice our players encountered the same people too often. To counter this we give a match a lower score if players encountered each other in the previous match as well. If they encountered each other as opponents but are now teammates (or vice-versa) we give that a slightly better score than if they meet each other in the same situation (opponents before, opponents now; or teammates before, teammates now). This gives the matchmaker a slight tendency to swap teams around if despite this rule it ends up making another match with the same players. This rule has a very low weight since we value the other rules more, but still this rule improved the situation significantly and we got much fewer complaints from players about this.
• Two possible servers. Since Awesomenauts is peer-to-peer, each match should have a player who can connect with everyone in the match, so that this player can be the server. Ideally there is also a second player in the match who can connect with everyone. This way if the first server-player drops out of the match, host migration can happen and the match can continue for the remaining players. This rule is only needed for peer-to-peer games: in a game with dedicated servers or relay servers this rule is irrelevant.

There's one important factor missing here: ping with teammates. Why is this not taken into account? The reason for this is that for every factor we add, the others become a little bit less important. In Awesomenauts a bad connection with your teammates is usually not a big problem because you never need to dodge their bullets. A bad connection with an opponent is much worse, because dodging becomes really difficult if the ping is too high. By ignoring the ping with teammates, we make the ping with opponents a lot more important.

Something to think about when calculating scores is whether we want them to be linear. For example, is a ping improvement from 210ms to 200ms as worthwhile as one from 110ms to 100ms? Both are 10ms improvements, but the latter is relatively twice as big. In Awesomenauts we've indeed tweaked our scoring formulas to better match the perceived difference instead of the absolute difference.

Another consideration is limiting the range of the scores. For example, in Awesomenauts the highest ranked players have a skill of over 20,000, but only 0.3% of all players are above 18,000. In other words: there's a huge difference in skill score in the very top, but this is quite useless to the matchmaker since there are too few players with such a high score to actually take this into account. So before calculating the skill scores we cap them to workable ranges. This way the matchmaker will consider a match-up between two players with skill 18,000 to be as good as one between an 18,000 player and a 20,000 player. Ideally you don't cap anything, but since we have to balance between all the different matchmaking goals capping at some points will make other ranges and aspects more important.

Combining scores

For each of the above 6 rules we now have a score from 0% (really bad match-up) to 100% (really good), but we need 1 score, not 6. To get this 1 score we take the average of all the scores. We use a weighted average for this so that we can make certain things more important than other things. For example, in a fast-paced game like Awesomenauts, ping should be pretty important. Opponent variation on the other hand is a detail that we don't want to stand in the way of good ping, so that one gets a pretty low weight.

An interesting thing that happens here is that some of these scores react much more extremely than others. Getting a 0% skill score requires having 3 pro players and 3 beginners in a match, which is extremely rare. Getting a 0% premade score however happens much more easily, since all that's required for that is having a three player premade play against three solo players. Some scores reacting more subtly than others is something we need to take into account when choosing the weights: since premades react so strongly, we need to give them a lower weight to keep them from overshadowing scores that respond more smoothly, like skill and ping.

Now that we have a single score for each match, we can calculate the totale score for the 100+ players we're matching. We do so by simply averaging the scores of all the matches. The result is one big megascore.



Since we look at the total score, the algorithm gets certain preferences. If swapping two players increases the score of one match by 5% but decreases the score of the other match by 10%, then it won't do that since in total that makes things worse. This sounds obvious, but in some cases one might want to deviate from this. For example, maybe you might prefer improving a match with a 50% score (which is really bad) to 55% at the cost of decreasing a 90% match to 80% (which is still pretty good). A way to achieve this might be to take the square root of all the match scores before averaging them. That way improving bad matches becomes relatively more important. For Awesomenauts we decided against this, because we think the individual scores already represent well enough how big a match quality improvement really is.

Finding the match-up with the best score

Now that we have a scoring system that defines what the best match-up is (the one with the highest score) we get to the algorithmic part of the problem: how to actually find that best match-up. With 100+ players the number of combinations we can make is insane so we can't brute-force our way out of this by simply trying all combinations.

I figured this might actually be a graph theory problem so I tried looking for node graph algorithms that could help me, but the problem turned out to be so specific that I didn't find any. Even after consulting a hardcore academic algorithmic expert nothing turned up, so I decided to look for some good guesstimate and just accept that the actual best match-up is probably not findable. The problem might even be NP-complete, but I didn't actually try to prove that. Finding a better algorithm might be a fun thesis topic for a computer science student somewhere.

The approach I ended up at is to first create a somewhat sensible match-up using a simple rule. I tried a couple of rules for this, like for example simply sorting all players by skill and then putting players in matches based on that sorting. So the top six players together get into one match, then the next six, etc. This is easy enough to build and produces the best possible skill scores. Since it ignores all the other scores entirely it performs really badly for ping and premades.

We now have 6 players per match, but we haven't decided who's in the red team and who's in the blue team yet. So the next step is how the six players in each match are divided over the two teams. Here we can brute-force the problem: we simply try every possible combination and select the best one. Here we take into account all 6 scores, so not just skill. The number of combinations is pretty low, especially since the order in which players are in each team doesn't matter. Since we try every combination we know for sure that the players will be split over the teams in the best possible way.

We now have a good starting point. The next step is to look for improvements: we're going to look for swaps of players that will improve the total score. We do this one swap at a time. Here we can again brute-force our way out of this problem: we simply try all swaps and then perform the best one. There are lots of different possible swaps and we also need to recalculate the split over the two teams for each swap, so this uses a lot of processing power. However, modern computers are super fast so with some optimisations we can do a few hundred swaps per second this way.

Especially the first swaps will improve the matches drastically, but we start seeing diminishing returns up to the point where no swaps can be found that are actually an improvement. There might still be some bad matches there, but no single swap will be an improvement overall. In other words: we've reached a local optimum. We almost certainly didn't find the best match-up possible for those 100+ players, but we can't improve this one any further with our current swapping algorithm.



To get out of that local optimum I tried a few things. The first thing I tried is to do a number of forced swaps where the players in the very worst positions are forced into other matches, despite the overall result becoming worse. After a dozen or so forced swaps we start doing normal swaps again. This indeed resulted in a slightly better score, but it did make the algorithm take much longer. Since processing is already taking seconds at this point, performance is very relevant. We don't want players to wait half a minute extra just for this algorithm to finish.

I then tried a different approach: I just leave the first result as it is, and start over again but from a different starting point. I semi-randomly generate a completely new match-up and start doing swaps on that. This will also bring us to a local optimum, but it will be a different one that might actually be better than the previous local optimum.

In practice within 5 seconds we can usually do a couple of retries this way (less if there are more players) and then we simply pick the best one we found. This turns out to produce much better results than trying to force ourselves out of a local optimum, so I threw away that approach and instead we just try a bunch of times. To limit waiting times we simply check how long we've been going so far and do another retry only if we haven't spent too much time yet.

To see how good this could get I tried letting my computer run for a whole night, constantly retrying from different starting points. This produced tens of thousands of different match-ups, all at different local optimums. It turns out that the difference between trying a couple of times and trying tens of thousands of times is actually surprisingly small. This showed us that doing just a couple of retries is already good enough.

Downsides

So far I've explained how we do matchmaking and why. Now that Galactron has been running for almost two years in this way it's also a good moment to look back: did our approach work out as nicely as hoped? Overall I'm pretty happy with the result, but there is one big choice in here for which I'm not sure whether it's actually the best one: matching everyone at the same time. Doing this gives our matchmaking algorithm the most flexibility to find the best match-ups, but it turns out that players are spread out over the world even more unevenly than I had estimated beforehand, causing problems here.

For example, if we look at a matchmaking round at 20:00 western European time, then the vast majority of those players will be from Europe and the rest of the players will be spread out over the rest of the world. That means that in a matchmaking round of 100 players, there are only a few Australians or South Americans. Those players won't get a good match in terms of ping because there simply aren't enough players in their region in that round.

Solving this requires having even larger numbers of players per round: I think around 300 would be ideal. However, this means either having extremely long waiting times, or having a gigantic playerbase. We'd love to have a playerbase as large as League of Legends, but unfortunately that's not realistic for all but the biggest hit games. The result is that we chose a middle ground: we wait for those 100 players, which can take 5 minutes or even more, and then we just perform matchmaking and accept that the result won't be as good as we'd want for some people. If the number of players is too low then at some point we just perform matchmaking anyway so that waiting times never go beyond a maximum.

While building Galactron we've found very little information on how the big multiplayer games do their matchmaking exactly, but a talk by a designer of Heroes of the Storm (which I unfortunately can't find anymore on YouTube) suggested that at some point in that game, the system was that a match would be started as soon as it could be created with a good enough match quality. In our case this would mean that if there are 6 players who are similar enough in skill and have a good ping with each other, then the match is started immediately. If it takes too long to fill a match, then the requirements are gradually decreased. At some point a player who has waited too long becomes top priority and just gets the 5 most fitting players that the matchmaker can find, even if match quality is lower than desired.



The benefit of such an approach is flexibility: in areas with lots of players one would get matches much more quickly, while areas with fewer players would get longer waiting times. This kind of flexibility is also nice for pro players: one can make them wait longer so that they only play against others of almost exactly the same skill. Our own algorithm can't diversify like that since players from the whole world are matchmade simultaneously at fixed moments.

In comparison I expect our own method produces slightly better results, because it can juggle with a lot more players at once to find the best match-ups. However, this benefit might not be big enough to weigh up against the benefit of having lower waiting times.

There is one really nice benefit that we get from our fixed matchmaking moments: we can tell the player beforehand exactly how long they'll have to wait for matchmaking to happen. I think waiting is more endurable if you know how much longer you have to wait. Showing how much longer you need to wait is a really nice touch that few other games have.

A direct comparison between the live matchmaking algorithms of Awesomenauts and Heroes of the Storm is unfortunately not possible because Heroes of the Storm has so many more players. Any matchmaking algorithm will do better with more players, so it is to be suspected that regardless of the algorithm, the matchmaking quality in a big game like Heroes of the Storm will be much higher than in Awesomenauts anyway.

Conclusion

Building a matchmaking system starts with defining what good matchmaking is. No algorithm can produce good match-ups if you don't give the computer rules for what that actually means. In this post I've discussed the rules we use, which are mostly based around skill, ping and premades. Depending on what's important for your game you can add more rules or define them differently.

Just keep in mind that the more rules you have, the less important the other ones become, and every rule you weigh heavier makes the others less important. Restraint and balancing are key here. Also, you probably need a while to finetune the rules further once your matchmaking is live, like we've done with the Galactron matchmaker in Awesomenauts by adding the 'opponent variation' rule later on.

Entertaining players while waiting for matchmaking

In a game where high quality matchmaking is important (like competitive team-based games), waiting times to find suitable opponents are often unavoidable. Today I'd like to discuss the problems we had in Awesomenauts with our initial approach to this and how we made waiting a bit less boring later on.

Until the launch of the Galactron update in 2016, this is how it worked in Awesomenauts: when you started searching you got into a matchroom right away. You selected your character, and then you had to wait in the loading screen until the match was full with 6 players. This could take several minutes and there was no interaction possible whatsoever: just a loading screen that says the match isn't full yet. To show progress we did have six tickboxes that showed how many players had joined and whether they had already selected their characters, but that was it.



A static screen with no interaction possible is probably the most boring type of waiting imaginable. Compare this to current Awesomenauts, where you wait in the menus instead of in a loading screen. Still not ideal, but this means that while waiting you can chat with other players, check out the skills and items of all characters and watch replays and live matches. Awesomenauts has a system where it lists the ten most exciting matches happening right now and also automatically selects a match of the day that you can watch. We hardly advertise watching replays while waiting though, so I think many players don't realise that you can watch a live match or replay while waiting for matchmaking.

An important thing to realise when thinking about waiting times is that on PC, players can simply alt+tab and browse the internet while waiting. This isn't possible on console, but there's always that other option: mobile phones. I don't have any statistics on this, but I wouldn't be surprised if a lot of players ignore menus, replays and chat during waiting times and just randomly browse the internet instead.

While I think current Awesomenauts is a huge improvement in terms of diversions while waiting for matchmaking, I don't think it's ideal. Waiting times for a proper match can be rather long (which is unavoidable if you don't have a giant playerbase and prefer better match-ups over quicker matchmaking) and clicking through menus isn't the best entertainment in the world. I like the solution that PlayerUnknown's Battlegrounds has: players get thrown into a little area where they can walk and jump around, punch each other, voice chat and generally goof around a bit. It's still clearly a waiting area, but it's so much more fun and interactive than a menu or a replay.



Even better is to have some kind of mini-game while waiting. During early development of Awesomenauts we experimented with having a little death match arena where you played until the real match started. It's similar to what PlayerUnknown's Battlegrounds does, but even more game-like since it's real deathmatch. I can't quite remember why we decided not to put this into the game, but I imagine the mean reason was probably a lack of time: we were making a really big game with a small team and needed to cut features in order to finish it before funds ran out.

There might also have been concerns internally that it might be frustrating to be in a deathmatch game and then we thrown out into the main game all of a sudden. Knowing that it can end any moment might potentially make that deathmatch more frustrating than fun. In Swords & Soldiers we solved this frustration by letting the player do single player while waiting and storing the state of that single player as soon as an online match started. This way the player could continue the single player session after having finished the online match. However, this approach doesn't work with an online deathmatch. Still, I think having gameplay instead of menus is probably preferable anyway.



Despite all of this I do think we had some nice touches in the original waiting screen in Awesomenauts. One is that we showed the maximum waiting time left. Waiting for a few minutes is a lot more acceptable if you actually know how long you're going to be waiting than if you don't know whether you need to wait one more minute or ten. If after a few minutes the match was still not full, it would just start with fewer players and add bots to complete the teams. This way the player had a guarantee that when the timer ran out, gameplay would definitely start.

In the new matchmaking system we also want players to know how long they're going to wait, so we still show a number that says how long it takes until the next matchmaking round happens. Only now it's shown in the main menu instead of in the loading screen. This can be frustrating if the time is long (worst case it can be as much as 7 minutes), but at least you know what you're getting into.

In the waiting screen of the old version of Awesomenauts you also heard the theme song of the character you had selected. Every character in Awesomenauts has their own jingle and this adds loads of personality to the game. Since this was basically the only entertainment during this waiting screen we even made these songs longer: jingles for the initial characters were only around 20 seconds, while later on we started aiming for 1:30 minutes to reduce repetition.

In the current system where you don't select a character until the match is full we do miss this great opportunity to let the character jingles shine. Instead you now only hear a short bit of the character jingle after selecting the character. The other way to hear these jingles is in the menus while browsing character stats. Having less focus on these awesome theme songs is a pity.


Ideally your multiplayer game has an infinite number of players and you can find equally skilled opponents right away. However, very few games live in this ideal world, so everyone else needs to think cleverly about how their matchmaking works. Just building basic matchmaking and assuming it'll work out isn't going to cut it. If you're interested in reading more on this topic I recommend reading my previous post on designing matchmaking for non-gigantic communities as well.

In the end I think the main reason we ended up with that waiting screen in Awesomenauts for years is that we underestimated how hardcore our community would be. We had designed Awesomenauts as a casual, fun alternative to the other MOBA games. We thought starting with bots right away and adding players as soon as they came online was way more fun than waiting for a full match. Our players disagreed: joining a match that's already in progress is a disadvantage since the bots might have already lost some ground by the time you replace them. Matchmaking issues quickly became the number one complaint about Awesomenauts in our community, so we increased the waiting time before a match actually starts to 2 minutes (or less if the match is already full). This did increase the chance of everyone being there at the start, but we still got complaints about matches not always being full, so we increased the waiting time even further to 4 minutes.

Since we thought we were making a more casual game we hadn't anticipated that our players would prefer long waiting times to be able to play in a more competitive way. We did adjust the system based on this feedback, but the result was waiting in a loading screen. I'm glad we got to improve that situation when Galactron launched later on.

The limitations of matchmaking without server logic

I've previously explained how Steam, Microsoft, Sony and Nintendo all offer similar generic matchmaking systems based on rooms. These systems are great for quickly getting things going and are fine for most smallish games. However, if matchmaking is very important for your game and the requirements for matchmaking become more complex, these generic systems turn out to be pretty limiting. Today I'd like to discuss how we tried several approaches to get the best out of such generic systems in our game Awesomenauts and how the problems we encountered ultimately led us to develop our own matchmaking servers altogether.

Keep in mind that whether these problems are truly important depends on the game. If your game is a bit more casual, or has fewer players per match, or less focus on online multiplayer, or allows late-join, then the requirements for matchmaking become a lot less stringent. Depending on how demanding the matchmaking for your game is, none of these problems might be enough reason to skip the great convenience of using an existing matchmaking system.



The main problem is that these generic matchmaking systems don't allow serious logic to run on the server. The server keeps a list of gamerooms that the player might join, but the decision which room to actually join is made by the client. With this in mind, let's have a look at some of the client-based matchmaking algorithms we tried in Awesomenauts.

The 'growing search range' approach

The standard approach to client-based matchmaking is to have a growing search range. The client starts by asking the server for a list of all the rooms that can be joined and then checks whether any of those rooms are near geographically (to achieve low ping) and are similar in terms of skill. A good starting point might for example be that the other players in the matchroom need to be in the same country/state and differ by at most 500 skillpoints (in a system where 10,000 is the average skill of all players). If several suitable rooms are found, then we join the one closest.

This algorithm is simple enough, but the challenge is what to do when no suitable rooms are found. If you're adamant on always having high quality match-ups, then you might choose to simply wait for more suitable players to appear. However, unless your game has an enormous player count this probably results in really long waiting times for the user, or even in never finding a full match at all.

Therefore we increase the search distance over time. For example, we might double the allowed geographical distance and skill difference every 20 seconds. We keep doing this until we've found a suitable match. If at the biggest search range we still haven't found anyone to play with, then we simply keep waiting and searching.

An important note here is that while searching, the player also has her own room open for others to join. As soon as someone joins your room, you stop searching and just wait until enough players have joined for the match to be started. If you join someone else's room, you close your own room. To avoid problematic edge cases like two players joining each other's rooms simultaneously we add a simple rule: if the other room has only one player in it then we only join it if that room has a lower roomID than our own room. This may seem limiting but in practice doesn't matter much: if I can join your room because you have similar skill, then you can (eventually) join my room as well. It doesn't matter who joins who as long as we end up together.

This is roughly the matchmaking algorithm that Awesomenauts had at launch. What we didn't know back then, is that in many cases the result is little better than just joining a random room. For example, let's say a new player starts searching every 5 seconds. In a game with matches of 20 minutes, this means we have 240 concurrent players, which is pretty nice for a smaller indie game (multiplayer games that are an okay success on Steam are generally in the range of 400 to 2000 concurrent players). In the case of Awesomenauts a full match needs 6 players, which means it would take 30 seconds for a complete group to start searching. For decent matchmaking we don't want to play with just anyone though: we want a good match in terms of ping and skill. Only 1 in 4 players will be even remotely acceptable to play with (in practice probably even less), so we need to wait 2 minutes for enough players for a match. However, this won't happen: by the time we've waited that long we will have grown our search radius so far that we will have joined some other match already.

The result is that this algorithm isn't as good at finding a good matchup as one might think initially. We tried playing around with how quickly the search range grows and such, but we couldn't find a way that achieved a big enough improvement.

League-based matchmaking

Back then we got a lot of complaints about matchmaking not producing good match-ups, so we concluded that we needed something different. What we came up with was to add a league system to Awesomenauts and limit matchmaking based on that. We added this to the game in September 2012, a few months after launch. The idea is that we divide the leaderboards into 9 leagues, based on player skill. These leagues weren't just intended for matchmaking, but also for the player experience: it's cooler to be in league 2 than to be in place 4735, even though that might actually be the same thing.



The matchmaking rule we combined with these leagues is that players could only join matches in the 3 nearest leagues. So a league 4 player could join matches that are leagues 3, 4 or 5. A league 1 player (the top league) could only join matches in leagues 1, 2 and 3. Having such a hard limit should keep players from experiencing extremely big skill differences with their opponents or teammates.

For this to work you need to know the player's skill pretty well, so an important question is how well the game actually knows each player's skill. This is an important issue in any matchmaking system and is such a big topic that I'm not going to discuss this today. For today's blogpost let's just assume that the game has a somewhat decent idea what a player's actual skill is.

Another important issue is what to do if there are few players online. Having hard limits on who can join who means that in some cases you might not find opponents quickly enough. For this reason we stretched the allowed league range a bit further when few players were playing at that specific moment. The Steam API contains this information so implementing this is simple.

An easy mistake here is to simply split the leaderboards evenly. So for example if there are 90,000 players in the leaderboards, then numbers 80,001 to 90,000 are league 9. However, this doesn't work: some players play much more than others and players who stopped playing are still in the leaderboards. If these would be evenly spread out over the community this wouldn't be a problem, but of course they aren't: top players play much more than bottom players.

Our solution was that we measured how many matches were actually being played each day in each league and adjusted the league sizes based on that. Our implementation was bit clunky so we had to adjust this by hand occasionally, but it worked most of the time. The result is that league 2 is only a few percent of the leaderboards while league 9 is around 55 percent, resulting in equal numbers of matches being played every day in each league.



The league system was a big improvement for the matchmaking in Awesomenauts: not only did it make the matchmaker much better at matching similar players, it also provided a fun extra reward system, since wanting to be in a certain league is a fun challenge.

One specific problem with league based matchmaking is that going up a league makes too big a difference. If you go from the top of league 3 to the bottom of league 2 you suddenly get way better opponents, even though your own skill might have only marginally improved. Also, the skill difference between the top of a league and the bottom of that same league can be really huge, especially in the top leagues. While our matchmaker also looked for the best room to join in terms of exact skill (besides just the league requirements), it wasn't very good at this because of the problems with the growing-search-region algorithm I described earlier in this blogpost. A way to lessen this problem is to have more and smaller leagues, but we felt having dozens of leagues makes league climbing less fun for players.

The lack of server logic

While leagues were an improvement, this approach didn't fix the big problem that there's no logic running on the servers. Each client decides for themselves based on incomplete information. This is very limiting to how clever your matchmaking can be. There are lots of situations in which this algorithm will fail to produce optimal match-ups. Here's an example:



As you can see, in this case the algorithms described above fail to find the best possible match-up. Perfect matchmaking is impossible without huge player counts, but the matchmaker should do its best to find the best combinations with what's available.

Below is another example of a type of situation that this algorithm doesn't handle well. This one shows that this particular algorithm is particularly bad at matching premades with other premades: as soon as a single solo player joins, there's no room anymore for another premade that might start searching a little bit later.



Undoubtedly some super complex distributed algorithm is possible that can always find the optimal match-ups without any logic running on the servers. However, at that point the complexity of the algorithm becomes so high and debugging so difficult that I think running the matchmaking logic on a server instead is a much more viable solution. So we started developing our own matchmaking system (called Galactron) with our own servers. This was added to Awesomenauts in 2016.

In this post I've shown a number of problems we've encountered with generic room-based matchmaking systems and how we tried different approaches to work around those. However, not all of the issues with our early matchmaking approach were caused by the limitations of generic room-based matchmaking. Some issues were caused by how we had actually implemented our side of it. Those make for an interesting read by themselves so my next blogpost will be about the things we could have done better within the confines of generic room-based matchmaking.

Working with generic room-based matchmaking

When creating matchmaking for a game you can either build it all yourself, or use a generic system provided by the platform you're releasing on. Steam, Microsoft, Sony and Nintendo all offer similar room-based systems. The basic idea is that clients can create, search for and join game rooms, and the actual decision which room to join is entirely made by the client. There's very little actual logic happening in those matchmaking servers. Today I would like to discuss how they work and what benefits and limitations they come with.

As far as I know most smallish games build their matchmaking using these generic room-based systems. The big triple-A developers seem to usually build everything themselves from scratch, but that's too much work and too much server maintenance to be feasible for most smaller developers.

Our own game Awesomenauts initially launched using these basic room-based systems. We didn't develop our own matchmaking system Galactron until years later when we concluded that the requirements for a genre as demanding as a competitive MOBA don't sit well with the limitations of such generic room-based systems.



The specifics of the systems that Steam, Microsoft, Sony and Nintendo offer aren't public and thus can't be discussed here, so I'm not going into any detail on how they each differ from the generic ideas described here. However, they share enough that I feel this post is very relevant if you ever make matchmaking for any of these platforms, or another platform that's built on similar ideas.

The basic concept is that the platform holder runs servers that handle rooms. A room usually represents a single match with a bunch of players in it. For example, in Awesomenauts a room would have the 1 to 6 players in it that are playing together in a match. If a player wants to join a match then the game will ask the server for a list of all rooms that have a spot left for at least one more player. The game then chooses which room is the best fit and requests the server to join it. Upon joining the game receives a list of the other players in that match and can make a connection with them and start playing.

It's important to realise that the room doesn't actually run the game. It's not a gameplay server. Usually one of the players is the gameplay server. All that the room does is keep a list of who's in the match and allow players to search for matches to join. The room is only intended for finding and maintaining lists of matches and can do very little else.

Since the room doesn't handle the actual gameplay it can often be closed once the match has started and the players are connected. Only if you support drop-in do you really need to keep a room up once the match has already started. Whether closing the room once the match has started is actually how it works depends on the platform though.

The server that handles the room can do some basic logic, but generally not much. One of the most important things it does is limit how many people can be in the room. You usually set a limit and if two players try to join at the same time while there's only space for one, then the server will make sure only one actually succeeds. This solves one of the more hairy race conditions around matchmaking.

Rooms aren't just usable for matches: you can also use them for some other things. A simple example is a chatroom. This is simply a room that players use to exchange chat messages, for example if you have global chat somewhere in your game's menus. Whether rooms scale to large numbers of players for a true global chat varies per platform. This case also shows another use of rooms: often it's possible to send messages to other players in the same room through the rooms server, without actually connecting to other players.

Also, if the client is in a room it will get notifications of changes to the room and of messages being sent through the room. You shouldn't expect rooms to be able to handle high numbers of messages or send them quickly, but something like a chatroom where lag isn't really an issue might be suitable, depending on the platform.

A more common use of rooms outside gamerooms and chatrooms is setting up a party to play with: this is also usually a room. This kind of room starts out with just one player in it, and if that player invites friends to play together then they join her party. The party leader then does matchmaking to find a room with enough free spots for the entire party.



Joining a gameroom with a partyroom of several people is often a hairy thing to develop. If the platform holder didn't supply anything for this particular situation then there are a lot of things that can go wrong that you need to solve. For example, what if a party of 3 players starts joining a room that has 3 empty spots, but by the time two of the players in the party have joined someone from outside also joins that room, taking up the final spot? In that case you need to roll back the joining for the two players who already joined and go back to searching. The number of edge-cases that might occur here is pretty horrible and is just one example of why developing an online multiplayer game is so much more work than developing a single player game. (For some more reasons check this blogpost on why adding multiplayer makes game coding twice as much work.)

Another interesting challenge when using generic room-based matchmaking is what supposedly happened to one of the Gears of Wars games at launch. Rumour has it that their matchmaking broke down because lots of clients all decided that the same room was the best one to join. Only a few got in before it was full, and then the remaining players all tried joining the same next room. Since each time most would fail it often took so many tries to actually get into a room that the matchmaking took forever. This particular situation is difficult to test without a large playerbase and definitely something to consider during development.

Rooms allow for additional data to be stored in the room. This data can be used in any way the developer sees fit and will probably contain information on the game mode, the skill of the players in the match and the map on which the match takes place. This way when a player is searching for a specific game mode the game can look through the list of rooms for one that's the desired game mode. This also allows the game to check whether the other players in the room are at a similar skill level as the player. Rooms can hold arbitrary data as the developer sees fit, but they generally don't allow for a large amount of data, so only the essential stuff is stored here.

While it's often relatively easy and quick to use the matchmaking systems that platform holders provide, there are some downsides to this approach. One of the most important ones is that if you make a multiplatform game then you'll need to build this again for each platform. If you're doing simultaneous launch on 3 platforms then this is going to take a serious amount of programming work.

You might think you can reuse a lot of your code since the basic ideas behind these rooms are so similar, but in our experience the platforms have just enough differences that this approach quickly becomes a horror story. The old version of the Awesomenauts matchmaker supports 5 platforms: Steam, X1, X360, PS4 and PS3. We tried to abstract away the platform-specific details as much as possible, but adding code for all the weird edge cases each platform introduces has turned our old matchmaking codebase into one big tangled mess where any change might break any of the platforms. Designing a good architecture for handling the differences between these systems is a tough challenge, especially if you're doing it for the first time.



Note that the room systems on various platforms also can't connect to each other. So if you want to do cross-platform multiplayer then you basically need to write your own matchmaking servers.

Another problem of using these generic room systems is that they allow for very little server-side logic to be run. This is a huge limitation on how well you can match players. This is also the main reason we redid the entire Awesomenauts matchmaking code and built our own, called Galactron. This launched in 2016. The limitations on matchmaking imposed by a lack of server logic is a big enough topic that I'll write a separate blogpost about that soon.

The generic room-based matchmaking systems that platform holders provide make developing matchmaking a lot easier than without. They save you all the hassle of developing and maintaining your own servers, so unless you're doing a large production, chances are they are the best option to efficiently develop the matchmaking for your game.

The requirements of good matchmaking

Matchmaking is a highly important part of most modern multiplayer games. Build it badly and the user might be stomped by players way too good for them, get a really laggy connection or wait forever to get into a match. It's also a difficult topic to analyse and test before launch because there are so many unpredictable factors. For example, how are your players spread out over the globe? How much do match durations vary? How long are players willing to wait? How good are their internet connections? All of these and many other factors influence whether any given matchmaking implementation actually works well. Surprisingly few developers have written about the matchmaking in their games in detail, so I figured it would be useful to share our experiences. Over the years we've had several different approaches to matchmaking in Awesomenauts, and have iterated live on each of those, so let's have a look at what we've learned so far!



Matchmaking is such a big and complex topic that I'm going to write a series of posts about it. Today I'm going to kick off this topic by looking at the requirements of "ideal" matchmaking. In future posts I'm going to explore the actual matchmaking implementations we've used and their pros and cons. This will start with the standard matchmaking systems that the big platform manufacturers provide (like Valve, Sony, Microsoft and Sony), followed by an analysis of why we switched away from that and how we built our own. I'm also going to dive into some more controversial topics, like the psychological side of matchmaking and how we managed to massively reduce the percentage of people who leave a match before it finishes.

What defines good matchmaking? This depends on the game of course, but usually the matchmaker will try to achieve most of the following things:

• Low waiting times to get into a match.
• Low ping.
• Match players of similar skill.
• Match premades with premades.
• All players in a match start at the same time.

Many additional requirements are possible, such as:

• Match players who speak the same language.
• Punish bad apples by matching them with each other (for example leavers and griefers).
• Don't get matched with the same people too often.
• Gameplay variation: maybe you don't want to be in the same kind of matchup every time.
• Match players who use voice chat with others who also do so.
• Match based on team composition (if one team has a rare team composition then maybe match them with others who also don't have the standard set of tank/healer/harasser/etc).

It's rarely possible to achieve all of these requirements simultaneously. Even if your game is a big success and 100 people or more search for a match every minute, then some requirements might still not be doable with acceptable waiting times. For example, if you want to match Australians only with other Australians, then waiting times deep in the Australian night will be extremely long for all but the biggest games. What works for a game with a playerbase as large as League Of Legends or Call Of Duty doesn't work for most other games.



I've previously written a blogpost about how many players you need for quality matchmaking and the short summary is that 1000 concurrent players isn't much from a matchmaking perspective. However, it is for most games: especially indie games are considered a hit if they reach 1000 simultaneous players every day. Even if you're convinced your game will be an indie hit you still need to think long and hard about how to make the matchmaking work with that relatively small playerbase. And this isn't limited to indies either: even a big triple-A multiplayer production like Lawbreakers currently has only a few hundred simultaneous players.

Because it's usually impossible to achieve all matchmaking goals simultaneously, you need to think about how important each of them is. Especially waiting times suffer when you make other requirements more important. The more requirements you can ignore, the better you can do on the other ones. For example, in Awesomenauts we've chosen to ignore ping with teammates: the matchmaker only looks at ping with enemies. This is because you can't hit your teammates anyway, so if there's more lag there then that will be much less of a problem. In an ideal world we would also want a low ping with teammates, but in practice we can achieve better ping with enemies if we ignore ping with teammates, and thus in our case this is a good choice to make.

One important thing to keep in mind when designing a multiplayer game is that adding more gamemodes often means splitting the playerbase and thus increasing waiting times or reducing matchmaking quality. Think long and hard before adding an additional matchmade game mode!

Reducing the matchmaking requirements isn't the only way to reduce waiting times. For example, if you can design your game in a way that allows for drop-in-drop-out, then players will usually be able to get into a match right away. Or if that's not possible, then maybe if rounds are short enough players can spectate a round while waiting for it to finish and then play in the next one. This doesn't reduce the waiting time, but it does make waiting a lot more enjoyable. I've previously written a blogpost that explores a bunch of different approaches to designing a game so that multiplayer works with a smaller playerbase.



Since this series of posts is for a large part based on our own experiences, I will mostly talk about matchmaking for MOBAs. In other words: real-time team-based games where skill is extremely important, where matches take relatively long and where the same players should play a match together from start to finish. In a sense MOBAs are amongst the most demanding games in terms of matchmaking requirements.

It's important to realise that other types of games have different requirements. For example, in shooters skill is often mostly ignored. Instead, teams are reshuffled after every round to balance things out. Shooter players usually also expect to get into a match really quickly, while MOBA players are mostly okay with waiting a couple of minutes for a better match-up. Of course this varies from game to game: not all games in the same genre have the same matchmaking requirements.

In the end the most important thing is to take matchmaking seriously. Think about the requirements of your game and don't assume they are the same as for another game. In some cases you might even have to redesign your multiplayer experience to make matchmaking work, maybe by removing some game modes or adding something for players to do while waiting for matchmaking. Matchmaking can't be an afterthought if your multiplayer experience is to thrive!

Analysing the new Awesomenauts matchmaker

We're currently rebuilding the entire matchmaking, menus and party system of Awesomenauts. This rework is called Galactron. This is a huge operation: especially the matchmaking part is highly complex. We've done a bunch of betas for it so far and last week we let it run live for everyone for 48 hours. After that we turned it off again to analyse the results, read player feedback and fix bugs. Today I'd like to discuss the most interesting results.

I plan to do an overview of how Galactron works in a future blogpost, but today I'm just going to focus on what we've encountered in last week's beta.

Ragequitters and the removal of latejoin

This is probably the hottest topic in Galactron. Latejoin was probably the most hated feature of the old Awesomenauts so we've removed it altogether. In Galactron players are never put in a match that's already in progress. We've also added a rejoin option so if you're kicked out of a match due to a network error you can get right back into it. Finally, if someone leaves a public match, he can only rejoin that same match and can't get into another. A lot of players like this, but there's also a vocal group who is strongly against the removal of latejoin and the increased punishment for ragequitting.

I think there's no way to make both groups happy: they just want something different out of Awesomenauts. I've seen some people suggest we should make two gamemodes: one where latejoin and ragequit are allowed and one where they aren't. However, this is not an option because our playerbase is not big enough to split it like that. If we had the playercounts of League of Legends we could try things like that, but with our current playercounts we can't afford to split the community. Matchmaking in both versions would be worse if we did.

Instead of talking about opinions, let's have a look at cold hard facts. A common complaint about the removal of latejoin is that if someone leaves a matches and doesn't rejoin, the match remains 3vs2 until the end, whereas in the past the player would be replaced by someone else. This replacement didn't always happen though: in the old matchmaking matches weren't all 3vs3 all the time either. We gather metrics on this, so let's have a look:



As you can see the percentage of time that matches are 3vs3 is actually higher in Galactron than in the old matchmaking. In other words: while it may feel lame that leavers aren't replaced by latejoiners anymore, the overall result of Galactron is that you're playing 3vs3 more often. So while I can imagine that ragequitters dislike the new punishment they get for leaving, the complaint that matches remain incomplete more because leavers aren't replaced turns out to be unjustified.

High ping

The other major complaint about Galactron is high ping. While Galactron is supposed to make the average ping lower, players complain that ping is higher. Again, let's look at the metrics to see whether they're right:



Here it turns out that the stats agree with the complaining players: average ping has indeed increased significantly in Galactron. This is a big problem that we need to solve. It's also a big surprise to us: Galactron contains a lot of clever tech to match players better, why isn't that working?

We've been analysing this extensively these past few days and have a bunch of theories, but we don't have the final answer yet. One thing we've noticed is that it seems like Galactron doesn't get enough ping data. While players are waiting for matchmaking the game pings all other waiting players. We expected this would give us ping data on at least 75% of all possible connections, but in practice this turns out to often be around 50%. This means that Galactron doesn't have enough information to make a good matchup. Figuring out why this happens is our highest priority now.

Another possibility is that Galactron may care about skill too much right now. Galactron needs to find the best match-up based on a bunch of criteria around ping and skill. It can't satisfy them all perfectly all the time so it tries to find a balance. We can tweak how important each criterium is and I think we might have made skill too important right now.

A final issue we're looking into is how to handle people with horrible internet. High ping can happen when you're playing against someone on the other side of the globe, but sometimes your opponent just has such a horrible internet connection that he has high ping to everyone, even to people near him. No matter who Galactron matches such players to, the connection will always suck. We haven't decided how to handle that yet. One option is to make such players only play against others with crappy internet and keep them out of the regular matchmaking, but that might give odd results in the leaderboards as they would always be playing against the same people. We'll have to observe this group more once the other issues are fixed.

Skill matching

This is an aspect where Galactron seems highly successful: matches are very balanced. There are still situations where players of differing skill play against each other, but this is now usually caused by premades in which the players don't have the same skill. We can't split premades so this can't be solved. What we can do however is balance such premades against another similar premade, or against other players with similar skill patterns. Galactron turns out to be able to do so quite often. We also see premades play against other premades much more often now than before.

It would have been nice to compare this to our snowballing stats, but unfortunately those weren't recorded in the Galactron live beta due to an oversight. We normally collect stats about how often one team has much more kills than the other team, and how often the losing team didn't destroy any turrets. It would be interesting to see those stats for Galactron, so it's a pity they weren't recorded during the last beta.

Waiting times

I've seen some complaints regarding long waiting times in Galactron. Galactron performs a matchmaking round once every 5 minutes. That means that if you start searching for a match just before the round starts you'll get a match really quickly, while if you start searching just after you'll have to wait the full 5 minutes. On average this means a waiting time of 2.5 minutes. That isn't that much longer than before: in the old matchmaking it might take up to 4 minutes before a match stars, and longer if it isn't full yet at that point.

It's easy enough to reduce the waiting time: we can just switch from 5 minutes per matchmaking round to 4 minutes. However, we're not sure we should: a shorter waiting time will also mean lower match quality. There's a balance we need to strike here between speed and quality, and for the moment we think 5 minutes is fine. If we keep seeing a lot of complaints about this we'll reconsider and might change it to 4 minutes at some point. Feel free to share your opinion on this in the comments below.

Crashes

There have been some reports on crashes in Galactron. We've investigated most of these and so far they seem to be caused by driver issues and by broken Steam downloads. The latter can usually be fixed by running Steam's Verify Integrity Of Steam Cache.

As for the driver issues: my theory is that the crashes that we have data on are caused by alt+tab. Unfortunately our crashdumps don't contain a history that tells us whether alt+tab happened just before, so we can't tell for sure.

If alt+tab crashes than your videocard drivers are problematic. The solution is really simple: run in Borderless Window instead of Full Screen. With Borderless Window the game still covers the entire screen, it's just a different way of asking Windows to do so. Borderless Window is much more stable when it comes to alt+tab, and it's also much faster at it. In general we advice never running Full Screen unless Borderless Window gives you problems. I think we might rename these options to reflect this, since I expect some people might not understand what Borderless Window means.

We're still investigating some of the crash reports. If we encounter any that are caused by Galactron, we'll fix them of course.

Bugs

Besides the things mentioned above there were also a bunch of random bugs. Most of those we've already fixed by now, or are working on fixing. Some of the most notable ones:

• Global chat didn't work half the time.
• One menu screen around the tutorial didn't accept any button input, making it impossible to progress if you got to this screen (ARGH!).
• Some menu screens didn't support controllers properly; only keyboard+mouse worked there.

So there you have it, the current status of Galactron. If you have any further questions, complaints or suggestions on Galactron then please do post them below in the comments. We hope to do another beta 1.5 weeks from now. Hopefully that one will be good enough that we won't have to turn Galactron off again, but we'll have to see how it goes. Most important is that Galactron needs to be really good before it goes out of beta.

Designing matchmaking for non-gigantic communities

Most multiplayer games die within a week. Not because the multiplayer doesn't work, but because there aren't enough people playing online. If you look for opponents and don't find any, then the multiplayer mode might just as well not have been made. This is even more of a waste because online multiplayer is so much work to develop, as I recently explained. The vast majority of online multiplayer games ever made isn't playable any more because of this. Partially this is because not all games can be a success, but there is a much more important reason: often the matchmaking is poorly designed for a smaller playerbase. With clever choices multiplayer can flourish even with a tiny community.

For an average indie game that is doing okay but isn't a big hit it is pretty decent to have 40 simultaneous players a few weeks after launch. Our own game Awesomenauts did really well so our daily peak is usually above 1000 players, but expecting such numbers is unrealistic for most games.

Let's consider a four player multiplayer game that has three game modes. We press the quickmatch button and the game starts looking for opponents. If a match takes on average 30 minutes and there are 40 simultaneous players then a player starts looking for a match once every 45 seconds. Since there are three game modes your mode will have one player searching once every 135 seconds. Waiting for a complete match of four players then takes almost 7 minutes.

Seven minutes! Few players are willing to look at a "Searching" screen for that long, so most will just quit and play something else. If a player concludes your game is dead he probably won't try again later and is lost forever. Whatever playerbase you had will quickly disappear. So many multiplayer games die because there aren't enough players for a good experience.

However, with clever design choices this fate can be averted. 40 simultaneous players would have been plenty for a good experience if matchmaking had been designed in a different way.



Reduce game modes

Game modes split the community. If half of your players wants to play Domination while the other half wants to play Deathmatch, then they all have only half as many opponents to play with. In the example above removing two game modes would have reduced the waiting time to 2:15 minutes, which might already have done the trick. Even the gigantonormous League of Legends has removed less popular game modes because there weren't enough players for a good experience.

For exactly this reason Awesomenauts has only one core game mode. There is a custom game option where users can set up all kinds of weird game modes, but these aren't part of the core matchmaking and thus only playable through friend invites. This way there are extra options for players who want them, but there is only one core mode and it has plenty of players to matchmake with. (For a more detailed analysis of player spread, check my blogpost on why good matchmaking requires enormous player counts.)

If you really want to have more game modes, you can also choose to force players to play whatever is available. Maybe players can select a preferred game mode and the matchmaker looks at all the preferences and tries to find the best match-up. If there aren't enough players in your favourite game mode, then you just play a different game mode. The downside of this approach is that players would probably complain a lot that they aren't put in their favourite game mode. Not having an option at all might in some cases feel better than having an option that you can't actually play...

Allow solo play while waiting

The above approach tries to reduce the waiting times. An alternative is to make waiting more fun so that players are willing to wait longer. In our 2010 game Swords & Soldiers we allow players to play all single player content while waiting for an opponent. If you start the matchmaker a little icon appears in a corner of the screen and you can just play some skirmish against the computer until another human becomes available to play with. This works wonders: players are willing to wait much longer, increasing the chance they actually find an opponent.

This can be improved further by having content that is fun to play repeatedly. A story campaign isn't so much fun to play every time while waiting, but a highscore mode is. In Swords & Soldiers I usually played the endless Survival mode while waiting. Of course it sucks to be thrown out of your highscore run because an opponent is found, so our solution was to auto-save the solo match when starting a multiplayer match. You can continue your single player challenge after the multiplayer has ended.

A while ago a colleague of mine tried playing Swords & Soldiers online. Even though the Steam version of the game is almost five years old now and there are only a few people playing, he got three different opponents in just an hour. There were never more than four people playing Swords & Soldiers simultaneously that evening, so he got a surprisingly good multiplayer experience with such a microscopic community. Amazing!



Playing against bots

For Awesomenauts we chose a different approach. Here we let the match start even if there aren't enough players for a full 3vs3 match. The missing spots are filled with bots until more players are found. The result is that even if there are very few players online, you never have to wait long and can always start playing.

Initially Awesomenauts just started a match right away regardless of how many players were found, removing waiting times altogether. The big downside of this is that it doesn't work well with a competitive mentality: it feels unfair if bots already lost a tower before you even joined. For this reason we later added waiting time, but with a countdown timer that shows the maximum waiting time left. Since Awesomenauts has plenty of players most of the time the match will be full well before the timer runs out, so you rarely start with bots.

This drop-in aspect of the game is a constant source of complaints from our userbase, but players often don't realise that without it, Awesomenauts might not have survived its first months. It took nearly half a year before our playerbase reached a constant good amount on PC and even with those smaller numbers there were never long loading times.



Adding more players

So far I have focussed on making the most out of a your existing players. Another important approach is to maximise the numbers of players. This can be done in various ways. Many multiplayer games are free-to-play, increasing the size of the playerbase (although there are also plenty of free-to-play games that don't actually manage to gather a significant playerbase).

The developers of Verdun chose a hybrid approach: at some point their game had a free browser version and a paid Steam version and they all played together online. The free players didn't bring in a lot of money directly, but they made the game more fun for the Steam players, increasing sales there.

Even when not going free-to-play it is a good idea to not make a game too expensive. The average indie game seems to have recently gone up in price, but I would heavily advice against pricing a multiplayer-only game above $15: this will decrease the size of the playerbase. Selling more copies is more important for the long term income of a multiplayer game than the short term income of a higher price. Awesomenauts is $10 and we often drop its price heavily during sales, by as much as 90%. This boosts the playerbase enormously, more than making up for the low price.

Extreme measures

If you want to make a multiplayer game playable with below 10 simultaneous online players you might have to resort to extreme measures to keep something going. Several devs told me they promoted a specific moment, like one specific evening every week, as multiplayer evening. This way everyone comes online at the same moment, creating a somewhat lively community once per week. One dev even went as far as taking the multiplayer functionality offline altogether during weekdays, only allowing multiplayer during weekends.



I can imagine other, more subtle ways to get players online at the some moment. Like allowing players to send a challenge to other players who are near them in the leaderboards. The game then sets up a time for them to play against each other, giving them a good reason to come online at one particular time. Asynchronous multiplayer is also a good way to do this, but this only works with certain types of games. For highscore based games asynchronous multiplayer can be an excellent way of boosting the competition within the community.





There are many other ways to make the most out of a small community. For example, things like clever lobbies, a rematch option and showing the expected waiting time per game mode. Key is to consider matchmaking a real part of game design, something to brainstorm about and come up with creative solutions. If you just make something then you might end up killing your community much quicker than needed. If you aren't Blizzard or Riot then thinking about how to make the most out of a smaller online community is crucial to the success of your multiplayer game.

Why good matchmaking requires enormous player counts

Good matchmaking is an important part of creating an online multiplayer game. One thing you may not realise is that no matter how you build it, truly good matchmaking requires enormous numbers of players. Awesomenauts often has well over 1,000 people playing the game at the same time, which is very high and successful for an indie game. It certainly sounds like a lot to me, but this is only a fraction of what would be needed to do everything with matchmaking that we would want to do. Today I am going to explain why tens of thousands of concurrent players are needed for truly awesome matchmaking.

Matchmaking has two main goals. The first is to let people play together who will have a good internet connection to each other. We would like to avoid Australians playing together with Europeans because their ping will be very high. High ping decreases the quality of the game experience, especially in a fast and highly competitive game like Awesomenauts. Finding good connections is more complex than simply looking at distance: sometimes you have a worse connection to your neighbour than to someone on the other side of the continent. The internet is just very unpredictable and random when it comes to connection quality.

The second main goal of matchmaking is to let people play together who have similar skill. There is no fun in getting beaten by someone who is way better than you, and n00bstomping gets old really quickly as well.

Now that we know our core goals, let's try to estimate how much 1,000 concurrent players really is. I am going to simplify things and use a lot of assumptions, but I think the ballpark figures are realistic enough to communicate my point.

The first step is to look at how often these players are actually available for matchmaking. An Awesomenauts match takes on average somewhere around 20 minutes, so players are available for matchmaking once every twenty minutes:

1,000 / 20 = 50 players per minute

We don't want to matchmake with people who are too far away, and players are spread all over the globe. Let's say that the average player would have a good enough connection to one third of all players. In reality players are not spread equally, since Awesomenauts is more popular in some countries than in others, and because of time zones. Let's work with that one third though:

50 / 3 = 16.7 players per minute

Next step is skill. Let's say we consider one third of all players to be close enough in skill to make for a fun match:

16.7 / 3 = 5.6 players per minute

AUCH! We are already down from 1,000 concurrent players to only 5.6 suitable players per minute, and this is while looking at only the most basic of assumptions...

For perfect matchmaking I would want to split players further. A common request from Awesomenauts players is to have unranked and ranked multiplayer. If we would add this, the split between these modes would probably not be equal: one mode would likely get more players than the other. Let's assume one third of all players would play one mode, and two thirds would play the other mode. We need to split further from the already small numbers we have because even in unranked matchmaking we still want to match players based on similar skill to create fun matches. This is how few players we would have left in the smallest of the two modes:

5.6 / 3 = 1.9 players per minute

Another common request from the Awesomenauts community is to split pre-mades from solo-queuers. A pre-made is a group of people who form a team by hand in the lobby before the match, while solo-queuers are put together in a team with complete strangers by the matchmaker. Pre-mades potentially have a big advantage because they will likely do much better teamwork. In the ideal case pre-mades would therefore only play against other pre-mades. How many pre-mades there are varies wildly with the skill level of the players (highly skilled players generally play in pre-mades much more). Let's assume that on average one fourth of all players are in a pre-made:

1.9 / 4 = 0.46 players per minute

Since we are talking about perfect matchmaking, let's have another look at our skill-based matchmaking. Above I assumed that one third of all players is a good enough match in skill. In reality the top players are way too much better than the rest to make this ideal. The top 5% of players are an enormous amount better than the top 33% of players. The more precisely we could match based on skill, the better. I think we would need to do at least three times better than we did above for ideal matchmaking:

0.46 / 3 = 0.15 players per minute

I can imagine some more criteria for ideal matchmaking (like supporting more game modes in matchmaking), but I think the point is quite clear already. With 1,000 concurrent players it will take 30 minutes to fill a match! Obviously this is totally unacceptable. Here's a summary of all the criteria I have mentioned so far:



Let's say it is fine to let players wait for two minutes for a match to fill up, bringing us to 0.3 suitable players during the available time for matchmaking. To bring us to the required 6 we would therefore need 6 / 0.3 = 19 times as many concurrent players for good matchmaking. We started with 1,000, so we need 19,000 concurrent players. Of course there is a big daily fluctuation in the number of players (there are fewer players deep in the night and early in the morning), so to also have good matchmaking at the slow hours we would need three times more players still. Thus we would need 58,000 concurrent players at peak for good matchmaking. That probably equals over 5 million unique players each month. Holy cow that is a lot!

When building a multiplayer game it is important to think about this. Ideal matchmaking requires enormous player counts, and if your matchmaking is built assuming such player counts will be there you might make something that works really badly for more realistic numbers. Therefore the ideal matchmaking system is flexible: it brings perfect matchmaking when there are tons of players but also makes the best of a small player count.

Despite all of this we can still make big improvements to the matchmaking system in Awesomenauts. We are aware of this and are therefore rebuilding the entire matchmaking system from the ground up in a much smarter and much more flexible way than is currently in the game. I am sure this will bring a big improvement, but at the same time it is important to have realistic expectations: no matter how well we build our matchmaking, the player count required for 'perfect' matchmaking is unrealistic for all but the few most successful games in the world.