Taking live soundtrack improvisation to the next level

I've previously written about how Rene Derks and I did a live musical improvisation to someone playing Journey and Ori And The Blind Forest. This was a pretty unique experiment already, but recently we kicked it up a notch with an even bolder performance: instead of knowing what was coming, we let players choose whatever game they wanted to play and we had to improvise live music to that. Since this was an experiment not all of it worked well, but I think it was a success: a bunch of games turned out pretty awesome and we learned a lot. Today I'd like to explain how we approached each game and why some of them worked, and others turned out not to be a great fit for this format. I've included a couple of short videos from the performance, check 'm out!

The basic idea for these performances is that we mix live improvisation with gameplay. Instead of playing the original game's soundtrack we come up with new music on the spot and try to respond to whatever is happening in the game. The result is a truly adaptive soundtrack, by the magic of spontaneous improvisation. As always with improvisation, some of it sounds bad, a lot sounds okay, and some moments are pure magic that can only be experienced there and then.



For the previous performances we had chosen specific parts of the games Journey and Ori that would have a nice emotional arc to add music to, and we thought beforehand about the kinds of emotions and melodies we wanted to do for the various sections of those games. While that makes for better music, it's also limiting: we can't do a whole lot of games that way and experimentation on the spot is somewhat limited. So when we got a chance to try this again at Abunai in August 2017 we chose a different approach: we went in deliberately unprepared. We had a Switch and a laptop with some 40 Steam games installed and let players choose whatever they wanted to play. We improvised to whatever they did.


Beforehand we thought that if 25% of our performance was awesome, 25% was okay and 50% sucked, we would consider it a successful experiment. We may have done a little bit better than that, but not much: some games indeed didn't work, and for some games we only figured out what kind of musical approach worked after a few minutes of struggling.

Another change we made compared to the previous performance is that we played in a small hall in a corridor. During our previous performance at Animecon was had a big stage. That's great if a performance is a big success, but it creates a lot of distance to the audience and you need a lot of audience to make it work. At Animecon on average we had maybe 20 viewers, which made the hall look like a big empty failure. So at Abunai we requested something much simpler: a spot that people walk past on their way to other events at the con, and some couches. This reduces the distance to the audience and makes it easy for people to watch for a few minutes and then walk on. In total we reached way more people this way, and it was much more fun to do. There was even room for musically fooling around with the audience a bit.



The first player of the afternoon chose Limbo and this was also the most successful game for us. During the afternoon we could easily see whether we were doing well by checking whether passersby stayed to watch and listen. They did for Limbo.

The reason Limbo is so great for this kind of musical improvisation is that it has a slow pace with a strong emotional arc. First a player walks for something like 20 seconds, which is a nice duration to set a mood musically. Then he approaches a trap. We know the game so we can start increasing the tension by playing something eerie. Then the player interacts with the trap and might even die a couple of times, which are great opportunities for doing something special musically. In this case I played 2 high shrieking notes at every death, combined with a few hard hits on the djembe by Rene. By doing the same thing at every death the whole performance gets glued together.

Limbo is slow enough that Rene even saw room to play something similar to sound effects on his djembe. For example, he played taps that mimic footsteps, drum rolls during slides and a deep bass when the player lands after a slightly deeper drop.



The second game that was chosen was Ikaruga, an insane bullet hell shooter. This was very difficult for us to react to: the screen is filled with bullets practically all the time so there's isn't really any kind of arc that we could latch onto. We ended up just playing a fast beat and only really reacting during boss fights. It worked somewhat, but the whole idea of the music reacting to the game was kind of lost here and cello+djembe isn't the best possible combo for a fast beat.



The worst of the day and the only complete failure was Tekken 7. A player brought a laptop to show his skills at beating the final boss on the hardest difficulty. He was incredibly good at the game so that was impressive to see, but making music to this was practically impossible. The action was too fast to follow and the camera constantly jumped around to emphasise specific scenes. For a short bit we tried to play sound effects on the punches, but the game is so fast that we totally failed at seeing them coming.

Figuring out what to do with the next two games was really interesting. Spelunky is a game with a rather constant state during gameplay: every five seconds you kill an enemy, grab some loot and do some platforming. Individual events are too fast to react to and the flow of events is pretty much constant. However, Rene noticed that during each level the player is always going down. I tried following this progress by starting each cave on the highest string of my cello and then dropping to the next string as the player drops. While this idea is nice, I feel it was too subtle to be picked up by the audience. In hindsight it might have been better to react to how much health the player had left. Spelunky is a rogue-like game so the tension of being low on health is really high.



During The Binding Of Isaac we found a better trick. For each floor Rene chose a base rhythm that he expanded as the floor progressed, building towards the boss fight. During empty rooms he went back to the floor's basic rhythm. While Rene kept playing this beat on his djembe, I stopped playing altogether. Whenever the player cleared a room, I played a short melody; the same one for each room. I had another melody for opening a chest. And then as the player reaches the final boss of a floor, I started actually playing and driving up the tension. I think this worked really well and it shows a very different approach compared to what we did during the other games.

Next up was Broforce, in 3 player coop. Sjeez, that game is chaotic! Each level is just 40 seconds of constant gunfire and explosions. I had no idea where to look or what to respond to. This was almost as bad as Tekken 7. Ugh.

So I shivered when the next choice was Ultimate Chicken Horse: another super chaotic multiplayer game. However, here we found a fitting approach. A match of Ultimate Chicken Horse is a series of short rounds, where each round starts with everyone modifying the level a bit, and then everyone playing it. Each phase lasts around 15 seconds. Reacting musically to the actual events is impossible in this chaotic game, but instead we simply did something special for the phases. During the building phases I did some goofy plucking on my cello, deliberately playing melodies that sound a bit childish and dumb. And then during the action phase I played a faster, more aggressive variation with the bow. Rene made this contrast even stronger by not playing at all during the building phase and playing a really fast djembe rhythm during the action phase. I think this captured the back and forth of the gameplay phases quite well, and was a good fit for the wacky art of Ultimate Chicken Horse.



We ended with two slower single player games: Zelda and ABZÛ. The Legend of Zelda: Breath of the Wild is a great fit for improvising music. Most things the player does take long enough that you can react to them. Walk for half a minute, fight for half a minute, look at your loot for a bit, climb a hill, swim some water, another battle. Each of these steps takes long enough that you can really react to them without turning the music into a chaotic mess that goes all over the place.

ABZÛ is an even slower game. Like Journey it's a gorgeous game that's mostly about looking and experiencing, with very little challenge in the gameplay. The serene under water gameplay is a great fit for setting a mood through music, but I think our instruments weren't the best fit here. Djembe and cello are interesting for many things, but underwater moods are difficult with this setup. Also, after playing for around 2.5 hours the sound got a bit stale for us.


Nevertheless, during the final round of ABZÛ another interesting thing happened. The player had overlooked that he needed to free a little robot before a door could be opened. Instead, he repeatedly pressed a button on the door, hoping it would open. I played two simple notes each time to accentuate that it didn't work, and since the player kept trying, I kept playing those same two boring notes. This went on for way too long and the audience started laughing at how stupid that sounded. This was one of the nicest and most direct bits of audience interaction we had. It also taught me an important lesson: sometimes it's okay if it sounds bad musically if the interaction is fun or interesting in some other way.

Finally, after the performance we had dinner at Rene's parents. They happen to own a piano and Rene also plays that, so we started jamming some more, despite having played so long during the day already. This sounded quite wonderful and it taught me the biggest change we should do if we do this kind of performance again: we need a more diverse set of instruments. I can only play the cello, but since Rene plays both keyboards and percussion we should just bring both to diversify the sounds and moods we can play. Or maybe we should even get a third musician next time!


In the end I think this was a really interesting experiment. The quality of the music varied from really awesome to really bad and that's okay: the whole point of improvisation is that you don't know what's going to happen and the moments where it's awesome are true magic, happening right there and only then. The key lesson we learned is that different games require vastly different musical approaches and the trick is to find whatever arcs the game has going and respond to those through music.

The requirements of good matchmaking

Matchmaking is a highly important part of most modern multiplayer games. Build it badly and the user might be stomped by players way too good for them, get a really laggy connection or wait forever to get into a match. It's also a difficult topic to analyse and test before launch because there are so many unpredictable factors. For example, how are your players spread out over the globe? How much do match durations vary? How long are players willing to wait? How good are their internet connections? All of these and many other factors influence whether any given matchmaking implementation actually works well. Surprisingly few developers have written about the matchmaking in their games in detail, so I figured it would be useful to share our experiences. Over the years we've had several different approaches to matchmaking in Awesomenauts, and have iterated live on each of those, so let's have a look at what we've learned so far!



Matchmaking is such a big and complex topic that I'm going to write a series of posts about it. Today I'm going to kick off this topic by looking at the requirements of "ideal" matchmaking. In future posts I'm going to explore the actual matchmaking implementations we've used and their pros and cons. This will start with the standard matchmaking systems that the big platform manufacturers provide (like Valve, Sony, Microsoft and Sony), followed by an analysis of why we switched away from that and how we built our own. I'm also going to dive into some more controversial topics, like the psychological side of matchmaking and how we managed to massively reduce the percentage of people who leave a match before it finishes.

What defines good matchmaking? This depends on the game of course, but usually the matchmaker will try to achieve most of the following things:

• Low waiting times to get into a match.
• Low ping.
• Match players of similar skill.
• Match premades with premades.
• All players in a match start at the same time.

Many additional requirements are possible, such as:

• Match players who speak the same language.
• Punish bad apples by matching them with each other (for example leavers and griefers).
• Don't get matched with the same people too often.
• Gameplay variation: maybe you don't want to be in the same kind of matchup every time.
• Match players who use voice chat with others who also do so.
• Match based on team composition (if one team has a rare team composition then maybe match them with others who also don't have the standard set of tank/healer/harasser/etc).

It's rarely possible to achieve all of these requirements simultaneously. Even if your game is a big success and 100 people or more search for a match every minute, then some requirements might still not be doable with acceptable waiting times. For example, if you want to match Australians only with other Australians, then waiting times deep in the Australian night will be extremely long for all but the biggest games. What works for a game with a playerbase as large as League Of Legends or Call Of Duty doesn't work for most other games.



I've previously written a blogpost about how many players you need for quality matchmaking and the short summary is that 1000 concurrent players isn't much from a matchmaking perspective. However, it is for most games: especially indie games are considered a hit if they reach 1000 simultaneous players every day. Even if you're convinced your game will be an indie hit you still need to think long and hard about how to make the matchmaking work with that relatively small playerbase. And this isn't limited to indies either: even a big triple-A multiplayer production like Lawbreakers currently has only a few hundred simultaneous players.

Because it's usually impossible to achieve all matchmaking goals simultaneously, you need to think about how important each of them is. Especially waiting times suffer when you make other requirements more important. The more requirements you can ignore, the better you can do on the other ones. For example, in Awesomenauts we've chosen to ignore ping with teammates: the matchmaker only looks at ping with enemies. This is because you can't hit your teammates anyway, so if there's more lag there then that will be much less of a problem. In an ideal world we would also want a low ping with teammates, but in practice we can achieve better ping with enemies if we ignore ping with teammates, and thus in our case this is a good choice to make.

One important thing to keep in mind when designing a multiplayer game is that adding more gamemodes often means splitting the playerbase and thus increasing waiting times or reducing matchmaking quality. Think long and hard before adding an additional matchmade game mode!

Reducing the matchmaking requirements isn't the only way to reduce waiting times. For example, if you can design your game in a way that allows for drop-in-drop-out, then players will usually be able to get into a match right away. Or if that's not possible, then maybe if rounds are short enough players can spectate a round while waiting for it to finish and then play in the next one. This doesn't reduce the waiting time, but it does make waiting a lot more enjoyable. I've previously written a blogpost that explores a bunch of different approaches to designing a game so that multiplayer works with a smaller playerbase.



Since this series of posts is for a large part based on our own experiences, I will mostly talk about matchmaking for MOBAs. In other words: real-time team-based games where skill is extremely important, where matches take relatively long and where the same players should play a match together from start to finish. In a sense MOBAs are amongst the most demanding games in terms of matchmaking requirements.

It's important to realise that other types of games have different requirements. For example, in shooters skill is often mostly ignored. Instead, teams are reshuffled after every round to balance things out. Shooter players usually also expect to get into a match really quickly, while MOBA players are mostly okay with waiting a couple of minutes for a better match-up. Of course this varies from game to game: not all games in the same genre have the same matchmaking requirements.

In the end the most important thing is to take matchmaking seriously. Think about the requirements of your game and don't assume they are the same as for another game. In some cases you might even have to redesign your multiplayer experience to make matchmaking work, maybe by removing some game modes or adding something for players to do while waiting for matchmaking. Matchmaking can't be an afterthought if your multiplayer experience is to thrive!