Robo Maestro is coming to Steam!

Play with music like never before! Robo Maestro is the 'game' I'm currently working on and it's a procedural music toy. Guide an endless stream of new music or make detailed tweaks in Editor Mode. Create songs, be inspired and share your creations. Robo Maestro is coming to Steam, add it to your wishlist here!

Robo Maestro is a revolutionary new way of creating and experiencing music. You tell it what you want, and then the Maestro automatically generates music that fits that. Playing Robo Maestro requires no music knowledge at all: just have fun and hear what happens! Guide the Maestro and create awesome music together with it!

Here's how I made my computer write music

One of my current hobby projects is creating a procedural music system: a computer program that composes and performs music all by itself. It's still a work-in-progress in its early stages, but some recent improvements have gotten it to produce quite interesting music already, so I figured it's about time I share some of my thoughts around how it works. Let's start with a little video that lets you hear the amount of variation it can currently do:



Note that this video contains a selection of the nicer pieces, so it isn't all this good. Most of the other music it currently produces is a bit more boring, or too similar to these fragments. The program is a work-in-progress and it should improve a lot as I spend more time on it.

I started programming this for a rather odd reason: I wanted to practice improvising on my cello over complex chord schemes and figured a program that automatically produces those would do the trick. As usual with this kind of thing, it got out of hand and I've so far spent my time making it cooler instead of playing along to its tunes on my cello. I hope I can expand it to the point where it can be a game soundtrack, or a live streaming radio channel on Youtube that constantly produces new music.

My approach to procedural music is mostly based on my own ideas about composition. I'm deliberately steering away from a lot of common composition 'rules' since I expect that's been done already. I think I'll get something more interesting and more personal if I start from my own ideas instead. While I don't ignore music theory either, I do try to avoid rules like ending a chord progression with the dominant and then going back to the tonic or anything like that.

Just to be clear: my music generator is not an AI or a self-learning system or anything like that.

The procedural music generator consists of a bunch of elements that together produce the music:

Chords, rhythms and scales
To 'teach' my program basic music theory I've created a bunch of text files that define a lot of common and less common chords, rhythms and scales. You can see which it's using in the video: it might for example choose to play a C major chord in 3/4 rhythm.



Structure generators
The first thing needed to create a piece of music, is a basic structure. This part of the program randomly chooses the chord progression, the rhythm and the scales. It also chooses how individual instruments are allowed to deviate from the rhythm, so that there's some consistency between instruments without requiring them to do the exact same thing.

Instrument layer generators
These generate the notes for specific instruments. Each category of instruments has a role to fulfil and its own specific algorithm for that. For example, the generator for drums is completely different from the generator for bass. Most of these generators are still very basic so there's plenty of room for improvement there. A lot of the variation in the video above comes from having different combinations of instrument layer generators. For example, some have drums, others do not. I expect that having a lot of layer generators is where my procedural music will get most of its variation. Currently I have five, but I'd like to create at least a dozen. The most important missing layer at the moment is one for actual melodies, which is probably also the most challenging one to get right.

VST instruments
Actually performing the music is done using VST instruments. VST is a standard for digital instruments that is used by a lot of programs on Windows. A VST file is basically just a DLL and I've coded a simple music player that can handle them. For each instrument layer I'd like to have several VST instruments that can play that layer, so that I can get different sounds. Currently most of it sounds very digital, so it would be nice to also have VSTs for other types of instruments, like acoustic drums and guitars.

Song structure
This is an element that's currently completely missing: at the moment all music played by my music generator is simply a loop of 4 to 12 bars. I want to program systems for song structures with verse/chorus/bridge, but also for endlessly progressing music and for music that for example slowly builds up to a climax.

Carefully restricted randomness
All the systems in this generator contain a lot of randomness, but the randomness is limited to specific rules I've come up with. If those rules are too free then the result is noise, not music, while if the rules are too strict then the resulting music will always seem same, so carefully balancing the randomness is a big challenge here.



For future improvements I'd like to add more instrument layers and refine the current ones, add song structures and add more VST instruments. Since I've recently become a dad I don't have a whole lot of time to work on this, so I expect progress will be very slow. I hope to include a first basic version of my procedural music generator as a bonus feature in the home edition of Cello Fortress, whenever that's actually finished.

Let's conclude with a question: do you know any communities for open source or free VSTs? There are plenty of VSTs that can be downloaded for free, but in many cases they just specify that they're free to use, not free to distribute with commercial software. I've so far had a hard time finding usable instruments or getting into contact with (hobby) instrument programmers who might be willing to allow me to include their instruments. Leave a comment if you've got suggestions or are a digital instrument programmer yourself.

The level art tools for Swords & Soldiers II

For our new game Swords & Soldiers II (coming to Wii U on May 21st!) we wanted to create detailed and varied levels. Because of the number of maps and their large size we needed art tools that provide a lot of automation to be able to populate them quickly. At the same time we wanted to allow customisation and control over the details to create unique, designed places. Today I would like to show the tools that Ronimo programmer Machiel van Hooren developed for our artists. Check especially the awesome demo video at the end!

Our starting point was the level editing tools we made for Awesomenauts. Those are very flexible and not tied to a single game, so we could pretty much use all of them directly in Swords & Soldiers II. However, the shape of the levels in Swords & Soldiers II is very different: the Awesomenauts tools are built around rectangular boxes, not curving landscapes. Nevertheless they provided a really strong beginning. I made a short video montage about the Awesomenauts tools for a blogpost a couple of years ago, here it is again for those who hadn't seen it before:


The first thing we needed to add for Swords & soldiers II was curving terrain. The core gameplay takes place on a single line with little slopes and hills. So we made a new feature with which our artists place TerrainNodes in the level, and then the engine automatically generates a smooth terrain that goes through those TerrainNodes.

To calculate a smooth line through those points we use standard bezier splines. We also tried using slightly simpler Hermite splines, but we didn't like the look of those. Bezier splines usually come with additional control points for more control over the curvature. To simplify things we calculate the control points automatically based on the previous and next TerrainNode. This gives slightly less control, but that turned out to be fine for our artists and it makes the tools simpler and faster to use.



The next step is terrain layers. We wanted to add as much depth to our 2D art as possible. Having more terrain layers in the foreground and background is a good way to accomplish this.

The most obvious method is to just allow our artists to create more terrain layers by hand at different parallax depths. However, efficiency was also a goal, so we wanted to avoid creating all those layers by hand. Therefore we made it so that the parallax layers are copies of the main terrain. To be able to create some diversity we added settings for things like random noise, different textures and colour modifications. Not only is it faster to create levels with these copied terrain layers, it also means that if the main terrain is edited (and it often is), then the background and foreground layers automatically change with it. This makes iteration and modification much quicker.



The big limitation this creates is that the hills in the background are the same as those on the playground. Sometimes our artists wanted to break free from this pattern. We could have tried making the terrain tools more flexible to accommodate for this, but often the big background objects aren't hills, but an inn, or a wall, or something else. The solution was simple: the artists just placed a big texture of a hill in the background and that's it.



Plants, small vegetation and other props are all tied to the landscape. To help place those quickly we created ObjectDroppers. An ObjectDropper attaches one or more objects to a terrain layer, allowing an artist to drop a bunch of bushes or flowers on the landscape quickly. Props are dropped using a random seed and the artist can try different seeds to look for one that he likes. With a bunch of prop types a level can have its basic setting in place really quickly, after which the artist can focus on adding the unique flavours. Since the dropped props are tied to the landscape, modifying the terrain also automatically modifies the props, making tweaking the terrain that much faster.



Our level artist Ralph Rademakers even used ObjectDroppers to place bigger objects like trees. By working efficiently with these tools Ralph was able to dress up the simpler levels in just a few days per level, using the textures drawn bij Adam Daroszewski and Gijs Hermans.

The other main tools used for the level art are things we inherited from Awesomenauts: placing textures and animations in levels and recolouring them. You can find some good examples of the power of in-engine recolouring in this blogpost: Using 2D daylight assets to create a night level.

Now that I have discussed the tools, let's have a look at them in action in this little video:


I think the main lesson that can be learned from the Swords & Soldiers II level art tools, is that it's sometimes a good idea to limit control to make the workflow faster. The ObjectDroppers create a lot of objects at once, giving much less control than placing them by hand, but also being much, much faster. Similarly, the background and foreground terrain layers being copies of the main layer is limiting, but also very efficient. Where required our artists had enough other tools to break through the limitations by hand, resulting in a good balance between automation and old fashioned handwork.

Evolving level art from Swords & Soldiers 1 to 2

In Swords & Soldiers 2 we are stepping up the quality of everything compared to the original. Level art is one of the areas where we are improving the game a lot. With our improved tools we can make them look a lot more detailed and varied. We also approach level art in a very different way now. Today I would like to discuss how the approaches in both games differ and why we switched to different techniques.



In the original Swords & Soldiers the levels were mostly procedurally generated. The only parts that were placed by hand were the ground itself and the small props on it, like grass and flowers. The foreground and background layers were randomly filled with objects. Since this is essentially a 1D game I simply wrote an algorithm that placed mountains (or trees, or houses) at varying distances from each other. Each layer had a bunch of different textures and settings for how densely it should be filled with objects.



There are many arguments in favour of procedurally placing objects like that, but at the time only one really mattered: we didn't have any tools to place objects by hand. I was the only full-time programmer on Swords & Soldiers 1 and I had to build all the gameplay plus the entire engine, so there was no time to create such tools. I did have a couple of interns helping with programming, but in total there was just way too little time to implement any real tools for level creation.

Levels therefore had to be created in Notepad, as I previously described in this blogpost. This worked surprisingly fast and simple for our artists and designers, but of course precise control over graphics is not possible this way.



Our artists had a limited palette of options to differentiate levels and create varied places. They made six different art sets they could use for parts of levels: desert, snow, grass, rocks, jungle and cherry blossom. They could set a gradient for the sky colour and a second gradient to modify the colour of background props. They could set the weather to rain or snow, modify the density of background props, foreground props and clouds and place smaller props on the landscape by hand.

While all of these options together may sound like a decent set of tools to create unique levels, in practice this is not the case. Levels in Swords & Soldiers 1 quickly blend together and start to look similar. One reason for this is that levels often contain more than one environment type. They might for example vary between rocks and desert. An environment type always randomly varies between all the textures it has. The result is that in only a few levels we can have seen nearly all level art in the game. In fact, the very long Berserker Run level contains all six settings and thus contains nearly all level art the game has.



Another problem is that randomly placed objects may stand in a different spot in every level, but they all feel the same. No matter at what distance the same set of mountains is positioned, it always feels like the same place. This could of course be done better with a more advanced procedural algorithm and a smarter set of textures, but in practice procedural content nearly always suffers from this.

This problem even shows in a game like Diablo III, which has high quality procedural world generation at its core. Blizzard spent enormous amounts of time on creating sophisticated algorithms and tons of art assets for random world generation, and yet when I played through Diablo III it felt like I was constantly visiting the same places. Either a place is uninteresting, with some random trees and bushes, or it is unique with some special big props. The number of unique props in Diablo III is large but not infinite, so they start to feel repetitive quickly.

I have not seen a single game with procedural worlds where the visuals did not become repetitive really quickly. In that sense I am slightly afraid for No Man's Sky: the game looks mindblowingly amazing, but I can hardly imagine they solved this problem well enough to keep planet discovery visually interesting. (I sure hope they did though, because that game looks soooooo good!)

In Swords & Soldiers 2 we therefore concluded that we did not want to use this approach anymore. We also don't need to work around a lack of tools anymore, since we now have the impressive toolset we created for Awesomenauts. All of that is at our disposal for Swords & Soldiers 2 as well.

By placing most objects by hand our artists can now create real compositions. The way trees are arranged can create forests, small clearings and green fields. This way even without additional assets it is possible to create places that feel much more real and unique than any procedural algorithm can. Of course one could add procedural rules to create such places, but there are limits to how much can be done this way, especially because every rule that creates a specific kind of place can be used only a couple of items if repetitiveness is to be avoided. With a good artist there really are no limits to how much variation and uniqueness can be created. Most of the levels in Swords & Soldiers 2 are filled with art by Ronimo artist Ralph and he is a true master at creating interesting compositions and little micro-stories through object placement.



Another big difference between Swords & Soldiers 1 and 2 is that in 2 we have much bigger props. In the original game mountains and trees were relatively small, so you always saw a lot of them. In the sequel we have scaled them much larger, so you might see only one or two really large mountains in the background, instead of dozens. This makes it all feel a lot larger, drawing the player more into the world instead of looking at funny miniatures. At the same time this makes the backgrounds look less repetitive: if you don't see ten mountains at the same time, then it is also much less obvious that several of them use the same texture.

This is strengthened by another big improvement in Swords & Soldiers 2: there are many more unique assets for specific levels. For example, one level plays in a Viking golf court and there are golf related objects everywhere. These props are not used in any of the other levels, making this level unique and memorable. This way the player will remember more specific things than in Swords & Soldiers 1, where all the levels looked quite similar. A level doesn't even need all that many unique props: since background objects are so much larger now, having one really big prop can sometimes already define a level.



Another improvement that I would like to mention today is the art style. The original game had a very simple cartoony look, while the sequel is much more detailed and painterly. Our artists went for a style reminiscent of French animation and it works beautifully. Most of the level art in Swords & Soldiers 2 is drawn by Adam Daroszewski. He worked as a freelancer for us and did an amazing job at creating a painterly and coherent look.

I have spent most of this blogpost explaining how Swords & Soldiers 2's level art is better than the original. There is however also one big downside: it is much more work to make. With the lack of proper tools and the simple style we had when we made Swords & Soldiers 1 it took very little time to decorate more levels. In part 2 everything requires much more work to do. It really shows, but it is an important trade-off to be aware of. Not having proper tools means that there are a lot of things that artists simply cannot do and this saves huge amounts of time.

Limitations like that are a very effective way of limiting the amount of time it takes to make a game. The better your tools, the bigger the risk of using them too much. We have already spent much more time making Swords & Soldiers 2 than originally planned and the game isn't even done yet. At the same time it is looking so good that spending so much time is totally worth it. I hope that when the game launches you will be as enthusiastic about it as we are! ^_^

In conclusion, I would like to say that I think procedural level generation is great for games that evolve around it, but for a designed experience having hand-made levels works much better. Levels in a game like Swords & Soldiers 2 look better when made by hand than when generated.

Designing levels without tools

One of the most interesting challenges I constantly meet when developing games, is to make cool things with a minimum effort. For Proun I am all alone and at Ronimo Games we have relatively too few coders to really work comfortably. So getting the maximum result out of a minimum effort is incredibly important.

When we built Swords & Soldiers, we didn't have a level editor yet. We have our own 2D engine at Ronimo Games, the RoniTech, but the RoniTech 1 that is used in Swords & Soldiers was still very bare bones at the time. We are currently working with and on the RoniTech 2 and have lots of cool in-game level editing and graphics editing tools, but for Swords & Soldiers we had to come up with some smart workarounds to design level graphics without having a real tool to do it with.

The core information of a level is where goldmines and towers are, what the height of the landscape is, and what type of ground it is (rock, snow, grass, etc.). We made designing this possible in a crude and simple way: in a Notepad file, each line contains information about one of these categories. Since the game essentially plays on a line, this is actually very simple to understand and edit and worked really quick for our designers.



To give our artists some room to create places of interest in the level, I gave them the possibility to define the exact position of props as well. They had a couple more lines for this and a prop sheet with a character per prop. By typing these characters in Notepad, they placed props in the level. This has an incredibly hardcore hacking feel to it, but I simply didn't have the time to create anything better (creating an entire console game with only one full-time programmer is, euhm, interesting...). It did give our artists enough freedom to make some fun areas, though, so the results were good.



The final part of the levels is the backgrounds (and foregrounds). We have lots of parallax layers and each layer is at a different distance from the camera. Also, the background layers often contain loads of objects per level. Since perspective is involved here, Notepad was out of the question. And without a graphical tool to design that, we had to choose a different solution here.

We came up with a procedural system where artists would create background elements for different layers, and the game would place these elements at random positions in their layers. Artists described to me which layers they wanted (nearby trees waving in the wind, medium distance hills, far away mountains, clouds, etc.) and I created code to place them. I did give our artists the possibility to remove layers or make them extremely dense, so that they could create deserts, mountain areas and forests.



The obvious downside of this approach is that it is very difficult to create unique places per level. With a real graphics level editor, artists could have placed specific eye-catchers in certain levels, like a city in the distance, or a palace just behind the field of war. This was impossible with the lack of tools for Swords & Soldiers.

Since we have a limited amount of background elements to generate levels from, repetition looms. So the artists requested two forms of control over that: one is simply a background gradient per level. The other is an atmospheric fog gradient, also per level, that is overlayed over the objects that are further in the background.



Despite the limitations, we managed to create a game with rich graphics that got great review scores. The smart choices described above made it possible to achieve that with only one programmer, which I am really proud of. Obviously, though, the real credit goes to our art team, who made some really compelling artwork. To quote IGN: "One of the best-looking games on Wii."

(Did I already mention I feel very proud of Swords & Soldiers? ;) )



PS. For those interested in some of the most hardcore programming topics: Jelle van der Beek, lead engine programmer at Vanguard Games (formerly known as W! Games) made a cool tool to analyse memory allocations and memory fragmentation. Although I have to admit that none of the problems he solves with this tool are relevant for the scale of games that we make at Ronimo (even though we do make real multiplatform console games), he shows some clear examples of the complexities of the topic, making this a very interesting presentation indeed for hardcore coders: Heap Inspector Demo video