Why Scrum is fundamentally broken (but we still use it)

Scrum is one of the most used project management methodologies in the games industry. I know many companies that use it, including we ourselves at Ronimo. Most companies don't use pure Scrum: they change parts to fit their needs. This is so common that there is even an official name for it: ScrumBut (as in: "We use Scrum, but ..."). Any Scrum purists I have met are strongly opposed to ScrumBut, but when talking to them I learned that their views are so rigid and stubborn that they don't understand that one of the core concepts of Scrum is fundamentally broken when used for game development. The purists I met simply deny the problem and come up with fancy crap like this:

"ScrumButs are reasons why teams can’t take full advantage of Scrum to solve their problems and realize the full benefits of product development using Scrum. ... A ScrumBut retains the problem while modifying Scrum to make it invisible so that the dysfunction is no longer a thorn in the side of the team." (source)

Purists want you to believe that only pure Scrum works, but Scrum is fundamentally flawed and therefore not a full solution to project management. I think Scrum is still the best methodology around, but it is crucial to understand its problems and to not overestimate what it can do.

Scrum is mostly know for the stand-up meetings and the blackboards full of post-its, which are both great concepts. The big problem is in how Scrum relates to the overall process. Project management methodologies have many goals, like internal communication, empowering developers, facilitating innovation, communicating with stakeholders and the happiness of the team. Scrum is really good at each of these, but it falls flat on the face in the area that people most often associate with project management: meeting deadlines.



Scrum works with Sprints. A Sprint is a period of typically 1 to 4 weeks in which a certain feature or set of features is built. At the start of every Sprint it is decided which things to work on, and at the end of the Sprint the product is reviewed. Scrum deliberately asks the team not to look too far ahead: the core question when deciding what to work on during a Sprint is what the biggest improvement to the product is at that moment. During the Sprint this is then build and completely finished: Scrum emphasises that whatever is build during a Sprint should be finished and potentially shippable by the end of the Sprint.

This idea of having a fixed period of development and then evaluating what is most useful to work on next is a very good concept for game development: it is often impossible to really know what works until you are playing it in game, and once you are playing it you will learn all kinds of things about what the game actually is and where to bring it next. Regularly evaluating what works and what does not is really important for making good games and Scrum facilitates this really well.

The fixed period of a Sprint also manages to fix the big downside of constantly evaluating: it can be pretty frustrating if the plan changes from day to day and you cannot finish what you are working on because the plan changed again. Scrum forces that the plan cannot change during the Sprint, so it gives a certain amount of stability to the developers. This way they know where they stand during the Sprint.

The emphasis on re-evaluating what is needed next at the beginning of every Sprint brings up an important question: how do we know when the product will be finished? How to meet deadlines? Scrum has a short and simple answer to this: since every Sprint requires the things made in that Sprint to be finished, and since the most important things are always made first, the product is potentially shippable after every Sprint. The product gets better during every next Sprint and at the end of every Sprint it is again potentially shippable. Shipping dates are therefore always met according to Scrum: just stop developing at the deadline and release the product in whatever state it was in at the end of the last Sprint.

This is where Scrum fails utterly and brutally for game development, as this only works in theory.

Let's have a look at a couple of examples of why this does not work:

-To be able to ship a game on console it needs to abide by the console certification rules. These rules are things like showing an error if the harddisk is full when the game wants to save, and pausing the game if the controller is unplugged. Console manufacturers provide long lists of certification requirements. If the game fails to pass their tests, it cannot release. Since the number of requirements is so large, implementing all of them is a very big task. For a game to be "potentially shippable" after every Sprint (as Scrum requires) the certification requirements should be implemented first. Which is a horrible idea. One could in theory indeed start there, but in practice one really should not. Early in the project the most important thing is to get the core gameplay playable as quickly as possible and to set up the tools needed for the artists and designers, so that they can do their work. If the programmers start by focusing on certification requirements, then the artists and designers will waste several months with no game to work on. To enable the artists and designers as much as possible, gameplay programming should have priority and certification requirements should not be implemented until relatively late in the development period. In other words: the game cannot be 'potentially shippable' until very late in development.

-Many larger features also don't work with the concept of being 'potentially shippable' after every Sprint. Some things you either do entirely or not at all, like telling a story through cutscenes. Doing this the Scrum way probably means first very roughly animating all the cutscenes and putting them in-game. This is indeed a good idea for prototyping story flow early on. However, after that the next practical step is to just make each of those animations in turn. This means that halfway through that process, half of the animations are finished and the other half are still rough. Theoretically that might be a 'shippable product', but in practice you really cannot ship until all the rough animations have been replaced.

-Finally, the goal of game development is not "to finish a game" but "to finish a good game". Meeting the release deadline by just releasing whatever you have at that point is a pretty worthless strategy because it does not say anything about the quality of what you release. I can imagine that if the project has completely derailed and failed then you might want to release whatever you have and forget about it, but in all somewhat normal cases I am not interested in just "meeting deadlines": what I am interested in is "making quality games on time". Scrum helps a lot in making quality games, but it hardly helps in making them on time.

Absolutely baffling is that when I discussed this topic with a couple of official Scrum trainers, they simply denied that the problem existed. They claimed that if the game is not in a shippable state after every Sprint, then you are simply doing Scrum wrong. The above examples clearly show their error, but they just kept repeating that if Scrum is done right, the problem does not exist. This is why despite that we use Scrum and I like many aspects of it, I strongly dislike Scrum purists. Good thing not all Scrum trainers are like that!



Most project management methodologies are bloated with too many rules and too many claims of being able to solve almost any problem. Scrum is no exception, and Scrum's concept of ending every Sprint with a 'potentially shippable' product just doesn't provide a solid solution to the problem of meeting release deadlines. However, none of the other methodologies I have seen does this much better, so I think that despite this major flaw, Scrum remains the best project management methodology for games. Just be sure to keep in mind its flaws (and to always distrust Scrum purists).

The Maarten Principle: a golden rule of planning

I have been making schedules and work estimates at Ronimo for years now and although I won't claim that they are always correct, I have definitely learned a lot so far. One of the most important things I learned is what I like to refer to as The Maarten Principle, which I would like to discuss today. The Maarten Principle is named after Ronimo programmer Maarten who first coined this concept.

An important part of planning is trying to figure out how long it is going to take to build something. Scrum extremists might have you believe that you don't need to do this since your game is always finished and just getting better over time, but in practice there are a lot of things that you just have to do to make the game releasable. Implementing console certification requirements is a good example of a task that cannot be skipped, but usually there is also a vision for the game that just requires a certain amount of features and assets to make it a good game.

To have any idea how much work is left to finish the game, we need some kind of schedule. At Ronimo the core of the schedule is basically just a long list of everything that still needs to be done. We might change this schedule at any time, but we try to always keep it up to date with our current plans for the game.

To use this list to estimate how much work is left, each task needs an estimate of how long it is going to take to complete. Coming up with good estimates is the most difficult part of making a schedule, and we have gathered a bunch of tricks and guidelines through the years to help us make decent assessments. Unfortunately our estimates are still often not correct, but at least they are a lot better than they would be without these guidelines.

The most important of these guidelines is The Maarten Principe, the topic of today's blogpost. Its basic formulation is sarcastic:



The more elaborate way of saying the same thing is this:



The point of this rule is that when estimating a complex task that takes a lot of work (like for example implementing matchmaking in an online multiplayer game), it is impossible to make a decent estimate for the task as a whole. Two weeks feels like a long period in which you can do a lot of work, so for many big tasks it feels like it can be completed within two weeks, even if the feature in practice turn out to take much longer to implement. Distinguishing between tasks that take two weeks, three weeks or one month is incredibly difficult.

We learned this lesson the hard way. Halfway development of Awesomenauts we made a task overview with all the coding work we had left. This schedule contained a lot of large tasks (PS3 matchmaking, 360 matchmaking, bandwidth optimisation, performance improvements, menus, etc.) and in the end every single one of them turned out to take way longer than we had estimated in that overview. Development of Awesomenauts turned out to take three full years while we had planned this to be much less, so this was a huge mistake that back then caused our studio a lot of problems.

The problem with this schedule became very clear whenever we arrived at an actual task. Say for example we had estimated that implementing menus took 10 days of work (I don't know the exact estimate we had at the time, this is just an example). As soon as someone was to actually work on this task, we split it up into all the screens the menu needed and all the general systems we needed to make menu screens. We then estimated how much time each of these smaller tasks would take. It turned out that they added up to much more than the original estimation for the job as a whole!

This is a pattern that we have seen time and again: whenever a big task is split into smaller tasks, it turns out to take much more time than expected, often twice as much or even more. If we split up the task into smaller subtasks much earlier, we get a way better estimate of how long it will really take to implement. In our experience the problem is not so much that the task is too far away to give a good estimate, but more that the task is too big to grasp its complexity well enough to give an estimate.



Since I have so far mostly been responsible for the coding schedules at Ronimo, one might assume that this is just a problem with programmers (or maybe even just a problem with programmers at Ronimo). However, I recently helped our lead artist Gijs to make the art planning and I saw the exact same thing happening. We wanted to split several big tasks of dozens of days each into smaller tasks, and adding those all up immediately grew the total estimated time a lot. Just as it does with the coding estimates, splitting a big task into smaller subtasks immediately grew the expected time to finish it.

I still cannot claim that The Maarten Principle goes for any game studio, but I can definitely say that it does not apply only to programmers.

Based on The Maarten Principle we always try to split bigger tasks into ones that take at most 3 days to complete. At a maximum of three days per task it is still very doable to imagine all the things that need to be done to complete that task, and thus to make a decent estimate of how long it takes to complete. For tasks bigger than that, we try to split them into smaller tasks until each is at most 3 days. Of course this is sometimes impossible to do, as more information might be needed to have any idea how to split a big task. But in practice, most tasks can be split into smaller tasks quite easily.



Following The Maarten Principle does not grand one the power to make perfect schedules, but this rule does help us a lot in making better estimates!