Sunday, 23 March 2014

Why Scrum is fundamentally broken (but we still use it)

Scrum is one of the most used project management methodologies in the games industry. I know many companies that use it, including we ourselves at Ronimo. Most companies don't use pure Scrum: they change parts to fit their needs. This is so common that there is even an official name for it: ScrumBut (as in: "We use Scrum, but ..."). Any Scrum purists I have met are strongly opposed to ScrumBut, but when talking to them I learned that their views are so rigid and stubborn that they don't understand that one of the core concepts of Scrum is fundamentally broken when used for game development. The purists I met simply deny the problem and come up with fancy crap like this:

"ScrumButs are reasons why teams can’t take full advantage of Scrum to solve their problems and realize the full benefits of product development using Scrum. ... A ScrumBut retains the problem while modifying Scrum to make it invisible so that the dysfunction is no longer a thorn in the side of the team." (source)

Purists want you to believe that only pure Scrum works, but Scrum is fundamentally flawed and therefore not a full solution to project management. I think Scrum is still the best methodology around, but it is crucial to understand its problems and to not overestimate what it can do.

Scrum is mostly know for the stand-up meetings and the blackboards full of post-its, which are both great concepts. The big problem is in how Scrum relates to the overall process. Project management methodologies have many goals, like internal communication, empowering developers, facilitating innovation, communicating with stakeholders and the happiness of the team. Scrum is really good at each of these, but it falls flat on the face in the area that people most often associate with project management: meeting deadlines.

Scrum works with Sprints. A Sprint is a period of typically 1 to 4 weeks in which a certain feature or set of features is built. At the start of every Sprint it is decided which things to work on, and at the end of the Sprint the product is reviewed. Scrum deliberately asks the team not to look too far ahead: the core question when deciding what to work on during a Sprint is what the biggest improvement to the product is at that moment. During the Sprint this is then build and completely finished: Scrum emphasises that whatever is build during a Sprint should be finished and potentially shippable by the end of the Sprint.

This idea of having a fixed period of development and then evaluating what is most useful to work on next is a very good concept for game development: it is often impossible to really know what works until you are playing it in game, and once you are playing it you will learn all kinds of things about what the game actually is and where to bring it next. Regularly evaluating what works and what does not is really important for making good games and Scrum facilitates this really well.

The fixed period of a Sprint also manages to fix the big downside of constantly evaluating: it can be pretty frustrating if the plan changes from day to day and you cannot finish what you are working on because the plan changed again. Scrum forces that the plan cannot change during the Sprint, so it gives a certain amount of stability to the developers. This way they know where they stand during the Sprint.

The emphasis on re-evaluating what is needed next at the beginning of every Sprint brings up an important question: how do we know when the product will be finished? How to meet deadlines? Scrum has a short and simple answer to this: since every Sprint requires the things made in that Sprint to be finished, and since the most important things are always made first, the product is potentially shippable after every Sprint. The product gets better during every next Sprint and at the end of every Sprint it is again potentially shippable. Shipping dates are therefore always met according to Scrum: just stop developing at the deadline and release the product in whatever state it was in at the end of the last Sprint.

This is where Scrum fails utterly and brutally for game development, as this only works in theory.

Let's have a look at a couple of examples of why this does not work:

-To be able to ship a game on console it needs to abide by the console certification rules. These rules are things like showing an error if the harddisk is full when the game wants to save, and pausing the game if the controller is unplugged. Console manufacturers provide long lists of certification requirements. If the game fails to pass their tests, it cannot release. Since the number of requirements is so large, implementing all of them is a very big task. For a game to be "potentially shippable" after every Sprint (as Scrum requires) the certification requirements should be implemented first. Which is a horrible idea. One could in theory indeed start there, but in practice one really should not. Early in the project the most important thing is to get the core gameplay playable as quickly as possible and to set up the tools needed for the artists and designers, so that they can do their work. If the programmers start by focusing on certification requirements, then the artists and designers will waste several months with no game to work on. To enable the artists and designers as much as possible, gameplay programming should have priority and certification requirements should not be implemented until relatively late in the development period. In other words: the game cannot be 'potentially shippable' until very late in development.

-Many larger features also don't work with the concept of being 'potentially shippable' after every Sprint. Some things you either do entirely or not at all, like telling a story through cutscenes. Doing this the Scrum way probably means first very roughly animating all the cutscenes and putting them in-game. This is indeed a good idea for prototyping story flow early on. However, after that the next practical step is to just make each of those animations in turn. This means that halfway through that process, half of the animations are finished and the other half are still rough. Theoretically that might be a 'shippable product', but in practice you really cannot ship until all the rough animations have been replaced.

-Finally, the goal of game development is not "to finish a game" but "to finish a good game". Meeting the release deadline by just releasing whatever you have at that point is a pretty worthless strategy because it does not say anything about the quality of what you release. I can imagine that if the project has completely derailed and failed then you might want to release whatever you have and forget about it, but in all somewhat normal cases I am not interested in just "meeting deadlines": what I am interested in is "making quality games on time". Scrum helps a lot in making quality games, but it hardly helps in making them on time.

Absolutely baffling is that when I discussed this topic with a couple of official Scrum trainers, they simply denied that the problem existed. They claimed that if the game is not in a shippable state after every Sprint, then you are simply doing Scrum wrong. The above examples clearly show their error, but they just kept repeating that if Scrum is done right, the problem does not exist. This is why despite that we use Scrum and I like many aspects of it, I strongly dislike Scrum purists. Good thing not all Scrum trainers are like that!

Most project management methodologies are bloated with too many rules and too many claims of being able to solve almost any problem. Scrum is no exception, and Scrum's concept of ending every Sprint with a 'potentially shippable' product just doesn't provide a solid solution to the problem of meeting release deadlines. However, none of the other methodologies I have seen does this much better, so I think that despite this major flaw, Scrum remains the best project management methodology for games. Just be sure to keep in mind its flaws (and to always distrust Scrum purists).