Having constructed the Potemkin villages that are game environments, game developers just don’t see games the same way that the general public sees them. Every game developer I know has, at some point (and usually, often) sat around with co-workers, playing the latest and greatest game and critically ripping it apart for problems that are both easily seen and those that are more subtle. “You can totally see a seam on that texture.” “Oh, great, another FedEx quest.” “This UI is terrible.” “That cutscene sequence was dumb.” “Weapon X is totally worthless.” “This game is so unbalanced.” There are probably a million more complaints that you could come up with. Knowing what goes on behind the curtain makes us understand limitations of game engines, where corners have been cut, and seemingly inexplicable design decisions better than most.
When the Playstation 2 was released in the US, I sat in on our company demo of the new console. Partly due to the underwhelming launch titles, and partly just because programmers tend to be grouchy, we had plenty of fodder for complaint and discussion. But one of the more excited and forgiving souls in the room rolled his eyes and pronounced that, “Man, listening to you guys is like going to see a movie with film majors.” Truer words were never spoken.
I recently read this great blog post about Mass Effect (a game I have not yet played), in which another game developer points out some of the tell-tale signs of triage, content cutting, technical challenges, and design changes. In addition, there’s also some discussion of how these situations come into being. Having been a big part of a large, ambitious game project myself, I can vouch for a lot of the bullet points discussed, and wanted to comment a little bit further on things.
It is extremely rare for games to ship with the full feature set envisioned early on in the project — so typically, features and content are classified as either vital or non-vital. If, during development, a feature or piece of content is not working out, it can be cut if it is non-vital. The need to classify work in this manner early in the project’s life span, however, also begets a lot of the problems mentioned in that blog post.
- “Undue or odd simplicity.” Game mechanics that sound great in design discussions may not work out so well once they are later implemented. (For example, the boss battles in Guitar Hero III have been pretty much universally panned.) Since a lot of resources may have been poured into these ideas, the preference (rightly or wrongly) is usually to keep trying to make them work somehow, rather than just cutting them. These triage efforts may take the form of simplification, or providing alternate methods to “get around” the balky mechanic. The latter category includes the “you’ve failed this part 5 times, do you want to just skip it?” crutch that has become popular in recent games.
- “Significant differences between demos and the actual game.” Even if you do decide to cut a feature or some content, if you’ve ever showed it to the public or talked about it in the press, people are going to be wondering where it went in the finished product. And if you leave those vestiges on the shipped disc, you never know what might happen…
- “Sudden narrative leaps and loose ends.” Cutting non-vital story content and dialogue can still leave awkward gaps in a story, even if it doesn’t totally compromise it. Characters that simply show up or disappear without a trace, deus ex machina, and sudden jumps in location or disposition all tend to be signs that some non-vital content was cut and patched up in a low-risk fashion.
I also have to throw in some additional RPG-specific commentary on the “lots of bugs” category. RPGs tend to be games of large scope. They try to provide an experience that is more customizable (in terms of choices having an impact on gameplay) than almost any other genre, an engrossing and lengthy story, and interesting gameplay on top. If you imagine the gameplay experience translated into a directed graph of a player’s actions, RPGs tend to have very high branching factors compared to other game genres. This variability drastically increases the amount of test paths that must be checked, and even if you have a massive army of testers, it is extremely difficult to ensure 100% test coverage. I would go so far as to say that no modern RPG can achieve this. (The continual refinement and patching process of MMORPGs is a testament to this.)
An anecdote of the complexity of RPG games was related to me by a former co-worker at Obsidian. He told me that, several years ago at Black Isle, there was some discussion of adopting the then-new Dungeons and Dragons 3rd edition rules (widely viewed as very successful, and which simplified many arcane aspects of earlier editions) into a game that was already in development and using an existing engine. As part of the feasibility study, effort was put into determining how the combat system would need to be changed. One of the programmers diagrammed the combat resolution system on whiteboards, and wound up using something like 54 square feet of whiteboard space on an incomplete flowchart of the system, before the decision was made that it not feasible to adopt the 3rd edition rule set. The vast number of feats, spells, and items that exist in modern D&D take a massive amount of effort to implement, and oftentimes have subtle, unspecified, or ill-defined interactions with each other. Granted, as far as complexity goes, D&D is fairly high on the scale, but even simpler rules systems are tricky to get right.
Mass Effect is, by all accounts, not a short game, so I’m not surprised that it may seem buggy. But if game developers and publishers want to continue producing epic, complex RPGs, I believe that the tools for producing and testing gameplay content must become much better, and automated testing of gameplay scripting must somehow become a reality. Ensuring that changes and additions do not break progression in an RPG can sometimes be a very time-consuming process, because a bug may only affect certain character types, later parts of the game, or (worst of all) be randomly-occurring. To put this in perspective, for NWN2, we had testers who were “sprinters,” whose job was to simply go through the game as quickly as possible, to ensure that the “critical path” of the story was working and viable. If I recall correctly, “sprinting” through the game took about 20 hours. When you need a multiple-day test to really ensure that a change hasn’t caused another problem somewhere else, your iteration time and desire to make polish changes suffer.
One source of direction and inspiration is Inform, the interactive fiction authoring system whose latest incarnation uses a natural language system to construct gameplay. I’ll probably have more to write about Inform later — I think it’s fascinating and a tantalizing glimpse of the possibilities of higher-level gameplay tools.