GDC talk on Obsessive Compulsive Development now FREE on the Vault, WATCH THIS!
This is one of the best, most honest and moving, talks that I saw at GDC this year. You should all watch it, because Matt asked it to be free on the Vault so that you could.
http://www.gdcvault.com/play/1017963/Obsessive-Compulsive-Development-Retro-Grade
http://www.gdcvault.com/play/1017963/Obsessive-Compulsive-Development-Retro-Grade
Thanked by 1TheFuntastic
Comments
Very pleased to have watched this talk. Hope that other devs will find his hard-learned lessons useful.
Hearing him go through everything he learned during that process in his talk was really cathartic. I guess that's one of the reasons I've been rather hardcore about not wasting effort recently.
I think that it's far too easy for people to focus on their technical achievements or even to spend time listing bullet-point reasons to like a game that they're working on, only to find that they're not talking in ways that help people relate to their game. For instance, I can talk about Desktop Dungeons by saying "Random Level Generation!" and that being a huge feature, but that doesn't mean shit to a random player. What I should be saying is "Never get bored playing the same thing twice! Infinite replayability!"
We went on a sales/marketing course a couple of years ago that said take your technical bullet points and rephrase them as problems that you're solving for people. It seems like a really good thing to practice.
Even now working on my own games I'm finding it SO. DAMN. HARD. to not get mired in technical details. I suppose it's the fear of failure speaking - if you're technically competent you already know how to win with technology. But probably insecure about your ability to make decent gameplay people will pay hard earned cash for. So your mind sends you chasing technical dragons.
Still not really sure how to call that line between "whats totally necessary to figure out" and "self indulgence because you like making complicated things work."
What I want to know, is why, if he was taking the game to conventions and PAX etc. did he not listen to anyone?
There can't have been no-one who said: "Yeah, the 3D graphics don't improve things for me. I'd rather have more songs."
I feel like his talk is missing a vital lesson he really should have learned. His concept of "we were making the game for ourselves" seems to me to be wrong-headed, and he doesn't exactly face that in his talk. The closest he comes is saying "Think how things impact on your players", but he really should have learned to listen to them (I'm not sure if that was implied, but he did seem to present the feedback from reviewers after the game went on sale as if it was the first time he'd heard it).
Listen to your players. Start up conversations with them and demand really harsh criticism. Pitch ideas to them and see how they respond.
(Obviously that won't help with initial concepts or far out ideas, but doing the right amount of polish, and polishing the right things are relatively trivial decisions, just ones that developers are always too close to their own product to evaluate).
People are far more likely to go "Aw man, this would be so awesome if there was a big boss battle in it!" and then you write that down and grin because you've already been planning something like that. A short play session, especially when you design a game to be short-play because you're showing it on an expo floor, is rarely going to hit people's frustration points with anything other than interface and core gameplay loop problems. People would pretty much just assume that there was more content in the game because they're only seeing a limited cross-section right now anyway.
In fact I was surprised by some of the Amaze responses and have reassessed a few things as a result.
But even if expos are a terrible place to discuss your game, which expos other than Amaze might be, that doesn't make not getting any feedback about things like "10 songs is not a lot of value for a game of this type" during the development more excusable. There are loads of avenues developers have towards getting useful feedback early.
I don't think what he proposes about thinking about "how features impact on the player" is nearly as good a solution to his problem as talking to an actual player. Or at least, it's a good thing to do, but on its own it doesn't address the full extent of the problem. Because when you're thinking about what a player wants it's still through murky goggles covered in the emotional splooge of days/weeks/months/years/eons of development.
I am a big proponent of open development methodology of course. I feel that open development is nearly always the best path for Indies. The Retro-Grade guy was pretty open about trying to apply AAA development practices to Indie Development and admitting that AAA practices came up short. Getting more feedback is certainly something I would put emphasis on in a talk like that, but that's me.
So maybe getting more feedback is something he intends to do, but for whatever reason doesn't feel it's a major enough thing to include in the talk.
I'm just saying that feedback on length-of-play constraints is weird to get at an expo because it's already a given. You're trying to create the feeling of people wanting to play more, but they can't. At best, someone's going to ask you how many songs there are in the game... So I think I can understand why he might not have felt that player feedback is the best way to discover that problem - at least, not convention feedback (which presumably found all sorts of other issues).
For length and other complaints, you need full playtesting. Which I totally think everyone making a game should have. Obvs.
The type of conversation I find that happens a lot goes like this:
Player: So is *Game X" still in development.
Developer: Yes. We're planning to work on it a whole lot more.
Player: How many songs do you intend to include.
Developer: Just the ten songs you see here, but we intend to create challenges so you can play through them 100s of times.
Player: Useful feedback.
I don't think expectations for features, or appeal of features, are particularly hard to gauge. At least to a limited degree of accuracy, and it requires that you ask the right questions and speak to someone who isn't an idiot.
Assuming the game isn't a new genre unto itself and players can't predict their responses, in which case you just have to make it yourself before it can be tested.
Obviously full playtesting is best and developers should all do it, and not just right before release. That's where the most accurate most complete feedback is going to happen (like you say).
I learnt this the hard way with Pocket RPG. Because the game sort of grew as we went along, and it was always 2 months from completion, some of the features designed early in its development made no sense upon release (the fact that every quest reset you to level 1 with no weapons is really the feature I'm thinking about it).
We'd done some full playtesting, but only with the developer present. We saw the problem, and tried to smooth it out (as that was less effort than redoing all the stuff). But on release, when strangers played the game, it was revealed how serious a design flaw this was. And by then it was too late.
We needed to gauge reactions much much earlier. And probably be much more sensitive to the feedback we got.
Currently I try to build vertical slices of things and test ideas with strangers before we infest hugely in anything. Not sure if I'm doing it right, but certainly better, and it really helps to be doing this on PC.
Long story short: I'm not convinced that the 10 song choice, or the expensive 3D artstyle choice are problems so difficult that needed full playtesting to evaluate.
But I agree with you that certain problems basically do need full playtesting. Particularly the overarching design choices, like story and meta-game.