1/10 games - Is it even possible?
A thread on Neogaf became pretty interesting the other day. People were submitting their lists of games they classify as 1/10 material.
There were two arguments, one being that a 1/10 can be whatever you want it to be, and another stating it had to fill objective criteria like be so broken that it cant be played without user modification, and other people going so far as to state that 1/10 games are games that actually harm your PC (like virus'es). Others just said a 1/10 is anything that had lots of technical merit, but was an emotionally frustrating, boring chore to play through (using Metal Gear Solid IV as an example).
If you were to judge a game as 1/10, what does 10% quality mean anyway? I'm asking the question because on Neogaf noone approached the question from a Developers/artists point of view, only as gamers. I'd like to hear some game developers views on the matter if possible.
There were two arguments, one being that a 1/10 can be whatever you want it to be, and another stating it had to fill objective criteria like be so broken that it cant be played without user modification, and other people going so far as to state that 1/10 games are games that actually harm your PC (like virus'es). Others just said a 1/10 is anything that had lots of technical merit, but was an emotionally frustrating, boring chore to play through (using Metal Gear Solid IV as an example).
If you were to judge a game as 1/10, what does 10% quality mean anyway? I'm asking the question because on Neogaf noone approached the question from a Developers/artists point of view, only as gamers. I'd like to hear some game developers views on the matter if possible.
Comments
Agonising over what 0.01% Oppinion Points represents to your audience is a waste of valuable dev time. Try your best to make something you're proud of and would be happy to buy/play yourself. Full stop. :P
As you pointed out in your OP the users of NeoGaf could not reach a consensus over what were the criteria of 1/10 game. This is the point that Gaz was going for: It is impossible to create an objective criteria for what a worst game would be! Games are subjective and trying to define a worst game is not really possible.
(Ok it is technically possible by a statistical sampling of what developer and players at large find to be bad along certain sets of criteria, but this again will loop into how do we define aspects of games as there will be things that do not fit neatly into a definition of those criteria)
---
But to answer your question via your question by an observation of how metacritic rates games. Games are rated on two parts: 50% for technical competency and 50% emotional enjoyment. So I take a metacritic score of a game minus 50% and times it by 2 and to me that would be the rating if the game were a film. Now this is a rough and rather poor means, but for my personal musings over it I find it to be good enough.
Now given that 50% is given to a game that simply runs, how do we get a game that is so glitched that it would not even run? Well to my mind the most logical answer would to for there to be a game which must be compiled from source code and cannot be complied. So the technical standard is so low that you cannot even create a playable game.
---
But now I must ask the same question as @Tuism did. Why is this discussion relevant? What are you trying to gain by having it. Since I believe any method of trying to analytically measure games will just yield answers similar to what @Gazza_N gave: games are subjective and as such we cannot provide a good means of giving a numeric rating for them.
So why did I answer the question?
Well I find it part proof by contradiction, part criticism of videogame rating, and part satire of metacritic in general.
Ultimately it's about perception of quality. A score of 0.251%-out-of-10 pointstars is intended to communicate quality. What is quality? What individuals say it is. Hence "cult classic", "best game no-one played", "forgotten gem", arguments in the comments over "omissions" from top-ten-gaemz-of-all-teh-tiemz-evrar lists, fandom wars, your GAF thread, and all the other exciting artifacts of people judging the quality of entertainment differently.
Basically, you give me a formal definition of the ISO standard unit for quality, and I'll tell you what 1/10 means. :P
EDIT: Karuji also raises the technical performance of a game above. Barring games that don't launch at all (at which point they can't be meaningfully classified, objectively OR subjectively, because for all functional purposes they don't actually EXIST), people also have subjective tolerances for crashes, flickering textures, looping sounds, blah. Too many variables, man. Too many variables. ;_;
Gazza also reclarified his stance, that an objective measure of subjective interpretation is futile. It sounds nice, and I’ll be the first to admit that both of you are certainly more eloquent than I with your command over the English language. I’m not sure either of you are correct in your views though.
I personally think the time I use to think about this I could be using to make a better game :)
You don't have to share your views if they are just echoing whats already been said Tuism, but if you have something meaningful to contribute I would appreciate it.
Also, say I upload a horror game and ask people to rate it. People who don't like horror games at all will likely vote lower than those who do (granted it's any good). How do you solve this? You cannot simply tell one of the two groups not to participate. This still won't be objective. Asking people to look past their personal preference and rate the game on technical merits also doesn't make sense, because some games are a lot more than just the technical side. I mean, when you read a book, it can put you off if it's written badly, but provided that it's written well enough, the main criteria would be, How good is the story? You aren't going to tell your friends to "read this book, the story sucks and is boring, but I didn't even get one spelling or grammatical mistake". And then the question to be asked is, will each person rate the story equally good? Is there such a thing as an objectively good story? I bet you will always be able to find an audience that would be in disagreement about what story is good or not. And I think you can apply the same to games.
Hence, I do not think objectivity is always worth aspiring to.
I could likely compile quite a few examples of where ratings — metacritic scores — are flawed or meaningless. But I'm trying to be succinct here ;)
Now one might think that this only applies to emotive games or ones that aspire to create a certain feeling. But all games fall into certain aestetics even COD gives satisfaction for one's ability to make a skilled shot.
How do you find an objective means to measure what will always be emotive. Yes we can look what occurs beneath the hood of the game while we play it, but that only furthers an understanding.
The contradiction lies in this nub. Analysis is cold and clinical. Art is emotional, for this case warm. How can one analyse what is innately emotive without feeling it? Without being objective with a rating that rating has no meaning. So then what is the point of a rating if it has no meaning?
You could base it on the same "fitness for use" criterion that producers of physical goods do - does it fulfil the purpose for which it was designed? From a pure software standpoint, that's easy enough to answer, I guess - it launches, is largely stable, reproduces source assets in a functionally and aesthetically correct (subjective) manner, is systemically sound in its application of the game rules, has an interface that's clear (subjective) and consistently functional.
The threshold beyond which these can be deemed "fit for use" is highly subjective though, since there's no standard or defined metric by which we can measure all these factors other than personal opinion. And that only gives us part of the story. Whether a game runs is irrelevant - a user likely (see? MORE SUBJECTIVE UNCERTAINTY) isn't primarily interested in whether it launches or not, or whether the textures judder occasionally. They're interested in the unquantifiables of the game's implementation.
Those unquantifiables stem from an artistic and mechanical implementation standpoint, and there be dragons. We're lost. Citation: any forum thread or comments section related to any given game in the history of the Internet. Taste in visual aesthetics generally causes issues (right down to friggin' pixel count). The sound and music may not be "good enough". The narrative (or one of its many, many component parts) may not appeal to the player. Whether it even HAS a narrative will factor into a player's enjoyment. And that's not even getting into the rules and mechanics of the game itself and opinions over how well it's been implemented. Movement speed may be off according to one player, perfect for another. The same weapon is OP or totally balanced depending on who you speak to. Certain rules "aren't fair" for some people. The AI's awesome. The AI's terrible. This costs too much. No, it's perfectly priced, shut up. That spell's hax. No it's not. Level flow is terrible here. Level flow is amazing here. This game's amazing! No it isn't, OTHER game is amazing!
You can try to tie a number to qualify the warm feelings you get playing a game, sure. You can even break it down into the warm feelings you get from the graphics or audio or whatever, but that number will NEVER apply to everyone, because everyone's tastes are different, and everyone's interpretation of relative quality is different based on those tastes. Trying to measure these aspects objectively using the nonexistent SI Unit for Universal Narrative Enjoyment or ISO Standard Art-style Like-itude Quotient is not possible in any way.
Emotion comes from three ways I’ve identified.
1) Acknowledgment of relatable life experience (sports sim for sports lover)
2) Stimulation of the senses (audio visual experience for non-mechanically focused gamers)
3) Power of the mind (emotion through amount of invested effort, i.e working towards a high score)
Now, both Karuji and Gazza have echoed your argument, that emotion is inherent in games, and leaving it out for cold, critical analysis is pointless as games are experienced. Does that mean we shouldn’t try understand emotion though? I’ve just broken it into three main categories, all of which operate in the realm of video games. Sure I might have to acknowledge that our own experiences of emotions are also different, i.e some feel more guilt than others for the same actions, but I don’t believe that argument means we shouldn’t try develop a baseline of emotional experience. Every day millions of people experience the reality of shared emotions, otherwise chain mail with emotional topics (and the Facebook variant) wouldn’t have become so popular.
Re: Technical analysis. (and partly a response to Gazza), here’s how I would define a quality system for games.
Establish a hierarchy for emotional experience (based on the main ways emotion is experienced) and work from there.
But it is reasonable to imagine, that if our world is (essentially) deterministic, an algorithm could be devised (in the future) to choose the best game for a particular player (given enough information, processing time etc.) So in principle then, by this standard, quality can be quantified. Once we have a bot like that for an individual, we can construct other measures of quality (such as average over human population). We can also work in other useful properties, such as surprise (the bot can be made to randomize its results somewhat; the more random, the more surprise), or "investment utility" (best buy for given amount of money).
Of course, the psychology and AI people have a long way to go, so for the time being, I won't take numeric evaluations of games too seriously.
You could conceivably measure emotional reaction in individuals, sure. Build a statistical model for a large population? Sure. But applying it to an absolute rating system that is universally applicable isn't possible. The best you can do is a variant of the system Herman posits, which samples an individual's tastes and works from there, doing individual ratings based on past behaviour (which isn't necessarily a reliable basis for prediction).
Suggesting that a game element can be engineered to consistently invoke, say, 13.9 Horror Units for all conceivable users at all conceivable times is, in my humblest of opinions, patently ridiculous. We know this just through discussing horror games we've played with friends. And what about desensitisation? What scared me, or awed me, or touched me (assuming I experience these emotions at all) the first time around may not again, or may scare/awe/touch me even more, depending on how my brain rewired itself according to that stimulus. How would you predict that? How would you adjust your measurement methodology for that?
I keep coming back to the same point - there are too many wildly divergent variables, in too many permutations and combinations, and they're impossible to predict or quantify reliably enough to formulate a universally applicable metric, which is the very point of having an objective scale of measurement. Also, game reviewers will have to wear ECG and EEG gear, with frequent blood tests, and I don't think they'd like that. :P
I'd like to touch on something Karuji also said earlier 'The contradiction (of art and analysis) lies in this nub. Analysis is cold and clinical. Art is emotional, for this case warm.' I understand where you're coming from, but I think you're not looking at all the possible sources of emotion that can come from a gameplay experience.
Objectivity is a tricky thing, because you can't remove human experience from any equation where people are involved. Descartes, in his Meditations, speaks about how we can't separate ourselves from our experiences, and even if we could, that new viewpoint would still be experienced subjectively (Descartes isn't the most relevant or eloquent example, but a) dead white guy cred, and b) this idea isn't new at all, it's centuries old and isn't just the product of weird French postmodernists). Scales and rubrics are fine, but a human still has to make them, and another human has to interpret them in applying them. Even AI has to be programmed by a human, and will be informed by their experience and subjectivity. If we commit too fiercely to objectivity, then we get comment sections on articles collectively flipping their shit over 0.5 points of difference in review scores (take the Gamespot GTA V issue, where the comment section devolved into a cesspool of bigoted trolls because the reviewer commented on the misogyny present in the game).
So maybe objectivity isn't the thing we should be striving for so much as sensitivity to as many subjectivities as possible? Personally, I don't put too much faith into review scores, because they don't tell me much about the reviewer's experience playing the game, or really anything about the game aside from a numerical abstraction of how strongly the writer liked it. I guess what I'm trying to say is that video game reviews serve a worthwhile purpose, but barring game journalism's wholesale overhaul of their current MO, we ought to be aware and critical of the shortcomings of sticking numbers onto the ends of these things.
I'd also like to get a bit finicky about the cold analysis/warm art dichotomy, which I don't think is nearly as simple as that. This might be a mostly semantic issue, and my understanding of this is different because I'm a student of the humanities (see? Subjectivity and lived experience at play right here :P ). Analysis is (at least in cases where we look at something as a cultural artefact) performed by humans, and ends up being influenced by their subjectivity. Good research in the humanities is almost always prefaced by a statement of the analyst's potential biases, because elements of a text or image can be more salient to people from different backgrounds, or be interpreted differently based on the writer's identity. On the other hand, art isn't just a touchy-feely, totally unconscious activity. In general, art is iterative, and the final arrangement of its elements is more a product of design than intuition. The green light in The Great Gatsby isn't a happy coincidence, Fitzgerald placed it in particular scenes for a particular reason. So I don't think that we should necessarily keep art and analysis on the opposite ends of a spectrum, because they blend together, there's self-analysis in art and a lot of going-with-a-gut-feeling in analysis.
Well, this turned into a bit of a text wall and I'm not sure if it's going to be useful to anyone. Sorry about that :/
First I asked the question, is a 1/10 possible, and what would the criteria be? (Looking for some developer input).
First batch of responses - video games are so subjective, there's no real point in trying to be objective.
My reply - Maybe there is a disgusting level which all video games can achieve which makes them 1/10. Tuism asks me if I plan to become some type of video game cop (I was merely asserting that perhaps a 1/10 game could be a game which is indefensible by shared humans standards).
Twist - Karuji & Denzil bring up the idea of subjective experience, Karuji says 'How do you find an objective means to measure what will always be emotive.'. This challenged me, and I'd never thought to unpack the kinds of emotions people can experience while they game. (Side note: the last five years I've been obsessed about charting the mechanical and visual evolution of games - http://playerversuscpu.blogspot.com/, and to these ends have tried to put aside all feelings of nostalgia and non-interactive stimulation when doing my work). Putting aside these emotions has helped me so far, but Karuji's words have made me realize that my research lacks a degree of emotion, and maybe I shouldn't be so scared of relating my personal experience to the games on screen.
Soon after this you joined in, starting your views on review scores and the pro's and con's of the number system. I think you hit the nail on the head when you said this - 'maybe objectivity isn't the thing we should be striving for so much as sensitivity to as many subjectivities as possible?'. Very eloquent.
Maybe we can close the thread at this point? I've got a lot of good feedback here, and I appreciate the contributions of the above members.
But as a developer I really didn't like that it amounted to a cash in. It was the bare minimum of effort exerted to produce a product that could be sold. Whomever was making the decisions at Gearbox was prepared to burn fans and the Duke Nukem brand, and I presume their own team of developers, in order to make some cash.
If a score is numerically represented as a number out of 10. Then 1 out of 10 is the lowest whole number that isn't zero. Which is the bare minimum score. And that's what I'd give Duke Nukem Forever.
Also, I'm sorry if I'm entering this thread late, but in Silvaring's original post there is nothing about "objectivity", that appears to be a concept that Gazza_N introduced ???? Or has the OP been edited?
Personally I don't think the concept of objectivity is useful in discussions around art appreciation. But understanding how different groups of people relate to numeric values in relation to works of art can be useful (which seems to me to be what the original post was about).
I'm going to assume we're still talking about the game that players received when they bought Duke Nukem Forever.
In that case: That information would change my views somewhat on who is to blame (or who is the most to blame) for the state of the game when it launched.
But it would still be the same game. The same promises about the experience would still have been delivered. And the same shortfall in the quality of that experience would still be experienced by players.
My core of why I dislike Duke Nukem Forever so much as a developer is that I believe that in accepting someone's money you as a game developer are obligated to provide the best game possible. I perceive Gearbox to have shukked this responsibility.
This isn't something I take lightly. Game developers releasing poor quality games and obfuscating their quality to the consumer caused consumers to avoid video games in the 1980s. The great video game crash was causes by behaviour like Gearbox (and the other parties involved in Duke Nukem Forever) exhibited. I don't think we're headed to another crash, but heavily promoted shitty games have industry wide ramifications, and Duke Nukem Forever being shitty was no accident.
In your eyes then (I think), its maybe a more horrible sin to do that than to make a cheap looking, barebones game/demo with extremely limited resources. Its an interesting way of looking at things.
There probably are experiences I dislike more than Duke Nukem Forever, but you asked for what I thought was a 1/10 game as a developer.
I'm weighing up the game that is delivered against the game that was expected by the consumer. Personally I think a very basic barebones game is fine, so long as that's what the consumer is promised.
For instance Super Hexagon is a relatively minimalist game, but it is an excellent minimalist game for consumers who enjoy minimalist games. If Super Hexagon promised Skyrim to consumers, and charged $60, that'd be a problem. But instead most players feel they get a great game when they buy Super Hexagon.
As a player I don't enjoy the experience of Super Hexagon at all. But as a developer I think it's an excellent game.
I hope that helps clarify my opinion a little bit.
The developer sold the game on early access, but then never finished it and stopped developing it.
At least I think that's how the story went. I might be confused about which game that happened to.
I'm any case, accepting money and then quitting before delivering on your promises is something I feel is unacceptable.
I've never been a fan of this pre-order type culture, precisely because of stories like this. Shit can happen in peoples lives that derails projects, and then a few journalists latch onto a narrative and you forever get seen as a quitter or whatever other negative stereotype in the eyes of tons of gamers.
Remember what happened with Phil Fish? The guy became synonymous with a caricature of himself, tagged with the phrase 'eat a dick', and 'japanese games suck', even though with regards to the second phrase his colleague at the panel Jonathan Blow said worse things about Japanese games just seconds prior and didn't get the same amount of flak.
Maybe the Towns devs deserved it though? I'm not sure. In this sea of information, we pick up intermittent signals at best so its hard to tell.
Nevertheless I think Towns as a project is a big failure, whether the developers deserved it doesn't change that. The players who paid for it still got screwed.
Assuming the Towns developers didn't deserve it (sometimes shit just happens to you that you don't expect) I hope they are able to recover and can make good games in the future. I've heard of developers recovering from worse situations.
From what I gather in passing conversations Towns isn't the only Early Access horror story. I predict there is about to be a small backlash against paid access to games still in development.
Much the same way Phil Fish can have his reputation ruined through a few unfortunate events, Early Access is gathering detractors.
*drops mic*
(picks mic back up, places it on stand, slinks offstage)