Supercell was a noboday and now are huge and they came into what looked like an already saturated market as well. The samething is being said about facebook and Pretty Simple broke in with a super hit from nothing last year with Criminal Case.
This is actually what I'm warning against. I don't think holding up Supercell or Pretty Simple as examples of what to do is useful without knowing how many other companies were trying exactly the same thing and didn't make it. Don't try to emulate a luck-based success story, it won't work.
@dislekcia on your look at failures idea, good yes, but you have to compare with successes to understand where they failed as well. But there is also another school of thought that says look at companies that never do well who suddenly do better than they have done before. This means they have hit a cord with a market that exists and can be designed for.
Or they got lucky. Sometimes there really is a cause there, but I'm less and less convinced that that's actually the case in the mobile/F2P space.
Also, I mentioned the Red Bouncing Ball thing because that's just a tutorial template someone didn't do any more work to and posted to the app store. Apparently it did well, waaaaay better than a template should, by rights. Especially considering that the person who posted it did 0 design work... That's also relevant to the reskinning apps/games post that started all of this AND the question of "is it all just luck"? I mean, if it wasn't luck, then why did this particular template not succeed previously? It had all the same design elements that were cited as being useful in place back then already...
I'm worried that people aren't considering the impact of external factors, like what if this whole Flappy Bird and Red Bouncing Ball thing is just what it looks like when someone finds a new exploit for bots to farm downloads and reviews on the App Store? If that's the case, then mining these games for game design information to learn and try and implement skillfully in our own games is a complete waste of time.
I'm a bit out of touch from IOS. But when I was concerned about it what I did was look at all the new releases and try predict what their successes or failures would be like. When a big success like Flappy Bird came along, if it actually surprised, me then I'd try figure out why it succeeded.
Likewise, when a game's success was surprisingly disappointing, then I'd try figure out why it didn't succeed.
The mobile environment provides far better information about financial successes/failures than the PC marketplace, and vastly better information than consoles. (Look at sites like http://www.148apps.com/top-apps/ or http://www.appannie.com/ ). I think there is a lot of randomness in crafting a successful game (because making games is super hard), but accurate hit/flop prediction on mobile is well within human grasp. I don't think it is a case of games getting lucky, except where there are major unpredictable events (like a PewDiePie video).
I'd recommend not looking at notable failures (or developers who lamented failing but actually did really well like Gasketball), or notable successes... but rather look at ANY game which has a degree of success/failure that you wouldn't have predicted. The mobile environment tells you about all the new releases, you don't have to wait for a noteworthy story to analyze market success. It's a much better exercise if you do it without being guided by journalists.
If the success or failure of a game surprises you at all, that means you don't understand something about the way the market works.
(I mean, rationally, whenever you are surprised by an event it means that you didn't have enough information/understanding beforehand)
I'd also just mention that I think the mobile space is less external event driven than the PC/Console space (it is widely accepted that journalism has a smaller impact on the success of mobile games than in other markets, I'd assume that the same logic follows for other external factors).
I think it's less likely that external factors are playing a stronger role than I think, than I do not understand all the factors that drive commercial success in mobile games (because I have not actively been studying the marketplace and so have a very weak grasp on what mobile consumers want right now).
iPixelPierre said:
Could this not simply be attributed to Consumer perspective / Demand at different points in time?
@BlackShipsFillTheSky: That sounds like a much more informed way of studying the App Store. I only picked stuff like GasketBall because it was the only game I could think of at the time, so yes, monitoring new lists and trying to predict would be a good start. Thanks for the ideas :)
@Bladesway: No, that's not what I was talking about at all. I was pointing out how a zero-effort template put up by someone with no reputation fits into your model of game quality driving sales. I don't think it's anywhere near as clear cut or powerful a driver as you seem to think.
Could this not simply be attributed to Consumer perspective / Demand at different points in time?
How do we do that simply? Consumer demand/perspective is a complex thing, driven so many variables that there's almost certainly no clear-cut way to predict it. If there were, people would already do that with the stock market... And yet the best evidence we have so far indicates that complex feedback systems like the stock market (and I would argue the App Store is similar) have a huge random element to any success in them. That's what I'm getting at: How much random is there and how do we actually plan for that sort of stuff?
The advice I got long ago was "release on PC first, then do mobile once people recognise the game" and that's good advice. It's also not advice that has anything to do with Flappy Bird or Red Ball Spike Bounce, Angry Birds, etc.
@dislekcia lol, well if I knew all this stuff for sure I would be super rich and roling in dough :P but these are the assumtions I build on bases on what I read, hear in talks and ppl around me in the faith that if my design is good and has justified market appeal and I manage to hit the correct timing, that if all the stars align to make the market take note, my game won't just fade into the abyss along with so many other short lived successes. :)
Could this not simply be attributed to Consumer perspective / Demand at different points in time?
@dislekcia I also thought "Simply" was a poor choice of words. I suspect the iPixelPierre mean "Simply" in the Occam's Razor sense (as in simpler than assuming the App Store is even more complicated by explaining the difficulty of predicting success/failure on extra external forces).
I'd agree that the App Store consumer demand probably looks like it behaves similarly to a stock market. It also has feedback systems. I honestly don't understand the stock market well enough to compare them off hand.
If you assume that App Store success is somewhat predictable. Let's say more predictable than the stock market (though less well studied). Then the big problem from a game developer perspective is that we can only actually experiment with trying to play the App Store if we can craft games with less margin of error than the error margin for success.
Even assuming a luckless App Store, that doesn't make a guarantee of success. That still will be incredibly risky for most developers if they're attempting a flappy-bird success. Probably risky for even the best developers for a flappy-bird. Though flappy-bird style success is the most extremely risky form of success on the App Store, and I know you don't have to be number 1 on the charts to make a lot of money.
(I don't mean to sound like trying make a Flappy Bird is easy/safe. Flappy Bird not like something like Ridiculous Fishing which has far more avenues of appeal, but obviously Ridiculous Fishing is vastly more difficult to develop. I guess I'm trying to say that the simpler your game, the more perfect your understanding of the market has to be, and I'd concede that this possibly extends beyond human ability, but then I couldn't prove otherwise anyway)
I know I said studying only noteworthy examples of games is very limiting in terms of the information you receive and the context of that information... but... I thought Cut-The-Rope was an interesting case.
Chillingo predicted that Cut-The-Rope would beat Angry Birds (which was completely dominant at the time), well before the release of Cut-The-Rope, and they pushed a lot marketing behind it, and the game really did dominate for a while. Obviously Chillingo didn't develop it, and chose it instead of a lot of other hopefuls, but they DID know the App Store well enough to call it.
I thought it was interesting at the time, because I didn't call it. At the time I thought Cut-The-Rope wouldn't make number 1. (I was biased in favour of games with very open ended puzzles, like Angry Birds, I think in hindsight)
Your method has intrigued me over the last few days @BlackShipsFilltheSky. It seems like if you could keep track over time of your own prediction capability, then you could do some calculations and take very calculated risks (exactly like the stock market :P) (not sure how it could be used in your own game design though; I would imagine it's more useful as a publishing strategy). In any case, then I realised but this must be exactly the kind of thing publishers must be doing. (Or do they really just use their gut?) And then their success must be telling of the method as such. (If you can live with several assumptions you need to make to fill in the blanks.)
Super Crunchers (http://www.amazon.com/Super-Crunchers-Why-Thinking-By-Numbers-Smart/dp/0553384732) has a cool example about movies and if I recall correctly books too - where similar bets need to be made. I think one of the conclusions was that professionals don't have more prediction accuracy than the average man on the street (I will need to verify though). Isn't it fascinating that we have consumed art for thousands of years and we still don't know what people will like?
Your "gut" surely refers to your intuition? Honestly I wouldn't be able to make App Store predictions logically. What I mean is I couldn't verbally state my reasons for coming to a conclusion of this complexity, I would use my "gut". And I would train my "gut" by repeatedly testing it.
But I don't see any reason why it couldn't be done logically instead (or a combination of course). But I was talking about using your "gut". In my understanding of the term.
@hermantulleken And yes, I expect publishers are doing this, or should be, though it seems to me that many mobile publishers are surprisingly poor at predictions nevertheless, or at least they have a unique set of constraints that make them look stupid).
(Or maybe it is that professionals don't really get better at predicting taste, like you suggest might be the case, and that makes publishers look stupid)
(Or maybe very few of them are actively trying to get better at predicting taste/consumer demand)
hermantulleken said:
not sure how it could be used in your own game design though
I try to use it to model the reactions of potential players. It's nowhere near as insightful as actual playtesting and feedback. But if people are responding differently than I expect to birds (for instance) on the App Store within a certain demographic, then I know my perception of birds needs to be modeled differently when I'm trying to anticipate a player of that demographic's reaction. I can do this by reviewing my predictions of a lot of games with birds in it, and a lot of games with other animals (and trying to compensate for other factors).
@hermantulleken I'd be curious to know if the professionals are not significantly better at predictions where consumer demand is the object. (Not having read the book it appears to be about predicting numbers in a more logistic sense).
@BlackShipsFilltheSky I was in a talk from Antti Hattara a few months back where he explained his methods, He was the person in Digital Chocolate that decided which games to do next. Basically what he said was that he and his analysts would not only look at the top grossing, but look at the top grossing over the history of the app store, then they would plot out the cycles of success and determine which game types were destined for a come back soon. At the same time they would look at sites like Kongregate and New Grounds for trends and cycles in the same way, because trends are not limited to a specific market place but can be seen throughout similar industry, by that I mean casual, Kongregate and Facebook have more in common with Mobile than say steam, but it does have differences and it is important to understand those differences. That is in play time and cycle. mobile players need flexible sessions lengths, short on the train, long when watching tv, but facebook is always short because they are played during work hours for the most part.
The bottom line is that this approach then allows you to see trends which you can work with. I might mention that Gameloft uses a similar system fairly successfully. This system is also a lot closer to how economists look at the stock market and in fact the guys doing this for big companies are people that were trained in stock market analysis. Your system of self prediction is great and I like it, but I think taking a broader look at the market is also very important.
but I think taking a broader look at the market is also very important.
As you've described it, I don't really see how the Digital Chocolate method is broader. In fact, I'd suggest it's narrower as they were only looking at success stories and I was advocating looking at failures as well (which helps identify bad practices, over saturation, and negative trends).
Actually I'd suggest the only difference between what I described and what you described of Digital Chocolate is their only looking at successes, and the phrasing.
I didn't mean to advocate not looking at other areas of gaming for trends/insight, nor indeed non-gaming culture. (If that's what you thought I meant)
@BlackShipsFilltheSky actually I meant by broader a historical analytic look rather than just for gut. I didn't mean to disagree with you, just pointing out the value of trends. The value of failures are seen in contrast to the successes. Trends go up and they go down, so this way you are essentially following the timing of success and failure if that makes sense?
I skimmed through some of the comments here because there's so many, but I read the article and recognise a LOT of it from previous work and discussions where companies "offer a genuine experience (NOT)" while minimising worktime and maximising profits.
I'm for working easier; in this industry, you could sweat and bleed for months, or years, while never receiving the recognition, and ultimately the sales, that will make your next project easier.
But I am for quality of work and ACTUAL games as a product. If this nar-nar thinks he can justify his life's work in front of the good Lord and call himself a "games designer", he's got another thing coming. He may be able to pay for his kids' tuition, but he's no Peter Molyneux.
Ultimately though, there is one winning formula. Fail, fail and fail again till you get it right, and don't let your first or second failed attempt stop you. That's where the Carmack's, Romero's, Blezinski's, Jaffe's, Williamson's, Miyamoto's and every other game designer have done, and we acknowledge those failed individuals as heroes and successes.
Funny enough, as I'm reading Masters of Doom for the first time, I'm realising that there's a fair comparison between id Software's work and Minecraft that made both products a success: give them the world, and let them shape it as they see fit. Idea?
Right now that's not a bad thing, in my mind, he's gone off the deep end with Godus :p but yes I have mad respect for his past work.
I do agree with you though, player customization will always be a big road to success. There is simply no better content generation than players. Of course, a good game needs to be the predicate :)
@BlackShipsFilltheSky I reread the relevant parts in the book. Mostly, it draws comparisons between human versus machine analysis (and not expert versus layman), and provides many cases where machine outperforms humans (and in particular experts). The theme of the book is in fact that statistical models do better than the gut. (On second reading, the way they argue irritated me, even though I believe the basic premise (it should be obvious that you can always build a statistical model that can outperform any other model - if it doesn't, simply plug the other thing in as an extra variable)). Anyways, the examples are still interesting.
(It's also important to understand in which sense these algorithms outperform experts - it's similar to the way stock, insurance, and gambling works: you only really "win" if you play a lot. So their value is not in making in single predictions, but a series of predictions. It is for this reason that I cannot see how such a method can be applied to game design, unless you make lots and lots of games. But I understand of course this not what you are doing.)
In the case of the movie industry, it gives the example of data mining software developed by http://www.epagogix.com/, which outperforms typical success rates of big studios. The software analyses the plot, actors, directors, etc. (Apparently actors and directors make little difference; setting makes a lot of difference).
The authors did some analysis on predicting best sellers (books) by using titles, and then also papers by analyzing the full text.
----
The bits on experts is also interesting. It highlights some biases humans suffer: assigning more weight to "interesting" things than boring things - and similarly overestimating the occurrence of interesting events while underestimating the occurrence of boring ones; reluctance to change beliefs, even in the face of evidence; and most dangerously, IMHO, overconfidence in beliefs.
Since there are also more successes than failures, they are more interesting; which explains why many people would think they are more important to study than failures, and then cling on to believe that even when faced with compelling counter arguments :P (I'm not too serious with this comment!)
@Bladesway I'm not disputing the occurrence of trends in markets, nor their usefulness once identified in predicting successes/failures, is what I meant.
Of course looking for trends is something you'd have to do if you want to predict future successes/failures., or even imminent successes/failures. Otherwise you'd only be able to predict the past. <- This is me agreeing : ) (I feel like I'm expressing myself poorly)
(Though again, I'm not stock market savvy)
@hermantulleken Thanks for the info! I find it fascinating how unreliable our brains are in certain activities.
@hermantulleken I'm trying to imagine a way to use a machine to enhance game design. This is going to get a bit amateur sci-fi ... but then this discussion is well off topic already...
I can see a use in a machine that can take game design decisions as an input and come up with probabilities of market success through crunching masses of data (that would be difficult for a human).
But even with a really accurate statistical model, with a ton of data (that a human would have to accurately add to the database), and a lot of games produced in order to benefit from the higher probability of accurate prediction, and even assuming zero bias from the designers in imputing the values, and a perfect development team that can perfectly execute the designs... the tool still could only be used to analyze decisions that the designers come up with... the designers still have to learn to foresee good decisions.
I don't think we're at the point where a machine could create guidelines for game design that lead to enjoyable games... except maybe where the developer is prepared to clone/reskin games.
Because Bejewelled + Hamsters != Good Game.
Not necessarily anyway, because the content needs to reinforce the mechanics (like Flappy Bird works better as a game about a bird than a game about a helicopter, even if helicopters are trending), and I don't think a machine could understand that yet.
Obviously this is just me hypothesizing. Developing such a machine would be incredibly expensive and not feasible for small teams.
Though I'm probably being very optimistic about what a 2014 machine could achieve...
We're probably talking about a fairly dumb machine if it were made in 2014... a trend spotter with no understanding of game design is probably the highest a machine could aspire to in the next few years. It probably could do some extra accurate trend spotting, but I think it would be nowhere near as useful as a human mind applied to the same task.
Sorry if I've now gone all amateur sci-fi on this discussion.
@BlackShipsFilltheSky I'm glad you did because I was very tempted to do it myself :P (I have been watching Lie to me and just thought how cool a playtest lab would be equipped with all their face / body / language analysis stuff! Perhaps the actual "interview" room could just be a tad less clinical).
My immediate reaction was that machines can help in a lot of ways - but their success in other art forms is not very encouraging - and therefor I must agree with you. The one thing I like about many of these data mining examples is that some of the trends are very unintuitive (such as that actors don't matter much to success; I saw somewhere else that books that use more complicated language actually are more successful than ones that don't); it would be interesting with what trends an unemotional machine will come up with, and then how these can be leveraged by human designers.
the tool still could only be used to analyze decisions that the designers come up with... the designers still have to learn to foresee good decisions
...except if we can encode a game so that all decisions are built into the equation. We can do this at least theoretically; since all games that execute on a machine occupy if fixed number of bytes, all games (smaller than X) is a subset of all bit strings (smaller than X); so all decisions are just bit-wise decisions. There are of course many reasons such a bit-wise analysis is dumb...
I wonder if it would be interesting to work with a very narrow set of games - say all puzzle games made on grids with a set of 50 mechanics (toggle neighbors) and maybe 50 "support" mechanics (combo score) and maybe 50 goals (overtake territory), and perhaps adding some other variables... just to see what pops up. This can be done in 2014...
Comments
Also, I mentioned the Red Bouncing Ball thing because that's just a tutorial template someone didn't do any more work to and posted to the app store. Apparently it did well, waaaaay better than a template should, by rights. Especially considering that the person who posted it did 0 design work... That's also relevant to the reskinning apps/games post that started all of this AND the question of "is it all just luck"? I mean, if it wasn't luck, then why did this particular template not succeed previously? It had all the same design elements that were cited as being useful in place back then already...
I'm worried that people aren't considering the impact of external factors, like what if this whole Flappy Bird and Red Bouncing Ball thing is just what it looks like when someone finds a new exploit for bots to farm downloads and reviews on the App Store? If that's the case, then mining these games for game design information to learn and try and implement skillfully in our own games is a complete waste of time.
I'm a bit out of touch from IOS. But when I was concerned about it what I did was look at all the new releases and try predict what their successes or failures would be like. When a big success like Flappy Bird came along, if it actually surprised, me then I'd try figure out why it succeeded.
Likewise, when a game's success was surprisingly disappointing, then I'd try figure out why it didn't succeed.
The mobile environment provides far better information about financial successes/failures than the PC marketplace, and vastly better information than consoles. (Look at sites like http://www.148apps.com/top-apps/ or http://www.appannie.com/ ). I think there is a lot of randomness in crafting a successful game (because making games is super hard), but accurate hit/flop prediction on mobile is well within human grasp. I don't think it is a case of games getting lucky, except where there are major unpredictable events (like a PewDiePie video).
I'd recommend not looking at notable failures (or developers who lamented failing but actually did really well like Gasketball), or notable successes... but rather look at ANY game which has a degree of success/failure that you wouldn't have predicted. The mobile environment tells you about all the new releases, you don't have to wait for a noteworthy story to analyze market success. It's a much better exercise if you do it without being guided by journalists.
If the success or failure of a game surprises you at all, that means you don't understand something about the way the market works.
(I mean, rationally, whenever you are surprised by an event it means that you didn't have enough information/understanding beforehand)
@BlackShipsFillTheSky spot on <3 sounds like a good plan
I think it's less likely that external factors are playing a stronger role than I think, than I do not understand all the factors that drive commercial success in mobile games (because I have not actively been studying the marketplace and so have a very weak grasp on what mobile consumers want right now). I agree.
@Bladesway: No, that's not what I was talking about at all. I was pointing out how a zero-effort template put up by someone with no reputation fits into your model of game quality driving sales. I don't think it's anywhere near as clear cut or powerful a driver as you seem to think. How do we do that simply? Consumer demand/perspective is a complex thing, driven so many variables that there's almost certainly no clear-cut way to predict it. If there were, people would already do that with the stock market... And yet the best evidence we have so far indicates that complex feedback systems like the stock market (and I would argue the App Store is similar) have a huge random element to any success in them. That's what I'm getting at: How much random is there and how do we actually plan for that sort of stuff?
The advice I got long ago was "release on PC first, then do mobile once people recognise the game" and that's good advice. It's also not advice that has anything to do with Flappy Bird or Red Ball Spike Bounce, Angry Birds, etc.
I'd agree that the App Store consumer demand probably looks like it behaves similarly to a stock market. It also has feedback systems. I honestly don't understand the stock market well enough to compare them off hand.
If you assume that App Store success is somewhat predictable. Let's say more predictable than the stock market (though less well studied). Then the big problem from a game developer perspective is that we can only actually experiment with trying to play the App Store if we can craft games with less margin of error than the error margin for success.
Even assuming a luckless App Store, that doesn't make a guarantee of success. That still will be incredibly risky for most developers if they're attempting a flappy-bird success. Probably risky for even the best developers for a flappy-bird. Though flappy-bird style success is the most extremely risky form of success on the App Store, and I know you don't have to be number 1 on the charts to make a lot of money.
(I don't mean to sound like trying make a Flappy Bird is easy/safe. Flappy Bird not like something like Ridiculous Fishing which has far more avenues of appeal, but obviously Ridiculous Fishing is vastly more difficult to develop. I guess I'm trying to say that the simpler your game, the more perfect your understanding of the market has to be, and I'd concede that this possibly extends beyond human ability, but then I couldn't prove otherwise anyway)
I know I said studying only noteworthy examples of games is very limiting in terms of the information you receive and the context of that information... but... I thought Cut-The-Rope was an interesting case.
Chillingo predicted that Cut-The-Rope would beat Angry Birds (which was completely dominant at the time), well before the release of Cut-The-Rope, and they pushed a lot marketing behind it, and the game really did dominate for a while. Obviously Chillingo didn't develop it, and chose it instead of a lot of other hopefuls, but they DID know the App Store well enough to call it.
I thought it was interesting at the time, because I didn't call it. At the time I thought Cut-The-Rope wouldn't make number 1. (I was biased in favour of games with very open ended puzzles, like Angry Birds, I think in hindsight)
Super Crunchers (http://www.amazon.com/Super-Crunchers-Why-Thinking-By-Numbers-Smart/dp/0553384732) has a cool example about movies and if I recall correctly books too - where similar bets need to be made. I think one of the conclusions was that professionals don't have more prediction accuracy than the average man on the street (I will need to verify though). Isn't it fascinating that we have consumed art for thousands of years and we still don't know what people will like?
But I don't see any reason why it couldn't be done logically instead (or a combination of course). But I was talking about using your "gut". In my understanding of the term.
@hermantulleken And yes, I expect publishers are doing this, or should be, though it seems to me that many mobile publishers are surprisingly poor at predictions nevertheless, or at least they have a unique set of constraints that make them look stupid).
(Or maybe it is that professionals don't really get better at predicting taste, like you suggest might be the case, and that makes publishers look stupid)
(Or maybe very few of them are actively trying to get better at predicting taste/consumer demand) I try to use it to model the reactions of potential players. It's nowhere near as insightful as actual playtesting and feedback. But if people are responding differently than I expect to birds (for instance) on the App Store within a certain demographic, then I know my perception of birds needs to be modeled differently when I'm trying to anticipate a player of that demographic's reaction. I can do this by reviewing my predictions of a lot of games with birds in it, and a lot of games with other animals (and trying to compensate for other factors).
@hermantulleken I'd be curious to know if the professionals are not significantly better at predictions where consumer demand is the object. (Not having read the book it appears to be about predicting numbers in a more logistic sense).
The bottom line is that this approach then allows you to see trends which you can work with. I might mention that Gameloft uses a similar system fairly successfully. This system is also a lot closer to how economists look at the stock market and in fact the guys doing this for big companies are people that were trained in stock market analysis. Your system of self prediction is great and I like it, but I think taking a broader look at the market is also very important.
Actually I'd suggest the only difference between what I described and what you described of Digital Chocolate is their only looking at successes, and the phrasing.
I didn't mean to advocate not looking at other areas of gaming for trends/insight, nor indeed non-gaming culture. (If that's what you thought I meant)
I skimmed through some of the comments here because there's so many, but I read the article and recognise a LOT of it from previous work and discussions where companies "offer a genuine experience (NOT)" while minimising worktime and maximising profits.
I'm for working easier; in this industry, you could sweat and bleed for months, or years, while never receiving the recognition, and ultimately the sales, that will make your next project easier.
But I am for quality of work and ACTUAL games as a product. If this nar-nar thinks he can justify his life's work in front of the good Lord and call himself a "games designer", he's got another thing coming. He may be able to pay for his kids' tuition, but he's no Peter Molyneux.
Ultimately though, there is one winning formula. Fail, fail and fail again till you get it right, and don't let your first or second failed attempt stop you. That's where the Carmack's, Romero's, Blezinski's, Jaffe's, Williamson's, Miyamoto's and every other game designer have done, and we acknowledge those failed individuals as heroes and successes.
Funny enough, as I'm reading Masters of Doom for the first time, I'm realising that there's a fair comparison between id Software's work and Minecraft that made both products a success: give them the world, and let them shape it as they see fit. Idea?
I do agree with you though, player customization will always be a big road to success. There is simply no better content generation than players. Of course, a good game needs to be the predicate :)
(It's also important to understand in which sense these algorithms outperform experts - it's similar to the way stock, insurance, and gambling works: you only really "win" if you play a lot. So their value is not in making in single predictions, but a series of predictions. It is for this reason that I cannot see how such a method can be applied to game design, unless you make lots and lots of games. But I understand of course this not what you are doing.)
In the case of the movie industry, it gives the example of data mining software developed by http://www.epagogix.com/, which outperforms typical success rates of big studios. The software analyses the plot, actors, directors, etc. (Apparently actors and directors make little difference; setting makes a lot of difference).
The authors did some analysis on predicting best sellers (books) by using titles, and then also papers by analyzing the full text.
----
The bits on experts is also interesting. It highlights some biases humans suffer: assigning more weight to "interesting" things than boring things - and similarly overestimating the occurrence of interesting events while underestimating the occurrence of boring ones; reluctance to change beliefs, even in the face of evidence; and most dangerously, IMHO, overconfidence in beliefs.
Since there are also more successes than failures, they are more interesting; which explains why many people would think they are more important to study than failures, and then cling on to believe that even when faced with compelling counter arguments :P (I'm not too serious with this comment!)
Of course looking for trends is something you'd have to do if you want to predict future successes/failures., or even imminent successes/failures. Otherwise you'd only be able to predict the past. <- This is me agreeing : ) (I feel like I'm expressing myself poorly)
(Though again, I'm not stock market savvy)
@hermantulleken Thanks for the info! I find it fascinating how unreliable our brains are in certain activities.
I can see a use in a machine that can take game design decisions as an input and come up with probabilities of market success through crunching masses of data (that would be difficult for a human).
But even with a really accurate statistical model, with a ton of data (that a human would have to accurately add to the database), and a lot of games produced in order to benefit from the higher probability of accurate prediction, and even assuming zero bias from the designers in imputing the values, and a perfect development team that can perfectly execute the designs... the tool still could only be used to analyze decisions that the designers come up with... the designers still have to learn to foresee good decisions.
I don't think we're at the point where a machine could create guidelines for game design that lead to enjoyable games... except maybe where the developer is prepared to clone/reskin games.
Because Bejewelled + Hamsters != Good Game.
Not necessarily anyway, because the content needs to reinforce the mechanics (like Flappy Bird works better as a game about a bird than a game about a helicopter, even if helicopters are trending), and I don't think a machine could understand that yet.
Obviously this is just me hypothesizing. Developing such a machine would be incredibly expensive and not feasible for small teams.
Though I'm probably being very optimistic about what a 2014 machine could achieve...
We're probably talking about a fairly dumb machine if it were made in 2014... a trend spotter with no understanding of game design is probably the highest a machine could aspire to in the next few years. It probably could do some extra accurate trend spotting, but I think it would be nowhere near as useful as a human mind applied to the same task.
Sorry if I've now gone all amateur sci-fi on this discussion.
My immediate reaction was that machines can help in a lot of ways - but their success in other art forms is not very encouraging - and therefor I must agree with you. The one thing I like about many of these data mining examples is that some of the trends are very unintuitive (such as that actors don't matter much to success; I saw somewhere else that books that use more complicated language actually are more successful than ones that don't); it would be interesting with what trends an unemotional machine will come up with, and then how these can be leveraged by human designers. ...except if we can encode a game so that all decisions are built into the equation. We can do this at least theoretically; since all games that execute on a machine occupy if fixed number of bytes, all games (smaller than X) is a subset of all bit strings (smaller than X); so all decisions are just bit-wise decisions. There are of course many reasons such a bit-wise analysis is dumb...
I wonder if it would be interesting to work with a very narrow set of games - say all puzzle games made on grids with a set of 50 mechanics (toggle neighbors) and maybe 50 "support" mechanics (combo score) and maybe 50 goals (overtake territory), and perhaps adding some other variables... just to see what pops up. This can be done in 2014...