Idiotic reports and the need for scietific methodology...

People say you can make stats say anything you want. I personally don't subscribe to this line of thinking. I believe that it's more accurate to say that if your suppositions are wrong (on purpose or by accident) then you can go looking for the answer you want and support it through biased research. It's a constant danger in the scientific profession and one that you're supposedly trained to learn how to avoid - though i'd say we're all guilty of it to some degree.

The danger isn't so much in scientific circles though as bad research will get a name for itself that can be hard to shake off the authors' backs. No, the danger is in trusting figures from 'closed' environments that conceal how they do their research and have no peer (or otherwise) review process or even worse, from people who think they know how unbiased and fair research is conducted but have no training in the field, leading themselves and others into false assumptions.

While this isn't what this particular post is about, i thought i'd make the observation due to a recent experience elsewhere on the internet...... and it's also tangentially related to the content in this post :)

Ars has an article up about game sales in the US, UK and Japan and comes to the conclusion that if you want a blockbuster title you must do well in the 'all important' US market. Frankly the assumptions that seem to have gone into this conclusion seem idiotic.

First off there's the relative market size difference between the three territories. The population of the US is approximately 300 million, the UK is approximately 60 million and Japan is approximately 127 million. This ties in with the fact that if you want to take a trend or sales figure and compare it with how well a game does in each market then you have to account for the differences in population. Saying that a game sold more in a country with more consumers in that particular market is... well, it's kinda dumb and obvious. If you want to compare the UK to the US, compare it to a state with similar population.

Then there's the games they've compared. Madden '09, Wii Fit, The Force Unleashed, Mario Kart Wii and the latest Pokemon game. Ars makes the astute observation that Madden sold best in the US and conversely Pokemon sold best in Japan while Wii Fit and Mario Kart sold in each territory but TFU only sold in US and the UK:

The number one seller, for instance, was Madden 2009, with 2,958,000 units sold in the US, out of 2,994,000 worldwide. In other words, the US propelled that game to the top of the charts alone; nothing else came close. Number two was Wii Fit, where 61.4 percent of sales were in the US, and number three is Star Wars: Force Unleashed, where 81.5 percent of sales came from the US. That sends a powerful message: games that hit big in the US hit bigger than in any other market. The sole counterexample is the 1,482,000 units of Pokemon Platinum sold in Japan; that particular entry in the long-running series has yet to be released anywhere else.

This is a classic case of not understanding how things work and then coming to false conclusions. Sure, like i said above, a larger market will have more sales of a product.... not to mention that a game that has appeal in only one market of those surveyed will undoubtedly perform best there - I'm looking at you, Madden. No one else cares about American Football so it's a wonder it sold in the UK at all. Then there are the other games on the list: Pokemon is only available in Japan. It hasn't been released in any other territory yet - similarly, TFU had a staggered release date with the US receiving it before the EU (which would be the comparable territory to the 'US') and it hasn't even been released in Japan yet.

The other discrepancy in numbers is easily explainable with the difference in market size between the three countries and would paint a more accurate picture. Take for instance Wii Fit.

Out of a population of 300 million, Wii Fit sold 1, 283, 000 copies in the US. That's a 4.28 x10(-3) copies per person. The UK had 460, 000 sales which is 7.67 x10(-3) sales per person while Japan had 346 000 sales leading to 2.72 x10(-3) copies per person. Now looking at the relative sizes of the market has shown that in fact the UK has a stronger games market for Wii Fit than either the US or Japan. There may be less sales in the UK but because of the relative strength of the pound and the higher percentage likelihood that a person will buy a game in the UK it could be argued that a blockbuster game is more likely to be made if you target what the UK tastes are because they are broader than just the US tastes in games.

I'd love for this article to actually compare all sales in the EU and then see how the numbers come out... and of course compare Fifa '09 or Pro Evolution Soccer (it's football you idiots!) worldwide sales numbers in each territory and then see how that changes their conclusion.


It's this kind of unscientific, biased reporting that gets the industry into a mess. Undoubtedly this report will be debunked by a portion of its audience.... but how much more damage will be done by people who follow along thinking that they should target all their games (or from a consumer point of view, all games should be targeted) to the US audience? This mentality then filters out and provides harmful repercussions. A similar example to this is the argument that the 360 and PS3 developers shouldn't have to support SDTVs with their games.... an argument that is similarly unfounded and based on lack of thought and understanding of the situation.

I'd like to finish the article with the same paragraph that Ars does and i want you to see if you can stop yourself laughing or crying at the state of reporting in the world today:

Take a look at how many of the biggest games either come from, or are propelled by, sales in the United States. It used to be that a system had to conquer Japan for it to be a success. That time has passed.