Results ‘n stuff
Cool, came in 5th in a compo that I almost abandoned halfway through due to isometric sorting woes 
But looking at the scores in general has reminded me just how much the Humor category needs to be nuked from orbit!
Being hit with a load of 1′s and 2′s in this category (which isn’t relevant to a lot of games) isn’t much fun. It’s obviously not just my games that suffer from this – even Notch’s epic Metagun was given a couple of 1′s in this category, and that was a solid 4-5 game. How about just removing the humor category, if a game would have scored high here, that should add to the ‘fun’ score, shouldn’t it? (Maybe even score Fun out of 10, instead of 5?)
Also, there’s been a bit of what I can only describe as ‘troll scoring’ going on – with solid 4+ games being given 1s and 2s across the board by at least one voter (in a lame attempt to vote down the competition?)
I’d almost suggest filtering out the highest and lowest couple of ratings for each game when calculating the final results?
Discuss…
Humor does not affect your Overall score. In fact, none of the other scores do – Overall is calculated purely from what each person enters in that specific category. So it’s possible to get 1′s in every other category and still “win” LD.
Ah, cool… Ok, I’m a complete noob then
(I’ve only been doing these since LD#1, and never actually paid too much attention to how the scoring system works – other than the low scores people get in that particular category!)
Just as a note before I jump into this, I definitely don’t mean to imply that the results for this LD are unfair in any way. They aren’t. It doesn’t hurt to look at ways we can make the voting work better for future competitions.
“Also, there’s been a bit of what I can only describe as ‘troll scoring’ going on – with solid 4+ games being given 1s and 2s across the board by at least one voter (in a lame attempt to vote down the competition?)”
Along these lines it does concern me a little that a game that 2/3rds of people absolutely adore and 1/3rd hate will generally fair worse than a game that everyone thinks is merely okay.
Perhaps a partial solution would be to include a brief second round of voting on a shortlist of the top 10 (20?), run in a “Which game/s do you think is/are best?” style. It would mean extra faff though, and it’s not clear if it’d actually improve things anyway.
Ultimately any voting system is going to have flaws in it :-/
Ah well, the whole process was massive fun. The winners deserved it, (congratualtions!) and I’m really looking forward to December’s rematch
“it does concern me a little that a game that 2/3rds of people absolutely adore and 1/3rd hate will generally fair worse than a game that everyone thinks is merely okay.”
Hmm well, if 2/3rds of people give 5 stars, 1/3rd give 1 star, that makes an average superior to 3 stars.
The voting categories could be simplified. How about:
- Concept (= fun + innovation + humor + theme + whatever)
- Production (= graphics + sound + the old technical and polish categories that were removed, even though they are much more interesting categories than ‘humor’)
We don’t really need all those categories. I never know what to vote for innovation, humor and theme anyway, I find those impossible to judge in isolation.
The community category is just a popularity contest and it should really be gone, actually I’d rather have the occasional weird category like food photo or timelapse that community was supposed to replace. But those weird categories should last for only one LD and would provide an alternative way to compete for (and they should be available to the jam people too).
Simplifying the categories would probably be a good thing, as it’d make the reviewing/scoring process a bit easier. Concept and Production do sound like a decent suggestion. Although it does mean that there’s less ‘winners’ – it’s nice to have, for example, a ‘best audio’ game, that might not have made the top 20 or so overall, but gets some attention for what it did well
If there’s less categories, then maybe increase it to points out of 10 in each category, instead of just 5? – the limited range of scores makes scoring difficult ‘hmm, that’s better than the other games I’ve given a 4, but is it really worth the full 5?’
I’ve always quite liked the ‘community’ category, I like seeing timelapses and frequent work-in-progress screenshots, and ususally make an effort to post a good few progress updates. But it gets a bit silly when you start thinking ‘will I lose points for not taking a photo of my lunch’!
For me the important categories are:
- innovation because it’s nice to see new things
- theme because it’s the basis of the competition and the only way to check that the game wasn’t made before the compo in 3 months (short of making detailed timelapses mandatory and having someone check them frame by frame.)
- fun for obvious reasons
In my mind, “overall” kind of duplicates “fun”, but it can be used to take into account the technical side of things or any other non-official category I suppose (I too was surprised not to see any technical category.)
Community and humour are probably there because someone wanted to see games with humour and many blog posts…?
While I agree with the theory, in practice I find the innovation and theme categories just don’t work well.
Innovation, unless you’re really an avid gamer that plays thousands of game, you just can’t know if a game is innovative or just a copy of an innovative game you don’t know. The reverse is also possible, the entry may be reinventing something that already exists. There is also the question of whether an innovative idea that does not work should be rewarded.
For example, I once gave a 5 in innovation to a Tower Defense game as I had no idea that it had become an established genre of casual gaming and I thought it was very original.
“Theme” also leads to weird paradoxes, games that have the theme at the core of their gameplay get low theme scores while more games using more conventional gameplay but using the theme as a cosmetic element get the top rankings. So it seems it’s actually discouraging people from basing their games on the theme.
“Also, there’s been a bit of what I can only describe as ‘troll scoring’ going on – with solid 4+ games being given 1s and 2s across the board by at least one voter”
There’s definitely been some troll voting going on. I’ve seen a number of entries with a line of all 1s, that otherwise had decent ratings. I was lucky enough to have two such lines. Having two votes like this out of a total of 20 drops my score by 10% in every category, which makes a significant difference in the rankings. The same thing has happened to a few other people.