Comments on: Bayesian Ratings: Your Salvation for User-Generated Content

By: Swenor

Swenor — Wed, 17 Mar 2010 15:37:32 +0000

I’ve seen progression in every post. Your newer posts are simply wonderful compared to your posts in the past. Keep up the good work!

By: Gavin

Gavin — Mon, 08 Mar 2010 08:58:09 +0000

Andy, interesting post. Rankings/ratings and that paraphernalia happen to be my daily work, so thought I’d chime in.

One way I’ve used recently on a multi-country emerging market study is conjoint analysis; that’s where people have a list of choices and must also rank those choices. Along with your Bayesian, this can give greater nuance (i.e. switches from a one-dimensional list sorted in some way to a two-dimensional grid, also sorted in some way)

The one I prefer for everyday use, though, is a self-benchmarked relative rating system (a demo of which can be downloaded from my website).

Without going into the maths, consider the process that Amazon uses to recommend books to users: their ranking is based both on what you purchased (but may not have enjoyed) as well as on what you searched for (but didn’t buy, and probably weren’t interested in). In other words, while this type of rankings system appears to be sophisticated (and does increase sales) it still does a poor job of recommending things I will really enjoy.

The alternative is for a system that has all possible books in it grouped into segments that are relevant to me and, whenever I’m interested in finding a new book, I compare all books in the system to the type of book I have read before and am looking for now.

Feel free to email me if you find this approach interesting.

By: Yupa

Yupa — Thu, 18 Feb 2010 10:56:52 +0000

Great article, 7/10! ;)

By: links for 2010-02-16 « Blarney Fellow

links for 2010-02-16 « Blarney Fellow — Wed, 17 Feb 2010 01:31:51 +0000

[...] Bayesian Ratings: Your Salvation for User-Generated Content » Andy Moore (tags: bayes ranking recomendation) [...]

By: Alex Schearer

Alex Schearer — Tue, 16 Feb 2010 23:45:28 +0000

Great post thanks for shedding some light on how you manage your community. Another avenue which might be worth exploring is collaborative filtering. You could potentially have your players help find each other find levels they like — sort of similar to how Netflix will recommend a movie for you based on your past rentals. Though it’s an open question whether such an improved filtering mechanism would be worth requiring users to enter additional metadata about each level.

By: Andy Moore

Andy Moore — Tue, 16 Feb 2010 16:36:08 +0000

Unfortunately, no! I’m hoping I can change that someday, though. :)

By: Colm

Colm — Tue, 16 Feb 2010 14:04:18 +0000

That’s a pretty interesting use of Bayesian weighting. Did any of this go into Fantastic Contraption too?

By: Andy Moore

Andy Moore — Tue, 16 Feb 2010 01:47:58 +0000

And since I forgot to address it (and it’s not really bayesian specific) – Protonaut uses a simple +/- voting system with no neutral vote. People have to have an opinion, no matter how slight!

I was careful to phrase them nicely, too: “Worth Playing” or “Not Worth Playing”. It helps to stem preference issues; Some people think “10 stars” should mean “best level ever” and some take it to mean “great job! one of the best!”. Not having properly labelled ratings is suicide! You need to bracket your votes with context.

To get over the lack-of-voting you get when you have neutral votes, there is a super-secret set of gold coins you collect per-vote. When this system goes public it’ll help motivate more people to get involved.