Rating System - nuwen.net

Warm as sunlight, cold as frost in the stars

Other Rating Systems

There are many different rating systems for individual opinions of content (i.e. books, games, movies, etc.), and most of them suck because they are either too fine or too coarse. These are the common rating systems and the problems that they suffer:

These are all the same rating system, just viewed at different resolutions - they summarize quality with a linear rating. Consider what quality is: it is a continuum, much of which is occupied by horrible or boring stuff. Some stuff is good, and a few things are really awesome. Directly mapping this continuum to a linear rating wastes a lot of ratings on the horrible and boring stuff, and insufficiently distinguishes the merely good stuff from the really awesome stuff.

What is the real purpose of a rating system? It's not to summarize quality, but to advocate action. Actions are discrete. The simplest possible rating system would be binary, advocating for each item of content either "avoid" or "consume" (i.e. "read", "play", "watch", etc.). In practice, more ratings are necessary. "Avoid" needs no further elaboration, but there's always content that's on the frustrating edge of not being awful enough to recommend avoiding outright, but also not being good enough to actually recommend. This content should get a "neutral" rating. (No one will agree with a reviewer all of the time, so a reviewer should know when both "avoid" and "consume" are going to be wrong for a lot of people.) Also, it is useful to distinguish merely good from really awesome, so that they can be prioritized; really awesome content should be consumed as soon as possible, while merely good content can be consumed when there's nothing better to do.

(Note that the Flick Filosopher's ratings of "green for go", "yellow for maybe", "red for no" advocate action - yay!)

My Rating System

I use five ratings, each of which recommends a different level of action. For convenience, I express them with stars, but they are not a linear measure of quality. Using books as an example of content, my ratings are:

Image Meaning Recommended Action
1 Star - BAD BAD Avoid this book.
2 Stars - OKAY OKAY None.
3 Stars - NOTABLE NOTABLE Read this book; low priority.
4 Stars - EXCELLENT EXCELLENT Read this book; medium priority.
5 Stars - TRANSCENDENT TRANSCENDENT Read this book; highest priority.

I rate most science fiction novels highly (three and four stars) because I'm reasonably good at choosing SF novels that I'll like. One and two stars are uncommon; every once in a while, I take a chance on an author who turns out to be bad, or I'm surprised by a bad novel from a good author. Five stars are also uncommon; this rating is very special.


http://nuwen.net/stars.html (updated 9/23/2007)
Stephan T. Lavavej
Home: stl@nuwen.net
Work: stl@microsoft.com
This is my personal website. I work for Microsoft, but I don't speak for them.