Ratings are serious, but maybe they shouldn’t be. I dunno, sometimes I just kinda go with it. Sure if someone is a reviewer for a professional review facility (ie. AllMediaGuide) then the rating isn’t really open to change. Most cases, pliable ratings are okay

Don’t mind me; just dropping a “I’m not dead” comment.

You make a great point about ‘exceptional’ state. I sort of threw it out there, simply based on what I felt a 10, the highest rating, could possibly mean. Although it is the ‘experience’ being rated, it’s tough to break through user preference. A person enjoying a book at the same level as enjoying a movie might rate the experience of the book higher than that of the movie because they prefer the enjoyment of reading vs watching over the course of a couple hours. It’s a predicament of trying to have a 10 equal a 10 for every medium; it came out that way, but wasn’t quite my stimulus.

Rather than ‘being exceptional in every medium’ I feel a 10, for myself, is something that can be enjoyed/satisfying even beyond the medium; not necessarily a 10 outside the realm, but enjoyable. It’s almost like a recommendation. My personal thoughts on recommendation would be similar; if I recommend something, it is most likely across all media.

Case in point: Cowboy Bebop. Perhaps the only thing I would recommend without a thought. It possesses a level of cultural familiarity non-anime fans should be able to identify with, a fresh exposition, great character dynamics, etc etc.

Within the medium, the experience is a 10. Outside the medium and if I wasn’t an anime-enthusiast, I still feel it’s a satisfying experience.

It’s kind of difficult to push that meaning of a 10 on others, but that’s how the dynamic rating system comes into play. The concept is rather tough to think through, but it’s quite the same as a fixed-point system for the most part. It’s extremely difficult to rate everything on it’s own tier in an infinite list, but defining what level X means ahead of time helps to sort these thoughts.

Allowing users to define their rating system is optimal, and what matters in the overall is something of this nature: “the height at which an item appears in a list, over the system of all lists, determines the relation of said items to all other items in the system.”

If an item is in the top tier on every list in which it appears, it should be on the top tier in the overall… almost, it is surely more complex, but that is the general idea.

Oh god. Not again.

I know, sorry.

I like the focus on the quality of the experience, which ties into the whole ‘experience is not bound to medium’ thing. Though I suppose the idea of works which are ‘exceptional in and beyond the medium’ might tangle the hypothetical raters up in questions about the relative status of different media. (If a book is exceptional for its medium, is it likely to be automatically superior to all television, and so on . . .)

Since I was writing about the ways that two sites present their respective rating systems, rather than how or why we should actually rate things, I didn’t really feel best-placed to judge between them and/or other regimes . . . but I’m inclined to agree that a high-to-low system with an indefinite, expanding number of tiers is more practical, though it takes a bit of time to get my head around it.

Ratings should be allowed constant adjustment as well. If you glance back at this post, you’ll see one of the main advantages of dynamic rating (if you change one rating, you have the ability to change all other ratings fluidly). Example:

If suddenly a rater feels the need for a level between 6 and 7, they can rate a new item as a 7, and shift anything which was 7 or above, upward.

Another rare case would be if something comes along that blows all other experiences away. Yea, so it’s a 10, but shouldn’t it be rated higher than other 10s. So rating an 11 is totally viable.

There isn’t alway a need to do it, but the ability is there. It also doesn’t matter if two user’s scales have more or less levels; it’s transformable.

There were plenty of times I’d mark something on MAL an 8, 9, or 10, only to later reflect more (harder?) and pull back the score because it was swayed more by a great finale than the whole, pure experience.

I think the implication is that ratings that are handed out rarely are the ones that have the greater impact.

Precisely.

linear drop-off between “7″ and “9″ and a fast drop off between “9″ and “10″.

I did say exponential decay, but this is similar enough. The main idea is that certain ratings do become rarer at some point in the scale.

Also, review ratings within a closed system, such as a review site or blog tend to be accompanied with explanations, which is advantageous imo. Open-rating systems such as ANN, etc have a much harder time, because individuals cannot represent the way the would rate things or what a certain value means to the specific user.

This is a big stimulus for myself in creating a method of dynamic open-rating, where users may rate with any number of levels as they please, but still be relevant system-wide (math math math) simply by following the basic two principles stated (though no.2 requires more explanation).

Well there is no defining too many but generally, yes. For instance, an extreme case (or possibly base-case), a user who rates everything the exact same number. This is the inverse of the optimal rating, where every item has it’s own level.

No matter what the “value” of the rating may be, the user’s ratings are flat. What would one think if they saw a user who rated everything a 10? Whether the ratings are serious or not, it does not help in figuring out where items stand relative to each other, and is quite similar to “crying wolf.” If that makes sense.

From another’s point-of-view, how useful are those ratings…

Given, this is not necessarily the user’s fault, should not matter to the user (unless they intend to make their ratings useful) and could be attributed to the limits of a fixed rating system.

I think the implication is that ratings that are handed out rarely are the ones that have the greater impact. It’s like the idea of advertisements trying to come up with new superlatives to describe their products. If everyone starts by saying “this product is good, you should buy it”, after a while people will be inoculated to the word “good” and think, when they hear it “well, ‘good’ is fine, but it’s not that great”. So advertisers will move on to “great”, then “wonderful”, then “terrific”, then “sublime”, etc, and each time one of these words becomes overused, it’s impact and original meaning will be reduced. Which is why I think it’s makes sense to keep the ratings that you want to use to make a statement in rare use. If you want people to sit up and take notice when you award something with a “10″, then the way you do that is to only award it extremely rarely. That’s why “10″ isn’t redundant if used extremely rarely… if anything, it’s meaning and impact increases the less it’s used. Same goes for “1″ or, as we have on our site “0″ (which is even more rare than “10″).

Coincidentally, the rating system we have on our site compares very closely to the “hybridized” system, except that we also have a “0″. That kinda lends it to a sense of symmetry, with “5″ for “Average” at the middle. It doesn’t work that way, though, we are, after all, anime fans, so the weight of the distribution is on the positive side. I’m not sure I agree with the idea that there should be an exponential decay between “6″ and “10″. On our site, the most numerous rating is “7″, and there’s a linear drop-off between “7″ and “9″ and a fast drop off between “9″ and “10″. Personally, I’d have preferred “6″ to be the most numerous rating, but that kinda can’t be helped. Reviewers prefer to review the series they like.

The 1st fundamental understanding makes sense to me, but with the 2nd, are you saying that having too many shows with a 7 (for example), desensitizes the meaning of the 7?