lol BELL CURVE. Well, the thing about using more levels doesn’t really change the fact that a user only has 100% to deal with. 100% is the best a user can give any item. It’s broken down more so the system doesn’t have to support the tenth or hundredths places. There is no 8.5, but something might get a potential of 85% depending on the levels the user is using.

What does occur, is that if a new level is placed at the top, say 11, everything else is automagically shifted to appropriate percentages.

That’s what I’ve mentioned as potential, which is more useful in user-to-user evaluations than calculating a overall system rating. For that, we would use the energy/kinetic scoring. The difference is, kinetic scoring considers the depth and diversity of a user’s ratings and adds the outcome to an absolute value for a given item.

The main difference in thinking about this sort of scoring, is transition.

Item X deserves grade Q.

This is the current way.

Item X should be grouped in grade/level Q because of the set Y (other items in Q).

This is better for dynamic ratings. It’s about grouping items by grade, but not specifically caring what that grade value is… the algorithm will yield something relevant, which can then be looked at between items in the system. (all items are evaluated the same through connecting the dynamic user ratings).

The most important part is how the grade-levels are related greater,lesser,equal (equal implies the the two levels are the same level, level 1 = level 1).

Basic example using your 5 to 9 ratings:

5 = 5
6 = 4
7 = 3
8 = 2
9 = 1

5 levels. Now here are my levels, as possibly related to the above:

10 = 5
9 = 4.25
8 = 3.75
7 = 3.3
6 = 2.8
5 = 2.4
4 = 2.0
3 = 1.6
2 = 1.25
1 = 1

The actual numbers would be different, but don’t evenly divide out because it matters how many items are on various levels. (similar to matching the centers of the bell curves)

At some point the numbers really don’t matter… which is the objective. What matters is the position of a level and how many levels+items are above and below it. This could be perfectly used with a 10-point system, but hopefully it is clear why the results would be different. Also, bottom offsets would be good, or rather, having a list size, but allowing blank bottom slots where wanted…. no need for top offset since down-boosting is meh, and we care about the list in only one direction.

Right, everyone uses a different scale. A ‘7′ does not mean the same for everyone. I do not care much for a title’s overall rating, but I would like to see this implemented for generating user-compatibility-ratings (in MAL).

Using offsets does sound like a possible solution to “translating” a user’s scale. On the other hand, I refuse to support a dynamic-point system. Ten is enough. Any fucker who needs more than ten needs to quit inflating his/her goddamn scores.

I personally use scores from ‘5′ to ‘9′ (similar to letter grades). ‘6′ is the neutral line. ‘7′ is where I start saving the series. ‘9′ is reserved to only two series, so ‘8′ is generally the most I give.

Of course, there are additional things which would make it more accurate, like finding an intersection series, and then showing the relative distance of other series compared between two lists, while accounting for the difference in tiers (ie. I have 54 tiers, Michael has 10 I believe).

Naturally, just because something ends up on the lowest tier of a smaller list, doesn't mean it should end up on the bottom of a larger list as well; it might end up somewhere in the middle, depending on a possible offset. This sort of thing isn't implemented at this time, but is possible.

@Michael, ^_^ MAL does what it does. I would have used MAL if it wasn't AniDB (thought MAL is less robust with better UI), which I was already fed up with since logging episodic watches and nothing more is quite useless. melative takes a different, less anal approach to experience, there is states and history.

Also, Michael, if you use Firefox and have time, come play with the Stream. http://melative.com/stream (might have to reset pass)

Edit/Addition: @ghostlightning Not sure if you’ve come across this explanation, but I feel it might make more sense as to why this ratings system requires a different perspective about ratings (in terms of energy and potential). [link]

Converting to a 10-point system is indeed possible, but it is closest related to potential (or aggregate position on all lists). The issue is that if the aggregate position is the bottom 10%, that is technically a 1/10 on a fixed point system. Thinking in a fixed-point mindset here will cause trouble, since the true implication is that if there are 10 levels, said item appears on the bottom level. It’s not exactly the same as rating it a absolute 1/10… it just means its at the bottom of all items in the system…. no real number needs to be attached… it’s relative.

10/10 for the post.

Is it ‘good’ 7? Or, as you say, a ‘pity’ 7?

Not everyone can be articulate, as I don’t think even I can easily work through a meaning for each number (never mind the decimals, if any). So, having one prepared as a default that can be opted-out from is still a good idea I think.

Addition: thinking on the system-wide level is important for standards, but giving user-level abilities to change the way the default works is something I like. A good [tangential] example would be the visibility coded into the stream, which allows users to have a completely private or friends-only microblog, but still maintain the ability to publish items with public visibility.

In this dynamic algo I’ve been developing, the experiences of the user, mix their ratings of each experience play into a larger calculation which results in 2 aggregate statistics.

The thing is, each user does not have equal weighting power. A user rating everything a 9 or 10 is not going to contribute much for a series as opposed to a user who has few things rated 10, and the series was rated 10 (as well as a full spectrum of ratings).

What a user grades a given series is important, but all other grades for other series are taken into consideration as well.

A large part of these dynamic ratings is generating accurate ratings for what ratings are given, relative to all the items within the system.

I believe AniDB does use those standards, but then again when considering the subset of Masterpieces, they can still be organized and rated. IMO, it’s incomplete to leave it in such fashion. The other issue is that it can never be an exact rating even if going for a 9.786, since the more precision added the less we actually get what the hundredths and thousandths places mean. Using a sorted list nullifies the need for this decimal precision, but it’s not to say we can’t place things on the same level.

Everyone does not have to comply. This is true, which is even more push for dynamic ratings which allow users their own compliances, completely separate from other users, but at the same time, having an algorithm which makes aggregate sense of these various rating systems.

The tiers meaning: A>B, A<B, A=B … that is it.

If a user imports their ratings from 10-point system, it’s automatically in the standard. Over time they may branch out the ratings (especially in the upper tiers), but don’t need to. If there happens to be more than 10 tiers, the list might be more expressive, such as a hybrid between a top X list and standard point-ratings. The list can change over time, becoming larger and more complex or smaller and less complex, or it doesn’t need to do anything since normal 10-point ratings will work just the same individually; the difference is in calculating the aggregate.

10: Masterpiece, can be recommended to almost anyone. Lack of appreciation will be rare and almost always happen due to extenuating circumstances.
1: A total discredit to anime as a medium. Will only please collectors of terrible experiences.

…etc.

Everyone does not have to comply! One just has to opt-out of the system, which means one’s ratings won’t be part of the aggregate ratings.

All this does is to derive real conclusions from ratings. The conceptual heavy lifting will be in determining what the standards/ratings tiers/meanings will be.

If I look at a rating history of a user, let’s say on a 5 point scale, and I see nearly 100 4’s and 60 5’s, it really doesn’t make any sense. That’s 160 series which would have scored 8/10 or better on MAL. Is this good information about the titles [or the user]?

At some point these ratings lose significance. Dynamic ratings are an attempt at maintaining significance (mathematically, system-wide) even with such ratings. For the individual scope, it solely allows pliability (yes, one can have an 11 out of 10 if they want to) while staying relevant to other users (those who don’t have the 11th tier).

@moritheil always subjectivity, but what are your thoughts on varying degrees? Or is it, like digital, either zero or one. As for the Naruto is a 10 ordeal which later must be re-evaluated, I wrote on the pliability of ratings history earlier this year. [entry]

Finally, this notion of relative rating doesn’t require a different way about rating. It can be thought of directly comparing series, but users may still rating with a 10/10 scale, just as it would be done anywhere else. The difference comes in flexibility, customization, and calculated significance throughout the entire system.

I’d be interested to hear what changes you guys think might be beneficial to fixed-point systems. I begin these endeavors because I don’t feel the available tools are adequate (multiple sites for media, fixed-point rating systems), which is the same for the microblogging stream on melative; allows status updates about externally defined topics (pure content message with none of that, #SeriesNameTag 05: … garbage in the message).

Or another thought, I could translate your ratings and show exactly what they would look like in the system… this would be interesting with two users, which would bring in divergence/conformity though it’s better with more users.

Which effectively makes it subjective to the reader. There could be a distinction, I suppose, but categorically I think recognizing the subjectivity of ratings is essential. The nature of subjectivity is that people try hard to be objective and still fail.

I don’t think that someone who insists that Naruto is a 10 is necessarily lying or deluded; it might be that Naruto has whatever they’re looking for at the moment. This will definitely not be what everyone is looking for, and it might not be what they themselves appreciate most in a few years. In no way do those qualifiers detract from the sincerity of the rating.