Now available: Player Ratings

[bert_the_turtle]

Experimental, of course: http://dedcon.homelinux.net/ratings/
Currently, only the Duel and TotallyRandom ratings are filled, and the combined ratings in "All". For lack of rated games, the scores are widely inaccurate; a single won six player game was all it took to catapult Lynchfinger to the top of the TotallyRandom list. Usually, I'd filter out players where not enough data is available yet, but then the list would be empty right now.

What do you have to do to get on the rankings or improve your score there? Well, obviously, you'll have to play on the DedCon Test servers, but that's not all. You need to stick around til the end end then make the scores known to the server (the server doesn't simulate the game and can't know the scores, and the clients don't tell the server what they are). Use the /setscore command to enter the scores, it is used like this:
/setscore <name part> <score>
Where <score> is the score of a player, and <name part> is some unique part of the player's name. If two players have too similar names, use /setscoreid instead.
When all the scores are entered, you need to sign them. Just say /signscores. The more players sign the scores, the better (one signature suffices for the current rating system); it will probably required to say /signscores twice because the system won't let you sign scores you haven't seen yet.

The rows of the tables are of the form
<rating> <id> <name>
where <rating> is the player's rating, <id> is the token the player is identified by (IP address or KeyID) and <name> is the name the player used last. The rating has to be interpreted in the following way: if player A has a rating that is x points higher than player B, then player A has a chance of beating player B in a direct confrontation of ( 1+tanh(X/100) )/2. tanh is just a randomly picked special function that goes to -1 if the argument goes to -infinity and to +1 if the argument goes to +infinity. With concrete numbers: if A has 100 points more than B, he will beat B roughly 90% of the time. If A has 50 points more than B, he'll win 70% of the time, and with 25 points more only 60% of the games.

Very roughly speaking, the system can be understood do do this: it counts how many games you won, and it logs against whom you played. It then organizes the ranking in such a way that the number of games you played against people below you on the list is equal to the number of games you won. There is some smoothing involved using the tanh function, but that's a little hard to explain, maybe some other time. Anyway, this means that the most effective way to make your position on the list more accurate (and make it rise, hopefully) is to play against people close to you on the list. If you win against someone way below you or lose against someone way above you, your position does not change. When the list has stabilized, you should pick opponents within a radius of 100 points from your current position.

As you see from the description, the absolute ranking values don't matter; only ranking differences are well defined. Nevertheless, the ranking table is fixed at a certain point: there is an invisible player at 800 points that wins against everyone with 90% probability that is taken into account with a very low statistical weight. It roughly serves the following purpose: a score of 800 or more means that you are in the top 10% of players. I figured that would be a good place to anchor the ratings; it should be more stable than the best or worst players and make the higher scores comparable even at different times.

Todo:
- prettier appearance
- filtering out players that haven't played enough for a stable position
- determine error bars
- make the displayed score the lower bound of the error bar

[shinygerbil]

Yaaay!

(now to actually get back into playing Defcon again!)

[walrus47]

Now i feel i have to play on your server a lot more now

[bert_the_turtle]

Now i feel i have to play on our server a lot more now

You just have to remember to use /setscore

I forgot to tell you how games with more than two players are handled: they are treated as if the winner of the game won against each of the other players in a 1 vs 1. In other words, only losing the least counts, losing the second least is as bad as losing the most. The system will be released for other DedCon admins, of course, and then there will be more flexible choices available; however, I think everything that rewards more than one player invites shinybonsais.

[Deamon]

Wonderful.

But I can't get the feeling that IV has gotten owned by one person with their whole staff, even if they did make the game and didn't work on this particular feature.

Well done bert.

[Eat_The_Path]

But I can't get the feeling that IV has gotten owned by one person with their whole staff, even if they did make the game and didn't work on this particular feature.

Not to belittle Bert's accomplishment, but isn't IV's devteam all of three people? I could be wrong here, but it's not like they're some massive studio with 30 coders hammering away. Anyway, I hope there are some serious customization options in this thing, because in 3v3 no defection, it doesn't really make sense to me to have the top scorer ranked as beating his team-mates too. For that matter, if you're going by total team score, the top scorer could actually have lost the game. Otherwise, damned cool.

[xander]

May I also point out that bert_the_turtle has been getting help from the IV dev team in order to get DedCon off the ground. He didn't pwn anyone, and I imagine that the whole project would have been much more difficult without IV's help.

xander

[shinygerbil]

I'd like to see bert making a rating system for A GAME WHICH DOESN'T EXIST.

>_>

<_<

Kudos to you though, bert. You've done a lot of haxxing which I happen to feel was very beneficial to the community -- almost necessary, even.

[bert_the_turtle]

xander is entirely correct. Plus, without a dedicated server, rankings would have no place to be (unless they are sent to a central server, which gives rise to manipulation and a whole lot of other problems), so there really was little sense in adding them for IV. Now what they really could be doing to help a lot (nudge, nudge) would be to let the clients send the scores to the server so you don't have to do that manually.

Eeat_the_path: If you want full automation, on servers without defection, that could be possible. Ingame alliance changes are not trackable reliably because there's the timeout period you have to get exactly right; lobby alliance changes are easier, and of course if you want team scores, you'll also want to form the teams before the game starts. There'll be the additional problem of how the teams shall be identified... Either way, it's a problem for later, right now I'd be happy if more rated non-team games appeared in the database.
There is a handmade solution that would work even now: each team elects a fixed captain and at the end of the game, the scores are added up and declared as the captain's score only.

Anyway, now that I played myself successfully to the bottom of the score table (but at least I'm in! It takes a statistically significant amount of won games to get there ), I'll be on vacation for the next ten days and leave my servers to themselves. They're in the hands of a capable banbot, so don't worry, but also don't test too often whether the chatspam block still works. Two days ago, I found the server without power in the morning; I hope that won't happen again while I'm away. See you then.

[walrus47]

Am I the only one who actuall use the /setscore and /signscore?

[Ace Rimmer]

I forget to when I play.

Perhaps a reminder could be posted by the bot after the game is over?

[Hyperion]

Am I the only one who actuall use the /setscore and /signscore?

It's a nice idea but i don't take the trouble to do this because i know almost everyone else won't either...which really isnt a helping attitude i know but still the point is valid

[bert_the_turtle]

Perhaps a reminder could be posted by the bot after the game is over?

Small problem there, the server only knows the game is over when all the players have left :/

[Amerigo Vespucci]

The whole Upper/lower/middle/shishkabob thing is a bit confusing for me... I can't tell what it means. Would it be possible to just do a 1/2/3/4/5/6 ranking system separated by the number of players in the game? You've done a great job on all this, and it's really impressive. Just wanted to give my two cents.

[bert_the_turtle]

The whole Upper/lower/middle/shishkabob thing is a bit confusing for me... I can't tell what it means.

That's easy: the middle score is your most probable score. However, it is far from certain your actual score IS that value, it just lies around the middle value. And the upper and lower bound scores tell you in which interval your score very probably (about 90%) is, based on the uncertainty that always goes with limited statistical samples.

Would it be possible to just do a 1/2/3/4/5/6 ranking system separated by the number of players in the game?

That hasn't got anything to do with the upper/lower/middle score But yes, it would be possible and even makes sense, because a three player game is somewhat different from a six player game. Right now, I'd be happy to get enough data to generate one table, so a separation isn't really feasible. At a later time, perhaps.

Oh yeah, there has been a small change in the algorithm: it now also takes score differences into account, albeit at a very low weight: Total annihilation of five territories with no own deaths is worth the same as one win. The main purpose is to give the second and third placed in a six player game some positive points.