Now available: Player Ratings

General discussion about Defcon

Moderator: Defcon moderators

User avatar
bert_the_turtle
level5
level5
Posts: 4795
Joined: Fri Oct 13, 2006 6:11 pm
Location: Cologne
Contact:

Now available: Player Ratings

Postby bert_the_turtle » Sun Aug 12, 2007 12:51 am

Experimental, of course: http://dedcon.homelinux.net/ratings/
Currently, only the Duel and TotallyRandom ratings are filled, and the combined ratings in "All". For lack of rated games, the scores are widely inaccurate; a single won six player game was all it took to catapult Lynchfinger to the top of the TotallyRandom list. Usually, I'd filter out players where not enough data is available yet, but then the list would be empty right now.

What do you have to do to get on the rankings or improve your score there? Well, obviously, you'll have to play on the DedCon Test servers, but that's not all. You need to stick around til the end end then make the scores known to the server (the server doesn't simulate the game and can't know the scores, and the clients don't tell the server what they are). Use the /setscore command to enter the scores, it is used like this:
/setscore <name part> <score>
Where <score> is the score of a player, and <name part> is some unique part of the player's name. If two players have too similar names, use /setscoreid instead.
When all the scores are entered, you need to sign them. Just say /signscores. The more players sign the scores, the better (one signature suffices for the current rating system); it will probably required to say /signscores twice because the system won't let you sign scores you haven't seen yet.

The rows of the tables are of the form
<rating> <id> <name>
where <rating> is the player's rating, <id> is the token the player is identified by (IP address or KeyID) and <name> is the name the player used last. The rating has to be interpreted in the following way: if player A has a rating that is x points higher than player B, then player A has a chance of beating player B in a direct confrontation of ( 1+tanh(X/100) )/2. tanh is just a randomly picked special function that goes to -1 if the argument goes to -infinity and to +1 if the argument goes to +infinity. With concrete numbers: if A has 100 points more than B, he will beat B roughly 90% of the time. If A has 50 points more than B, he'll win 70% of the time, and with 25 points more only 60% of the games.

Very roughly speaking, the system can be understood do do this: it counts how many games you won, and it logs against whom you played. It then organizes the ranking in such a way that the number of games you played against people below you on the list is equal to the number of games you won. There is some smoothing involved using the tanh function, but that's a little hard to explain, maybe some other time. Anyway, this means that the most effective way to make your position on the list more accurate (and make it rise, hopefully) is to play against people close to you on the list. If you win against someone way below you or lose against someone way above you, your position does not change. When the list has stabilized, you should pick opponents within a radius of 100 points from your current position.

As you see from the description, the absolute ranking values don't matter; only ranking differences are well defined. Nevertheless, the ranking table is fixed at a certain point: there is an invisible player at 800 points that wins against everyone with 90% probability that is taken into account with a very low statistical weight. It roughly serves the following purpose: a score of 800 or more means that you are in the top 10% of players. I figured that would be a good place to anchor the ratings; it should be more stable than the best or worst players and make the higher scores comparable even at different times.

Todo:
- prettier appearance
- filtering out players that haven't played enough for a stable position
- determine error bars
- make the displayed score the lower bound of the error bar
User avatar
shinygerbil
level5
level5
Posts: 4667
Joined: Wed Dec 22, 2004 10:14 pm
Location: Out, finding my own food. Also, doing the shinyBonsai Manoeuvre(tm)
Contact:

Postby shinygerbil » Sun Aug 12, 2007 12:57 am

Yaaay! :D

(now to actually get back into playing Defcon again!)
Here is my signature. Make of it what you will.
Image
User avatar
walrus47
level4
level4
Posts: 559
Joined: Sat Jul 28, 2007 10:56 am

Postby walrus47 » Sun Aug 12, 2007 1:10 am

Now i feel i have to play on your server a lot more now :P
User avatar
bert_the_turtle
level5
level5
Posts: 4795
Joined: Fri Oct 13, 2006 6:11 pm
Location: Cologne
Contact:

Postby bert_the_turtle » Sun Aug 12, 2007 9:07 am

walrus47 wrote:Now i feel i have to play on our server a lot more now :P
You just have to remember to use /setscore :)

I forgot to tell you how games with more than two players are handled: they are treated as if the winner of the game won against each of the other players in a 1 vs 1. In other words, only losing the least counts, losing the second least is as bad as losing the most. The system will be released for other DedCon admins, of course, and then there will be more flexible choices available; however, I think everything that rewards more than one player invites shinybonsais.
User avatar
Deamon
level2
level2
Posts: 240
Joined: Sun Dec 10, 2006 7:09 pm
Location: Éire

Postby Deamon » Sun Aug 12, 2007 7:19 pm

Wonderful.

But I can't get the feeling that IV has gotten owned by one person with their whole staff, even if they did make the game and didn't work on this particular feature.

Well done bert.
Image
Eat_The_Path
level1
level1
Posts: 67
Joined: Tue Jun 19, 2007 7:02 am
Location: The Past

Postby Eat_The_Path » Sun Aug 12, 2007 7:43 pm

Deamon wrote:But I can't get the feeling that IV has gotten owned by one person with their whole staff, even if they did make the game and didn't work on this particular feature.


Not to belittle Bert's accomplishment, but isn't IV's devteam all of three people? I could be wrong here, but it's not like they're some massive studio with 30 coders hammering away. Anyway, I hope there are some serious customization options in this thing, because in 3v3 no defection, it doesn't really make sense to me to have the top scorer ranked as beating his team-mates too. For that matter, if you're going by total team score, the top scorer could actually have lost the game. Otherwise, damned cool.
Last edited by Eat_The_Path on Sun Aug 12, 2007 8:11 pm, edited 1 time in total.
User avatar
xander
level5
level5
Posts: 16869
Joined: Thu Oct 21, 2004 11:41 pm
Location: Highland, CA, USA
Contact:

Postby xander » Sun Aug 12, 2007 7:50 pm

May I also point out that bert_the_turtle has been getting help from the IV dev team in order to get DedCon off the ground. He didn't pwn anyone, and I imagine that the whole project would have been much more difficult without IV's help.

xander
User avatar
shinygerbil
level5
level5
Posts: 4667
Joined: Wed Dec 22, 2004 10:14 pm
Location: Out, finding my own food. Also, doing the shinyBonsai Manoeuvre(tm)
Contact:

Postby shinygerbil » Sun Aug 12, 2007 8:34 pm

I'd like to see bert making a rating system for A GAME WHICH DOESN'T EXIST.

>_>

<_<

Kudos to you though, bert. You've done a lot of haxxing which I happen to feel was very beneficial to the community -- almost necessary, even. :)
Here is my signature. Make of it what you will.

Image
User avatar
bert_the_turtle
level5
level5
Posts: 4795
Joined: Fri Oct 13, 2006 6:11 pm
Location: Cologne
Contact:

Postby bert_the_turtle » Sun Aug 12, 2007 11:34 pm

xander is entirely correct. Plus, without a dedicated server, rankings would have no place to be (unless they are sent to a central server, which gives rise to manipulation and a whole lot of other problems), so there really was little sense in adding them for IV. Now what they really could be doing to help a lot (nudge, nudge) would be to let the clients send the scores to the server so you don't have to do that manually.

Eeat_the_path: If you want full automation, on servers without defection, that could be possible. Ingame alliance changes are not trackable reliably because there's the timeout period you have to get exactly right; lobby alliance changes are easier, and of course if you want team scores, you'll also want to form the teams before the game starts. There'll be the additional problem of how the teams shall be identified... Either way, it's a problem for later, right now I'd be happy if more rated non-team games appeared in the database.
There is a handmade solution that would work even now: each team elects a fixed captain and at the end of the game, the scores are added up and declared as the captain's score only.

Anyway, now that I played myself successfully to the bottom of the score table (but at least I'm in! It takes a statistically significant amount of won games to get there :) ), I'll be on vacation for the next ten days and leave my servers to themselves. They're in the hands of a capable banbot, so don't worry, but also don't test too often whether the chatspam block still works. Two days ago, I found the server without power in the morning; I hope that won't happen again while I'm away. See you then.
User avatar
walrus47
level4
level4
Posts: 559
Joined: Sat Jul 28, 2007 10:56 am

Postby walrus47 » Fri Aug 17, 2007 1:46 pm

Am I the only one who actuall use the /setscore and /signscore?
User avatar
Ace Rimmer
level5
level5
Posts: 10803
Joined: Thu Dec 07, 2006 9:46 pm
Location: The Multiverse

Postby Ace Rimmer » Fri Aug 17, 2007 2:34 pm

I forget to when I play. :cry:

Perhaps a reminder could be posted by the bot after the game is over?
Smoke me a kipper, I'll be back for breakfast...
User avatar
Hyperion
level5
level5
Posts: 2102
Joined: Sun Dec 10, 2006 4:26 am
Location: England, UK

Postby Hyperion » Fri Aug 17, 2007 7:51 pm

walrus47 wrote:Am I the only one who actuall use the /setscore and /signscore?


It's a nice idea but i don't take the trouble to do this because i know almost everyone else won't either...which really isnt a helping attitude i know but still the point is valid :roll:
User avatar
bert_the_turtle
level5
level5
Posts: 4795
Joined: Fri Oct 13, 2006 6:11 pm
Location: Cologne
Contact:

Postby bert_the_turtle » Thu Aug 23, 2007 9:59 pm

Ace Rimmer wrote:Perhaps a reminder could be posted by the bot after the game is over?
Small problem there, the server only knows the game is over when all the players have left :/
Amerigo Vespucci
level1
level1
Posts: 70
Joined: Mon Oct 02, 2006 2:02 am

Postby Amerigo Vespucci » Sun Aug 26, 2007 8:19 pm

The whole Upper/lower/middle/shishkabob thing is a bit confusing for me... I can't tell what it means. Would it be possible to just do a 1/2/3/4/5/6 ranking system separated by the number of players in the game? You've done a great job on all this, and it's really impressive. Just wanted to give my two cents.
User avatar
bert_the_turtle
level5
level5
Posts: 4795
Joined: Fri Oct 13, 2006 6:11 pm
Location: Cologne
Contact:

Postby bert_the_turtle » Sun Aug 26, 2007 10:47 pm

Amerigo Vespucci wrote:The whole Upper/lower/middle/shishkabob thing is a bit confusing for me... I can't tell what it means.
That's easy: the middle score is your most probable score. However, it is far from certain your actual score IS that value, it just lies around the middle value. And the upper and lower bound scores tell you in which interval your score very probably (about 90%) is, based on the uncertainty that always goes with limited statistical samples.


Amerigo Vespucci wrote:Would it be possible to just do a 1/2/3/4/5/6 ranking system separated by the number of players in the game?
That hasn't got anything to do with the upper/lower/middle score :) But yes, it would be possible and even makes sense, because a three player game is somewhat different from a six player game. Right now, I'd be happy to get enough data to generate one table, so a separation isn't really feasible. At a later time, perhaps.

Oh yeah, there has been a small change in the algorithm: it now also takes score differences into account, albeit at a very low weight: Total annihilation of five territories with no own deaths is worth the same as one win. The main purpose is to give the second and third placed in a six player game some positive points.

Return to “General”

Who is online

Users browsing this forum: No registered users and 1 guest