Site scoring system (Feb 2nd 2013)

[super_dipsy]

Thank you BlueBorjigin - I sometimes get a bit weary of saying the same things again

Regarding the potential of losing points from a draw, does this depend on the ratings of all the players in the game, or just those remaining in the draw?

it has no relevance who remains - only who played.

I think it would be quite bad if there is any meta-gaming incentive to team up with this player over that player.

There is ALWAYs such an incentive - there was on the old system, and there will be on the new system. That is why we have tools that look for meta-gaming and that is why we hand out warnings and bans. If you are worried, play anonymous games.

Can you give us some idea of how much better a solo is compared to a two-way draw (without telling us the algorithm obviously). For example, if you have evenly rated players in a game, how much better would winning the game be compared to a two-way draw?

You will see soon enough

Is it possible to get a high rating without ever getting a solo? Relatedly, how do players who frequently draw, but never solo, compare to players who rarely draw, but do better than 1/7 for solos? For example, who is the rating system likely to favour: a player who draws 50% of the time, but never solos, or a player who solos 20% of the time, but never draws (holding fixed the ratings of all the players).

Yes; it depends; it depends.

Sorry to be a bit unhelpful, but really I think you may have misunderstood the way the system works. Your rating in any game depends on your rating and those who you play. So asking how players who frequently draw or frequently solo do is nonsensical because in each case it depends on the ratings of the other players. Do you see what I mean? Even asking how high you can get if you never solo does not make any sense. If you play every game against 6 players who are much more highly rated than you, then every draw will give you a good lift, and assuming you are able to continue finding such games then there is no limit (literally) to how high you can go. Of course, there IS a limit because eventually you would be unable to find 6 players much higher than you to play, unless we had an infinite pool of players. There has to be a ceiling. But yes, you can theoretically get a high rating if you never solo - will you be able to in practice with our pool of players? Don't know - the maths is way too complicated, and anyway it is completely subjective since 'high' could mean greater than 1500 or greater than 2000 or anything really

[super_dipsy]

Sorry that may have sounded a bit snappy - I wasn't meaning to cause any offence. I guess I just want to get the solution live (only 3 days to go!) . I'm sure things will be clearer for everyone once they can see it working (or not if I have left too many bugs in )

[TheProfessional]

Thank you BlueBorjigin - I sometimes get a bit weary of saying the same things again

My mistake, I must have missed it.

There is ALWAYs such an incentive - there was on the old system, and there will be on the new system. That is why we have tools that look for meta-gaming and that is why we hand out warnings and bans. If you are worried, play anonymous games.

I guess I meant "non-cheating" forms of meta-gaming, but perhaps there is no such thing. Also, I'm less concerned about people intentionally meta-gaming as I am about the system rewarding subconsciously making choices that cause imbalances and meta-gaming play. As far as I can see, not all forms of meta-gaming are bad. For example, I haven't heard complaints about doing research on a player by looking at his or her previous games to gauge how they might play against you (this is no different from studying your opponent's games in chess). But this, of course, can create unintentional forms of bad meta-gaming, such as players ganging up on someone they perceive from examining previous games as being a very good player. Also, I am not so much concerned with my own particular games, as I am with the culture that the rating system encourages, so anonymous games (putting aside the fact that they're only available to premium players) are not entirely helpful (at least not with discouraging people with settling for a draw).

Yes; it depends; it depends.

Sorry to be a bit unhelpful, but really I think you may have misunderstood the way the system works. Your rating in any game depends on your rating and those who you play. So asking how players who frequently draw or frequently solo do is nonsensical because in each case it depends on the ratings of the other players. Do you see what I mean? Even asking how high you can get if you never solo does not make any sense. If you play every game against 6 players who are much more highly rated than you, then every draw will give you a good lift, and assuming you are able to continue finding such games then there is no limit (literally) to how high you can go. Of course, there IS a limit because eventually you would be unable to find 6 players much higher than you to play, unless we had an infinite pool of players. There has to be a ceiling. But yes, you can theoretically get a high rating if you never solo - will you be able to in practice with our pool of players? Don't know - the maths is way too complicated, and anyway it is completely subjective since 'high' could mean greater than 1500 or greater than 2000 or anything really

Well... I don't think I misunderstood the way the system works, but I appreciate that it is complicated. By 'high' I meant something like this: if you tend to play against players of a rating with a certain interval (say between 1600-1900) then what would be your chance of getting to the top of that interval if you had a tendency to draw instead of solo? More precisely: if you start at the bottom of that interval, how often would you have to two-way draw in order to get to the top of the interval? Similarly: how does some frequency of (for example) two-way drawing compare to some other frequency of soloing? Or: how often do you have to solo if you want to do just as well rating-wise as a player (initially of equal rating to you) who two-way draws fifty percent of the time, when playing against people of the same rating as you? Sure these are complicated question, with a lot of variables, but I do think they make sense within the new system, once we fix the variables. I don't expect you to have answers, but I thought you might have mathematical hunches regarding these sorts of questions. I guess I was hoping (though again not expecting) that the rating system might have been designed with these sorts of questions in mind.

The "snappiness" didn't bother me. I appreciate all the work that's been put into the project regardless of whether it addresses my concerns about players treating draws as if they are themselves worthy of aiming at.

[super_dipsy]

Well... I don't think I misunderstood the way the system works, but I appreciate that it is complicated. By 'high' I meant something like this: if you tend to play against players of a rating with a certain interval (say between 1600-1900) then what would be your chance of getting to the top of that interval if you had a tendency to draw instead of solo? More precisely: if you start at the bottom of that interval, how often would you have to two-way draw in order to get to the top of the interval? Similarly: how does some frequency of (for example) two-way drawing compare to some other frequency of soloing? Or: how often do you have to solo if you want to do just as well rating-wise as a player (initially of equal rating to you) who two-way draws fifty percent of the time, when playing against people of the same rating as you? Sure these are complicated question, with a lot of variables, but I do think they make sense within the new system, once we fix the variables. I don't expect you to have answers, but I thought you might have mathematical hunches regarding these sorts of questions. I guess I was hoping (though again not expecting) that the rating system might have been designed with these sorts of questions in mind.

Hmmm. I think part of the problem is you are hoping for fixed ratios. It doesn't work like that. I do have 'hunches' - indeed, I have the benefit of having been able to use our historical data to do a lot of modeling and tuning, although of course I know this is rather suspect since games may have been chosen and played differently under the new system than the old. That is why I specifically pointed out that solos are weighted more heavily than draws, and gave the example that a solo is more than 2 2-way draws etc..

I am not trying to be unhelpful, but I am reluctant to go into too much detail. I am a realist - I know that there will always be people who play the system rather than the game, and for these people it is important that they understand the nuances of how the scoring system works so they can maximize their chances of placing as high as possible. But I am against encouraging this. My ideal would be people would play the game and the ratings would take care of themselves, and for many people I think that is the way they will play when based on a set of basic guidelines. The basic guidelines I want everyone to be aware are
- soloing is going to give you a better score faster and is weighted more highly than draws
- your rating adjustment each game will take into account the ratings of your opposition compared to yourself; ie good performances against stronger rated players will be recognized with a higher adjustment etc
- picking up surrendered countries when your shield is charged (eg you have started a rated game since you last used it) is a smart thing to do now and then because you get a free play (no negative score possible)

However, even if I was prepared to go into more detail, there are too many variables even now to answer your questions as phrased. You ask what hunches do I have? My hunch is players who solo a lot will get to the highest ratings, and their ratings will climb faster. Does that help?

[Mortiferus Rosa]

So, this is elo essentially? I approve!

[TheProfessional]

However, even if I was prepared to go into more detail, there are too many variables even now to answer your questions as phrased. You ask what hunches do I have? My hunch is players who solo a lot will get to the highest ratings, and their ratings will climb faster. Does that help?

Well, that was true even in the old system, so not really, but I take your point. I agree with you entirely about keeping the algorithm secret, and answering my questions in an informative way may jeopardise that, so I'll stop bugging you.

[Kemev]

Could you elaborate on why your country doesn't factor into your points earned per game?

Considering there's pretty good data to suggest that all countries are not equally likely to win or draw, it seems like your country should affect your Elo rating. So a solo playing Italy should be worth more than a solo win with France, and getting eliminated in Austria should hurt less than getting eliminated with Turkey.

[asudevil]

Could you elaborate on why your country doesn't factor into your points earned per game?

Considering there's pretty good data to suggest that all countries are not equally likely to win or draw, it seems like your country should affect your Elo rating. So a solo playing Italy should be worth more than a solo win with France, and getting eliminated in Austria should hurt less than getting eliminated with Turkey.

Because on average, every player will play every country approximately equally. And nation-wide dip stats say that Russia solos the most...but here, its Turkey...so whose difficulty to solo stats do we use?

But that's why we aren't incorporating that.

[Kemev]

Because on average, every player will play every country approximately equally.

Really? It seems like the preferences and first-come/first-served options would skew the country distribution. Are there so few games with those options that they're irrelevant? It also seems likely that most players won't finish enough games to have a decent sample, which could further bias their ratings.

And nation-wide dip stats say that Russia solos the most...but here, its Turkey...so whose difficulty to solo stats do we use?

Could do both; use the playdiplomacy stats for the site ranking, and a broader stat set for the puredip rating.

[asudevil]

Could, but then we are back to so many options and making it even harder for people to understand the way scoring works.

I think preferences/first come are about 20-25% of games, so a decent amount, but again...most players are going to get a relatively equal distribution.