Site scoring update (July 2017)

[super_dipsy]

The site scoring system is an elo-like system, and is described in some detail here http://www.playdiplomacy.com/forum/viewtopic.php?f=565&t=34913.

This post is to document changes made to the site scoring system in July 2017. The following summarizes the changes made:

- After 4 years experience with the old algorithm, it has now been tweaked to reflect the lessons learnt
- The new algoritm has been reapplied to the historic games to give a consistent position since the rating system started (Feb 2013)
- Player ratings are now available for a number of different categories (ranked games only):
--- Standard (Diplomacy on the classic map with classic rules, full communications and no variants)
--- All-game (all games no matter what the game options)
--- Fog of War
--- Gunboat
--- All games on the Classic map
--- All games on the Milan map
--- All games on the Ancient Med map
--- All games on the 1900 map
--- All games on the Versailles map

More details follow in subsequent posts

[super_dipsy]

WHY the changes?
There were two main drivers for these changes. The first was that extensive experience with the first iteration of the rating algorithm had showed up some issues that were affecting players and the second was that players had for a long time been asking to see how they performed in the major ' groupings' (eg different maps, gunboat games, fog games, pure Diplomacy as written in the Avalon Hill rules).

Feedback on the old rating algorithm had coalesced around a number of specific points. Players felt that it was too easy to zoom up to the top echelon of ratings after only a handful of games (this was because the rating swings in early games were very large). Players also felt that after playing many games on site, ratings were too static; wins and losses had minimal effects on ratings, making it hard to see how you could climb much higher and limiting the risk of falling much lower. There was also a specific concern about possible 'gaming' of the system by joining games with a load of newbies and picking up easy wins for lots of points.

As a result, the new algorithm, while still elo-like, has been adjusted to reduce the extent of early swings but increase the 'steady state' benefits (and penalties) for wins and losses that you see when you have played a good number of games. This makes ratings more fluid, meaning that you still have scope to climb well even after playing a lot of games, but of course also increasing the vulnerability of higher rated players with heavier losses. It also puts more importance on playing more games, making it harder to leap up to the highest ratings with just a handful of games.

[super_dipsy]

Q & A
1. Why did my rating drop so much?
If your rating dropped a lot, there are probably two most likely explanations. The first is that if you have only played a small number of games but done well in them, the smaller 'early player' bonus and the larger 'frequent player' bonus will have had a greater effect on you.
2. My rating didn't drop much, so why did my ranking fall much lower?
Others may have moved up or down too, but the most likely reason may be you may have dropped below 1000. Because 1000 is where players start, there is a large bulk of players at or around 1000, and therefore in ranking terms a big difference between 1100 and 900.
3. Why did you drop different maps from the Standard rating? Why is it not the same as the old Puerdip rating?
There are a few reasons for this. One is that it is the one rating where non-premiums can be judged on a level playing field with premium members, but by far the most important is that by keeping to the single map together with full communications and no variants, this rating reflects the 'real' Diplomacy game as laid down in the official rules.
4. I heard about people getting +400 for their first solo, but my first solo is half that. How is that fair?
Yes, people would have got around +400 for soloing their first game in the old system. But don't panic - because all the ratings have been reworked with the new algorithm, these historic results all reflect the new system, and hence those 400s obtained in the past will have halved just as yours has.
5. Since you were changing things, why did you not change to take into account how many SCs you have left at the end of the game, or how long you survived?
This is a common question from people used to playing torunaments, where such scoring is frequently used. But this is because in a torunament you have to finish games after a set time, and only have a small number of rounds, so you need some way of differntiating players apart from the norm. The major issue with having any sort of result mechanism that takes into account how long you survive / how many SCs you had at the end is it makes a completely different game, and we feel the changes would be negative rather than positive.
6. Will you ever restart the ratings? Start everyone off again at 1000?
There are definite attractions to this, giving everyone a fresh challenge and perhaps enabling players who have grown in expewrience the chance to start afresh. However, if you have put a lot of effort into getting a high rating then seeing it reset to the same as everyone else might be rather demotivating. At this point, we have no plans to do this.