Scoring System for Second PDL Season

Suggestions to improve the concept of a PlayDiplomacy League

Moderator: mjparrett

Re: Scoring System for Second PDL Season

Postby hedge trimmer » 26 Mar 2018, 12:54

-Overall the current system of doubling the points for every elimination is simple and easy to understand, if you ignore the center counts that aren't really used. Perhaps it encourages the relatively lucrative 3WDs too much though?
-Currently having 2WDs to be twice as valuable as 3WDs might push people towards eliminating the third player, moving the game away from the stable 3WD. However, I don't think this has happened too often, except in League 3. Even there the reason for doing that hasn't been because of the scoring system anyway, as far as I can tell.
-Penalizing surrenders doesn't really matter, as I doubt the people who surrender care enough about PDL to play in the remaining games.
-Even in the current system, if you solo you're almost certainly going to get promoted, and in very good position to win the league.

So perhaps the current system encourages 3WDs too much, and I suppose this change might encourage more solos, but part of that is that 4WDs & 5WDs aren't as heavily penalized in comparison to 3WDs. In other words it might be harder to set up a solid stop-the-leader alliance if there are smaller powers that are too busy fighting each other to stop the solo. Of course, that's assuming there somehow are less eliminations, as it might just be that league/tournament setting encourages small draws.

...I don't know what I'm talking about either, do I have to vote? I don't see anything wrong with the suggestion, but I think we need to see more rounds before deciding if something is really broken or not.
Silent and deadly Loud and annoying
User avatar
hedge trimmer
 
Posts: 128
Joined: 27 Jul 2017, 21:36
Location: Finland
Class: Star Ambassador
Standard rating: 1382
All-game rating: 1400
Timezone: GMT+2

Re: Scoring System for Second PDL Season

Postby mhsmith0 » 27 Mar 2018, 05:27

The nature of the current system is that it substantially incentivizes draw whittling, not just because the % share you get goes up, but that the size of the points pool itself goes way up with every elimination:

solo or 2-way draw: 40 points total
3-way draw: 30 points total
4-way draw: 20 points total
5-way draw: 10points total
6/7-way draw: 0 points total

The fundamental question, I think, is whether or not that's something we like or whether that's something we want to change.
Proud holder of the Superior Tophat of Solving, an item entrusted with the forum's most prominent smartass
User avatar
mhsmith0
 
Posts: 3612
Joined: 11 Dec 2015, 06:55
Location: Phoenix, AZ
Class: Star Ambassador
Standard rating: (1269)
All-game rating: (1439)
Timezone: GMT-7

Re: Scoring System for Second PDL Season

Postby duckling » 27 Mar 2018, 08:35

That’s something I like, I think.

If I’m thinking about a solo, taking on 3 others alone is hard.
Seems to me it is easier to open up for a solo chance this way.

my curiosity is triggered....I have to go simulate some rounds now to see how scoring is affected.
duckling
 
Posts: 142
Joined: 06 Jan 2010, 17:48
Class: Star Ambassador
Standard rating: (1130)
All-game rating: (1176)
Timezone: GMT+1

Re: Scoring System for Second PDL Season

Postby mjparrett » 27 Mar 2018, 16:17

Let us know how the simulations go!

I appreciate the post refers to bringing this in for the second season (lets hope there is one!) as I don't like making changes half way. But I also agree with another post that says we need a little more analysis - it certainly isn't broken just now. It might not be perfect, but it isn't the worst. And I think the league format will have a bigger impact on draws/solos rather than how many specific points you get for each (which is the same for everyone). Basically what I mean is, I don't think it will make any difference (unless you did something mental like keep the current structure except 1000 points for a solo).
mjparrett
 
Posts: 369
Joined: 01 Mar 2017, 20:05
Location: Scotland
Class: Star Ambassador
Standard rating: 1488
All-game rating: 1524
Timezone: GMT

Re: Scoring System for Second PDL Season

Postby mhsmith0 » 27 Mar 2018, 23:15

duckling wrote:That’s something I like, I think.

If I’m thinking about a solo, taking on 3 others alone is hard.
Seems to me it is easier to open up for a solo chance this way.

my curiosity is triggered....I have to go simulate some rounds now to see how scoring is affected.


fwiw, conventional wisdom is that the more players who are alive, the easier it is to solo. Essentially the key reasons being that

1) it is logistically harder to coordinate a defense even among motivated allies when you have more independent powers doing different things
2) relatively minor powers that are very likely to get shut out of a final result regardless of whether it's a solo or a small draw are relatively likelier to throw the game to the aspiring solo'ist instead of just rolling over for the smaller alliance that's trying to kill them anyway

You can even look at my first PDL game (PDL 2, round 1, game #1, https://www.playdiplomacy.com/game_play ... _id=138643 ) as an example. Fall 1908:
Image
England was the top power, and decided to stab me (France) and take a chance at either whittling the draw or maybe even hitting a solo. He ended up losing outright because F/A/T made an organized resistance against him. Imagine if Russia and/or Italy was a rump power that had the ability to help England move towards a solo by making it difficult or impossible to resist him (as reference, the one rump power on the board, Germany, WAS in fact a thorn in the side of the counter-alliance, as in 1909 he stabbed me and took Belgium making it more difficult and time-consuming to shut England down, even though it happened in the end).
Proud holder of the Superior Tophat of Solving, an item entrusted with the forum's most prominent smartass
User avatar
mhsmith0
 
Posts: 3612
Joined: 11 Dec 2015, 06:55
Location: Phoenix, AZ
Class: Star Ambassador
Standard rating: (1269)
All-game rating: (1439)
Timezone: GMT-7

Re: Scoring System for Second PDL Season

Postby Riles » 29 Mar 2018, 07:48

mjparrett wrote:Let us know how the simulations go!

I appreciate the post refers to bringing this in for the second season (lets hope there is one!) as I don't like making changes half way. But I also agree with another post that says we need a little more analysis - it certainly isn't broken just now. It might not be perfect, but it isn't the worst. And I think the league format will have a bigger impact on draws/solos rather than how many specific points you get for each (which is the same for everyone). Basically what I mean is, I don't think it will make any difference (unless you did something mental like keep the current structure except 1000 points for a solo).


I would disagree that it won't make a difference. In general, scoring systems differ the most not where things seem logical (solos > draw), but where you get into close cases where it is not obvious who should be ranked higher than who. For example, let's compare two players, Alice has a 2-way, a 5-way, and two losses. Bob has 3 4-way's and a loss. Who should be ranked higher than who?

Using the current method, Alice would have 20+2+0+0=22 points while Bob had 5+5+5+0=15 points. Alice is ranked higher.
Using the proposed method of OP, Alice would have 30+12+0+0=42 points while Bob had 15+15+15+0=45 points. Bob is ranked higher.

Close cases are going to be what probably separates who gets promoted/relegated and who doesn't, so these intricacies do matter. Someone that solos 3 times out of 4 will win their league in any scoring method while someone who loses 4 times will always be relegated.

In the current method, someone that solos once and loses the rest will be tied by someone that had 4 3-way's. Does that sound like a fair trade of equal effectiveness in the league?
In the proposed method, the person with 4 3-way's would beat the 1-time soloist. Did the person with 4 3-way's actually perform better in the league?

I'm not taking a stance one way or another on any of this, it just depends on what the league considers "better league play" than another.

As for saying give 1000 to a solo, I wouldn't necessarily call that mental. Perhaps the league wants to focus on solos being the primary point of Diplomacy. So a person that solos once can lose the other three and still be ranked higher than someone that had 4 two-way draws. After all, that 4-time draw-er didn't actually ever win a game of Diplomacy in the league. It just depends on what the league wants to value and that value should be directly tied to the scoring system chosen. On the other hand, perhaps the league wants to value surviving over all else, so awards -1000 points if you end up with 0 SC's by the end of the game. If you can convince everyone to NOT eliminate you, doesn't that say something too? In that case, maybe the 4 time 2-way draw-er would crush the single soloist because while the soloist was able to solo, they got eliminated the other three times (probably by the 2-way draw-er at least once).

I would also challenge that 2-way draws are always better than 3-way draws are always better than 4-way.... After all, if a player is able to get to 17 centers on a board with 4 other non-eliminated players of generally equal power, didn't that player generally dominate compared to the others? If there's a 2-way 17-17 split then that's two people that dominated, not a single person. Perhaps the single person should be rewarded more than the pair? That's one of the philosophies behind the Sum-of-Squares scoring method. Scoring 3-ways as better than 4-ways in all cases also encourages games to last longer as three major powers take several game years trying to eliminate that pesky small fourth power where a different scoring method may find that eliminating that fourth power would actually hurt you if you are not the one taking the fourth's SC's (Sum-of-Squares) or that the elimination of the fourth power barely affects you at all (C-Diplo).

If the proposed method is adopted by the league that says that the league wants 4 3-way draws to be better than a single solo with 3 losses instead of them being tied; and that Bob should be ranked higher than Alice. These are the types of nuances that could separate someone from being promoted to the next league.

mhsmith0 wrote:The nature of the current system is that it substantially incentivizes draw whittling, not just because the % share you get goes up, but that the size of the points pool itself goes way up with every elimination:

solo or 2-way draw: 40 points total
3-way draw: 30 points total
4-way draw: 20 points total
5-way draw: 10points total
6/7-way draw: 0 points total

The fundamental question, I think, is whether or not that's something we like or whether that's something we want to change.


A corollary I would add is, what does the league want to value? Do we want to value eliminations by making 3-ways always better than 4-ways? Or do we want to value general board dominance more than eliminations? How would we want solos to compare to all this? How much should we value just surviving (using the example of PDL 3 round 1 game 1 where England survived but was not a part of the draw)? If a solo happens in 10 game years, should that be scored different than a solo that took 12 game years? How could any of this effect other factors like average game length, frequency of stabs (or likelihood of "BFF" alliances), or kingmaking?

Regardless, I think it would be wise that at the end of the league to apply a lot of different scoring methods to the end results (ignoring the fact that some people may play the game differently if we had used another scoring system). The current method would be the official one for promotion/relegation/etc purposes of course, but I think applying a lot of different methods (including methods that do not depend on the size of the draw) may help us see what method generally fits with what the league wants to value (or make it look like it really doesn't matter because they all returned the same rankings anyway).

I have a fascination with how different leagues are set up in general (tournaments, sports leagues, even voting methods for elections), so I possibly am making too big a deal of this.

To re-iterate, I'm not favoring one option over another. I just want to make sure that the scoring system is in line with what we want the league to value as a whole. The current method is certainly good enough and doesn't have any major flaws that I see, it just emphasizes small draws a lot, which emphasizes draw-whittling, which may or may not be what we want (to echo mhsmith).
Riles
 
Posts: 36
Joined: 25 Jan 2018, 06:14
Class: Ambassador
Standard rating: (1164)
All-game rating: (1362)
Timezone: GMT-5

Re: Scoring System for Second PDL Season

Postby Custer » 29 Mar 2018, 11:48

Here, here.....like he said!

The SHIV
First..........get off my lawn! Second........it's a dashing self portrait! Courtesy of The Craw. Third.....I am still SHIV, Keeper of the Stone Tablets! Go Pack! And behold the power of cheese!
User avatar
Custer
Premium Member
 
Posts: 3245
Joined: 24 Jan 2009, 20:29
Location: Sailing somewhere in the Rockies on the Black Pearl but originally a Yooper!
Class: Ambassador
Standard rating: 1099
All-game rating: 1088
Timezone: GMT-6

Re: Scoring System for Second PDL Season

Postby Saturos » 29 Mar 2018, 16:52

I'm almost down for 1000 points (or just even more points) for a solo. My only concern is that 2 way draws are already very unappealing, since it is often so easy to grab the solo at the end (and who wouldn't if it is so much more?)

I'd almost say it should be a sharper spike, with 2 way still awarding quite a bit and then 3 way dropping off significantly.

Something like:

solo - 100
2 way - 65 each
3 way - 30 each
4 way - 12 each
5 way - 7 each
6 way - 3 each
7 way - 0 each

I do think there needs to be some change to make 3 way less appealing, and both solo and 2 way more appealing.

Currently the risk reward for 3 way is just way too good compared to everything else. And I think only focusing on improving solo points will change it to 3 way or 1 way. Therefore, I think something like I suggested might help increase variation more evenly across solo/2 way/3 way.

Personally, in a 7 player game, I feel like 4 or higher draws feel cheap since it is the majority and should be discouraged. That's why I like a more "exponential" curve.
Saturos
 
Posts: 34
Joined: 17 Aug 2017, 23:13
Class: Ambassador
Standard rating: (1374)
All-game rating: (1497)
Timezone: GMT

Re: Scoring System for Second PDL Season

Postby mjparrett » 29 Mar 2018, 21:25

All very good points. A fascinating post just there about the different scoring systems. Obviously you are right and different methods will make some players change league position - what I mean is though that whatever method is applied is the same for all and there is no use in saying "I would have been better than you under a different system". Because that is very subjective, and the easy comeback of "I would have played differently under a different system and still beaten you" is fairly valid.

It gets really messy when you start saying a solo in 10 years is better than 12. And I don't overly agree with that. I would probably argue an Italian solo is better than a Russian one, but again I know some people disagree.

The scoring system was one of the more talked about points when I started this, and this thread is just backing up what we discovered then - you WON'T get a unanimous verdict on what is better. So you pick something and stick with it. The other danger with changing the system before round 2 is that there will likely be (using your examples above) some people who were relegated who wouldn't have been under the new system and will complain things are unfair. Again very messy.
mjparrett
 
Posts: 369
Joined: 01 Mar 2017, 20:05
Location: Scotland
Class: Star Ambassador
Standard rating: 1488
All-game rating: 1524
Timezone: GMT

Re: Scoring System for Second PDL Season

Postby Shyvve » 29 Mar 2018, 22:00

Riles wrote:A corollary I would add is, what does the league want to value? Do we want to value eliminations by making 3-ways always better than 4-ways? Or do we want to value general board dominance more than eliminations? How would we want solos to compare to all this? How much should we value just surviving (using the example of PDL 3 round 1 game 1 where England survived but was not a part of the draw)? If a solo happens in 10 game years, should that be scored different than a solo that took 12 game years? How could any of this effect other factors like average game length, frequency of stabs (or likelihood of "BFF" alliances), or kingmaking?

Regardless, I think it would be wise that at the end of the league to apply a lot of different scoring methods to the end results (ignoring the fact that some people may play the game differently if we had used another scoring system). The current method would be the official one for promotion/relegation/etc purposes of course, but I think applying a lot of different methods (including methods that do not depend on the size of the draw) may help us see what method generally fits with what the league wants to value (or make it look like it really doesn't matter because they all returned the same rankings anyway).

I have a fascination with how different leagues are set up in general (tournaments, sports leagues, even voting methods for elections)
*, so I possibly am making too big a deal of this.

To re-iterate, I'm not favoring one option over another. I just want to make sure that the scoring system is in line with what we want the league to value as a whole. The current method is certainly good enough and doesn't have any major flaws that I see, it just emphasizes small draws a lot, which emphasizes draw-whittling, which may or may not be what we want (to echo mhsmith).

*Emphasis added.

Adding some of my "gut-feeling" subjective thoughts:
-Yes, IMO 3WD's are always better than 4WD's, 4WD's>5WD's, etc. With the caveat being that no draw is a 'win'. I'm trying to find a solution to disincentivize the all too frequently seen 3WD's while also keeping the total points per game constant too (except at the extremes of a solo on one hand and 6-way or 7-way draws on the other).

-IMO, a player earning four 3WD's should NOT end up tied with a player who got one solo and three eliminations. The soloing player should score higher (something my proposed change doesn't allow since they'd end up tied as previously pointed out). Whoops!

-I say our current system of tiebreaker points addresses the issue of rewarding survival sufficiently.

-The idea of a sliding scale for the number of game years necessary to achieve a solo never even occurred to me. This is something that might be worth considering although I think that any increased points being given for a quicker solo should be very small in relation to the points given for the solo itself.

-I really like the idea of testing other scoring systems in a "what if, let's see" fashion at the end of our inaugural season. This would really give us an idea as to how we'd like to fine tune things going forward. I've never participated in any Elo-like systems (other than here for about 15 months now) and I'd be interested in helping crunch numbers on something like this at the end of the season. I'm at least casually familiar with SoS and C-Diplo systems having read about them some.

In addition, and although my OP didn't specifically mention this, when the League was first proposed there was some brief talk about a Championship Game for each League Division. I still think it would be great to have an event like this to cap things off and a SoS system would probably be ideal for such a one-off event. There would also be potential scheduling conflicts with the ensuing season of course. But I think the players who anticipated they were going to qualify for any Championship game could simply adjust their current number of games being played so as to not be overwhelmed when the first game of the new season and the Championship game are overlapping.
An Oldie and Gold Classicist. Moderator for the Classicist group.
User avatar
Shyvve
Premium Member
 
Posts: 425
Joined: 31 Dec 2016, 20:10
Class: Star Ambassador
Standard rating: 1351
All-game rating: 1381
Timezone: GMT-6

PreviousNext

Return to PDL suggestions

Who is online

Users browsing this forum: No registered users and 1 guest