All Diplomacy Scoring Stinks

[BrotherBored]

Here's an interesting new article by BunnyGo!

It's amazing how just the one topic of scoring generates as much talk as any of the actual game concepts.

[super_dipsy]

I have to admit I did not get the discussion of Utility points etc at all. The description given of bridge scoring for example bears no relation to any bridge rating system I know. When it comes to Diplomacy, although there is a fleeting mention of Elo, there is no discussion at all (as far as I can see) of the sort of Elo-like system we use at Playdip. Surely the essence of a Diplomacy scoring system in a none-tourney environment (and remember I haven't a clue what the three flavours he names mean in practice) is to reflect achievement vs expectation and provide a relative 'skill level' to other players also in the system?

[V]

I have to admit I did not get the discussion of Utility points etc at all. The description given of bridge scoring for example bears no relation to any bridge rating system I know. When it comes to Diplomacy, although there is a fleeting mention of Elo, there is no discussion at all (as far as I can see) of the sort of Elo-like system we use at Playdip. Surely the essence of a Diplomacy scoring system in a none-tourney environment (and remember I haven't a clue what the three flavours he names mean in practice) is to reflect achievement vs expectation and provide a relative 'skill level' to other players also in the system?

Agreed on all points “Super”. I’ve never heard such garbage relating to Bridge scoring. The author sounds confused & unable to articulate a reasoned argument. The PlayDip scoring system is well considered & excellent for purpose.

[jay65536]

While I do agree that the article could be better written--and also I think the author is webDip-centric and may not even be aware of Playdip's rating system--there is some substance in some of the points being made. So let me try to unpack what I think he means by some of the terms being thrown around.

The big thing that this article highlights is that not enough people make a distinction in their minds between a rating system and a scoring system. Rating systems are designed to be a measure of skill, taken across arbitrarily many games on an ongoing basis. (I don't play bridge, but my analogue is chess.) So when the author refers to "skill points", that's referring to points being used for rating systems. Scoring systems are designed for competitive tournaments played over a fixed-length string of games.

dipsy, since you said you didn't understand the terminology, maybe I can help a bit. In game theory, "utility" just means the value to you of a particular outcome. The crudest analogy would be if you were playing a game for money. Like let's say we're playing Diplomacy and each player puts in $6, making a pot of $42. If we're playing winner-take-all, then if you solo, you win $36, and if you lose to a solo, you're out $6. So the utility of a solo to you is +36, and the utility of a loss is -6. (Or you could say the utility of a solo is +42 and the utility of a loss is 0; they're equivalent.) But that's what is meant by "utility".

Where the terminology falls apart is that there are really only 2 different kinds of points: rating points (which he calls "skill points") and tournament scoring points. "Utility" is really an umbrella term that encompasses both of the different situations.

I'll withhold comments about the substance of the article itself (not generally a fan); hopefully what I said helps you understand the points being made.

[super_dipsy]

Thanks Jay. With your post as a pointer, I was able to decode enough of the article to understand what he was trying to say. As a bridge player, I found it very difficult to have any faith in the article because the jargon did not relate at all to anyone who has played bridge in tournaments or clubs. Now that you have explained the jargon, I can vaguely see how his discussion of bridge systems fit to reality. The problem with this sort of thing though is as a reader I don't want to have to trasnlate everything that is said; it makes it look like the author does not know what he is talking about at a first glance.

I totally agree that a ratings system is completely different to a tournament scoring system, as we have discussed before. Putting two types of points in to cover the latter again just detracts from the credibility of the article because of that, which may be unfair to it. But putting that aside, when I translate and reduce the article to what I believe it is trying to say, I think it comes down to
- Careful about mixing up rating systems and scoring systems (agree)
- Bridge and chess have cool systems but it is easier for them because Dip has many other interactions that have to be taken into account
- And that's about it other than a request to have people discuss these things more

As far as I could see, the article does not attempt to discuss the title of the piece ie the claim that all diplomacy scoring stinks.

On that basis, it seems there is little to discuss further about the article itself other than the confusing way it is laid out.

Incidentally, since you say you are not a bridge player, the reason that bridge scoring systems in tourneys / clubs work is that it is possible to eliminate a lot of the variables, which given the length of diplomacy games played to a conclusion it is not practical to do. To put it simply, the same hands are played by different pairs or teams, and it is therefore possible to look at how one pair/team did vs how another one did with the same cards. Interestingly, bridge rating systems have a particular problem to struggle with which was one we used to face at Playdip when we did pure points scoring per game (based on solo vs 2way vs 3way etc); while they can award 'points' to players who do well in a particular club or tourney event, players who play a lot build up more points. In the end, one approach used by bridge organizations is to have different classes of points, such as local and national and international points. Someone who plays three times a week at the local club can build up a huge number of local points, but you don't get national points for a normal club evening. Instead you have to participate in a national event.

Chess does not have the same problems of course because the only 'difference' is whether you are playing black or white. However, again in the article the author points to the 0/0.5/1 scheme in chess but does not reflect properly the way chess tournaments typically work, weighting your final score by ensuring you play people with a similar score each round.

As for Diplomacy, I have little to contribute to the discussions of scoring because I have no experience in Tournament Diplomacy. The one thing I would call out is to echo the article in saying that whatever system is chosen there is no doubt it will affect play, similar to the bridge discussion in the article (which decodes to talking about pairs or scoring). On rating systems, of course, I would just point to ours

[jay65536]

Interestingly, bridge rating systems have a particular problem to struggle with which was one we used to face at Playdip when we did pure points scoring per game (based on solo vs 2way vs 3way etc); while they can award 'points' to players who do well in a particular club or tourney event, players who play a lot build up more points.

Sorry, I'm totally ignorant of bridge--are you talking about a masterpoint-type system? If you are, there is some history trying that in North American FtF Diplomacy. The late Buz Eddy once tried to set up a system to assign points based on performance in all games played. It did not catch on well at all and has been totally discarded.

[TTBen]

Interestingly, bridge rating systems have a particular problem to struggle with which was one we used to face at Playdip when we did pure points scoring per game (based on solo vs 2way vs 3way etc); while they can award 'points' to players who do well in a particular club or tourney event, players who play a lot build up more points.

Sorry, I'm totally ignorant of bridge--are you talking about a masterpoint-type system? If you are, there is some history trying that in North American FtF Diplomacy. The late Buz Eddy once tried to set up a system to assign points based on performance in all games played. It did not catch on well at all and has been totally discarded.

Aren't most systems based to assign points based on all games played? Or are you meaning going back and rescoring old games once an opponent is proven better or worse than originally thought? That would certainly seem to be good in theory but very difficult to do accurately.

[super_dipsy]

Interestingly, bridge rating systems have a particular problem to struggle with which was one we used to face at Playdip when we did pure points scoring per game (based on solo vs 2way vs 3way etc); while they can award 'points' to players who do well in a particular club or tourney event, players who play a lot build up more points.

Sorry, I'm totally ignorant of bridge--are you talking about a masterpoint-type system? If you are, there is some history trying that in North American FtF Diplomacy. The late Buz Eddy once tried to set up a system to assign points based on performance in all games played. It did not catch on well at all and has been totally discarded.

Yes, that's right. However the key point that renders it from pointless (he he) to useful is that points are only awarded to a small number - in fact in Nationals you have to finish very high up to get any at all, So even if someone plays loads of tournaments, unless they are really good they wont get any points. So yes, the total is reflective of how many tournaments or events a player plays in, but not just plays in but excels in.

[jay65536]

Interestingly, bridge rating systems have a particular problem to struggle with which was one we used to face at Playdip when we did pure points scoring per game (based on solo vs 2way vs 3way etc); while they can award 'points' to players who do well in a particular club or tourney event, players who play a lot build up more points.

Sorry, I'm totally ignorant of bridge--are you talking about a masterpoint-type system? If you are, there is some history trying that in North American FtF Diplomacy. The late Buz Eddy once tried to set up a system to assign points based on performance in all games played. It did not catch on well at all and has been totally discarded.

Aren't most systems based to assign points based on all games played? Or are you meaning going back and rescoring old games once an opponent is proven better or worse than originally thought? That would certainly seem to be good in theory but very difficult to do accurately.

Tournament systems assign points based on all games played, but in a tournament setting, you can safely assume each player is playing the same number of games.

Rating systems are not supposed to do this. For example, if you are using a rating system based on Calhamer points, you don’t always want someone with 4 5way draws to be “rated” higher than someone with 1 3way and nothing else. You don’t want quantity to drown out quality. That’s what I meant by “all games”.

[BrotherBored]

The Diplomacy Dojo discusses! scoring!

(Figured it was better to continue this thread than start a new one)

I don't have much responses to the Bridge concepts and whatnot, since that's not my bag.