[Computer-go] win rate bias and CLOP

[Rémi Coulom]

It is important to understand that CLOP claims very little in terms of win rate. That is to say the win rate estimates it reports are all biased. Win rate over all samples underestimates the real win rate. Win rate near the maximum (central, and weighted) tend to be over-estimated.

CLOP finds the location in parameter space that has the highest win rate. It may be the highest because it is the best, but also because it is the most lucky. That's why it is necessarily biased toward optimistic values.

If the win rate over all samples is an improvement, then you can be sure you have an improvement. Otherwise you cannot be sure unless you actually play a lot of games with the suggested parameters.

Rémi

On 3 janv. 2012, at 14:09, Ingo Althöfer wrote:

> Hi David,
> 
> David Fotland on CLOP-optimization:
>> I tried it, but got no benefit so far.  It claimed to find better settings
>> for most parameters, but when I used them the program wasn’t any
>> stronger.
> 
> Interestant. Had it similar strength or did it even become weaker?
> How often did the move proposals by your "older" ManyFaces and the
> CLOP-MF differ?
> 
> Ingo.
> -- 
> NEU: FreePhone - 0ct/min Handyspartarif mit Geld-zurück-Garantie!		
> Jetzt informieren: http://www.gmx.net/de/go/freephone
> _______________________________________________
> Computer-go mailing list
> Computer-go at dvandva.org
> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go