[Computer-go] Beta-testing: feedback to bot owners

[Nick Wedd]

In message 
<AANLkTin6dYi0xTNuciKbD4JKscc7hR+HUG_Wc9sPCf2U at mail.gmail.com>, 
Willemien <wilemien at gmail.com> writes
>Go is in principle a drawing game.
>Proper komi (in my humble opinion) should result that optimal
>(perfect) play leads to a draw.
>
>That some programs cannot cope with integer komi/ draws/ jigo is a
>problem of those programs and  not an unfair advantage to the programs
>who can cope with it.
>
>(a  simple way to cope with it is to shift the komi 1/2 point in the
>programs advantage, (W +1/2, B -1/2) altough then the program will
>treat draws as win,  and possibly  forgo a real win)

isn't it better way to shift it in the other direction, and try and 
ensure a win by a sufficient margin to overcome the program's 
misunderstanding of jigo?

>Suppose soon that 2 programs arrive that play perfect on 9x9, do we
>prefer that they draw against eachother and draw or win against all
>other  programs or that they win or lose depending on how lucky they
>are with the colour assignment?

In principle you are right.  But I accept that, where the bot events on 
KGS are concerned, there is nothing I can do about it.  I am in the 
position of a particularly ineffective cat-herd.  The change to using 
the clean-up mechanism for KGS bots was made years ago, and I was told 
that bot programmers should find it easy to implement;  but there are 
still bots that haven't implemented it, or have implemented it wrong.

If I insist on running events with integer komi, I know what will 
happen.  Some bots, including GNU Go, already support it;  some will 
implement it correctly;  some will implement it wrong, so that strange 
things happen;  some will fail to support it, and thereby lose won games 
to weaker programs;  some may refuse to support it, and stop playing in 
the events that I organise.  I prefer to leave things as they are.

I announced earlier that I would be using integer komi of 7 for the 9x9 
KGS bot tournaments this year.  I have changed my mind, I will use 
half-integer komi throughout.  This is not an ideal decision, it is a 
pragmatic one.

Nick


>On Fri, Jan 21, 2011 at 11:51 AM, Nick Wedd <nick at maproom.co.uk> wrote:
>> This is boring - most of you will want to skip it.
>>
>> While beta-testing the improved tournament system on KGS, my task was to
>> report on the behaviour of the tournament-scheduler.  But I happened to
>> notice several things the bots did.  I report on these here.
>>
>> In the biggest tournament I ran, the komi was set to 7, allowing jigo. It
>> seemed that gnugo3pt7 (a pre-MC build of GNU Go, which I ran) understood
>> this, but StoneGrid and Orego12 did not.  As a result, gnugo3pt7 got several
>> undeserved wins against these stronger programs.
>>    I now think that using integer komi is a mistake.  I do not plan to use
>> it in future events.  And it will not be used in the computer events in the
>> European Go Congress this summer.
>>
>> The final test I did used 11x11 boards.  When StoneGrid joined its game, it
>> immediately and repeatedly disconnected and reconnected.  Indeed, it did
>> this so rapidly that I could deduce that Professor Drake lives rather close
>> to Portland, Oregon.  StoneGrid had played normally in the previous tests,
>> so I guess it dislikes non-standard board sizes.
>>
>> The clean-up phase was mishandled in at least two games between StoneGrid
>> and gnugo3pt7 (rounds 3 and 7).  I am fairly sure that GNU Go does clean-up
>> correctly, so I suspect that StoneGrid doesn't.
>>
>> TimeWaster (one of Aloril's delinquent bots) is somehow able to abuse the
>> clean-up system.  At the end of every game, it claims that all its
>> opponent's stones are dead, and that its own stone (it never has more than
>> one on the board) is alive.  Then the game enters the clean-up phase, there
>> is one pass, and the players make their claims again.  This repeats
>> indefinitely.
>>    My understanding is that this shouldn't be possible.  Once the game has
>> entered the clean-up phase, there should be no more claims, all stones still
>> on the board when play stops for the second time should be treated as alive.
>>
>> Nick
>> --
>> Nick Wedd    nick at maproom.co.uk
>> _______________________________________________
>> Computer-go mailing list
>> Computer-go at dvandva.org
>> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
>>
>_______________________________________________
>Computer-go mailing list
>Computer-go at dvandva.org
>http://dvandva.org/cgi-bin/mailman/listinfo/computer-go

-- 
Nick Wedd    nick at maproom.co.uk