[Computer-go] Beta-testing: feedback to bot owners

[Hideki Kato]

Nick Wedd: <bllFUKoNFuONFAOt at maproom.demon.co.uk>:
>In message 
><AANLkTin6dYi0xTNuciKbD4JKscc7hR+HUG_Wc9sPCf2U at mail.gmail.com>, 
>Willemien <wilemien at gmail.com> writes
>>Go is in principle a drawing game.
>>Proper komi (in my humble opinion) should result that optimal
>>(perfect) play leads to a draw.
>>
>>That some programs cannot cope with integer komi/ draws/ jigo is a
>>problem of those programs and  not an unfair advantage to the programs
>>who can cope with it.
>>
>>(a  simple way to cope with it is to shift the komi 1/2 point in the
>>programs advantage, (W +1/2, B -1/2) altough then the program will
>>treat draws as win,  and possibly  forgo a real win)
>
>isn't it better way to shift it in the other direction, and try and 
>ensure a win by a sufficient margin to overcome the program's 
>misunderstanding of jigo?
>
>>Suppose soon that 2 programs arrive that play perfect on 9x9, do we
>>prefer that they draw against eachother and draw or win against all
>>other  programs or that they win or lose depending on how lucky they
>>are with the colour assignment?
>
>In principle you are right.  But I accept that, where the bot events on 
>KGS are concerned, there is nothing I can do about it.  I am in the 
>position of a particularly ineffective cat-herd.  The change to using 
>the clean-up mechanism for KGS bots was made years ago, and I was told 
>that bot programmers should find it easy to implement;  but there are 
>still bots that haven't implemented it, or have implemented it wrong.
>
>If I insist on running events with integer komi, I know what will 
>happen.  Some bots, including GNU Go, already support it;  some will 
>implement it correctly;  some will implement it wrong, so that strange 
>things happen;  some will fail to support it, and thereby lose won games 
>to weaker programs;  some may refuse to support it, and stop playing in 
>the events that I organise.  I prefer to leave things as they are.
>
>I announced earlier that I would be using integer komi of 7 for the 9x9 
>KGS bot tournaments this year.  I have changed my mind, I will use 
>half-integer komi throughout.  This is not an ideal decision, it is a 
>pragmatic one.

Have you asked the reasons to the authors of the programs which can't 
handle integer komi?  I guess they didn't noticed (or just forgot) 
your announce.  GPW Cup uses integer komi for two years with no 
troubles.

Hideki

>Nick
>
>
>>On Fri, Jan 21, 2011 at 11:51 AM, Nick Wedd <nick at maproom.co.uk> wrote:
>>> This is boring - most of you will want to skip it.
>>>
>>> While beta-testing the improved tournament system on KGS, my task was to
>>> report on the behaviour of the tournament-scheduler.  But I happened to
>>> notice several things the bots did.  I report on these here.
>>>
>>> In the biggest tournament I ran, the komi was set to 7, allowing jigo. It
>>> seemed that gnugo3pt7 (a pre-MC build of GNU Go, which I ran) understood
>>> this, but StoneGrid and Orego12 did not.  As a result, gnugo3pt7 got several
>>> undeserved wins against these stronger programs.
>>>    I now think that using integer komi is a mistake.  I do not plan to use
>>> it in future events.  And it will not be used in the computer events in the
>>> European Go Congress this summer.
>>>
>>> The final test I did used 11x11 boards.  When StoneGrid joined its game, it
>>> immediately and repeatedly disconnected and reconnected.  Indeed, it did
>>> this so rapidly that I could deduce that Professor Drake lives rather close
>>> to Portland, Oregon.  StoneGrid had played normally in the previous tests,
>>> so I guess it dislikes non-standard board sizes.
>>>
>>> The clean-up phase was mishandled in at least two games between StoneGrid
>>> and gnugo3pt7 (rounds 3 and 7).  I am fairly sure that GNU Go does clean-up
>>> correctly, so I suspect that StoneGrid doesn't.
>>>
>>> TimeWaster (one of Aloril's delinquent bots) is somehow able to abuse the
>>> clean-up system.  At the end of every game, it claims that all its
>>> opponent's stones are dead, and that its own stone (it never has more than
>>> one on the board) is alive.  Then the game enters the clean-up phase, there
>>> is one pass, and the players make their claims again.  This repeats
>>> indefinitely.
>>>    My understanding is that this shouldn't be possible.  Once the game has
>>> entered the clean-up phase, there should be no more claims, all stones still
>>> on the board when play stops for the second time should be treated as alive.
>>>
>>> Nick
>>> --
>>> Nick Wedd    nick at maproom.co.uk
>>> _______________________________________________
>>> Computer-go mailing list
>>> Computer-go at dvandva.org
>>> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
>>>
>>_______________________________________________
>>Computer-go mailing list
>>Computer-go at dvandva.org
>>http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
-- 
Hideki Kato <mailto:hideki_katoh at ybb.ne.jp>