[Computer-go] Beta-testing: feedback to bot owners

Sat Jan 22 06:19:06 PST 2011

For years people have talked about how to make chess less drawish and many
proposals have been forwarded.    It appears to be considered  more ideal to
not draw.    Also, there are complaint of those who play for the draw and
how it can cause less interesting chess.    Also,  it's more desirable in
tournaments because draws tend to create many winners in some cases.

But in GO, we have the situation where we can define the rules in such a way
that draws are not possible.   It has to be done at the expense of one side
or the other however.    However I'm not sure it's known what the proper
komi should be although there are strong opinions on the subject.

But just accepting non-integer komi's to escape the perceived difficulties
of draws is not very satisfying.    I think programmers should experiment
heavily with this and try to determine if we can build bots that are not
compromised by the possibilities of draws.

This is not a criticism of Nicks decision of course.   I think it's a good
decision but it raised an issue that we as go program authors need to
continue to address.

Is there a problem with viewing each playout as 2 games,  either 2 wins,  2
losses or a win and a loss?   Nothing else would have to be changed but how
would it play?   Would some of the parameters need to be adjusted?    Of
course if floating point math is used then a draw is half a win.

Don

On Sat, Jan 22, 2011 at 8:53 AM, Nick Wedd <nick at maproom.co.uk> wrote:

> In message <AANLkTin6dYi0xTNuciKbD4JKscc7hR+HUG_Wc9sPCf2U at mail.gmail.com<AANLkTin6dYi0xTNuciKbD4JKscc7hR%2BHUG_Wc9sPCf2U at mail.gmail.com>>,
> Willemien <wilemien at gmail.com> writes
>
>  Go is in principle a drawing game.
>> Proper komi (in my humble opinion) should result that optimal
>> (perfect) play leads to a draw.
>>
>> That some programs cannot cope with integer komi/ draws/ jigo is a
>> problem of those programs and  not an unfair advantage to the programs
>> who can cope with it.
>>
>> (a  simple way to cope with it is to shift the komi 1/2 point in the
>> programs advantage, (W +1/2, B -1/2) altough then the program will
>> treat draws as win,  and possibly  forgo a real win)
>>
>
> isn't it better way to shift it in the other direction, and try and ensure
> a win by a sufficient margin to overcome the program's misunderstanding of
> jigo?
>
>
>  Suppose soon that 2 programs arrive that play perfect on 9x9, do we
>> prefer that they draw against eachother and draw or win against all
>> other  programs or that they win or lose depending on how lucky they
>> are with the colour assignment?
>>
>
> In principle you are right.  But I accept that, where the bot events on KGS
> are concerned, there is nothing I can do about it.  I am in the position of
> a particularly ineffective cat-herd.  The change to using the clean-up
> mechanism for KGS bots was made years ago, and I was told that bot
> programmers should find it easy to implement;  but there are still bots that
> haven't implemented it, or have implemented it wrong.
>
> If I insist on running events with integer komi, I know what will happen.
>  Some bots, including GNU Go, already support it;  some will implement it
> correctly;  some will implement it wrong, so that strange things happen;
>  some will fail to support it, and thereby lose won games to weaker
> programs;  some may refuse to support it, and stop playing in the events
> that I organise.  I prefer to leave things as they are.
>
> I announced earlier that I would be using integer komi of 7 for the 9x9 KGS
> bot tournaments this year.  I have changed my mind, I will use half-integer
> komi throughout.  This is not an ideal decision, it is a pragmatic one.
>
> Nick
>
>
>
>  On Fri, Jan 21, 2011 at 11:51 AM, Nick Wedd <nick at maproom.co.uk> wrote:
>>
>>> This is boring - most of you will want to skip it.
>>>
>>> While beta-testing the improved tournament system on KGS, my task was to
>>> report on the behaviour of the tournament-scheduler.  But I happened to
>>> notice several things the bots did.  I report on these here.
>>>
>>> In the biggest tournament I ran, the komi was set to 7, allowing jigo. It
>>> seemed that gnugo3pt7 (a pre-MC build of GNU Go, which I ran) understood
>>> this, but StoneGrid and Orego12 did not.  As a result, gnugo3pt7 got
>>> several
>>> undeserved wins against these stronger programs.
>>>    I now think that using integer komi is a mistake.  I do not plan to
>>> use
>>> it in future events.  And it will not be used in the computer events in
>>> the
>>> European Go Congress this summer.
>>>
>>> The final test I did used 11x11 boards.  When StoneGrid joined its game,
>>> it
>>> immediately and repeatedly disconnected and reconnected.  Indeed, it did
>>> this so rapidly that I could deduce that Professor Drake lives rather
>>> close
>>> to Portland, Oregon.  StoneGrid had played normally in the previous
>>> tests,
>>> so I guess it dislikes non-standard board sizes.
>>>
>>> The clean-up phase was mishandled in at least two games between StoneGrid
>>> and gnugo3pt7 (rounds 3 and 7).  I am fairly sure that GNU Go does
>>> clean-up
>>> correctly, so I suspect that StoneGrid doesn't.
>>>
>>> TimeWaster (one of Aloril's delinquent bots) is somehow able to abuse the
>>> clean-up system.  At the end of every game, it claims that all its
>>> opponent's stones are dead, and that its own stone (it never has more
>>> than
>>> one on the board) is alive.  Then the game enters the clean-up phase,
>>> there
>>> is one pass, and the players make their claims again.  This repeats
>>> indefinitely.
>>>    My understanding is that this shouldn't be possible.  Once the game
>>> has
>>> entered the clean-up phase, there should be no more claims, all stones
>>> still
>>> on the board when play stops for the second time should be treated as
>>> alive.
>>>
>>> Nick
>>> --
>>> Nick Wedd    nick at maproom.co.uk
>>> _______________________________________________
>>> Computer-go mailing list
>>> Computer-go at dvandva.org
>>> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
>>>
>>>  _______________________________________________
>> Computer-go mailing list
>> Computer-go at dvandva.org
>> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
>>
>
> --
> Nick Wedd    nick at maproom.co.uk
> _______________________________________________
> Computer-go mailing list
> Computer-go at dvandva.org
> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://computer-go.org/pipermail/computer-go/attachments/20110122/4fb67ec7/attachment.html>