[Computer-go] Running automated bot matches?

[David Doshay]

You should be very careful about assuming that a modification of a program that leads to improvement against the base code is really an improvement. It seems like a good assumption, and I made the same assumption many years ago, but I now believe that it is quite flawed. 

The best way to measure improvement of a Go playing program is how it plays against a variety of other Go programs. Check out CGOS.

Cheers,
David



On 11, Jan 2011, at 9:41 AM, Joona Kiiski wrote:

> Hi everyone,
> 
> During the last week I've been examining sources of different open source go-engines (fuego, pachi, orego).
> Now I'd like to start making some simple modifications to some of them (not yet decided which one) and 
> see how it goes (likely my 50 first tries will fail miserably, but it's okay).
> 
> In computer chess programming, it's nowadays a widely accepted fact that only reasonable way to test changes is to run a huge number of test games between original and modified version. I assume that same applies also for go-programming.
> 
> So, let us have open-source program X and slightly modified version it X'. What is the easiest way to run say 1000 super-fast games between them? I hope there already exists some scripts or programs to do this. 
> 
> My OS is Linux if it matters.
> 
> Thanks for your help!
> _______________________________________________
> Computer-go mailing list
> Computer-go at dvandva.org
> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go