[Computer-go] semeai example of winning rate

[David Fotland]

The issue is testing time.  To see if a change makes a small improvement I
play 1000 or more test games.  To do a test in a day requires a large number
of cores or very short games.

 

David

 

From: computer-go-bounces at dvandva.org
[mailto:computer-go-bounces at dvandva.org] On Behalf Of terry mcintyre
Sent: Tuesday, January 18, 2011 6:43 PM
To: computer-go at dvandva.org
Subject: Re: [Computer-go] semeai example of winning rate

 

Old saying: make it correct first, then make it fast. 

 

Blitz games may give fast but wrong results, penalizing a new patch for lack
of speed. 
 

As a kibbitzer put it recently, referring to a program running on 64 nodes:
"64 wrongs don't make a right."

 

Terry McIntyre <terrymcintyre at yahoo.com>

Unix/Linux Systems Administration
Taking time to do it right saves having to do it twice.

 

 

  _____  

From: Darren Cook <darren at dcook.org>
To: computer-go at dvandva.org
Sent: Tue, January 18, 2011 6:23:09 PM
Subject: Re: [Computer-go] semeai example of winning rate

>> The main problem right now is that a lot of "smart" code actually
>> weakens the program.
> 
> ...
> we are here essentially trying to patch a bot weakness. The usual way to
> test strength is by playing other bots. Since they also have the
> weakness, and anyhow won't steer the game in that direction, patching
> that weakness does not make a huge difference against them, and can
> easily be overshadowed by any collateral change, such as being slightly
> slower. On the other hand, it might still make them stronger against
> humans.

I fully agree; in fact I'm getting increasingly passionate on this issue
:-) I feel some good ideas are getting thrown away (along with lots of
bad ones) because they are initially only considered on blitz time
settings, and only on self-play or against one opponent. I think Very
Large Test Suites are the way forward; supported with play on go servers.

Darren


-- 
Darren Cook, Software Researcher/Developer

http://dcook.org/work/ (About me and my work)
http://dcook.org/blogs.html (My blogs and articles)
_______________________________________________
Computer-go mailing list
Computer-go at dvandva.org
http://dvandva.org/cgi-bin/mailman/listinfo/computer-go

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://computer-go.org/pipermail/computer-go/attachments/20110118/bfc436b1/attachment.html>