A Blassic-based Warrior Benchmark Program ========================================= This is a simple corewar warrior benchmark program that tests all warriors in a specified directory against test warriors in another directory, score results are written to a report file which may be processed using the sort command (for example sort rep_bm07.txt) to list the top-scoring warriors. The program also prints the top-scoring warrior and the average score. Directory and file names are hard-coded in the program itself. An alternate version of this program that uses an INI file instead is included with the Maezumo evolver (http://www.corewar.info/maezumo/), that version might be easier to use if evolving for multiple coresizes that use many test sets. For simple uses it's just as easy or easier to make copies as needed (edit the run batch/script to run the correct .bas file) and keep the "workdir" directory name(s) constant to minimize editing, then benchmarks can be done by double-clicking the appropriate run script. The following listing is set up to benchmark nano warriors in the "workdir" directory, using the "nanoBM07" directory for the test set. Adapt as needed. The program is set to use pmars, typically the same one used for evolving, The pmars command line is configured to test nano warriors using the permutate option of pmars 0.9.2 and later, permutate is only appropriate for nano. If configuring for other core sizes use an adequate number of fixed rounds using -r [rounds] -f, make sure the number for -r matches the rounds setting. For example, for coresize 8000 edit to something like... [name it say benchstd.bas and edit the run batch/script accordingly] 140 testdir$="Wilkies" 'set to directory containing test set 150 testmask$="*.RED" 'set to match test warriors 160 report$="rep_wilkies.txt" 'set to name of file to write bench report to 170 rounds=250 'number of rounds 180 sim$="pmars -bks 8000 -p 8000 -c 80000 -l 200 -d 200 -r 250" 'sim comline Links to utilities... Blassic for running the program is at: http://blassic.org/ Pmars is at: http://www.koth.org/pmars/ or http://www.corewar.co.uk/pmars/ x86 Linux pmars binaries are at: http://newton.freehostia.com/cw/ Various benchmark test sets are at: http://www.corewar.co.uk/bench.htm Notes for Windows... The SDL pmars-server.exe version of pmars works well with Windows, for best results rename to pmars.exe and put it and blassic.exe in a path directory such as C:\Windows or see other docs for creating your own path directory. Otherwise pmars and blassic have to be in the same directory as the bench program or the bench program and scripts (and evolvers etc) edited to specify the locations of the binaries, it's much easier to put them in a path directory so they can be run using just "pmars" and "blassic". When copy/pasting code into Notepad to save to files, make sure the filename is quoted otherwise it adds a .txt extension. For best results set Windows so that it displays the file extensions or make sure you know what the extensions are based on the icon or file properties. Notes for Linux... Make sure to edit line 120 to say usewin=0. Make sure the strings exactly match case with the directories and extension. When saving the .sh scripts make sure they're converted to unix format (or type them in manually, the ` chars in the cd's are the key next to 1). I installed and use the "flip" package to convert: flip -u filename.ext The .sh files have to have properties set to allow executing. The blassic .deb file can usually be installed by right-clicking and selecting the package manager, Ubuntu also permits right-clicking and opening as an archive, then the blassic binary can be put in a path directory. Rename the pmars binary to "pmars" and put it in a path dir like /usr/local/bin (root privs required, or at a command line sudo mv pmars /usr/local/bin) Some systems support a local path directory in the ~/.local/bin directory. -------------- begin benchnano.bas ------------------------------------------- 100 REM Benchmark a directory of warriors against a benchmark set 102 REM by WTN 7/15/09 modified 8/22/09 110 REM --------------------------------------------------------------- 120 usewin=1 'set to 1 for Windows, 0 for Linux 130 soupdir$="workdir" 'set to directory containing warriors to test 140 testdir$="nanoBM07" 'set to directory containing test set 150 testmask$="*.RED" 'set to match test warriors 160 report$="rep_bm07.txt" 'set to name of file to write bench report to 170 rounds=142 'number of rounds 180 sim$="pmars -bks 80 -p 80 -c 800 -l 5 -d 5 -P" 'sim comline 190 REM --------------------------------------------------------------- 200 sim$=sim$+" -r "+str$(rounds)+" ":print "Press Esc to stop" 210 if usewin then dc$="dir /b /on ":ps$="\" else dc$="ls -1 ":ps$="/" 220 shell dc$+soupdir$+ps$+"*.red > souplist.tmp" 230 shell dc$+testdir$+ps$+testmask$+" > testlist.tmp" 240 open report$ for output as #1:topscore=0:top$="" 250 open "souplist.tmp" for input as #2 260 while not eof(2) and inkey$<>chr$(27) 270 score=0:n=0 280 line input #2,war$:if usewin then war$=soupdir$+ps$+war$ 290 open "testlist.tmp" for input as #3 300 while not eof(3) 310 line input #3,opp$:if usewin then opp$=testdir$+ps$+opp$ 320 shell sim$+war$+" "+opp$+" >scores.tmp" 330 open "scores.tmp" for input as #4 340 line input #4,a$:close #4 350 z=instr(a$," "):w=val(left$(a$,z)):t=val(mid$(a$,z)) 360 score=score+w*3+t:n=n+1 370 wend 380 close #3 390 score=(score/n)*(100/rounds) 400 if score>topscore then topscore=score:top$=war$ 410 sc$=mid$(str$(-1*(1000+score+.00001))+" ",3,6) 420 print sc$;" ";war$ 430 print #1,sc$;" ";war$ 435 totalscore=totalscore+score:nwars=nwars+1 440 wend 450 close #1:close #2 460 kill "souplist.tmp":kill "testlist.tmp":kill "scores.tmp" 470 print "Top score = ";str$(topscore);" by ";top$ 475 print "Average score = ";str$(totalscore/nwars) 480 system -------------- end benchnano.bas --------------------------------------------- Windows batch to run it... -------------- begin benchnano.bat -------------- @echo off blassic benchnano.bas pause -------------- end benchnano.bat ---------------- Linux script to run it... -------------- begin benchnano.sh --------------- #!/bin/bash if [ "$1" == "sub" ]; then cd `dirname $0` blassic benchnano.bas echo "--- press any key ---" read -n 1 nothing else xterm -e $0 sub fi -------------- end benchnano.sh ----------------- Put these in your top-level evolving directory, put benchmark directories in a directory under that. Individual runs (or the soup directory) should be placed in other directories under the top-level directory. Edit line 120 to usewin=1 for Windows or usewin=0 for Linux. Edit the string in line 130 to the name of the directory for the run to test. Edit the string in line 140 to the name of the benchmark directory. Edit the string in line 150 to match the extension of the benchmark warriors. Edit the string in line 160 to the name of the report file. Lines 170 and 180 are set to use pmars 0.9.2 or above with permutate, if using an older pmars set rounds to 1000 and replace -P in line 180 with -r 1000 -f. Legal... This software is public domain and can be used any way you want, including adapting and including with other evolving software. This software is provided as-is and without warranty. -------------------------------------------------------------------------- Last modified 8/22/09 by Terry Newton (wtn90125@yahoo.com)