RedMixer 1.2x5 (experimental) 7/2/09 ===================================== The following text describes only the new features, supplimenting the existing RedMixer documentation. This is an experimental version, it likely will change. The purpose of the modification was to incorporate an automatic benchmarking and extraction scheme, similar to that prototyped in the SEWAX program (http://newton.freehostia.com/cw/sewax.txt). This feature can be used to simply detect and save strong warriors, or optionally re-insert the top-scoring warrior back into the soup at defined intervals to indirectly guide evolution. Unlike SEWAX, the re-insertion point is the loser of the last battle at the time of re-insertion (overwriting the mutation from the winner but hopefully avoiding overwriting a known-good warrior, SEWAX had options to re-insert to a fixed or random soup position). Guided evolution can produce warriors that score high on a particular benchmark set but often not against warriors not in the test set (what I call the "focus" effect, exploiting weaknesses in specific warriors rather than finding generally strong strategies). Using a fairly large test set, and limiting the amount of guidance, helps reduce the focus effect to produce warriors that score well on other bench sets (and hopefully on a hill). This version also incorporates an enhanced version of the ANSI "movie" mod, when enabled periodically the soup display is dumped to an ANSI file, which can be played back to show the evolution of the soup at high speed. The INI parameters are the same as the original mod but the display also includes a line showing average size, average generation, last score and top score. The following new INI settings have been added... (these are the defaults) ... benchpermutate no ;if yes then use -P option (req's pmars 0.9.2+) ... ; auto-bench/reinsertion parameters benchinterval 0 ;soup cycles per bench (0=disable) insertinterval 0 ;soup cycles per reinsertion (0=disable) topwarrior top.red ;name of file for current top-scoring warrior savedir save ;directory to save all top-scoring warriors to savethresh 135 ;must score at least this to save to savedir ... ; ANSI movie settings... enablemovie no ;set to yes to enable ansi movie acshift 0 ;offset for all ansi colors axcadjust 45 ;multiplier for 2nd species field aycadjust 1 ;multiplier for 1st species field frameinterval 5000 ;number of battles between frames ansifile redmixer.ans ;ansi output file (appends) With the defaults it behaves like previous RedMixer versions, the new features have to be enabled if using. The benchpermutate option permits more accurate benchmarks, avoiding finding warriors that are more sensitive to the number of battle rounds. Note that benchfixed must also be yes, and for true permutate benchrounds must be set to the optimal permutate number which is 142 for nano warriors (coresize 80) and 1522 for tiny warriors (coresize 800). Permutate requires a recent version of pmars and is mainly useful for nano, marginally useful for tiny and totally impractical for coresize 8000 as that requires 15602 rounds (which would take a long time). The benchinterval setting controls how often the current winning warrior is tested, the smaller the number then the more likely and quickly it will find the "best" warrior form, but if too low will slow down evolution. Values from 200-1000 are in the range (500 seems about right). The insertinterval setting controls how often the top-scoring warrior is inserted back into the soup, replacing the current losing warrior and overwriting the mutation from the winner (not worth trying to eliminate the extra disk access - this is applied outside the core evolving code). If the number is too low then the soup will saturate with copies of the top-scoring warrior and impede free-form evolution which might result in stronger or different strategies, higher numbers have progressively less effect. Values from 500-10000 are in the range... in some experiments best results were obtained if evolution is began with it set high for minimal guidance (otherwise it tends to lock into an inferior form), once a decent form was produced insertinterval was lowered to better optimize the form. Other experiments suggest that setting insertinterval to about 2x soup size (around 3000) may be sufficient. This is still highly experimental, the effect of insertion is complex - the inserted warriors can mutate into stronger forms, and the inserted warriors can encourage production of warriors that beat the inserted warrior but don't necessarily score more. The topwarrior setting is the path/name of the file of the "best" warrior, savedir is the path/name to save all top warriors that score at least savethresh to numbered warrior files. The automatic benchmarking uses the existing benchmark parameters from the INI file however they are not changed if changed using the options menu, which only changes the parameters when benchmarking individual warriors using the interface. Thus the INI can be set to use a certain bench set for guided evolution, then if needed the bench set can be changed using options to see how the warriors perform against other test sets without affecting the evolution process (otherwise it would mess up the process due to the varying scores). Note - if changing benchdir in the INI or the contents of the bench directory during a guided run, edit the top.red warrior and change the bench score comment to make sure it reflects the score against the new bench set (lower is OK) or delete the top.red warrior to find a new top-scoring warrior. The general code strategy of the automatic bench and reinsertion is... ... benchinterval = how many soup battles between benchmarks (0 to disable) insertinterval = how many soup battles between insertions (0 to disable) savethresh = minimum bench score to save as numbered warrior ... topscore = 0 if benchinterval > 0 then (not arranged like this, but equivalent) make sure bench set is valid, make a temp list of bench warriors make sure save dir exists, if not make it nextsavenumber = first non-existent numbered warrior in save directory if top.red exists then topscore = last bench score in top.red end if set up pmars command line, rounds, etc for benchmarking end if benchcount = 0 : insertcount = 0 ... mainloop: ... (existing evolving code) IF benchinterval > 0 THEN benchcount = benchcount + 1 IF benchcount >= benchinterval THEN GOSUB BenchAndUpdate benchcount = 0 END IF END IF IF insertinterval > 0 THEN insertcount = insertcount + 1 IF insertcount >= insertinterval THEN GOSUB InsertTopWarrior insertcount = 0 END IF END IF ... GOTO mainloop ... BenchAndUpdate: if top.red doesn't exist or winning warrior is not a dup of top.red then benchmark the winning warrior if bench score > (topscore + 1E-2) then topscore = bench score copy winning warrior to top.red (add bench score info) if bench score >= savethresh then copy top.red to nexsavenumber.red in the save directory nextsavenumber = nextsavenumber + 1 end if end if end if return InsertTopWarrior: if top.red exists then copy top.red to current losing warrior end if return Results... An early version of RedMixer 1.2 produced the following warrior, edited with additional comments... ;redcode-nano ;name Dementis Proeliator ;author Terry Newton ;strategy Evolved 6/27/09 by RedMixer 1.2x ;strategy Testing new autobench/re-insertion techniques ;strategy NanoBM07 score = 152, Koenigstuhl top-20 = 148 ;origin 20_12.red ;parent 01_07.red ;generation 582 ;assert CORESIZE==80 spl.i # -4 , < 66 mov.i > 27 , { 79 mov.i > 9 , < 58 mov.i < 57 , } -1 djn.i $ -2 , < 37 end 0 ;species 13_26.red ;wins 3 ;score 196 ;bench score 150.3426 ;bench dir bench74 (74 warriors) ;original name 01_06.red (32.red) This warrior was produced using the 74 warrior benchmark set posted at http://newton.freehostia.com/cw/bench74.zip with insertioninterval initially set to around 5000, then lowered to 750 when a viable form appeared and eventially lowered to 500 towards the end of the run. The name translates to "mad warrior". It is fairly strong but not quite as strong as the best warriors produced by SEWAX. This could be because the parameters aren't as optimal as they could be, or just luck (every run is different, sometimes several runs are needed to produce very strong results). The current version also records the number of bench rounds in the bench dir comment - for nano usually set to 142 / permutate but useful info when evolving for larger coresizes to judge the accuracy of the bench score. ANSI movie notes... Make sure you want this before using, the output ANSI file can be dozens of megabytes for a long run. When running from a ram disk edit the ansifile parameter to add a path to point the file to hard disk storage instead to avoid running out of ramdisk space. Each frame requires about 13K. The default frameinterval setting of 5000 isn't too bad as far as file growth, perhaps a few megabytes for an overnight run, but be careful when using lower values. The program will refuse to run if under 50, if under 1000 requires confirming. To avoid problems with Dos or Windows ANSI viewers the file is always written with CRLF line ends even if running under Linux. ANSI is an "old" format that used to be very standard (almost all pre-internet BBS's used it), and still is standard for Linux systems, where a simple cat command will "play" the file although perhaps to rapidly. The cat command also works under the Cygwin environment for Windows. If needed a viewer program written in QBasic is in the original ANSI mod file at: http://newton.freehostia.com/cw/rmansi.txt Also refer to this file for instructions for setting up Windows XP to display ANSI files natively, essentially it involves editing config.nt to add ansi.sys then using command /c type redmixer.ans to display (ANSI only works in the dos-mode command.com shell, not in the default cmd shell). Despite producing large files, ANSI codes compress very nicely using zip or tar.gz and is a much solution for recording a run than say mpeg capture or other modern methods. Each frame is recorded exactly and if needed individual frames can easily be be extracted and displayed in any font size. The format allows for the possibility of writing a nice viewing program able to cursor through frames or play back at any speed. Source code usage instructions... QBasic-style FreeBasic source code configured for Linux is in the redmixer_linux.bas file (remove the .txt part if from the web). For Linux, get the FreeBasic Linux compiler and unless the shell command has been fixed, get my patch from http://newton.freehostia.com/fb020shellmod.tar.gz and replace the libfb.a file in the FreeBasic/lib/linux directory with the patched version. Otherwise with the stock FreeBasic 0.20 compiler it'll only run for a bit then fail. This only applies to Linux, the Windows version doesn't have this issue. Compile the redmixer_linux.bas file using the command line: [path/to/]fbc -e -lang qb redmixer_linux.bas (add path to fbc command if not in a path dir) For point-and-click operation under Linux make a shell script... #!/bin/bash # the following cd doesn't work if dir contains spaces, if that's # needed try cd "`dirname '$0'`" but doesn't work on all systems # (or just hard-code the cd directory) cd `dirname $0` # uncomment one of the following to select a terminal # konsole -e ./redmixer_linux # gnome-terminal -e ./redmixer_linux xterm -bg black -fg green -e ./redmixer_linux (that's the run_redmixer.sh file from the stock RedMixer Linux package, alternatively can be run from a terminal using the command ./redmixer_linux but don't just run the binary if not in a terminal - if you do then nothing will be seen, will sit consuming near-100% CPU doing nothing and you'll have to use a utility to kill the task - try killall redmixer_linux) For Windows, get the FreeBasic Windows compiler and install it. Rename the source code to (say) redmixer_win.bas and edit it to change pathsymbol$ (under initialize:) from "/" to "\". Also edit the title and change the (L) to (W) to indicate it's for Windows. Compile using a command line like: "C:\Program Files\FreeBasic\fbc" -e -lang qb redmixer_win.bas (adjust the path to reflect where fbc is) Shouldn't need a script to run it unless the mode bug strikes causing a flickering display, then make a "run_redmixer_win.bat" containing say: @echo off mode con lines=25 redmixer_win.exe Put a set of benchmark warriors (.red extension by default) in the benchmark directory (named nanotest by default but doesn't matter as long as it doesn't contain spaces). Run it (either run_redmixer.sh or redmixer_win.exe) to create the default INI file, then quit so the INI can be edited. Make sure the benchdir and benchmask settings are OK, pmars and pmarsv matches the names of the installed pmars binaries, etc, to bench using permutate set benchpermutate to yes and benchrounds to 142 (for nano). Make a directory where it's going to be run from (preferably on a ram disk and not on the disk containing the operating system, fairly disk intensive) and copy the redmixer binary, script used to run it, bench dir and INI file to the directory. Should work. Dev changes... 7/2/09 - 1.2x5 - removed unneeded ANSI color settings, fixed bug when reading top.war when insertinterval disabled, made insert not require bench. Added a check to make sure evol doesn't start if DeleteMe exists. 6/30/09 - 1.2x4 - little touches, nicer numbers in ANSI file. 6/29/09 - 1.2x3 - added ANSI-dump code, added validation for new parms. 6/28/09 - 1.2x2 - fixed unicode chars that somehow slipped in and broke it. Removed underscores from label names for QBasic-compatibility. 6/27/09 - 1.2x1 - added code to prevent dup high-scoring warriors. 6/27/09 - initial 1.2x version --------------------------------------------------------------------- RedMixer page: http://newton.freehostia.com/cw/redmixer.html Terry Newton (wtn90125@yahoo.com)