RedMixer is a program that evolves warriors for the game of
CoreWar...
Files and options...
Various tools I use (not exactly refined but work for me)...
Results and docs...
Corewar is a game where programs written in "redcode" battle each
other in a simulated "core". Punching "corewar" into a search engine
returns lots of informative web sites that explain the game, here it is
assumed that the reader is familiar with terms like instruction
modifier etc. Redcode has certain properties that make it suitable for
using genetic algorithms and other A-Life techniques to evolve warriors
from random beginnings: for every warrior there is another warrior that
can beat it; there are no invalid combinations of the various language
elements; and most importantly, changes to the warrior's code tend to
improve or weaken the warrior, quite unlike regular machine code where
any change tends to result in a non-functional program. Unlike some
A-Life systems when evolving warriors for Corewar, the reproduction
mechanism is not part of the organism but rather a feature of the
evolver program itself. The evolver makes mutated copies of successful
warriors to other locations within the "soup" (the collection of
warriors being evolved, not to be confused with replicating warriors
that make copies of their code throughout the core), replacing weaker
warriors with stronger ones. Keep in mind that strength is only in
relation to what already exists - for every code form there is another
that is stronger so the warriors in the soup may cycle through
different variations as the process continually tries to find ways to
beat the existing warriors. The overall strength of the soup warriors
tends to increase over time (up to a point) but not all the
oscillations do particularly well on benchmark tests. Evolved warriors
for core size 80 tend to be more competitive against hand-coded
warriors than warriors evolved for other core sizes but it is all
relative... although purely evolved warriors for larger core sizes tend
to be weak they are still fun to play against other evolved warriors,
and they provide better feedback for finding more effective evolving
techniques.
RedMixer is a grid-based evolver similar to REBS
with an interface added for examining the evolved warriors. The overall
evolving algorithm is simple - randomly select two neighboring warriors
from the soup (if either doesn't exist create a random
warrior), battle them in pMARS, replace the loser with a mutated copy
of the winner and repeat until the process is stopped. The mutations
includes operations such as
changing the instruction, modifier, A and B address modes and values,
inserting and deleting lines and changing where the warrior begins
executing from. Some operations involve various kinds of changes to the
same item such
as incrementing or decrementing an existing value or selecting a new
random value, which can be a small offset which can refer to warrior
locations or a large value which can refer to other locations within
the
core. The topology of the soup plays a fairly significant role in the
results by permitting "islands" of warriors to evolve
semi-independently from other warriors in the soup. Evolvers that have
no topology tend to find a local strength maximum and have difficulty
finding higher maximums unless other steps are taken such as using an
evaluation hill, "valhalla", structured crossover, etc. Introducing the
concept of locality causes warriors to only be compared with other
warriors around it, giving weaker forms more time to try various
mutations and possibly increase in strength or turn into entirely
different forms without immediately being wiped out by other stronger
warriors in the soup. I prefer the grid topology as it resembles a
"world", the RedMixer grid is wrapped on the left and right but not on
the top and bottom to form a cylinder of warriors. Another viable
topology for reducing excessive spread is the "fuzzy ring" where
warriors are linearly numbered and battle other warriors within a
certain range, wrapping at the ends. This topology is not directly
supported by RedMixer but can be approximated by using a short but wide
soup.
One of the main reasons I wrote RedMixer (besides wanting to add an
interface to the REBS concept) was to explore various
crossover methods. Picking just any warrior to mix with doesn't help
much, crossing with mostly similar code has a much greater chance of
producing an offspring which retains characteristics of the parents. To
assist with selecting similar code RedMixer records a species string
into each warrior, if more than a certain percentage of instructions
are changed then it selects a new string to create a new species that
will be ignored (when set to cross only like species). As with battle
selection, crossover mates are chosen from warriors immediately
surrounding the winning warrior. RedMixer also has an "attraction"
option that records the number of times a warrior wins a soup battle
and its average score so the program can select the most effective
warrior to cross with. How the redcode is mixed likely makes a
difference so a few options are included to affect the crossover
process. Once the warrior to cross with has been selected, a flag
variable is used to determine whether the lines being copied and
mutated come from the winning warrior or the warrior selected to cross
with, a setting determines the odds of the flag being initially set to
the winning warrior. Settings determine the chance of changing the
source warrior for each item in a line and the chance of changing the
source on line boundaries, this provides control of the average size of
code fragments from each warrior. In the present scheme each cross
partner contributes approximately half of the code of the code which
will be mutated to produce the new warrior.
Blog-like stuff, most recent entries first...
1/16/10 - I made a new evolver called MEVO
that's similar to RedMixer but a bit more refined and not dos-based, so
the FreeBasic code is cleaner (no QBasic-compatible ON ERROR GOTO
stuff) and it's more
well-behaved under Windows. No lib mods are needed to recompile for
Linux. Other advantages is it can multi-thread, mostly keeps the soup
in memory for less disk usage, can write a log file that can be used to
plot run progress, and has somewhat simpler options... but isn't quite
as versatile and doesn't seem quite as strong when evolving for tiny
and standard core sizes. Nano's good.
An interesting real-world evolver-related thing I stumbled upon is
circuit evolution, typically using a SPICE simulator to evaluate
mutations. I used a program called ASCO (A SPICE Circuit
Optimizer) to help find out if a filter network would be fully
tweakable
before designing a printed circuit board, ASCO found solutions I hadn't
thought of.
Evolving a netlist isn't that different from evolving warriors, so
hoping some of the techniques I've previously explored will be useful.
ASCO uses a technique called Differential Evolution, which tends to
converge to a single solution - I'm anxious to try a grid-based
approach to generate multiple competing solutions - kinda like circuit
wars where the better solution wins local battles but the process lets
spatially-removed competing solutions survive long enough to possibly
develop alternative solutions.
7/2/09 - 1.2x5 seems pretty stable so packaged up all the builds
(QBasic, Windows and Linux) into a tar.gz file (needed to preserve
attributes, to extract under Windows use an extractor that handles
tar.gz files). There still might be bugs... lots of changes were made.
The re-insertion feature is still very experimental, have not found
optimum settings yet and it can affect the evolution of the soup in
unpredictable ways - the inserted warriors might mutate and become
stronger and at the same time they encourage the evolution of warriors
that beat them which might or might not produce stronger benchmark
results. Re-insertion also kind of goes against "natural" evolution
(nature doesn't re-insert) but it can be disabled, using only the
automatic benchmark and extraction options - these alone makes this new
version a lot more convenient (assuming there are no drastic bugs) as
it eliminates having to benchmark the entire soup to find strong
warriors, comments are added to each extracted warrior with the bench
score, the test directory and number of test warriors, and how many
rounds were used.
The code has become too large to compile for dos (FirstBasic says
it's too big), and if running interpreted there may be problems running
a dos version of pmarsv but in my testing under DosEmu/FreeDos it still
works for nano so probably no worse than it was. This is not an issue
when running the QBasic version under Windows XP. A FreeBasic-compiled
version for Windows is also supplied, might be more compatible with
other versions of Windows and multitasks better but isn't as quite as
fast when evolving as the interpreted QBasic version.
6/30/09 - The 1.2 version is coming along... although it still
doesn't seem to be quite as strong at making nano warriors as the small
SEWAX program. Not sure why but I suspect it's because of simpler
parameters such as only using spl mov and djn instructions and only
simple mutations. Sometimes (quite often actually) complexity makes
things worse by presenting more choices to get wrong [or it could just
be the luck of the draw]. The latest
testing version incorporates an enhanced version of the ANSI mod
that also records the average size, generation, last score and top
score [does not have the color options from the original mod, not
needed].
6/27/09 - Lately I've been playing around with small evolvers, with the right options
sometimes these tiny little programs can perform better than big
evolvers, but more importantly they provide an easy way to try out new
ideas without the code weight of a large program. One idea I tried was
a way to perform automatic benchmarking and saving of strong warriors,
and to increase the scores periodically re-inserting the top-scoring
warrior back into the soup. This guides evolution without directly
evaluating the soup warriors against a bench set like some guided
evolvers do, rather the soup continues to evolve normally (battling
each other), but gives an adjustable advantage to warriors that bench
well. [...added similar code to an experimental RedMixer 1.2...]
6/4/09 - The new RedMixer 1.1a packages are up - the documentation
might still be rough but should be good enough for now (darn, forgot to
put a note in the zip version that stock pmars requires that some
coresize-8000 benchmark warriors have to be edited to add space after \
for proper benchmark results... oh well will try to catch it next time).
There isn't a lot of functional change from 1.0d to 1.1a, it's
basically the same except it is now compatible with FreeBasic with new benchmark code
that isn't dos-specific so can be run under Linux natively without
having to emulate dos. While in the code I fixed a couple of minor
glitches and added 3 new optional INI settings to better control the
warrior size when evolving for coresize 8000 (a maxpmlen in case the
pmars setting needs to be different than maxlen), compensate for
Linux's case sensitivity (a setting so I can say *.RED instead of
having to rename my existing bench warriors), and a setting to specify
the number of screen lines so it'll work better on my Asus 701SD that
has a 22-line Konsole terminal. Since it is no longer necessary to have
to use a dos version of pmarsv to emulate under Linux the pmarsv50
batch for running a dos pmarsv with a 50 line display has been removed.
Big soups work fine with the SDL "pmarsw" and for Linux I'm sure xterm
can be set to display more lines - but still need to test that. Big
soups never did a lot for me anyway besides taking longer to evolve.
5/17/09 - Solved the shell problem with FreeBasic [workaround is in fb020shellmod.tar.gz] to produce a
native Linux version of RedMixer. Yay! While at it made so I can press
Esc to exit and fixed a couple of very minor bugs - soup coloring is
now "correct" (but the incorrect method had little to no visible
effect) and increased the number of loops in the bits for clearing the
key buffer to try to avoid bouncing keys when running on my Asus (which
sometimes sends two keys for a keypress). The benchmark code has been
partially rewritten to not use dos batch processing. [outdated info
removed]
4/11/09 - Finally got a RedMixer warrior onto the SAL tiny hill, in
last place and it took a stone to soften it up first but whatever
works [didn't last long got knocked off :]. This is a close relative of
Clone #3 that I found after
retesting previous results using various additional benchmarks...
;redcode-tiny
;name Refocused Clone #7
;author Terry Newton
;strategy Evolved by RedMixer 1.0d3f-800 (as 17_15, gen 1138)
;strategy Cloned 09_18.red to all soup members and re-evolved
;strategy Franz score = 176, TinyBM04 score = 130
;origin 18_73.red
;parents 18_16.red 17_17.red
;generation 1138
;assert CORESIZE==800
mov.i < 709 , > 275
mov.b < 635 , { 572
mov.i { 336 , } 600
mov.i < 107 , { 743
spl.x # 226 , } 2
mov.i < 4 , { 3
mov.i > 272 , } -2
mov.i } 728 , { 0
djn.x $ 208 , < 561
end 0
;species 01_17
;wins 8
;score 180
Also similar to this one which (barely) beats all the Franz test
warriors...
;redcode-tiny
;name Fluttering EvolBits
;author Terry Newton
;strategy Evolved by RedMixer 1.0d3h-800
;origin 18_73.red
;parents 09_52.red 09_53.red --> 08_51.red
;generation 1173
;assert CORESIZE==800
mov.b < 135 , > 575
mov.i } 551 , < 48
mov.i { 450 , } 600
mov.i < 472 , { 772
spl.i # 225 , } 2
mov.i < 4 , { 3
mov.i > 272 , } -2
mov.i > 726 , { 0
djn.i $ 208 , < 556
end 0
;species 15_14
;wins 8
;score 172
Opponent Scores Results Performance of Fluttering EvolBits
--------------- --------- ----------- ---------=---------=---------=---------=
Tiny Wicked 1669 1174 504 339 157 **********************
Tiny AK-47 1561 1306 476 391 133 ********************
Tiny Barbarian 1192 1129 171 150 679 ***************
Tiny Brutality 2492 338 774 56 170 *********************************
Tiny Everclear 1670 1124 488 306 206 **********************
Golden Gate v4 2373 546 764 155 81 *******************************
Tiny Porter 1819 1060 566 313 121 ************************
Tiny Spiral 1638 708 328 18 654 *********************
Tiny Tiberius 1424 1361 403 382 215 ******************
D-Clear 2440 469 783 126 91 ********************************
--------------- --------- ----------- ---------=---------=---------=---------=
Adjusted Score: 182.7
This one also had the highest "score ratio" with the Franz test set,
the sum of the first
score column divided by the second column is almost 2 to 1. To assist
with the math I
made a program (SCRATIO.BAS) that takes a TESTALL report as
input and writes the scores and ratios to another file. Often the
warrior with the highest score ratio also has the highest score but not
always. To make it easier to test a group of candidates against many
benchmarks I use a batch file that calls several copies of TESTALL.BAT
each edited to a specific benchmark set then the batch calls a
modification of the score ratio program hard-coded to process all of
the reports in one step (by calling the main part of the code
reconfigured as a subroutine), then I can try to figure out which
warrior is "better". A somewhat difficult task when the hill contents
is not known. Although the strong warriors from this particular line
are fairly similar, the differences in numbers, addressing modes and
modifiers cause them to perform better or worse depending on the
warriors they're being played against. But I'm about ready for a
different line... got new experiments going...
3/29/09 - I've been trying to push a family of evolved replicators
(decended from "Experiment A" described in cwevol.pdf and in the
3/15/09 entry below) to the point they can make the SAL tiny hill, not
having much luck. So far this is the best I've come up with,
lacked less than 2 points of making the hill...
;redcode-tiny
;name Refocused Clone #3
;author Terry Newton
;strategy Evolved by RedMixer 1.0d3f-800 (as 21_36, gen 1096)
;strategy Cloned 09_18.red to all soup members and re-evolved
;strategy Franz score = 178, TinyBM04 score = 130
;origin 18_73.red
;parents 21_35.red 20_36.red
;generation 1096
;assert CORESIZE==800
mov.ba < 730 , > 694
mov.b # 332 , { 578
mov.i { 335 , } 600
mov.i < 92 , { 744
spl.ab # 226 , } 2
mov.i < 4 , { 3
mov.i > 272 , } -2
mov.i } 728 , { 0
djn.i $ 208 , < 665
end 0
;species 01_17
;wins 2
;score 146
It's quite similar to 02_34, which was cloned and evolved a bit to
generate 09_18 then cloned and evolved some more. But
02_34 only lacked about 4 points of making the hill so all the
shuffling and cloning isn't yielding that much improvement. The
experiment is still on-going, now back to plain evolving using the
strongest of the clone experiment warriors for the seed soup. Some of
the speciments are scoring as high as 183 against the Franz test set
but are not doing as well when benchmarked against other test sets, not
sure what's going to happen.
One of the benchmarks I use is the contents of the old 1/05
Sourceforge tiny hill and I noticed these warriors were getting an
unusually high score against one of the warriors, so thinking it might
be something I had been testing and forgot to remove I investigated.
Nope it was part of the old hill. Well this must be pretty weak warrior
I thought, but wrong again, the little bugger turned in fairly good
scores - 183 against the Franz set and 137 against the TinyBM04 set.
And it's a little bitty Fizzle warrior...
;redcode-tiny
;name 084.red 08-25-2004
;author Terry Newton
;assert CORESIZE == 800
ORG 1
MOV.AB $ 193 , }-2
MOV.I { 267 , @ 0
SPL.I # 80 , { 1
MOV.I } 147 , }-3
DJN.I $-1 , }-4
END
;evolved using Fizzle 1.2
;generation 736
;length 5
Huh. Just a "boring" stony thing, not much to see when it
runs. I chose to pursue the replicating forms because they're more
interesting from a code point of view, but they're at a disadvantage as
replicators tend to tie more against other replicators and lose to
scanners (and there seems to be a lot of hand-coded scanners these
days, a strategy that
doesn't show up much when evolving). Stone strategies tend to win or
lose, often gathering more points in the
process. The 02_08 nano warrior uses a similar boring strategy but that
works well for it.
The following charts illustrate the difference...
Opponent Scores Results Performance of 084.red
--------------- --------- ----------- ---------=---------=---------=---------=
Betadine 1792 1186 590 388 22 ***********************
Wilt ;) 493 2404 130 767 103 ******
Unknown 1720 1246 562 404 34 **********************
Ugly Uncle 859 2116 278 697 25 ***********
Tiny Zooom... 1838 1127 601 364 35 ************************
Table Scan 1880 1091 617 354 29 *************************
Tinyshot 1678 1246 534 390 76 **********************
tiny Blowrag 371 2549 97 823 80 ****
Tiny BiShot 2.0 1982 1010 658 334 8 **************************
Digital Swarm 745 2074 188 631 181 *********
Son of Darkness 946 1723 205 464 331 ************
Snowstorm 1786 1018 530 274 196 ***********************
Sneaky Spike 1833 1122 596 359 45 ************************
Seek and Destro 1698 1296 564 430 6 **********************
Provenance 1971 999 647 323 30 **************************
Potenza 668 2273 203 738 59 ********
Origin of Storm 1531 1438 500 469 31 ********************
Muskrat 1350 1623 441 532 27 ******************
livewire voodoo 887 2000 258 629 113 ***********
Hellish Glare 1393 1510 432 471 97 ******************
Lone Gunman 1851 1104 602 353 45 ************************
Where's Giles? 1218 1629 355 492 153 ****************
Diamondette 1329 1587 415 501 84 *****************
bestwar4.red 714 2265 231 748 21 *********
Four Winds 1826 1157 603 380 17 ************************
--------------- --------- ----------- ---------=---------=---------=---------=
Adjusted Score: 137.4
Opponent Scores Results Performance of Refocused Clone #3
--------------- --------- ----------- ---------=---------=---------=---------=
Betadine 1297 1498 364 431 205 *****************
Wilt ;) 1204 1540 316 428 256 ****************
Unknown 1504 1105 371 238 391 ********************
Ugly Uncle 1583 1214 460 337 203 *********************
Tiny Zooom... 1184 1721 363 542 95 ***************
Table Scan 881 1958 240 599 161 ***********
Tinyshot 1356 1503 405 454 141 ******************
tiny Blowrag 1215 1212 214 213 573 ****************
Tiny BiShot 2.0 1000 1840 280 560 160 *************
Digital Swarm 1140 1299 193 246 561 ***************
Son of Darkness 1047 1374 156 265 579 *************
Snowstorm 1280 1175 245 210 545 *****************
Sneaky Spike 1400 1442 414 428 158 ******************
Seek and Destro 1227 1599 351 475 174 ****************
Provenance 1154 1733 347 540 113 ***************
Potenza 1446 1281 391 336 273 *******************
Origin of Storm 1046 1814 302 558 140 *************
Muskrat 1845 1035 575 305 120 ************************
livewire voodoo 1353 1032 246 139 615 ******************
Hellish Glare 1454 1391 433 412 155 *******************
Lone Gunman 1539 1263 447 355 198 ********************
Where's Giles? 1210 1087 169 128 703 ****************
Diamondette 1928 767 541 154 305 *************************
bestwar4.red 1143 1662 316 489 195 ***************
Four Winds 1216 1633 355 494 151 ****************
--------------- --------- ----------- ---------=---------=---------=---------=
Adjusted Score: 130.6
084 is up and down without as many ties, and predictably does well
against the scanners and poorly against replicators. Refocused does
fairly well against bombers, not so good against the scanners and ties
a lot with the papers. Thus
confirming basic core war theory :-) but if this replicating form is
going to ever get anywhere it's going to have to toughen up. When
writing my evolving document I
dismissed the stones as one of the things I was attempting to
demonstrate was that the principles of evolution could lead to the
development of more
"intelligent" forms of code, but the dynamics of core war place those
forms at a disadvantage unless they develop strengths in other areas.
3/28/09 - A little easily-solved issue I haven't mentioned as it's a
specific OS bug (and I forgot about it) - under Ubuntu 8.04, the stock
DosEmu program fails with a memory map error and segmentation fault
because a security update that forbids access to the first 64KB of
system memory. The problem is described in this Ubuntu
launchpad entry along with various solutions. My solution was I
made a fix script which runs another script containing (this is for
Ubuntu 8.04 only) "echo 0 > /proc/sys/vm/mmap_min_addr" as root
(rigged so I don't have to bother with entering a password), then for
dosemu renamed the dosemu script to dosemu_stock and replaced with a
script that runs the fix then runs dosemu_stock "$@" to pass all parms.
I did it this way because my DosEmu was compiled from source so I can
mess with the files all I want, and at the time thought there might be
a reason for the security change so only wanted to reset it after
running DosEmu or another affected emulator. After reading the
discussion in the bug entry it appears that the memory map change
wasn't a real problem after all, so the suggested permanent "unfix"
that permits DosEmu to run unchanged might be a better solution if
affected. For later Ubuntu's the problem has been corrected (I think).
Just thought I'd mention this just in case someone installs DosEmu to
run RedMixer under Ubuntu and finds it doesn't work. I don't know if
other systems are affected but if a similar error occurs look for a
similar fix. It's just a bit of tweaking to sneak DosEmu past a
security change and for me is totally trivial compared to the
convenience of being able to run almost all my dos stuff from the past
15 years. My Linux GUI right-clicks for *.red files are full of dos
utilities for doing various things with warriors. Another possibly
helpful Ubuntu tip - under Gnome I use ROX Mime Editor to
define custom redcode type so the associated utilities don't show up
for every text file.
Updated the initial up-front links to include the various scattered
links plus a couple new links, the most significant is a link to "Using
Core War to Simulate Evolution", a document I wrote mainly to provide a
mostly self-contained snapshot of the present RedMixer program and what
it does, code
and all, in an easily preserved and distributable PDF file. This is the
first semi-proper document I've written about the subject, came out a
bit on the big side but includes what I wanted to include and says most
of the things I wanted to say. I was attempting to present a mostly
scientifically valid demonstration of the principles of evolution and
natural selection so I didn't use any tricks except for presenting one
warrior that was derived by re-evolving (02_34.red, see 3/15/09 entry
below) to show how unnatural selection can produce stronger code. Now
that the science is done (for now), almost all tricks available are
back on the table with the exception of seeding with already-strong
hand-written warriors. Even re-evolving other evolved warriors is fair
game provided it improves the form (but not really interested in doing
that). One thing I've been doing is taking a single strong warrior and
copying it to all positions in the soup then running the evolver a bit
to generate numerous varients (some stronger some weaker), then
repeating with the new strongest warriors. Several points of
improvement can be obtained this way. The "clone" program I use is very
crude...
line input "clone file: ";clone$
clone$ = ltrim$(rtrim$(clone$))
if clone$ = "" then system
line input "soup dir: ";soup$
soup$ = ltrim$(rtrim$(soup$))
if soup$ = "" then system
shell "md " + soup$
for x = 1 to 77
for y = 1 to 21
yname$ = right$("0" + ltrim$(rtrim$(str$(y))),2)
xname$ = right$("0" + ltrim$(rtrim$(str$(x))),2)
dest$ = soup$ + "\" + yname$ + "_" + xname$ + ".red"
shell "copy " + clone$ + " " + dest$ + " > nul"
next y
next x
system
3/20/08 - I've been playing around with the (3/2/09) size-800 settings, seems that it works about
the same (give or take due to random varience) when "attraction" is
turned off, where it's simply randomly picking a another surrounding
warrior of the same species to cross with (or just mutate if no
same-species warriors around). As of about generation 600 the present
run with attraction off is right in the middle score-wise as two other
runs with attraction on but otherwise identical settings, all produced
warriors scoring in the 160-170 range against the Franz tiny test set.
Seems the "attraction" idea isn't as effective as I was hoping it would
be, oh well at least it has a switch. Once again the simpler method
seems to be just as good (but we'll see, let it run for awhile [after
another couple hundred generations scores were down by almost 20 points
- but that might possibly be a result of using only 30 rounds - tricky
stuff, if rounds are more than 50 then replicators don't seem to
develop]). On the
other hand, some sort of similar-code selection mechanism such as
same-species (same length and similar base instructions) appears to be
necessary when using crossover, tried evolving with the crossmode set
to any (where it just picks a surrounding warrior with no regard to
content) and the result was a stony mess. No similar-code islands
developed, evolution occured but scores remained low. Haven't tried yet
but I'm guessing same-origin (a warrior decended from the same original
randomly-created warrior) would also work, at least to a degree, should
permit islands of similar code to develop. Question remains if the
islands will remain or if it'll all degenerate into a hodgepodge of
broken code.
3/15/09 - I rediscovered a little thing I forgot... ANSI can be
enabled in Windows XP, provided "command" is used to run the type
command or app using ANSI. To enable, check to see if the ansi.sys file
is present in the Windows system32 directory, if it is add the
following line to the CONFIG.NT file in the same directory...
device=%SystemRoot%\system32\ansi.sys
...then it should be possible to open a dos window where an ANSI
file is and display it using the command...
command /c type filename.ans
The "no ANSI" limitation in WinXP only applies to things running
under cmd, the "dos" command.com shell seems to process ANSI codes just
fine. This can be put into a batch file so (say) .ans files can be
associated to the batch and displayed by double-clicking them, but it
is a little tricky as the filename parameter passed by Windows is
quoted for long filenames but command (like any dos program) only
understands unquoted short filenames. The quotes can be removed with a
FOR command but there's still the matter of long filenames. Typing
"help for" provided a FOR option that converts the filename into a
short name, problem solved...
:: typefile.bat - displays file in a dos window (WinXP etc)
:: if ansi.sys installed in config.nt then displays ansi file
:: when "run" by associating .ans files with this batch
@echo off
for %%a in (%1) do set file=%%~sa
command /c type %file%
pause > nul
Also added to the ansi movie mod file (see 3/3/09 entry).
On evolving size 800 warriors... what I'm finding is the number of
soup battle rounds makes a big difference. With 30 rounds I get papers
but if rounds is set to 75 or 100 I only get boring rocks. Apparently
evaluation error is needed to foster the development of more
"intelligent" code, my theory is because it starts out relatively weak
and requires "luck" to evolve to the point it can compete with the
simpler but deadlier programs. Once a population of papers has taken
root, then rounds can be increased more more accurate evaluation and
stronger code. Another trick is once the strongest warriors in a soup
don't seem to be getting any stronger, take a few of the strongest
members and put them in a new empty soup. Upon re-evolving they'll
quickly take over the initial random code and create a soupful of
varients of the desired form, usually increasing the score of the
strongest members by a few points (but check often as after so long the
top scores return to the "normal maximum" for the parameters - but it's
a random process so results will vary). Using these techniques and the
red800.ini file with rounds first increased from 30 to 50 then to 100 I
ended up with this specimen...
;redcode
;name Pushed Replicator (02_34)
;author Terry Newton
;strategy Evolved by RedMixer 1.0d3d-800
;origin 18_73.red
;parents 01_35.red 02_35.red
;generation 1084
;assert CORESIZE==800
mov.ba < 730 , > 694
mov.b { 332 , { 264
mov.i { 335 , } 600
mov.i < 472 , { 744
spl.ab # 226 , } 2
mov.i < 4 , { 3
mov.i > 272 , } -2
mov.i } 728 , { 0
djn.i $ 208 , < 665
end 0
;species 01_17
;wins 4
;score 150
Opponent Scores Results Performance of 02_34.red
--------------- --------- ----------- ---------=---------=---------=---------=
Tiny Wicked 1470 1395 445 420 135 *******************
Tiny AK-47 1496 1379 457 418 125 *******************
Tiny Barbarian 1226 1163 205 184 611 ****************
Tiny Brutality 2449 364 754 59 187 ********************************
Tiny Everclear 1601 1220 474 347 179 *********************
Golden Gate v4 2366 545 759 152 89 *******************************
Tiny Porter 1708 1180 532 356 112 **********************
Tiny Spiral 1684 688 352 20 628 **********************
Tiny Tiberius 1392 1371 385 378 237 ******************
D-Clear 2440 475 785 130 85 ********************************
--------------- --------- ----------- ---------=---------=---------=---------=
Adjusted Score: 178.3
Opponent Scores Results Performance of 02_34.red
--------------- --------- ----------- ---------=---------=---------=---------=
Betadine 1131 1734 332 533 135 ***************
Wilt ;) 1203 1530 312 421 267 ****************
Unknown 1435 1162 344 253 403 *******************
Ugly Uncle 1558 1258 458 358 184 ********************
Tiny Zooom... 1157 1721 345 533 122 ***************
Table Scan 903 1917 241 579 180 ************
Tinyshot 1207 1672 362 517 121 ****************
tiny Blowrag 1143 1254 180 217 603 ***************
Tiny BiShot 2.0 923 1919 255 587 158 ************
Digital Swarm 1176 1218 190 204 606 ***************
Son of Darkness 1072 1336 160 248 592 **************
Snowstorm 1290 1173 251 212 537 *****************
Sneaky Spike 1369 1498 412 455 133 ******************
Seek and Destro 1143 1689 325 507 168 ***************
Provenance 1020 1863 301 582 117 *************
Potenza 1378 1312 356 334 310 ******************
Origin of Storm 1067 1814 316 565 119 **************
Muskrat 1900 1009 603 306 91 *************************
livewire voodoo 1258 1117 211 164 625 ****************
Hellish Glare 1353 1518 408 463 129 ******************
Lone Gunman 1449 1362 420 391 189 *******************
Where's Giles? 1241 1061 181 121 698 ****************
Diamondette 1923 774 540 157 303 *************************
bestwar4.red 1163 1634 320 477 203 ***************
Four Winds 1058 1772 296 534 170 **************
--------------- --------- ----------- ---------=---------=---------=---------=
Adjusted Score: 126
Not too bad but I don't think I've found the optimum parameters yet.
This is one of the warriors I started with, evolved with rounds set
to 30 initially then increased to 50...
;redcode
;name 15_38.red
;author Terry Newton
;strategy Evolved by RedMixer 1.0d3b-800
;origin 18_73.red
;parents 15_39.red 14_40.red
;generation 931
;assert 1
mov.i * 458 , < 178
mov.i { 25 , } 601
spl.ba > 660 , < 440
mov.ab { 470 , { 380
spl.ab # 223 , } 2
mov.i < 4 , { 3
mov.i > 272 , } -2
mov.i $ 773 , { 0
djn.x $ 208 , { 704
end 1
;species 09_19
;wins 3
;score 164
Scores 172.3 with the Franz test set, 114.7 with the tinybm04 test
set. Here's another one used for the re-evolving experiment that scores
119.8 against the tinybm04 test set and 162.8 against the Franz set...
;redcode
;name 15_62.red
;author Terry Newton
;strategy Evolved by RedMixer 1.0d3b-800
;origin 18_73.red
;parents 14_63.red 15_63.red
;generation 1043
;assert 1
mov.i { 95 , } 600
mov.i > 510 , { 535
spl.b # 452 , } 0
mov.i < 4 , { 3
mov.i > 39 , } -2
mov.i { 763 , { 352
djn.i $ 208 , < 95
end 0
;species 02_01
Probably the biggest "problem" I'm having with RedMixer is it is a
manually-driven program with no automated data-collecting features - I
have to stop it when I think it's about time then manually run the test
program and gather the data. Not that this is really a flaw as an
evolver (wasn't part of the plan when I wrote it) and it works fine if
I just run it and mostly leave it alone other than occasionally
checking, but makes it difficult to use for gathering scientific data -
score increase over time, comparing various settings, etc.
Contemplating how to go about adding a "statistics" mod. For best
results and to avoid missing a rare strong specimen the entire soup
needs to be benchmarked but this takes a long time, usually an hour or
two depending on benchmark rounds. Fewer rounds is faster but not as
accurate so the top-scoring warriors need to be tested at higher rounds
(typically 1000) for valid results. Another approach might be to
occasionally benchmark just a small random sampling, which would
probably work as far as catching strong code but wouldn't produce
scientifically accurate results. In addition, the entire soup needs to
be periodically archived to permit further study afterwards. Thinking
about how best to do all this without impacting the program too much,
preferably using a copy/paste kind of mod. Subject to change (or not
doing at all) but thinking along the lines of settings for
testinterval, testpercent, savedir, savescore [or maybe just save the
top few], archivedir, archiveprog
and the archiver better not require too much memory. Separate paths for
the archives and saved warriors permit them to be saved to the hard
disk to avoid using up ramdisk space using some kind of auto-numbering
scheme. Just a thought.
3/3/09 - This is fun if into this sort of stuff... this ANSI Movie Mod for RedMixer adds code that
appends an ANSI "frame" to a file at specified intervals, when the file
is played back in an ANSI viewer (or in Linux or Cygwin displayed using
the cat command) it produces an animation of soup progress. The mod
file includes a simple QBasic ANSI viewer. Unlike earlier REBS
experiments which wrote everything to the file, writing frames at
intervals results in much smaller files and it didn't take much effort
to make a program that can play the file back with adjustable delay
between frames. For now the mod only records the soup display,
practically identical to the output of DUMPSOUP.BAS only appends.
Perhaps in the future other information can be included but for now I
just want to see what happened while I'm away, or just enjoy watching
the creatures do their thing.
3/2/09 - I've been fooling around with trying to evolve coresize 800
warriors (having almost given up on coresize 8000). Here's a coresize 800 INI I've been experimenting
with, the 3/2/09 version is much better... at least it makes some
replicators instead of all bombers like the 3/1/09 settings file did.
Here are a couple of "one-off" programs I've been using to help document RedMixer runs...
SCOREANA.BAS - takes a TEST scorefile
of a RedMixer soup and reports average score, maximum score and by
which warrior(s), and plots an ascii chart of the soup showing where
the strong warriors are located.
SOUPANA.BAS - reads all warriors in a
RedMixer soup and reports min/avg/max generation, min/avg/max size, how
many warriors have score data (having won at least one battle) and of
those reports the average wins and the warrior(s) with the max wins,
and average accumulated score and the warrior(s) with the max
accumulated score (the usefulness of this data is dubious, often only
indicates warriors surrounded by weak warriors), and outputs an ascii
chart of the warrior sizes.
DUMPSOUP.BAS - writes an "ansi" file
with a representation of a RedMixer soup colored by species. The colors
are different but close enough to (usually) show where the species
boundaries are. Ansi files can be viewed in Linux using the "cat"
command, or by a dos "type" command if the dos can support ansi.sys and
it's installed. Multiple frames can be appended together (i.e. cat
file1.ans file2.ans etc > file.ans) to make an animated "movie" of
an evolving run. For Windows perhaps an ANSI viewer can be found, or
use cat under Cygwin. [Note... until more error checking is added to
account for non-existent warriors, dumpsoup.bas is restricted to
dumping complete soups only, such code can be found in the "Movie Mod"
file which also has an ANSI viewer which can be used under
Windows/QBasic to view ANSI files produced by these techniques.]
[Another note... WinXP can view ANSI directly without using external
software... see 3/15/09 entry]
2/27/09 - RedMixer package updated. Still 1.0d, no changes to the
program itself (I like it!) but reworked the readme.txt to include more
scripts and platform-specific usage information, and for Windows users
named the run batch to a more descriptive Run_RedMixer.bat and added a
Stop_RedMixer batch.
I did some more tests under
Windows and it seems it is mainly the dos DJGPP builds that cause
control-C grief under Windows, the SDL pmars-server version might be a
bit slow to respond to control-C when cranking but no weird bugs occur,
no need to panic just use the right software. If running RedMixer under
Windows use the SDL versions of pmars/pmarsv, they're pretty much the
only binaries that do work well under Windows. I now rate my DJGPP dos
builds only for dos (or rather DosEmu/FreeDos which is their intended
purpose), they work under Windows but the darn the break bugs. I did
some timed benchmark tests under both DosEmu/FreeDos and Windows XP and
found the speed differences were not as great as it seemed - at 500
rounds benchmarking the same warrior, in DosEmu using the pm092-O3.exe
dos binary it took 31 seconds, shelling to Linux to run the pmars92N1
binary took 28 seconds and the old dos 0.8.0 binary took 39 seconds.
Under Windows XP using the SDL pmars-server binary the same test took
27 seconds and the pm092-O3 binary took 29 seconds. Windows XP runs
QBasic much faster than DosEmu so at very small round numbers or at the
start of evolution it really flies giving the illusion it's about 5x
faster but once the warriors start spending much more time in pmars
than in the evolver the speeds mostly even out. Another possible factor
was under Windows the dos binary likely has less startup time (no
switching out of the dos emu) than the Windows-specific version, making
it seem speedier but not once pmars has to actually do work.
2/26/09 - Here is my patched version of
pmars 0.9.2 with the continuation fix. Includes binaries for
Linux/x86 pmars and pmarsv, and a couple of DJGPP-compiled pmars server
binaries for dos
(one is speed-optimized, the -O3 version seems faster on my systems but
this might not be the case with all systems depending on the processor
cache etc). These binaries are mainly to use with RedMixer running
under DosEmu/FreeDos running under Linux. For Windows there are
probably [no, are] better Windows-specific binaries that can be used
(even if a
few test set warriors need to be edited to avoid the continuation
"feature") but even if a bit on the unstable side the new binaries seem
faster than what I was using [but it was an illusion... blathering
removed].
I ran into an issue using the old dos pmarsv.exe from RedMixer when
running interpreted in QBasic.exe - not enough memory to run coresize
8000 warriors (only works for nanos). At least I think it's lack of dos
memory causing the problem. The compiled redmixer.exe version runs
pmarsv.exe just fine with coresize 8000 but only supports a 25 by 80
"dos window". Essentially this means pmarsv50.bat for running pmarsv
when using a 50-line screen is only useful for nano warriors, haven't
tested with tiny warriors yet. For Windows this isn't an issue at all,
just use the SDL version and set pmarsv to pmarsw.exe which doesn't
care what the dos window size is as it runs in its own window. Every
non-SDL version of pmarsv.exe I tried under Windows goes to full-screen
mode when used, if used with a 50-line display from pmarsv50.bat messes
up the RedMixer display until popped back into a window (alt-enter).
And then there are out of environment space issues requiring editing
Windows' config.nt file (or specifying command /e:4096 /c pmarsv50.bat
for the pmarsv setting). Too much trouble when the SDL version works
great. For my Ubuntu DosEmu/FreeDos setup I solved the problem by
adding scripts to run my new Linux version of pmarsv instead, like
running the Linux pmars from RedMixer it's a hack but once set up and
going works ok, I only need it for my "big 8K" setup, a 46 by 77 soup
of coresize 8000 warriors. Details are explained in the red8kbig.ini file I'm presently
experimenting with. The evolution parameters are not optimized, neither
are they for the other 8K ini's yet - evolving for coresize 8000 is
hard!
2/23/09 - A few things. I've been benchmark nanos using TEST
set to 250 to 500 rounds fixed when sweeping my soups to find strong
warriors, for about 1500 warriors and a fairly large test set this
takes a fair amount of time but not too bad, maybe an hour or two. Once
a report has been generated I choose the top 10 or 20 then subject them
to further tests using TESTALL.BAT with TESTASET.BAT set to 1000 rounds to
generate a "warrior report", typically using the nanobm07 test set, the
koen090112 set presently included with redmixer.zip, and a composite
test set combining them and samples of my
nano warriors to try to
simulate the SAL nano hill. These tests
only
take a couple minutes each as there aren't that many picks to test.
Turns out reasonably accurate results can be had by running only 142
battles using the -P option instead of -f but it only works with pmars
version 0.9.2. Ok so I drag out DJGPP
and make me a dos version of pmars 0.9.2 and almost immediately
discover that it turned in incorrect scores for coresize-8000 warriors
- thought I had unusually strong warriors but upon further
investigation discovered 3 out of 24 warriors in the Wilkies and Wilmoo
test sets didn't execute correctly. To make a long story short
eventually discovered that around version 0.8.6 a change was made to
make pmars treat "\" at the end of a line as a continuation mark -
breaking warriors that uses "\" and "/" in the comments to point out
blocks of code. So fixed that with a modification
to the asm.c file. The makefile
I used was modified to replace some whitespace with tabs to make it
work, set to make a "server" version and use -O3 optimization to make
it go as fast as possible in my DosEmu environment. Overall the new
binary is better, seems a lot faster but doesn't respond to control-C
as well as the pmars.exe from pmars08.zip (from www.koth.org/pmars),
have to press control-C a few times to get my evolver to stop. Worse on
my Eee PC so for now sticking with the old 0.8.0 dos version which on
that platform is plenty fast enough for evolving nano warriors. For my
Ubuntu system I've mostly switched to the new version where I'm
evolving warriors for coresize 8000 and need all the speed I can get,
plus that's where I do most of my analysis and can use the -P option in
TESTALL.BAT. Using -P in TEST will take modifying the program itself as
it provides no access to the command line, a pending change as that
would give me faster and more accurate soup sweeps.
I've been trying to evolve coresize-8000 warriors using RedMixer, so
far the best I've gotten were "RepliBomber"
things that scored up to
about 90 with the Wilkies test set and 70 or so against the Wilmoo test
set. Weak but somewhat better than I usually get from simple evolvers.
I ran the soup for some time but never got anything over the
low '90's on the Wilkies scale. So trying something else. Wrote a redcode analyser (warning it's crude and
rude) that reports the number of instructions, modifiers, address modes
and data mixes in both the A and B fields. I used CONVRC.BAT (and a similar adaptation for
nano warriors) to convert the warriors to "load" format, otherwise
labels would make the number counts useless. One problem was some
warriors had huge sequences of DAT 0,0 lines which an evolver would not
likely generate so ignored these in the counts. The analyser gave me nano stats and standard
stats based on the benchmark code that I used to make a new INI
file for evolving coresize-8000
warriors [latest adjustments here]. The
INI also includes a deeply convulted hack for running my
native Linux pmars to try to get more speed, but the new dos 0.9.2
binary I made is just as fast or faster so that was mostly useless
other than showing how it can be done, at least until I make a
super-optimized Linux version of pmars. I need to make new Linux
binaries anyway, my pmarsv is broken in that it won't let me break into
the debugger and need to apply the "\" mod to both pmars and pmarsv.
Oh, and removed the Evolver Design page... everyone's got there own
ideas about how it should be done and I think my theories are
sufficiently documented elsewhere (like the program code itself). Plus
it's a moving target. The grid-like structure I use does help keep the
soup from going stale, I'm pretty sure now crossover has some benefits
(but still unsure if from higher mutation and more broken code or from
the blending of code), and weighting the instructions and other items
is very helpful. But nano warriors are easy to evolve, trying to evolve
for coresize 8000 is a much more difficult task. I'm starting to think
that to evolve stronger standard warriors it's going to take more than
blind weighting, will see how the current experiment goes but thinking
along the lines of weighting the modifiers, modes and data for each
instruction, and weighting the instructions on the previous
instructions to attempt to build code that more resembles human-written
code. Similar tactics have been and are being used with good results
but it takes a lot of code and often multiple computers to pull it off
- so the task is to simplify! eliminate the complicated aspects as much
as possible but such a thing is probably beyond the framework of the
RedMixer program... the INI would be huge as separate data would have
to be specified for each instruction, including a weighted instruction
set for the following instruction. I probably won't have time to do all
that anytime soon but maybe the idea can be useful for other
evolver-makers.
2/20/09 - So far so good with 1.0d but the default 5 percent
specieschanges setting probably needs increasing for non-nano warriors,
trying 10 for somewhat bigger islands of "similar" species with my
coresize 8000 experiment. Updated the posted readme.txt to include
more/better coresize 8000 information, will add to the zip at some
point. The current zip instructions for coresize 800/8000 forgot DJN
[updated readme posted but not in the zip yet],
but just as well as often one-line DJN streams take over the soup and
hinder the development of more sophisticated strategies - for coresize
8000 warriors I recommend leaving out the DJN instruction until the
soup develops then add it to the list.
2/19/09 - Updated to RedMixer version 1.0d with a new specieschanges
setting which sets the percentage of instructions that can be different
and still consider the offspring warrior to be of the same species. The
default is 5(%), so with nano warriors if any instruction changes it's
a new species but with a 80-line warrior up to 4 instructions can be
different and still count as the same species. Fixed up a few other
things like not covering up the results with a "press any key" when
battling warriors, added C and S indicators to show when crossover and
species changes occur, and modified so it actually checks to see if a
different instruction was actually selected before counting as a change
- previously it counted if it picked the same instruction again (this
changes the behavior slightly from previous versions - hopefully the
fix doesn't hurt anything... sometimes errors can have unexpected
beneficial results).
The coresize 8000 soup is coming along, up to around generation 350
and full of paper-like warriors. Changed to use cross by species method
now that it is more likely to work (was crossing by size), there are a
lot more warriors of the same species to cross with now that the
species change rules have been loosened.
2/16/09 - the 77x46 soup of nanos didn't work out, scores never got
any better than the high 130's, not competition-strength. Oh well that
happens, had set initial minimum size to 3 but perhaps the default of 1
is better. Started a new soup on my 701SD using the default settings.
The 8000 soup is getting stronger, full of paper/bomber things that get
Wilkies scores up into the 90's, nowhere near what would be considered
strong but a bit stronger than what I typically see from evolvers.
2/15/09 [edited 2/16/09] - Version 1.0c has a fixed pmarsv50.bat
file that doesn't leave a null file (used to Linux, in Dos it's nul not
null), fixed yet another validate bug in RedMixer and added new options
to permit crossing by size (works better with coresize 8000) and
control how the species tags are selected.
I'm currently running two experiments... my Eee PC 701SD is busy
churning away at a 77x46 soup of nano warriors and after running
overnight is up to the 100-150 generation point with bench scores in
the 130-140 range. By SPL MOV MOV MOV MOV code, this soup seems to be
developing something besides SPL MOV MOV MOV DJN. On my Ubuntu system
I've been evolving a 77x21 soup of standard warriors for coresize 8000,
after a few overnight runs it's up to generation 150-220 with warriors
scoring in the 60's and 70's against the Wilkies test set. Not going so
well but they generate cool fireworks in pmarsv. To speed up initial
evolution I had rounds set to 5 random but last night increased to 10
doublefixed. Color by species just makes a messy display as there
essentially are no islands of species, most warriors are mutated enough
to count (by my methods) as a new species and merely take on the color
of the soup position but there are still enough like species to trigger
crossover about 10-20% of the mutations. Probably mostly "headless
chicken crossover" where the selected cross partner isn't actually
similar code but merely has the same tag. Since the species tags are
just the soup position where the mutation occured if a warrior
reproduces without base instruction change then gets replaced by a
warrior with a different sequence then if the previous offspring
reproduces it might select the warrior with a different sequence as it
carries the same tag. This isn't as bad as it sounds since most of the
soup is now decended from the same original random warrior and kind of
sort of already similar so there's a chance the cross will work anyway,
just makes for a useless soup display when colored by species. Coloring
by size or origin (at least until totally saturated with one origin)
provides a somewhat more useful display.
2/14/09 - and I thought I was done :-) The main reasons for the
[1.0b] update was to speed
up the soup display when using the user interface - discovered that my
FirstBASIC compiler doesn't support large screens so to run a "big"
77x46 soup I had to run the program interpreted in QBasic and under
DosEmu screen updates were taking about 9 seconds every time I listed a
warrior or anything, so fixed it so the first time it records stuff to
an array then reads the array to refresh the soup display rather than
rescanning roughly 3000 warriors. Now it only takes a couple of
seconds, much better. On Windows the refresh is almost instant (but
even the original code only took a second or so). While in the code I
cleaned up a few other things like fixing the built-in
warrior/benchmark file viewer so it displays more lines if a bigger
soup display is enabled (goes by the ysize setting). The other reason
for the update was to update the readme.txt and include a batch file
for running - after updating my WinXP system to SP3 and the latest
updates I was greeted by a horrible flickering display when RedMixer
was evolving, was working fine but the update messed up the "dos"
window making it react badly to the changing display. Fortunately all
it took was mode con lines=25 in a batch file to fix, whew! Sometimes
modern stuff hates my old stuff - if they made modern stuff that works
as well as old stuff then maybe I'd use it, but a BASIC compiler that
can't shell more than a few dozen times just won't cut it and I'll be
damned if I inflict my readers with code for a compiler that doesn't
have bounds checking... sometimes I make mistakes and a mistake in C
can crash the system. Not to mention having to spend weeks writing a
program instead of a few days. QBasic is relatively safe and easy...
Ran into another issue... when running in 50 line mode the version
of pmarsv.exe I use resets the text mode to 25 lines when it exits, and
if set for text mode doesn't work at all unless in 25 line mode. The
zip now includes pmarsv50.bat to fix that. This isn't an issue when
using the SDL "pmarsw" version under Windows.
[outdated notes removed, initial code posted 2/11/09, user interface
and zip package added 2/12/09]