Sequences of bit-net matrixes from BNRIDER ------------------------------------------ This project is described in detail at: http://www.nc5.infi.net/~wtnewton/otherwld/robot2.html In case you found this file in isolation, here are some definitions... A bit-net is my term for a simple OR-network that connects a set of inputs to a set of outputs, similar to a simulated matrix of diodes except the bit-net uses neural-network-like rules to spontaneously rewire the net in response to a training signal. BNRIDER is a bit-net for a PIC processor, designed to control a walking robot based on circuits invented by the famed Mark Tilden. Cool stuff! The "microcore" is an arrangement of four of Mark's patented "nervous neuron" circuits arranged in a loop to form a rhythm generator capable of driving a two-motor walking robot. Layout and numbering... .-------------- Both Feelers Microcore Layout... |.------------- Right Feeler (arrows show forward mode) ||.------------ Right Dark |||.----------- Right Light front motor ||||.---------- Middle Light .--. |||||.--------- Left Light left .-(+ -)-. right ||||||.-------- Left Dark | `--' | |||||||.------- Left Feeler .> 2 . . 4 <. |||||||| | \ / | ------------ | \/ | | ** |----- Reverse | /\ | | |----- Control 1 | / \ | | ** * |----- Control 2 `- 1 < > 3 -' | |----- Control 3 | .--. | | * ** |----- Control 4 `-(+ -)-' | * ** |----- Increase Effect `--' | * * |----- Increase Time rear motor ------------ Note - only one Feeler, Dark and Light input can be active at any one time, at the most three input signals. The train signal is applied to the last move when a feeler remains depressed, or was depressed in the move before last. Reverse is only selected when training if a feeler is touching. The signal from the front switch (and it's pulse-stretcher) is applied to both the left and right feelers and appears as Both Feelers. For these tests, the direct connection from the front switch to the reverser circuit has been disconnected forcing it to learn to back up by itself. Running in the larger box... ------------------------------------------------------------------- | ** | ** | ** | ** | ** * | ** * | | | | ** | | * | * | | ** * | ** * | ** * | * * | ** * | ** * | | | * | * | * | ** | *** | | * ** | * * | * ** | * * | * ** | * ** | | * ** | * ** | * | * ** * | * ** | * ** | | * * | * | * *** | * * | * * * | * * * | ------------------------------------------------------------------- | ** * | * * | * * | ** * | ** * | ** * | | ** * | * * | ** * | * ** * | ** * | * ** * | | ** * | ** * * | ** * * | * * * | ** * | * ** * | | * | * * | * * | * * | * | | | * ** | * ** | * ** | * ** | * ** | ** | | * *** | * * | * * | * * * | *** | * | | * * * | ** ** * | ** ** | * * * | * * * | * ** * | ------------------------------------------------------------------- | ** * | ** * | ** * | ** * | ** * | ** * | | * ** * | *** * | * * * | * * * | * * * | * * | | * * * | * * * | * * * | * * * | * * | * * * | | * | * ** | * ** * | * **** | * * * | * * | | *** | ** * | ** * | ** ** | ** ** | ***** | | *** | * | ** | * | * | * | | * * | ** * * | * * * | * ** * | * ** * | * * * | ------------------------------------------------------------------- | ** * | ** * | ** * | | | | | * * * | * * | * * * | | | | | * * * | * * * | * * * * | | | | | * * | * * | * * | | | | | ** ** | ** * | ** * | | | | | * * | * * | * | | | | | * * * | * ** * | * ** * | | | | ------------------------------------------------------------------- Connection Sums and Average... ------------ 19 21 0 0 0 0 0 17 | ** * | 2 0 1 12 8 10 0 17 | * * | 9 21 0 6 8 4 0 17 | * * | 8 0 8 5 4 15 0 0 | * | 1 21 9 2 17 15 0 0 | * ** | 8 0 1 9 11 16 0 1 | ** | 2 20 1 11 12 5 1 16 | * ** * | ------------ The following is from the same run, after spending the night in a smaller box, half-breaking a leg, and having it fixed. In the first five frames it was still in the smaller box, moved it to the larger box (with varying light levels) to complete the sequence... -------------------------------------------------------- | ** * | ** * | ** * | ** * | ** * | | * * * | ** * | ** * | * * | * | | * * | * | * | * | ** | | * * * | * * *** | * * *** | * ***** | * ***** | | **** | *** | *** | * *** | * *** | | | * ** | * ** | ** | ** | | | ** | ** | * ** | ** | -------------------------------------------------------- | ** * | ** * | ** * | ** * | * * | | | * | * * | * * * | * * | | * | ** | *** | * | * | | * ***** | ***** | ** | * ** | ** | | **** | * **** | * ** | * | ** | | ** | ** | * ** | ** | ** | | * ** | ** | **** | * ** | * ** | -------------------------------------------------------- | ** * | ** * | ** * | * * | ** * | | * | | * | | * | | ** | ** | ** | * * | * | | * ** | * *** | *** | * ** | ** | | * | * | * | *** | *** | | ** | *** | * *** | * *** | *** | | * ** | * *** | ** ** | ** ** | ** ** | -------------------------------------------------------- ------------ 12 15 1 0 0 0 0 15 | ** * | 6 5 1 1 4 5 0 1 | | 1 0 5 6 4 8 0 0 | * | 0 6 3 8 4 9 14 14 | * *** | 4 0 9 10 15 4 0 0 | *** | 2 0 0 2 1 4 14 14 | ** | 2 0 5 4 2 2 14 14 | ** | ------------ The only really constant thing that has been learned is to go in reverse whenever a feeler touches, but that makes sense - it is the only reliable action available. A microcore walker can only barely steer, it more or less goes where it wants to making it an interesting test of re-enforcement learning ideas. Controlling a wheeled robot would be a breeze by comparison. The robot has been "alive" for over two days, but I need to add some debugging code so I can check out the good/bad ratios of each map. Before I do, here's one more look... -------------------------------------------------------- | ** * | ** * | ** * | ** * | ** * | | ** ** | ** *** | *** *** | *** ** | * ** | | ** * | ** * | **** | * * | ** | | * * | * * | *** | ** * * | ** * | | * * | * * | ** * | * * | * * | | * * *** | * * *** | * * * | * ** ** | * * * ** | | * * * | * * * | ** * * * | * * * | * ** * | -------------------------------------------------------- ------------ 5 5 0 0 0 0 0 5 | ** * | 1 5 4 1 0 2 5 5 | ** ** | 0 0 4 4 3 3 0 0 | **** | 0 1 5 2 1 3 0 2 | * * | 2 0 5 1 2 1 0 0 | * | 5 0 5 1 3 2 5 2 | * * * * | 1 5 0 2 4 1 0 5 | * * * | ------------ Looks like it picked up some associations this time around. How's it doing? Here are some results from a new run that recorded when training signals occurred, allowing the good-to-bad ratio for each frame to be computed... -------------------------------------------------------- | ** | ** | | * * | * * | | * ** | * | * * | | | | * * * | * * | * | * * | | | * * | * | * | * * | *** * | | * ** | * * | * | ** | * ** | | * * * | * | * | * ** | ** | | * * * | * | * | * *** | ** * ** | -------------------------------------------------------- | ** * | ** | ** | ** * | ** * | | * * * * | * * * | *** * * | ** * * | ** * * | | * ** | * ** | ** *** * | * * | * ** | | ** * | ** | ** | ** * | * * | | **** | **** * | * * * | * ** * | * * | | * * * | * * * | * * * | * * | * * | | * * | * * * | * * * | * ** | * | -------------------------------------------------------- | ** * | ** * | ** * | ** * | ** * | | ** ** * | ** ** * | ** ** * | ** ** * | ***** * | | * * | * * | * * | * * | * | | * * | * * | * | * | * * | | ** * * | ** * * | ** * | ** * * | ** ** * | | * * | * * | * * * | * *** | * *** | | * * | * * | * ** | * ** | * ** | -------------------------------------------------------- | ** * | * * | ** * | * * | ** | | ***** * | ***** * | ***** * | * * * | * * | | *** * | *** * | *** * | ** | * *** | | * | * | * | * * | ** * | | * * * | * * * | * * * | * * | * ** | | * *** * | ** *** * | * **** * | ** * | * | | ** * | ** * | ** * | ** * * | * * * | -------------------------------------------------------- | ** | ** | ** * | ** * | * * | | * * | * | * * | * * | * * | | * *** | * *** | *** * | * *** | ** * * | | ** * | * | ** | *** | * ** | | * ** | * ** | * | * | * * | | * | * * * | | * | * | | * * * | * * | * * | * * | * * | -------------------------------------------------------- ------------ 24 19 0 0 0 0 0 17 | ** * | 1 13 15 9 10 18 0 18 | ** * * | 3 15 10 13 10 11 9 1 | * * | 1 1 6 13 6 2 16 5 | * * | 7 15 4 8 9 11 7 12 | * | 3 15 1 11 8 16 3 8 | * * | 13 15 1 15 6 9 3 4 | ** * | ------------ Performance... (moves) 1: 76% (46) 2: 60% (64) 3: 53% (32) 4: 56% (58) 5: 87% (16) 6: 41% (36) 7: 71% (14) 8: 71% (46) 9: 56% (16) 10: 50% (6) 11: 50% (20) 12: 100% (6) 13: 94% (54) 14: 83% (6) 15: 93% (30) 16: 86% (61) 17: 100% (6) 18: 100% (6) 19: 82% (64) 20: 69% (56) 21: 100% (6) 22: 83% (54) 23: 86% (30) 24: 86% (44) 25: 66% (30) There isn't a direct correlation between the percentage-good scores and how good the corresponding pattern is, the percentages are simply a measure of how much training occurred to arrive at the pattern. The pattern's effectiveness is reflected in how successful it is in the next period, or how much the pattern remains the same. Convergence to the environment appears to occur sometime during period 11, comes and goes after that. This data doesn't mean much beyond surface examination of the acquired memories over time. The percentage-good scores measure not only performance, but the amount of change in its environment. For a good part of the last frame it was upside down having a leg soldered on. The whole time it had to walk around dragging a data log tether. I'm sure that affected things. And in low light it sometimes only walks for six cycles, those 100% scores are probably good luck. ... The 1016b code has a few changes that will hopefully result in better performance: For me, the minimum awake time has been increased to 14 cycles. The dark input bits now turn off the photo-bits, further reducing the input space to no more than two active inputs at once. RL algorithm change - now if a repeating bad signal occurs it trains not only the current in/out set but also the previous. Before it had to get all of the twist from the current move, ignoring the short ineffective backup that preceded it. Much better now. The following frames are not continuous, the log data contains many gaps. It had already adapted before I started recording, between some of the periods the robot had some traumatic experiences including almost being discharged (wouldn't budge, between 6 and 7). There seems to be some clumping in later frames, possibly due to the tug of the data tether or other environmental influences. -------------------------------------------------------- | ** * | ** * | ** * | ** * | ** * | | * ** | * ** | * | * * * | * * ** | | * * ** | * * * | * * | * | * | | ** | ** * | *** ** | * * ** | * **** | | ** * * | *** ** | ** **** | ** *** | ** * * | | * * | * * | * * | * * | * * | | ** **** | * ** | *** * | ** ** | ** * | -------------------------------------------------------- | ** * | ** * | ** * | ** * | ** * | | * **** | * ** | * ** | * * | ** * | | * * | * * * | * * * | * * | * * | | * **** | **** * | **** * | ****** | ****** | | ** * * | * * ** | * * ** | * * * | * *** * | | * * | | | | | | ** * | **** | **** | * *** | * * | -------------------------------------------------------- | ** * | ** * | ** * | | | | * ** | * * | ** * | | | | * * * | * * | * * | | | | **** * | ****** | ****** | | | | * * ** | * * * | * *** * | | | | | | | | | | **** | * *** | * * | | | -------------------------------------------------------- Connection Sums and Average... ------------ 13 13 0 0 0 0 0 13 | ** * | 6 0 4 12 1 7 8 0 | * ** | 12 0 2 4 4 0 7 1 | * * | 0 0 11 10 13 9 8 12 | ****** | 12 6 2 8 7 6 7 8 | * ** ** | 0 6 0 0 0 0 0 6 | | 1 6 11 4 6 9 9 1 | * ** | ------------ Performance... (moves) 1: 67% (43) 2: 60% (25) 3: 82% (58) 4: 95% (46) 5: 84% (38) 6: 63% (22) 7: 85% (60) 8: 100% (14) 9: 81% (43) 10: 81% (66) 11: 100% (14) 12: 81% (43) 13: 81% (66) The 1017 code changes were minor, just how long to sleep, however the diode connecting the reverse "neuron" to the reverser circuit has been reconnected. This sequence was recorded from power-up and contains no gaps... ------------------------------------------------------------------- | ** * | * * | ** * | * | * * | * * | | **** ** | ** * * | *** * | * ** | * ** | * ** * | | **** * | * * * | * * * | * * * | * * * | * *** | | *** **** | * * **** | * *** | * * | * * | * * * | | | * * | ** * | * * * | * * | * ** | | * | * | ** | ** | ** | ** | | * * * | * ** | *** | * ** | * ** | * ** | ------------------------------------------------------------------- | ** * | ** * | * * | ** * | ** * | ** * | | ** | ** * | ** * | * ** * | * ** * | * ** * | | * *** | * * * | *** * | *** * | *** * | *** * | | * * | * * * | ** * * | ** * * | ** * | ** * | | * ** | * * | * * | ** * | *** | *** | | | | | * | * | * | | * * | * * | * | * | ** | ** | ------------------------------------------------------------------- Connection Sums and Average... ------------ 9 9 0 0 0 0 0 12 | ** * | 7 3 12 10 2 6 1 3 | * ** * | 0 1 12 5 12 2 12 0 | * * * | 2 1 12 4 2 7 3 12 | * * * | 2 2 0 11 3 4 7 0 | * * | 8 5 0 0 0 0 0 0 | * | 8 0 0 6 7 5 2 0 | * ** | ------------ Performance... (moves) 1: 82% (45) 2: 90% (30) 3: 57% (14) 4: 83% (30) 5: 55% (18) 6: 92% (14) 7: 90% (52) 8: 96% (33) 9: 89% (39) 10: 64% (14) 11: 81% (16) 12: 100% (14) Even though the hard-wired reverse reflex is present it still prefers to send its own reverse signal. This time around I tried to prop the data collection wire up so not to exert as much drag. Might have worked... at least the distribution looks more natural. ----- from Terry Newton last modified October 17, 1997