by Terry Newton
(caution... new page... under contruction... proceed at own risk...)
This article explores various ways to control and add apparent intelligence to walking robots based on Mark Tilden's patented robotic control system, refered to as a "nervous network". The primary focus here is on networks using four neurons in a loop structure, commonly refered to as a "microcore", typically connected to two geared motors to provide back-and-forth rhythms capable of moving a robot across irregular terrain. Such robots are inexpensive to construct, and perform similarly to or better than much more sophisticated designs that use computers to attempt to figure out where to place each leg. A nervous network with simple mechanics can "compute" walking all by itself, application of very simple electrical and mechanical principles that result in self-adapting walking action. One could accomplish the same thing without using nervous networks but the patented circuits have additional adaptive properties that are very desirable for autonomous robots.
Index of material so far...
The nature of intelligence as it applies to robots
A short primer on influencing Nervous Nets
Brains for Nervous Net Walkers
Random direction when a feeler touches
Using another Nervous Net for control
Bit-net associative memory in hardware
Another idea for an associative memory
Please keep in mind that this stuff is highly experimental, most of the ideas will require additional design work and have not been tested. Build at your own risk..
The nature of intelligence as it applies to robots
Read the definitive Living Machines paper, and don't let the big claims for such a little circuit throw you, once you see one in action you'll realize it's all true, even if you wouldn't of put it quite that way. Truth is, a motor with springy legs and series resistance can be called "a high complience adaptive leg able to compute its motion in parallel with the other legs". It is. Yes, it's that simple! For walkers anyway. You don't need to measure the current of a motor to determine where it should stop - no, just soften it up and it stops by itself. A nice property of the microcore is (with proper circuit elements) it can "feel" the motors to a certain degree and adjust its timing accordingly. There is debate on how much this matters but electrical feedback is just one property that can be used to enhance performance, springy legs and "soft" motors and drivers do most of the "computational" work and produce interesting adaptive effects even with no feedback at all. So far I've made two of them, the first walker has lots of feedback and a cpu brain, the second walker originally had almost no feedback but worked much better thanks to its superior motor drivers - except for an annoying property of head-butting walls, it needs a brain to give it some direction. For a time I layered on a version of my PIC brain to associate the feelers and photocells to the four microcore inputs and reverse. Later I used a simple hardware bitnet made from a couple chips and a bunch of diodes, alas I think that might come off for being too few bits in too big of a space, works reasonably well (for what it is) but makes an otherwise cool walker way too tall and ugly. Back to hardwired for this one, at least until the next brainy design comes along.
Judging from posts to the BEAM discussion list, it seems people have extremely different ideas about what constitutes "intelligence" and "intelligent" (subtly different words). When I say my walker shows intelligence, what I mean is my walker shows some ability to process information, learn from it and use that information to better deal with the world around it. Lots easier to say it has some intelligence. Smarts or Learning Ability are workable substitutes for intelligence, at least in the context of small autonomous robots. However, I would not say my walker is intelligent without also stating what I was comparing it too. On good days my walker is more intelligent than the typical rock.
Although I don't count them in the "brain" catagory as defined here, hard-wired behaviors layered on top of the basic walking circuit can go a long way to improve a walker's ability to solve problems, thus increasing its apparent intelligence. It looks smarter, never mind it's just a bunch of diodes and stuff with no memory. What these are doing is transfering some of our human intelligence into it, at least as far as what we think it ought to do. Often the solutions work better than what the robot could have learned on its own, at least until the robot finds itself in a predicament that the designer did not anticipate.
What I consider a robot "brain" should be able to figure out workable associations on its own, without the builder specifying what should activate what but let the robot decide for itself. Even though self-learning brains don't always produce the best solution, they often come up with activations the builder wouldn't have thought of. I've seen my microprocessor controlled robot turn in place in a radius about the size of its body, quite a feat if you've ever tried to steer one of these things manually. That happened with the complex Heiserman-algorithm brain (that had a problem of having too much personality). To me anyway, a robot is more interesting when it learns to do stuff on its own, displaying what I term intelligence. No requirement whatsoever that it meet what a human considers "intelligent" - all it has to do is learn to do its own thing without specifically telling it what to do.
A short primer on influencing Nervous Nets
Controlling a nervous network is easy, just connect a high-value resistance to the neuron's trigger input, paralleling the existing resistor. Applying small amounts of positive bias can also be done, but the results are not as useful since more than a little stops the net. The resistors can be applied using tri-state microprocessor pins, FETs or analog switches, diodes arranged to only conduct if the activation out is low, however you wish to switch it. But whatever you use should have an extremely high "off" resistance, hundreds of megohms, or you'll have drift problems. Fortunately most tri-states and diodes are that way and work fine. That diode might as well not be there when the cathode is more positive than the anode. The resistors leading to the nervous network should be sized to influence the network but not slam the motors against the stops (if present, if not good luck). Values from 2M to 22M seem useful, experiment.
Microprocessors make it easy to simulate almost any size resistor from one value, oscillators and switches can be used to do the same thing. The principle is easy... a 1 meg resistor that's connected only 25% of the time has the same effect as a 4 meg resistor, provided the pulse rate is substantially larger than the time over which the effect is averaged. For neurons firing about every quarter of a second, pulse rates over 40 or so a second should be ok.
Brains for Nervous Net Walkers
What should a walker brain do? Fortunately, it doesn't have to much, just enable the robot to better survive and navigate its environment. Two motor walkers using the microcore architecture can't really steer well enough to have the kind of lofty goals often proposed by robotologists... map the surroundings, be able to navigate to any arbitrary point, taking into account anything that has changed since the last time... forget it, it ain't gonna happen at this level. That's not what they do (that clever mapping robot would probably have a hard time navigating across a cluttered bench, a feat the microcore accomplishes with minimum fuss). A more practical approach to nervous network walker intelligence is "am I touching anything? if I am then I shouldn't be, do something, whatever it takes, to not be touching". More complex versions of this concept make use of memory - when the robot touches (or sees) something it looks up in memory to see what it should do. If whatever it remembered didn't work (still touching) it modifies that memory and tries again. The solution remains in memory when it gets away from whatever, to be used again when the same situation presents itself in the future.
Traditionally, stored program computers in the form of microcontroller chips have been used to increase the intelligence of autonomous robots, this works as long as the goals of the computer do not expect more than the locomotion platform can deliver, or more information than the environment has to offer. Old algorithms that practically predate neural nets do wonders for wheeled robots or other repeatable systems, the same algorithms don't fair as well when applied to a microcore, simply because it has a mind of its own and a controller must be patient with it to achieve results. Once I devised my own really simple algorithm that was more compatible, I got better results. Not great, but about what I'd expect given the input/output restraints of the design. Refer to the brains section of the robot2 walker page for more details.
Lately I've been mind-exploring alternative paths to simulated intelligence for nervous net walkers. This is all new territory and has not been tested. Consider everything to follow as suspect, but hopefully it will click some ideas around here and out there, comments welcome. (wtnewton@nc5.infi.net)
Random direction when a feeler touches...
from feelers
.-- 4.7M --.
O---|<|---*-- 4.7m -- + _|_ |
| e| .- 10K -*----------.
O---|<|---*-- 4.7m -|< pnp | _|_ |
| | | -.- 2.2u |
O---|<|---' .- 100K --*--|>|--' _|_ |
_|_ | ______ | to
| _______ | |-' microcore
`-----| |---| 4066 |--|<|-- R --->
| 4 bit |---|analog|--|<|-- R --->
|counter|---|switch|--|<|-- R --->
|_______|---|______|--|<|-- R --->
Every time any feeler closes the counter increments and the analog switch connects the four bits to the microcore, it bumps and increments the counter until it finally hits something that works which remains connected for a few seconds then removed. I seem to have misplaced my handy radio shack circuit book, I'm pretty sure the counter can be only one chip. Not bad... if it works.
A variation of this method would be instead of treating all feelers alike, duplicate the circuit three times, one per feeler and apply separately - even better don't count the first feeler pulse (gate it out until at least one more pulse arrives over a given period), that way the counters will act as memories of what to do for each feeler. Got to find that circuit book...
Using another Nervous Net for control...
Another idea is to use another nervous network running in parallel to the microcore, the feelers introduce and modify pulses in this control network which then tugs on the microcore to induce effects. This might be the best way, certainly more fitting with beam-technology, but more thought will be needed to implement. Here's a thought... use an eight node nervous net running in the opposite direction, each delay twice as fast so that it approximently matches the speed of the microcore. An interesting thing happens as the pulses circulate in opposite directions... they always meet again at the same spot. Such a network, if perfectly synced, could theoretically indefinitely hold a given activation pattern (i.e a combo of resistances applied to the microcore trigger inputs), but perfect syncronisity probably isn't desired, the shifting phase of the control signals is bound to produce interesting behavior. Something like...
.-------------------------------.
| .-------|----- Nu
`--- 4 <-- 3 <-- 2 <-- 1 <------|-.
| |
.--> 1 --------> 2 ---. | |
| | | | | | Feelers activate to Nu's
`----|-----------|----|--. | | pairs of outer loop are
M M | | | | connected to microcore
.----|-----------|----' | | | through diodes and
| | | | | | high-value resistors
`--> 3 --------> 4 ------' | |
| |
.--- 8 <-- 7 <-- 6 <-- 5 <------' |
| `---------|--- Nu
`---------------------------------'
Big hmmm... that might be the ticket.
At least for short term memory... perhaps there should be another net encircling the control net to act as a longer term memory. Chaos soon takes over, it's difficult to predict just what will happen with such arrangements once they extend beyond a couple of layers. To really make things interesting, couple the outs between the layers so that they can influence each other's timings, about 4.7K should do something. What, I'm not sure...
The following animation from Richard Diaz demonstrates the concept nicely:
With this particular phasing, it can be seen that the pulse is impacts the top two neurons, each at 50% of maximum. What this does not show is how the change of speed will cause the two loops to lose sync, thus activating different inner neurons. An idea to help to not lose sync... Wire the microcore neurons to control the outer neurons by an equivalent amount. The outputs of two outer neurons wire through resistors and diodes to the input of one inner neuron, and the output of the inner neuron wires to the two control neurons, also through diodes and resistors. Each connection would be something like:
neuron output >------ R -----|<|------> neuron input
R would be something in the 10 meg range. The diode only allows control when the controlling output is low, paralleling resistance thus speeding up the controlled neuron's propagation delay. The trick is to get the neuron under control to then alter the timing of the outer loop by the same amount to keep things in phase.
And then you get the analog equivalent to a byte's worth of memory. I can see this won't be easy...
Bit-net associative memory in hardware...
This design is in the process of evolving, hopefully it will result in a simple circuit that is capable of learning how to map a pair of feelers to the microcore in a way that minimises excessive feeler activity. In other words learn to respond to the feelers in a way that navigates the robot away from obstacles. This is a hardware expression of a simple Bit-Net, a simple neural network architecture I devised because traditional neural nets were too hard to program onto a simple microcontroller that knows no floating point. Rather, the bit-net simulates a control network where either a connection exists or it doesn't, no multi-valued "weights". Each input to the network triggers a memory for that input. Although such networks cannot solve problems involving multiple inputs, they work fine for simple stuff like determining and remembering which microcore nodes to tug when a feeler touches something.
Here is the circuit configuration...
to microcore .--||--100--*---|>|----*--100K-- + /
input nodes _|_ 33u *--100K- + | / feeler
.--|<|--* 3 ____ *------------O O-.
a <--Re--------*-|>|-------*--*--|<|--|--| | | 2.2u |
| | .--|<|--* 4| | *---100---||------*
b <--Re------*-|-|>|-----*-|--*--|<|--|--| | | |
| | | | .--|<|--* 5| |14`-|<|-1.5M-*--10M-|-*- +
c <--Re----*-|-|-|>|---*-|-|--*--|<|--|--| |--- + __ 1 *-|>|--|-'
| | | | | | .--|<|--' 6| |1 3 / /--*--||--*
d <--Re--*-|-|-|-|>|-*-|-|-|--*--|<|-----|____|---*-o{ ( 2 33u_|_
| | | | R R R R _|_2 | \__\---*-----------.
| | | | _|_|_|_|_ |250p | 5 __ |
Re=1.5M, | | | | `-||-*-15K-' .--\ \ 4 |
R=1M in | | | | gates - 4001 diodes - 1N914 | |6 ) }o-'
proto | | | | dual counter - 74HCT393 `--15K -*--/__/
| | | | 14`--- +
Re=0, | | | | .--||--100--*---|>|----*--100K-- + /
R=4.7M | | | | _|_ 33u *--100K- + | / feeler
would be | | | | .--|<|--*11 ____ *------------O O-.
better | | | `-|>|-------*--*--|<|--|--| | | 2.2u |
| | | | .--|<|--*10| | *---100---||------*
| | `---|>|-----*-|--*--|<|--|--| | | |
| | | | .--|<|--* 9| | `-|<|-1.5M-*--10M-|-*- +
| `-----|>|---*-|-|--*--|<|--|--| | __ 13*-|>|--|-'
| | | | .--|<|--' 8| |13 11/ /--*--||--*
`-------|>|-*-|-|-|--*--|<|-----|____|---*-o{ ( 12 33u_|_
R R R R 12_|__|_7 | \__\---*-----------.
_|_|_|_|_ |250p | 9__ |
`-||-*-15K-' .--\ \10 |
June 9 1998 WTN | | 8 ) }o-'
`--15K -*--/__/
_|_7
The circuit consists of two binary counters that are wired using diodes to apply "tug" signals to the microcore whenever associated feelers are activated. Capacitors stretch the duration of the feeler signals to allow enough time for the output to take effect. When a counter out and the feeler are both low, a resistor (4.7M) is applied through a diode to influence the nervous net neuron. Thus, the counters serve as memories, storing a pattern to be applied when its feeler goes low from touching something. To achieve learning, 4001 quad nor gates are wired as a pair of oscillators that only pulse whenever their feelers touch excessively, like when it's doing something wrong. Additional capacitors are arranged so each feeler touch pulls a little more juice out of it, when it goes low enough to be read as logic 0 the oscillator fires and dials up a random memory for the feeler, which remains in effect until the next time the feeler experiences too much activity.
There is a potential problem with the design... if the oscillator triggers the same number of pulses every time, learning is restricted. The correct answer might not come until after many tries, if at all. One solution is to use a quantum noise source instead of a fast oscillator, however most such circuits have a hard time working at 3.6 volts because they amplify the noise of a "frying" transistor, takes about 6 volts at least. I am amazed at how difficult it is to contruct a simple low power rail-rail noise circuit, must be a cruel joke of electronics that noise appears only when you don't want it. A more practical solution is to make use of the environment to introduce randomness, either in the oscillator period or the time the counter clock is applied. If the clock is very fast compared to the learning signal duration, then phase noise and temperature drift in the oscillator or variations in the duration of the learning signal (influenced by the environment) should introduce sufficient randomness in the selections to ensure the correct solution will eventually be found.
I'm using a HCT part for the counter (R.S.), despite being rated for 4.5 volts minimum, it works fine at 3. A 74HC393 would probably be better but I don't think it matters unless running under 3 volts. To avoid confusing myself (and because this is the only schematic of this), parts values reflect what's installed even if it's wrong, I'll update it when I aquire correct values.
For what it is I'm impressed, but the 2x4 memory space is limiting. I miss the photocell inputs of the programmed version which provided continuous stimulation and gave the robot much of its "personality". While the circuit works for feeler interpretation, when the feelers are not touching anything it's boring. And... the most difficult part of construction was the diode array - I think a pair of 4066s would have been much less wiring and space but what the heck, it's just a technology demonstrator before blowing bucks on something big. Once I get the basics ironed out I'll simplify and expand, possibly to a 4x4 array.
A thought... the programmed bitnet was able to vary the duration and strength of the applied control signal, a very handy thing from the 'bot's point of view. If a short kick doesn't work, try a longer or stronger one. If control were restricted to two lines rather than four then half of the diodes (or whatever used to gate) would go away and the extra outputs could vary the effect strength and duration. Or more practically, just duration and or/gate the extra bits for a backup signal. This configuration would probably work better on a linear drive robot that can actually steer (like a photopopper) but if the walker can be tuned to adequately steer (move/thrash or whatever walkers do) then the extra bits sure could be put to use elsewhere. From what I've seen out of my walking platforms, it takes all four lines to do anything useful. Perhaps I need to make a better walker, one that can steer.
Another idea for an associative memory...
2-25-99... A hard-wired learning "brain" I've been thinking about but haven't fully designed yet - use a sram (2114 etc) and a counter fed by an oscillator (to serve as a random move maker). RL works by keeping the moves that work and randomising memories that don't so no need to store specific memories, just whatever it lands on. The read/write line of the ram and the random move enable (4066 whatever) tie together and become the learn input, pulse and the memory changes. When to pulse determines the behavior. Something like...
________
Left feeler ------|a d|------------*-----|<|---4.7M---
Left light ------|a d|----------*-|-----|<|---4.7M--- to
Center light ------|a 2114 d|--------*-|-|-----|<|---4.7M--- microcore
Right light ------|a d|------*-|-|-|-----|<|---4.7M--- inputs
Right feeler ------|a______w|o-. _|_|_|_|_
| | 4066 | or counter w/ oe
slow osc + -100K--* |_________|---------------------.
__ .1u | _|_|_|_|_ fast osc |
.---| \ .--|>o---*-||--* | counter | __ |
| | )o-* _|_ | |_________|-*---o<|--. / |---*
| .-|__/ | -.-2.2u| _|_ *-o( | |
| | `--220K--* | 100p -.- | \__|-. |
| `-----------------' | *---100K-' | |
`-------------------. | `-----------------' |
| `--|>o----------------------------'
| (bigtime guessing on values)
learn signal
low=good move, high=bad move
The light inputs assume a bit of processing, as worded only one light input is active at a time, telling the robot where the light is no matter what the average level is, op-amps come to mind. Denser encoding can be used, but no shortage of input (address) lines, ground unused ones or use them to change moods. The logic needed to define when learning happens isn't shown, diodes, resistors, caps and gates translate inputs you want to avoid into the learn signal. Only four data lines (all a 2114 has), reverse is assumed to be hard-wired.
To encourage it to learn to avoid feeler contact, make the learn signal high if either feeler touches more than briefly. Triggering on every feeler touch is too much, trigger if the feeler touches more than 5 seconds is more like it. To make phototropic, learn if the middle signal is not on (or whatever indicates it's not going in the right direction, a photocell in the back might work well for this). Of course the walker has to be able to steer somewhat to make use of directional conditions.
This is just an idea, haven't tried any of this. 2114 is probably a bad choice for low-power applications, a 6116 might be better and give more outs (like reverse). Something else besides a 4066 could gate the random move, an 8 bit bus chip with both 4 bit counter sections might be more practical. All is just for illustration. This idea is not just for walkers, I originally envisioned it for driving two motors with a '245, a setup reverses fine with only 4 bits.