First published 2 July 2012
On the Emergent Behaviors of a Robot Controlled by a Real-Time Evolving Neural Network
Walter O. Krawec
In this paper we apply a real-time evolving neural network which uses a hill-climbing algorithm capable of adapting not only a network's synaptic weights but also its topology (creating a recurrent neural network). We then apply this network to a robot in a simulated environment. By equipping the robot with a minimal set of instincts and a short-term memory system (to facilitate reinforcement learning), we observe that several strategies developed which pass the emergent behavior test of (Ronald et al., 1999). In particular, we see robots learning behaviors that are not rewarded by the environment.
Of course a hill-climbing algorithm is more likely than a genetic-algorithm to get stuck at a local optimum, we argue that, despite this, the method described here has several unique advantages. In particular, it allows us to create a single persistent robot that slowly learns and "grows up" as described in (Ross et al., 2003). With our system, it is an individual that learns not a population of individuals, and our learning is continual (e.g. there is no need to reset the robot to some starting position to evaluate the fitness of a particular network).
We conclude with several future problems and applications. For instance, we describe a simple mechanism allowing a network to be copied to embedded hardware whenever a network connection is available to a PC (which is responsible for the memory and time intensive task of evolving the network). This mechanism does not require a continual link to a PC. We also discuss the possibility of creating a distributed evolving neural network system.