Learning robot behaviours by observing and envisaging

Abstract

In this thesis, we apply machine learning to the problem of controlling mobile robots in difficult, complex terrain. The motivation for this research is the desire to have an autonomous rescue robot that is able to overcome rubble, kerbs and other obstacles and perform a task such as finding survivors.

Traditionally, this control problem has been solved by deriving control equations from mathematical models that encapsulate the interactions between the robot and the terrain. As the terrain becomes increasingly complex, these models become intractably difficult to construct.

We have developed three control agents for a mobile rescue robot. They observe the terrain through the robot’s on-board sensors and use machine learned models to decide on actions to take to achieve a goal. These models encapsulate information that is automatically extracted from the performance of a demonstrator. We use a human expert and an autonomous demonstrator based on a trial-and-error forward search in simulation.

The first agent uses a simple Situation-Action formulation and is related to Behavioural Cloning. It directly learns a model of the demonstrator’s behaviour. The second agent generalises this by learning from the demonstrator’s successes and failures. It learns a model for the desirability of each action. The third agent learns a probabilistic motion model that is used in a reinforcement learning style short-range planner.

We have evaluated the control agents on unseen terrain in simulation and reality. This includes evaluating, on the real robot, agents that were trained purely in simulation. We have demonstrated that we can train control agents that approach human levels of performance.

We conclude that it is possible to learn control agents for controlling mobile robots in these complex environments and that even a simple Situation-Action formulation can perform well on this task. The contributions of this thesis include the application of machine learning to a robot control task that cannot be easily modelled, an investigation of feature extractors for modelling complex terrain, a survey of learning techniques for modelling the decisions to be made in this domain and an autonomous search-based demonstrator from which we learn viable behaviours for the real robot.