Scaffolding for hierarchical interaction learning of invertible processes

This project studies “mediated interaction” learning, based on reinforcement learning methods. Mediated interaction defines a class of multi-object interactions where the desired effects cannot be created through direct interaction but by exploiting one object as a tool that interacts with a target object. This structure is seen in many “interesting” real world scenarios, such as moving a cup by moving its plate, or using a stick to move a distant object, but is rarely studied in the area of machine learning because of its high complexity.

In this context, one of the main questions is how to speed up the learning process of this kind of tasks by utilizing the potential of an important general principle that shapes human learning in powerful ways. This principle has been named “scaffolding” in psychology. It structures learning by a sequence of tasks with increasing difficulty, while aligning each step to the current capabilities of the learner.

Even though the goal of this project is to widen the applicability of robots to a new class of problems, simulations are the most promising way to get useful results and a clear outline of this research area.

In the first phase of this project, a suitable physics-based simulation environment is created, together with the built up of the extensible reinforcement learning architecture. Additionally, reasonable object-interaction scenarios had to be designed, where the learning of the agent can be evaluated. In a second phase, first studies are planned and conducted to investigate the influence of embodiment and attention control on the learning process.

Visualization of an “extension of reach” scenario within the designed simulation world. In this scenario, the agent (green arrow) has to learn how to navigate the target object (blue disc) into the goal area (red circle). In order to solve the task, the orange L-shaped object has to be exploited as a tool to fetch the target object that is placed outside the border of the agent's interaction range (black circle).

Related People

Fleer, SaschaContact

Ritter, HelgeContact