The visual detection and recognition of human actions by technical systems is a fundamental problem with many applications the human-computer interaction domain. Activities involving hand-object interactions and action sequences in goal-oriented tasks, such as manufacturing work, pose a particular challenge. We use deep learning to detect and recognize such actions in real-time, and we propose an assistance system that provides the user with feedback and guidance based on the recognized actions.
Because of the complex anatomy of the human hand, in the absence of external constraints a large number of postures and force combinations can be used to attain a stable grasp.
This 12-taxel fingertip sensor provides highly sensitive normal-force sensing for our Shadow Robot Hands - fully integrated with the mechanical and electrical design of the hand.
“Scaffolding” is a powerful principle taken from psychology for organizing and optimizing the progress of a learner. The goal in this project is to combine this powerful structuring of a learning process with recent advances in deep and hierarchical reinforcement learning in order to make learning of an artificial agent more efficient and feasible when interaction data is much more limited than considered in current studies.
Despite decades of research, intuitive and robust control of multi-joint prosthetic hands is still an unsolved problem, largely due to missing sensorization of the hand and a poor human-machine interface, that only barely can recognize the intent of the patient. In this project we will employ and evaluate a new generation of tactile sensors coupled with modern machine learning approaches to overcome both problems.
Within the EU-Project WearHap, we develop different wearable devices, most notably a tactile-sensing glove, based on our elastic, fabric-based tactile-sensor-technology. The Tactile Glove provides normal force/pressure measurement in approx 60 taxels spread across the whole palmar side of the hand and allows to record human interaction forces while grasping or manipulating objects.
The visual detection and recognition of human actions by technical systems is a fundamental problem with many applications the human-computer interaction domain. Activities involving hand-object interactions and action sequences in goal-oriented tasks, such as manufacturing work, pose a particular challenge. We use deep learning to detect and recognize such actions in real-time, and we propose an assistance system that provides the user with feedback and guidance based on the recognized actions.
Because of the complex anatomy of the human hand, in the absence of external constraints a large number of postures and force combinations can be used to attain a stable grasp.
This 12-taxel fingertip sensor provides highly sensitive normal-force sensing for our Shadow Robot Hands - fully integrated with the mechanical and electrical design of the hand.
Despite decades of research, intuitive and robust control of multi-joint prosthetic hands is still an unsolved problem, largely due to missing sensorization of the hand and a poor human-machine interface, that only barely can recognize the intent of the patient. In this project we will employ and evaluate a new generation of tactile sensors coupled with modern machine learning approaches to overcome both problems.
Within the EU-Project