2017

A System for Learning Continuous Human-Robot Interactions from Human-Human Demonstrations

David Vogt, Simon Stepputtis, Steve Grehl, Bernhard Jung, Heni Ben Amor
International Conference on Robotics and Automation 2017
Preliminary Version | Paper

View Abstract

We present a data-driven imitation learning system for learning human-robot interactions from human-human demonstrations. During training, the movements of two interaction partners are recorded through motion capture and an interaction model is learned. At runtime, the interaction model is used to continuously adapt the robot’s motion, both spatially and temporally, to the movements of the human interaction partner. We show the effectiveness of the approach on complex, sequential tasks by presenting two applications involving collaborative human-robot assembly. Experiments with varied object hand-over positions and task execution speeds confirm the capabilities for spatio-temporal adaption of the demonstrated behavior to the current situation.

2016

Traffic Light Status Detection Using Movement Patterns of Vehicles

Campbell, J; Ben Amor, H.; Ang, M.; Fainekos, G.
International Conference on Intelligent Transportation Systems 2016
Paper

View Abstract

Vision-based methods for detecting the status of traffic lights used in autonomous vehicles may be unreliable due to occluded views, poor lighting conditions, or a dependence on unavailable high-precision meta-data, which is troublesome in such a safety-critical application. This paper proposes a complementary detection approach based on an entirely new source of information: the movement patterns of other nearby vehicles. This approach is robust to traditional ources of error, and may serve as a viable supplemental detection method. Several different classification models are presented for inferring traffic light status based on these patterns. Their performance is evaluated over real and simulated data sets, resulting in up to 97% accuracy in each set.

Projecting Robot Intentions into Human Environments

Andersen, R.; Madsen, O.; Moeslund, B; Ben Amor, H.
International Symposium on Robot and Human Interactive Communication 2016
Paper

View Abstract

Trained human co-workers can often easily predict each other’s intentions based on prior experience. When collaborating with a robot coworker, however, intentions are hard or impossible to infer. This difficulty of mental introspection makes human-robot collaboration challenging and can lead to dangerous misunderstandings. In this paper, we present a novel, object-aware projection technique that allows robots to visualize task information and intentions on physical objects in the environment. The approach uses modern object tracking methods in order to display information at specific spatial locations taking into account the pose and shape of surrounding objects. As a result, a human co-worker can be informed in a timely manner about the safety of the workspace, the site of next robot manipulation tasks, and next subtasks to perform. A preliminary usability study compares the approach to collaboration approaches based on monitors and printed text. The study indicates that, on average, the user effectiveness and satisfaction is higher with the projection based approach.

Estimating Perturbations from Experience using Neural Networks and Information Transfer

Berger, E.; Vogt, D.; Grehl, S.l; Jung, B.; Ben Amor, H
International Conference on Robotics and Automation 2016
Paper

View Abstract

In order to ensure safe operation, robots must be able to reliably detect behavior perturbations that result from unexpected physical interactions with their environment and human co-workers. While some robots provide firmware force sensors that generate rough force estimates, more accurate force measurements are usually achieved with dedicated force-torque sensors. However, such sensors are often heavy, expensive and require an additional power supply. In the case of lightweight manipulators, the already limited payload capabilities may be reduced in a significant way. This paper presents an experience-based approach for accurately estimating external forces being applied to a robot without the need for a force-torque sensor. Using Information Transfer, a subset of sensors relevant to the executed behavior are identified from a larger set of internal sensors. Models mapping robot sensor data to force-torque measurements are learned using a neural network. These models can be used to predict the magnitude and direction of perturbations from affordable, proprioceptive sensors only. Experiments with a UR5 robot show that our method yields force estimates with accuracy comparable to a dedicated force-torque sensor. Moreover, our method yields a substantial improvement in accuracy over force-torque values provided by the robot firmware.

Directing Policy Search with Interactively Taught Via-Point

Schroecker, Y; Ben Amor, H., Thomaz, A.
International Conference on Autonomous Agents and Multiagent Systems 2016
Paper

View Abstract

Policy search has been successfully applied to robot motor learning problems. However, for moderately complex tasks the necessity of good heuristics or initialization still arises. One method that has been used to alleviate this problem is to utilize demonstrations obtained by a human teacher as a starting point for policy search in the space of trajectories. In this paper we describe an alternative way of giving demonstrations as soft via-points and show how they can be used for initialization as well as for active corrections during the learning process. With this approach, we restrict the search space to trajectories that will be close to the taught via-points at the taught time and thereby significantly reduce the number of samples necessary to learn a good policy. We show with a simulated robot arm that our method can efficiently learn to insert an object in a hole with just a minimal demonstration and evaluate our method further on a synthetic letter reproduction task.

Experience-based Torque Estimation for an Industrial Robot

Berger, E.; Grehl, S.; Vogt, D.; Jung, B.; Ben Amor, H.
International Conference on Robotics and Automation 2016
Paper

View Abstract

Robotic manipulation tasks often require the control of forces and torques exerted on external objects. This paper presents a machine learning approach for estimating forces when no force sensors are present on the robot platform. In the training phase, the robot executes the desired manipulation tasks under controlled conditions with systematically varied parameter sets. All internal sensor data, in the presented case from more than 100 sensors, as well as the force exerted by the robot are recorded. Using Transfer Entropy, a statistical model is learned that identifies the subset of sensors relevant for torque estimation in the given task. At runtime, the model is used to accurately estimate the torques exerted during manipulations of the demonstrated kind. The feasibility of the approach is shown in a setting where a robotic manipulator operates a torque wrench to fasten a screw nut. Torque estimates with an accuracy of well below ±1Nm are achieved. A strength of the presented model is that no prior knowledge of the robot’s kinematics, mass distribution or sensor instrumentation is required.

Sparse Latent Space Policy Search,

Luck, K.S.; Pajarinen, J.; Erik Berger, E.; Kyrki, V. ; Ben Amor, H.
Conference on Artificial Intelligence 2016
Paper | Website | Code

View Abstract

Computational agents often need to learn policies that involve many control variables, e.g., a robot needs to control several joints simultaneously. Learning a policy with a high number of parameters, however, usually requires a large number of training samples. We introduce a reinforcement learning method for sampleefficient policy search that exploits correlations between control variables. Such correlations are particularly frequent in motor skill learning tasks. The introduced method uses Variational Inference to estimate policy parameters, while at the same time uncovering a low-dimensional latent space of controls. Prior knowledge about the task and the structure of the learning agent can be provided by specifying groups of potentially correlated parameters. This information is then used to impose sparsity constraints on the mapping between the high-dimensional space of controls and a lowerdimensional latent space. In experiments with a simulated bi-manual manipulator, the new approach effectively identifies synergies between joints, performs efficient low-dimensional policy search, and outperforms state-of-the-art policy search methods.

2015

Estimation of Perturbations in Robot Behaviors using Dynamic Mode Decomposition

Berger, E.; Müller., D.; Vogt, D.; Jung, B.; Ben Amor, H.
Advanced Robotics, Robotics Society of Japan 2015
Paper | Video

View Abstract

Physical human–robot interaction tasks require robots that can detect and react to external perturbations caused by the human partner. In this contribution, we present a machine learning approach for detecting, estimating, and compensating for such external perturbations using only input from standard sensors. This machine learning approach makes use of Dynamic Mode Decomposition (DMD), a data processing technique developed in the field of fluid dynamics, which is applied to robotics for the first time. DMD is able to isolate the dynamics of a nonlinear system and is therefore well suited for separating noise from regular oscillations in sensor readings during cyclic robot movements. In a training phase, a DMD model for behavior-specific parameter configurations is learned. During task execution, the robot must estimate the external forces exerted by a human interaction partner. We compare the DMD-based approach to other interpolation schemes. A variant, sparsity promoting DMD, is particularly well suited for high-noise sensors. Results of a user study show that our DMD-based machine learning approach can be used to design physical human–robot interaction techniques that not only result in robust robot behavior but also enjoy a high usability.

Occlusion Aware Object Localization, Segmentation and Pose Estimation

Brahmbhatt, S.; Ben Amor, H., Christensen, H.
British Machine Vision Conference 2015
Paper

View Abstract

We present a learning approach for localization and segmentation of objects in an image in a manner that is robust to partial occlusion. Our algorithm produces a bounding box around the full extent of the object and labels pixels in the interior that belong to the object. Like existing segmentation aware detection approaches, we learn an appearance model of the object and consider regions that do not fit this model as potential occlusions. However, in addition to the established use of pairwise potentials for encouraging local consistency, we use higher order potentials which capture information at the level of im- age segments. We also propose an efficient loss function that targets both localization and segmentation performance. Our algorithm achieves 13.52% segmentation error and 0.81 area under the false-positive per image vs. recall curve on average over the challenging CMU Kitchen Occlusion Dataset. This is a 42.44% decrease in segmentation error and a 16.13% increase in localization performance compared to the state-of-the-art. Finally, we show that the visibility labelling produced by our algorithm can make full 3D pose estimation from a single image robust to occlusion.

A Taxonomy of Benchmark Tasks for Bimanual Manipulators

Quispe, A. H.; Ben Amor, H.; Henrik Christensen, H.
International Symposium on Robotics Research 2015

Exploiting Symmetries and Extrusions for Grasping Household Objects

Quispe, A. H.; Milville, B.; Gutierrez, M.; Erdogan, C.; Stilman, M; Christensen, H.; Ben Amor, H.
International Conference on Robotics and Automation 2015
Paper

View Abstract

In this paper we present an approach for creating complete shape representations from a single depth image for robot grasping. We introduce algorithms for completing partial point clouds based on the analysis of symmetry and extrusion patterns in observed shapes. Identified patterns are used to generate a complete mesh of the object, which is, in turn, used for grasp planning. The approach allows robots to predict the shape of objects and include invisible regions into the grasp planning step. We show that the identification of shape patterns, such as extrusions, can be used for fast generation and optimization of grasps. Finally, we present experiments performed with our humanoid robot executing pick-up tasks based on single depth images and discuss the applications and shortcomings of our approach.

Learning Multiple Collaborative Tasks with a Mixture of Interaction Primitives

Ewerton, M.; Neumann, G.; Lioutikov, R.; Ben Amor, H.; Peters, J.; Maeda, G.
International Conference on Robotics and Automation 2015
Paper

View Abstract

Robots that interact with humans must learn to not only adapt to different human partners but also to new interactions. Such a form of learning can be achieved by demonstrations and imitation. A recently introduced method to learn interactions from demonstrations is the framework of Interaction Primitives. While this framework is limited to represent and generalize a single interaction pattern, in practice, interactions between a human and a robot can consist of many different patterns. To overcome this limitation this paper proposes a Mixture of Interaction Primitives to learn multiple interaction patterns from unlabeled demonstrations. Specifically the proposed method uses Gaussian Mixture Models of Interaction Primitives to model nonlinear correlations between the movements of the different agents. We validate our algorithm with two experiments involving interactive tasks between a human and a lightweight robotic arm. In the first, we compare our proposed method with conventional Interaction Primitives in a toy problem scenario where the robot and the human are not linearly correlated. In the second, we present a proof-of-concept experiment where the robot assists a human in assembling a box.

2014

Special issue on autonomous grasping and manipulation

Ben Amor, H.; Saxena, A.; Hudson, N.; Peters, J.
Autonomous Robots Journal
Paper

View Abstract

Grasping and manipulation of objects are essential motor skills for robots to interact with their environment and perform meaningful, physical tasks. Since the dawn of robotics, grasping and manipulation have formed a core research field with a large number of dedicated publications. The field has reached an important milestone in recent years as various robots can now reliably perform basic grasps on unknown objects. However, these robots are still far from being capable of human-level manipulation skills including in-hand or bimanual manipulation of objects, interactions with non-rigid objects, and multi-object tasks such as stacking and tool-usage. Progress on such advanced manipulation skills is slowed down by requiring a successful combination of a multitude of different methods and technologies, e.g., robust vision, tactile feedback, grasp stability analysis, modeling of uncertainty, learning, long-term planning, and much more. In order to address these difficult issues, there have been an increasing number of governmental research programs such as the European projects DEXMART, GeRT and GRASP, and the American DARPA Autonomous Robotic Manipulation (ARM) project. This increased interest has become apparent in several international workshops at important robotics conferences, such as the well-attended workshop “Beyond Robot Grasping” at IROS 2012 in Portugal. Hence, this special issue of the Autonomous Robots journal aims at presenting important recent success stories in the development of advanced robot grasping and manipulation abilities. The issue covers a wide range of different papers that are representative of the current state-of-the-art within the field. Papers were solicited with an open call that was circulated in the 4 months preceding the deadline. As a result, we have received 37 submissions to the special issue which were rigorously reviewed by up to four reviewers as well as by at least one of the guest editors. Altogether twelve papers were selected for publication in this special issue. We are in particular happy to include four papers which detail the approach and goal of the DARPA ARM project as well as detailed descriptions of the developed methods.

Interaction Primitives for Human-Robot Cooperation Tasks

Ben Amor, H.; Neumann, G.; Kamthe, S.; Kroemer, O.; Peters, J.
International Conference on Robotics and Automation 2014
Paper

View Abstract

To engage in cooperative activities with human partners, robots have to possess basic interactive abilities and skills. However, programming such interactive skills is a challenging task, as each interaction partner can have different timing or an alternative way of executing movements. In this paper, we propose to learn interaction skills by observing how two humans engage in a similar task. To this end, we introduce a new representation called Interaction Primitives. Interaction primitives build on the framework of dynamic motor primitives (DMPs) by maintaining a distribution over the parameters of the DMP. With this distribution, we can learn the inherent correlations of cooperative activities which allow us to infer the behavior of the partner and to participate in the cooperation. We will provide algorithms for synchronizing and adapting the behavior of humans and robots during joint physical activities.

Transfer Entropy for Feature Extraction in Physical Human-Robot Interaction: Detecting Perturbations from Low-Cost Sensors

Berger, E.; Müller., D.; Vogt, D.; Jung, B.; Ben Amor, H.
International Conference on Humanoid Robots 2014
Paper

View Abstract

In physical human-robot interaction, robot behavior must be adjusted to forces applied by the human interaction partner. For measuring such forces, special-purpose sensors may be used, e.g. force-torque sensors, that are however often heavy, expensive and prone to noise. In contrast, we propose a machine learning approach for measuring external perturbations of robot behavior that uses commonly available, low-cost sensors only. During the training phase, behavior-specific statistical models of sensor measurements, so-called perturbation filters, are constructed using Principal Component Analysis, Transfer Entropy and Dynamic Mode Decomposition. During behavior execution, perturbation filters compare measured and predicted sensor values for estimating the amount and direction of forces applied by the human interaction partner. Such perturbation filters can therefore be regarded as virtual force sensors that produce continuous estimates of external forces.

Learning Interaction for Collaborative Tasks with Probabilistic Movement Primitives

Maeda, G.J.; Ewerton, M.; Lioutikov, R.; Ben Amor, H.; Peters, J.; Neumann, G.
International Conference on Humanoid Robots 2014
Paper

View Abstract

This paper proposes a probabilistic framework based on movement primitives for robots that work in collaboration with a human coworker. Since the human coworker can execute a variety of unforeseen tasks a requirement of our system is that the robot assistant must be able to adapt and learn new skills on-demand, without the need of an expert programmer. Thus, this paper leverages on the framework of imitation learning and its application to human-robot interaction using the concept of Interaction Primitives (IPs). We introduce the use of Probabilistic Movement Primitives (ProMPs) to devise an interaction method that both recognizes the action of a human and generates the appropriate movement primitive of the robot assistant. We evaluate our method on experiments using a lightweight arm interacting with a human partner and also using motion capture trajectories of two humans assembling a box. The advantages of ProMPs in relation to the original formulation for interaction are exposed and compared.

Online Multi-Camera Registration for Bimanual Workspace Trajectories

Dantam, N.; Ben Amor, H; Christensen, H.; Stilman, M.
International Conference on Humanoid Robots 2014
Paper

View Abstract

We demonstrate that millimeter-level bimanual manipulation accuracy can be achieved without the static camera registration typically required for visual servoing. We register multiple cameras online, converging in seconds, by visually tracking features on the robot hands and filtering the result. Then, we compute and track continuous-velocity relative workspace trajectories for the end-effector. We demonstrate the approach using Schunk LWA4 and SDH manipulators and Logitech C920 cameras, showing accurate relative positioning for pen-capping and object hand-off tasks. Our filtering software is available under a permissive license.

Latent Space Policy Search for Robotics

Luck, K.S.; Neumann, G.; Berger, E.; Peters, J.; Ben Amor, H.
International Conference on Intelligent Robots and Systems 2014
Paper

View Abstract


Learning motor skills for robots is a hard task. In particular, a high number of degrees-of-freedom in the robot can pose serious challenges to existing reinforcement learning methods, since it leads to a high-dimensional search space. However, complex robots are often intrinsically redundant systems and, therefore, can be controlled using a latent manifold of much smaller dimensionality. In this paper, we present a novel policy search method that performs efficient reinforcement learning by uncovering the low-dimensional latent space of actuator redundancies. In contrast to previous attempts at combining reinforcement learning and dimensionality reduction, our approach does not perform dimensionality reduction as a preprocessing step but naturally combines it with policy search. Our evaluations show that the new approach outperforms existing algorithms for learning motor skills with high-dimensional robots.

Online Camera Registration for Robot Manipulation

Dantam, N.; Ben Amor, H; Christensen, H.; Stilman, M.
International Symposium on Experimental Robotics 2014
Paper

View Abstract


We demonstrate that millimeter-level manipulation accuracy can be achieved without the static camera registration typically required for visual servoing. We register the camera online, converging in seconds, by visually tracking features on the robot and filtering the result. This online registration handles cases such as perturbed camera positions, wear and tear on camera mounts, and even a camera held by a human. We implement the approach on a Schunk LWA4 manipulator and Logitech C920 camera, servoing to target and pre-grasp configurations. Our filtering software is available under a permissive license.

Dynamic Mode Decomposition for Perturbation Estimation in Human-Robot Interaction

Berger, E.; Sastuba, M.; Vogt, D.; Jung, B.; Ben Amor, H.
International Symposium on Robot and Human Interactive Communication 2014
Paper

View Abstract


In many settings, e.g. physical human-robot interaction, robotic behavior must be made robust against more or less spontaneous application of external forces. Typically, this problem is tackled by means of special purpose force sensors which are, however, not available on many robotic platforms. In contrast, we propose a machine learning approach suitable for more common, although often noisy sensors. This machine learning approach makes use of Dynamic Mode Decomposition (DMD) which is able to extract the dynamics of a nonlinear system. It is therefore well suited to separate noise from regular oscillations in sensor readings during cyclic robot movements under different behavior configurations. We demonstrate the feasibility of our approach with an example where physical forces are exerted on a humanoid robot during walking. In a training phase, a snapshot based DMD model for behavior specific parameter configurations is learned. During task execution the robot must detect and estimate the external forces exerted by a human interaction partner. We compare the DMD-based approach to other interpolation schemes and show that the former outperforms the latter particularly in the presence of sensor noise. We conclude that DMD which has so far been mostly used in other fields of science, particularly fluid mechanics, is also a highly promising method for robotics.

A Data-Driven Method for Real-Time Character Animation in Human-Agent Interaction

Vogt, D.; Grehl, S.; Berger, E.; Ben Amor, H; Jung, B.
International Conference on Intelligent Virtual Agents 2014
Paper

View Abstract


We address the problem of creating believable animations for virtual humans that need to react to the body movements of a human interaction partner in real-time. Our data-driven approach uses prerecorded motion capture data of two interacting persons and performs motion adaptation during the live human-agent interaction. Extending the interaction mesh approach, our main contribution is a new scheme for efficient identification of motions in the prerecorded animation data that are similar to the live interaction. A global low-dimensional posture space serves to select the most similar interaction example, while local, more detail-rich posture spaces are used to identify poses closely matching the human motion. Using the interaction mesh of the selected motion example, an animation can then be synthesized that takes into account both spatial and temporal similarities between the prerecorded and live interactions.

2013

Probabilistic Movement Modeling for Intention Inference in Human-Robot Interaction

Wang, Z.; Muelling, K.; Deisenroth, M. P.; Ben Amor, H.; Vogt, D.; Schoelkopf, B.; Peters, J.
International Journal of Robotics Research, 32, 7, pp.841-858
Paper

View Abstract


Intention inference can be an essential step toward efficient human-robot interaction. For this purpose, we propose the Intention-Driven Dynamics Model (IDDM) to probabilistically model the generative process of movements that are directed by the intention. The IDDM allows to infer the intention from observed movements using Bayes’ theorem. The IDDM simultaneously finds a latent state representation of noisy and high dimensional observations, and models the intention-driven dynamics in the latent states. As most robotics applications are subject to real-time constraints, we develop an efficient online algorithm that allows for real-time intention inference. Two human-robot interaction scenarios, i.e., target prediction for robot table tennis and action recognition for interactive humanoid robots, are used to evaluate the performance of our inference algorithm. In both intention inference tasks, the proposed algorithm achieves substantial improvements over support vector machines and Gaussian processes.

Learning Responsive Robot Behavior by Imitation

Ben Amor, H.; Vogt, D.; Ewerton, M.; Berger, E.; Jung, B.; Peters, J.
International Conference on Intelligent Robots and Systems 2013
Paper

View Abstract


In this paper we present a new approach for learning responsive robot behavior by imitation of human interaction partners. Extending previous work on robot imitation learning, that has so far mostly concentrated on learning from demonstrations by a single actor, we simultaneously record the movements of two humans engaged in on-going interaction tasks and learn compact models of the interaction. Extracted interaction models can thereafter be used by a robot to engage in a similar interaction with a human partner. We present two algorithms for deriving interaction models from motion capture data as well as experimental results on a humanoid robot.

Inferring Guidance Information in Cooperative Human-Robot Tasks

Berger, E.; Vogt, D.; Haji-Ghassemi, N.; Jung, B.; Ben Amor, H.
International Conference on Humanoid Robots 2013
Paper

View Abstract


In many cooperative tasks between a human and a robotic assistant, the human guides the robot by exerting forces, either through direct physical interaction or indirectly via a jointly manipulated object. These physical forces perturb the robot’s behavior execution and need to be compensated for in order to successfully complete such tasks. Typically, this problem is tackled by means of special purpose force sensors which are, however, not available on many robotic platforms. In contrast, we propose a machine learning approach based on sensor data, such as accelerometer and pressure sensor information. In the training phase, a statistical model of behavior execution is learned that combines Gaussian Process Regression with a novel periodic kernel. During behavior execution, predictions from the statistical model are continuously compared with stability parameters derived from current sensor readings. Differences between predicted and measured values exceeding the variance of the statistical model are interpreted as guidance information and used to adapt the robot’s behavior. Several examples of cooperative tasks between a human and a humanoid NAO robot demonstrate the feasibility of our approach.

2012

Mutual Learning and Adaptation in Physical Human-Robot Interaction

Ikemoto, S.; Ben Amor, H. ;Minato, T. ; Ishiguro, H. ; Jung, B.
IEEE Robotics and Automation 2012
Paper

View Abstract


Close physical interaction between robots and humans is a particularly challenging aspect of robot development. For successful interaction and cooperation, the robot must have the ability to adapt its behavior to the human counterpart. Based on our earlier work, we present and evaluate a computationally efficient machine learning algorithm that is well suited for such close-contact interaction scenarios. We show that this algorithm helps to improve the quality of the interaction between a robot and a human caregiver. To this end, we present two human-in-the-loop learning scenarios that are inspired by human parenting behavior, namely, an assisted standing-up task and an assisted walking task.

XSAMPL3D – An Action Description Language for the Animation of Virtual Characters

Vitzthum, A.; Ben Amor, H.; Heumer, G.; Jung, B.
Journal of Virtual Reality and Broadcasting, 9, 1
Paper

View Abstract


In this paper we present XSAMPL3D, a novel language for the high-level representation of actions performed on objects by (virtual) humans. XSAMPL3D was designed to serve as action representation language in an imitation-based approach to character animation: First, a human demonstrates a sequence of object manipulations in an immersive Virtual Reality (VR) environment. From this demonstration, an XSAMPL3D description is automatically derived that represents the actions in terms of high-level action types and involved objects. The XSAMPL3D action description can then be used for the synthesis of animations where virtual humans of different body sizes and proportions reproduce the demonstrated action. Actions are encoded in a compact and human-readable XML-format. Thus, XSAMPL3D describtions are also amenable to manual authoring, e.g. for rapid prototyping of animations when no immersive VR environment is at the animator’s disposal. However, when XSAMPL3D descriptions are derived from VR interactions, they can accomodate many details of the demonstrated action, such as motion trajectiories, hand shapes and other hand-object relations during grasping. Such detail would be hard to specify with manual motion authoring techniques only. Through the inclusion of language features that allow the repre sentation of all relevant aspects of demonstrated object manipulations, XSAMPL3D is a suitable action representation language for the imitation-based approach to character animation.

Probabilistic Modeling of Human Movements for Intention Inference

Wang, Z.;Deisenroth, M; Ben Amor, H.; Vogt, D.; Schoelkopf, B.; Peters, J.
Robotics: Science and Systems
Paper | Video

View Abstract


Inference of human intention may be an essential step towards understanding human actions and is hence important for realizing efficient human-robot interaction. In this paper, we propose the Intention-Driven Dynamics Model (IDDM), a latent variable model for inferring unknown human intentions. We train the model based on observed human movements/actions. We introduce an efficient approximate inference algorithm to infer the human’s intention from an ongoing movement. We verify the feasibility of the IDDM in two scenarios, i.e., target inference in robot table tennis and action recognition for interactive humanoid robots. In both tasks, the IDDM achieves substantial improvements over state-of-the-art regression and classification.

Maximally Informative Interaction Learning for Scene Exploration

van Hoof, H.; Kroemer, O.;Ben Amor, H.; Peters, J.
International Conference on Robot Systems
Paper

View Abstract


Creating robots that can act autonomously in dynamic, unstructured environments is a major challenge. In such environments, learning to recognize and manipulate novel objects is an important capability. A truly autonomous robot acquires knowledge through interaction with its environment without using heuristics or prior information encoding human domain insights. Static images often provide insufficient information for inferring the relevant properties of the objects in a scene. Hence, a robot needs to explore these objects by interacting with them. However, there may be many exploratory actions possible, and a large portion of these actions may be non-informative. To learn quickly and efficiently, a robot must select actions that are expected to have the most informative outcomes. In the proposed bottom-up approach, the robot achieves this goal by quantifying the expected informativeness of its own actions. We use this approach to segment a scene into its constituent objects as a first step in learning the properties and affordances of objects. Evaluations showed that the proposed information-theoretic approach allows a robot to efficiently infer the composite structure of its environment.

Generalization of Human Grasping for Multi-Fingered Robot Hands

Ben Amor, H.; Kroemer, O.; Hillenbrand, U.; Neumann, G.; Peters, J.
International Conference on Robot Systems 2012
Paper | Video

View Abstract


Multi-fingered robot grasping is a challenging problem that is difficult to tackle using hand-coded programs. In this paper we present an imitation learning approach for learning and generalizing grasping skills based on human demonstrations. To this end, we split the task of synthesizing a grasping motion into three parts: (1) learning efficient grasp representations from human demonstrations, (2) warping contact points onto new objects, and (3) optimizing and executing the reach-and-grasp movements. We learn low-dimensional latent grasp spaces for different grasp types, which form the basis for a novel extension to dynamic motor primitives. These latent-space dynamic motor primitives are used to synthesize entire reach-and-grasp movements. We evaluated our method on a real humanoid robot. The results of the experiment demonstrate the robustness and versatility of our approach.

Latent Space Policy Search for Robotics

Kroemer, O.; Ben Amor, H.; Ewerton, M.; Peters, J.
International Conference on Humanoid Robots 2012
Paper

2009

Kinesthetic Bootstrapping: Teaching Motor Skills to Humanoid Robots through Physical Interaction

Ben Amor, H. ; Berger, E. ; Vogt, D. ; Jung, B.
KI 2009: 32nd Annual Conference on Artificial Intelligence
Paper | Video

View Abstract


Programming of complex motor skills for humanoid robots can be a time intensive task, particularly within conventional textual or GUI-driven programming paradigms. Addressing this drawback, we propose a new programming-by-demonstration method called Kinesthetic Bootstrapping for teaching motor skills to humanoid robots by means of intuitive physical interactions. Here, “programming” simply consists of manually moving the robot’s joints so as to demonstrate the skill in mind. The bootstrapping algorithm then generates a low-dimensional model of the demonstrated postures. To find a trajectory through this posture space that corresponds to a robust robot motion, a learning phase takes place in a physics-based virtual environment. The virtual robot’s motion is optimized via a genetic algorithm and the result is transferred back to the physical robot. The method has been successfully applied to the learning of various complex motor skills such as walking and standing up.

Physical Interaction Learning: Behavior Adaptation in Cooperative Human-Robot Tasks Involving Physical Contact

Ikemoto, S.; Ben Amor, H. ; Minato, T.; Ishiguro, H.; Jung, B.
International Symposium on Robot and Human Interactive Communication
CoTeSys Best Paper Award | Paper

View Abstract


In order for humans and robots to engage in direct physical interaction several requirements have to be met. Among others, robots need to be able to adapt their behavior in order to facilitate the interaction with a human partner. This can be achieved using machine learning techniques. However, most machine learning scenarios to-date do not address the question of how learning can be achieved for tightly coupled, physical touch interactions between the learning agent and a human partner. This paper presents an example for such human in-the-loop learning scenarios and proposes a computationally cheap learning algorithm for this purpose. The efficiency of this method is evaluated in an experiment, where human care givers help an android robot to stand up.

Identifying Motion Capture Tracking Markers with Self-Organizing Maps

Weber, M.; Ben Amor, H.; Alexander, T.
IEEE Virtual Reality
Paper

View Abstract


Motion Capture (MoCap) describes methods and technologies for the detection and measurement of human motion in all its intricacies. Most systems use markers to track points on a body. Especially with natural human motion data captured with passive systems (to not hinder the participant) deficiencies like low accuracy of tracked points or even occluded markers might occur. Additionally, such MoCap data is often unlabeled. In consequence, the system does not provide information about which body landmarks the points belong to. Self-organizing neural networks, especially self-organizing maps (SOMs), are capable of dealing with such problems. This work describes a method to model, initialize and train such SOMs to track and label potentially noisy motion capture data.

2008

Grasp Synthesis from Low-Dimensional Probabilistic Grasp Models

Ben Amor, H.; Heumer, G.; Jung, B.; Vitzthum, A.
Journal of Computer Animation and Virtual Worlds, 19
Paper

View Abstract


We propose a novel data-driven animation method for the synthesis of natural looking human grasping. Motion data captured from human grasp actions is used to train a probabilistic model of the human grasp space. This model greatly reduces the high number of degrees of freedom of the human hand to a few dimensions in a continuous grasp space. The low dimensionality of the grasp space in turn allows for efficient optimization when synthesizing grasps for arbitrary objects. The method requires only a short training phase with no need for preprocessing of graphical objects for which grasps are to be synthesized.

Grasp Recognition for Uncalibrated Data Gloves – A Machine Learning Approach, Presence

Heumer, G. ; Ben Amor, H.; Jung, B.
17, MIT Press
Paper

View Abstract


This paper presents a comparison of various machine learning methods applied to the problem of recognizing grasp types involved in object manipulations performed with a data glove. Conventional wisdom holds that data gloves need calibration in order to obtain accurate results. However, calibration is a time-consuming process, inherently user-specific, and its results are often not perfect. In contrast, the present study aims at evaluating recognition methods that do not require prior calibration of the data glove. Instead, raw sensor readings are used as input features that are directly mapped to different categories of hand shapes. An experiment was carried out in which test persons wearing a data glove had to grasp physical objects of different shapes corresponding to the various grasp types of the Schlesinger taxonomy. The collected data was comprehensively analyzed using numerous classification techniques provided in an open-source machine learning toolbox. Evaluated machine learning methods are composed of (a) 38 classifiers including different types of function learners, decision trees, rule-based learners, Bayes nets, and lazy learners; (b) data preprocessing using principal component analysis (PCA) with varying degrees of dimensionality reduction; and (c) five meta-learning algorithms under various configurations where selection of suitable base classifier combinations was informed by the results of the foregoing classifier evaluation. Classification performance was analyzed in six different settings, representing various application scenarios with differing generalization demands. The results of this work are twofold: (1) We show that a reasonably good to highly reliable recognition of grasp types can be achieved— depending on whether or not the glove user is among those training the classifier— even with uncalibrated data gloves. (2) We identify the best performing classification methods for the recognition of various grasp types. To conclude, cumbersome calibration processes before productive usage of data gloves can be spared in many situations.

2007

Grasp Recognition with Uncalibrated Data Gloves – A Comparison of Classification Methods

Heumer, G.; Ben Amor, H.; Weber, M.; Jung, B.
IEEE Virtual Reality, IEEE
Paper

View Abstract


This paper presents a comparison of various classification methods for the problem of recognizing grasp types involved in object manipulations performed with a data glove. Conventional wisdom holds that data gloves need calibration in order to obtain accurate results. However, calibration is a time-consuming process, inherently user-specific, and its results are often not perfect. In contrast, the present study aims at evaluating recognition methods that do not require prior calibration of the data glove, by using raw sensor readings as input features and mapping them directly to different categories of hand shapes. An experiment was carried out, where test persons wearing a data glove had to grasp physical objects of different shapes corresponding to the various grasp types of the Schlesinger taxonomy. The collected data was analyzed with 28 classifiers including different types of neural networks, decision trees, Bayes nets, and lazy learners. Each classifier was analyzed in six different settings, representing various application scenarios with differing generalization demands. The results of this work are twofold: (1) We show that a reasonably well to highly reliable recognition of grasp types can be achieved – depending on whether or not the glove user is among those training the classifier – even with uncalibrated data gloves. (2) We identify the best performing classification methods for recognition of various grasp types. To conclude, cumbersome calibration processes before productive usage of data gloves can be spared in many situations.

A Neural Framework for Robot Motor Learning based on Memory Consolidation

Ben Amor, H.; Ikemoto, S.; Minato, T. ; Jung, B.; Ishiguro, H.
International Conference on Adaptive and Natural Computing Algorithms
Paper

View Abstract


Fixed sized neural networks are a popular technique for learning the adaptive control of non-linear plants. When applied to the complex control of android robots, however, they suffer from serious limitations, such as the moving target problem i.e. the interference between old and newly learned knowledge. To overcome these problems, we propose the use of growing neural networks in a new learning framework based on the process of consolidation. The new framework is able to overcome the drawbacks of sigmoidal neural networks, while maintaining their power of generalization. In experiments the framework was successfully applied to the control of an android robot.

2006

An Animation System for Imitation of Object Grasping in Virtual Reality

Weber, M.; Heumer, G.; Ben Amor, H.; Jung, B.
Advances in Artificial Reality and Tele-Existence
Paper

View Abstract


Interactive virtual characters are nowadays commonplace in games, animations, and Virtual Reality (VR) applications. However, relatively few work has so far considered the animation of interactive object manipulations performed by virtual humans. In this paper, we first present a hierarchical control architecture incorporating plans, behaviors, and motor programs that enables virtual humans to accurately manipulate scene objects using different grasp types. Furthermore, as second main contribution, we introduce a method by which virtual humans learn to imitate object manipulations performed by human VR users. To this end, movements of the VR user are analyzed and processed into abstract actions. A new data structure called grasp events is used for storing information about user interactions with scene objects. High-level plans are generated from grasp events to drive the virtual humans’ animation. Due to their high-level representation, recorded manipulations often naturally adapt to new situations without losing plausibility.

From Motion Capture to Action Capture: A Review of Imitation Learning Techniques and their Application to VR-based Character Animation

Jung, B.; Ben Amor, H.; Heumer, G.; Weber, M.
Thirteenth ACM Symposium on Virtual Reality Software and Technology, ACM Press
Paper | Video

View Abstract


We present a novel method for virtual character animation that we call action capture. In this approach, virtual characters learn to imitate the actions of Virtual Reality (VR) users by tracking not only the users’ movements but also their interactions with scene objects. Action capture builds on conventional motion capture but differs from it in that higher-level action representations are transferred rather than low-level motion data. As an advantage, the learned actions can often be naturally applied to varying situations, thus avoiding retargetting problems of motion capture. The idea of action capture is inspired by human imitation learning; related methods have been investigated for a longer time in robotics. The paper reviews the relevant literature in these areas before framing the concept of action capture in the context of VR-based character animation. We also present an example in which the actions of a VR user are transferred to a virtual worker.

Learning Android Control using Growing Neural Networks

Ben Amor, H.; Ikemoto, S.; Minato, T.; Ishiguro, H.
Proceedings of JSME Robotics and Mechatronics Conference ROBOMEC

View Abstract


Fixed sized neural networks are a popular technique for learning the adaptive control of non-linear plants. When applied to the complex control of android robots, however, they suffer from serious limitations, such as the moving target problem i.e. the interference between old and newly learned knowledge. To overcome these problems, we propose the use of growing neural networks in a new learning framework based on the process of consolidation. The new framework is able to overcome the drawbacks of sigmoidal neural networks, while maintaining their power of generalization. In experiments the framework was successfully applied to the control of an android robot.

2005

Intelligent Exploration for Genetic Algorithms. Using Self-Organizing Maps in Evolutionary Computation

Ben Amor, H.; Rettinger, A.
Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, pp.1531-1538, ACM Press
Paper

View Abstract


Exploration vs. exploitation is a well known issue in Evolutionary Algorithms. Accordingly, an unbalanced search can lead to premature convergence. GASOM, a novel Genetic Algorithm, addresses this problem by intelligent exploration techniques. The approach uses Self-Organizing Maps to mine data from the evolution process. The information obtained is successfully utilized to enhance the search strategy and confront genetic drift. This way, local optima are avoided and exploratory power is maintained. The evaluation of GASOM on well known problems shows that it effectively prevents premature convergence and seeks the global optimum. Particularly on deceptive and misleading functions it showed outstanding performance. Additionally, representing the search history by the Self-Organizing Map provides a visually pleasing insight into the state and course of evolution.