Metrics for Human-Robot Interaction 2008

A workshop affiliated with the 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI 2008)

March 12th, 2008, 13:30 - 17:30, Felix Meritis, Amsterdam

New: Proceedings, invited talk, and link to measurement questionnaire posted (see below)

The evaluation of interactions between robots and humans, different types of interactions, and individual robot and human behaviors require adequate metrics and guidelines. These metrics should take into account a variety of factors, ranging from objective performance to social interaction. These metrics can code behaviors, ways of interaction, social and psychological aspects, and technical characteristics or objective measures (i.e. success rates, interaction time, error rates, etc.). There are metrics that can be acquired using objective measuring tools; others depend on the personal interpretation by the staff conducting and analysing experiments. When human beings are present, metrics for social human-robot interaction are of utmost interest in order to achieve robotic systems that can be intuitively handled by people without causing frustration and despair.

The following key questions are to be addressed by the intended workshop: Which guidelines should be followed for careful experimentation in HRI? Are there objective metrics applicable to HRI? Are there social metrics applicable to HRI? What is the relevance of subjective criteria for evaluation? Can subjectively categorized criteria be used to form objective metrics for social HRI? How can benchmarks (standardized tasks) be used to evaluate human-robot interactions?

The goal of the workshop is to propose guidelines for the analysis of human-robot experiments and forward a handbook of metrics that would be acceptable to the HRI community and allow researchers both to evaluate their own work and to better assess the progress of others. The format will combine information about different metrics and evaluation methods given in submitted and invited talks, panels, and moderated group discussions.

Invited talk
Julie A. Adams, Vanderbilt University, Statistical Validity Pitfalls, PDF, 604 KB

Conference Proceedings
Catherina R. Burghart & Aaron Steinfeld (Eds.), Proc. of Metrics for Human-Robot Interaction, a Workshop at ACM/IEEE HRI 2008,  Amsterdam, the Netherlands, 12 March 2008, Technical Report 471 University of Hertfordshire, Hatfield, UK. PDF, 8.3 MB

Proceedings Contents

Teams and Frameworks

Social and Physical Interaction

The proceeding were made available as a printed report and are available at the link above. The aim of the workshop is to come up with a set of guidelines for experimental evaluation and a handbook of different metrics. It is intended to publish the results as well as selected papers in a special issue of an international journal. Input from this workshop will also be used in the construction of the portal.

Program committee
The co-chairs were supported by a distinguished list of organizers, all of whom have had significant experience both in organization and in relevant research. The list includes:


Catherina R. Burghart
Institute of Process Control and Robotics
University of Karlsruhe
Aaron Steinfeld
Robotics Institute
Carnegie Mellon University

Valid HTML 4.0 Strict -   Valid CSS! -   Level Triple-A conformance icon, W3C-WAI Web Content Accessibility Guidelines 1.0