Engagement during Interaction with Assistive Robots

*Corresponding Author:

Grazia D’Onofrio
Fondazione Casa Sollievo della Sofferenza, Complex Unit of Geriatrics, Department of Medical Sciences, San Giovanni Rotondo, Foggia, Italy

Abstract

Background:

Among various solutions to aid the aging population, the research community is focusing on assistive robotics that may help to delay admission to care facilities by fostering independence of older people in their homes. This paper presents the Robot-Era project and experimentation aimed at investigating 1) the psychological attitude of users toward robotic platforms as assistive companions, by assessing their level of engagement and 2) how the robot’s technological performance could affect the users’ level of engagement during their interaction with it.

Method:

Twenty-six older adults (M=13, F=13) were invited to interact with the robot during the experimental session. Data collection was based on observational methods; video recording of users’ interactions with assistive robots were evaluated and coded to retrieve both behavioral data on users’ engagement, and dialogue quality and efficiency metrics on robot performance.

Results:

Behavioral data revealed an overall positive attitude toward the robot, with facial expressions and gestures serving also to increase the communicative strength of messages to the robot, suggesting a good degree of engagement during the interaction. Moreover, a lower frequency of robot utterances was found to be related to higher degrees of engagement, enjoyment, instrumental use, and exploration behavior.

Conclusion:

Results from our investigation show that a robotic platform’s technical features and performance significantly influence the user’s engagement level during interaction with the robot.

marsbahis marsbahis mobilbahis 1xbet onwin betist piabella casino holiganbet holiganbet holiganbet mostbet mariobet meritking meritking meritking betnano casinoplus casinoplus betriyal casinoplus

Keywords

Assistive robotics, Engagement, Interaction, Older patients

Background

One reason that aging is emerging as a key policy issue is that both the proportion and absolute number of older people in populations around the world are increasing dramatically. Currently, only Japan shows a population where the proportion of people over 65 represents more than 30%, and by the year 2050, it has been estimated that the same situation will be reached by Europe and North America [1]. Consequently, the global population of older individuals will significantly increase the absolute number of older people who are care-dependent, as well. In addition, due to the aging of the population and the expected decline in the availability of unpaid informal caregivers, the demand for paid care workers is expected to increase by 2050 [2]. Policies at different levels will need to be implemented to improve recruitment and retention, and increase productivity [3,4].

Regarding this last point, research is needed to exploit technology advancements [5] to foster the active aging process while addressing efforts to prevent age-related issues, and also searching for solutions to support caregiving activities. Some examples can be found in literature of robotics that foster physical activity among older adults [6,7], or robotic platform systems designed to encourage adherence for home exercise programs and taking medication, as well as to provide appointment reminders and clinician communication [8]. Moreover, an emerging issue related to the aging population has been raised by the research community as it attempts to find solutions and develop technologies that may help to delay admissions to care facilities by fostering independence of older people in their homes. This direction is increasingly pursued as research continues to show that for older people, remain in their homes later in life instead of living in elder care facilities is beneficial in terms of overall quality of life, if their needs are adequately addressed [9]. Specifically, regarding the use of assistive robotics for older people at home, key roles have been identified for technological solutions including support in coping with functional and physical decline, cognitive decline, health monitoring and medication management, and psychosocial support [10].

Whatever the intended application, adaptation of the personality characteristics of the robotic system to individual preferences is crucial, and assistance provided over time should align with the progress of each individual’s recovery [11]. Robot features are critical in fostering technology acceptability of healthcare systems. Furthermore, acceptability of a robot is related to its functions, and if older people judge the robot as useful, they are more likely to accept it [12]. Other factors can influence the degree of acceptability of a robot. For example, it has been shown that the way the robot interacts with the person can indicate whether they are encouraged to further engage with it [13].

Indeed, engagement during human-robot interaction becomes crucial in order to foster acceptability and, consequently, effectiveness of healthcare robotics. Generally speaking, the engagement is defined as “the process by which individuals involved in an interaction start, maintain and end their perceived connection to one another” [14]. Engagement is related to the user experience—to the perceived control, feedback, interactivity, attention—and its modulation during interaction conveys physiological changes and behavioral changes in the persons through verbal and non-verbal communication [15].

As in human-to-human interaction, engagement in human-robot interaction (HRI) must be supported by several measures. It is important to establish and maintain eye contact between people involved in interactions, since this is one of the key factors causing the “social engagement” [16], even if this, alone, seems not to be sufficient to make the person effectively feel a social connection with the robot. This is because feelings of co-presence seem to be felt by the partners as an increase of the mutual engagement during interaction [17].

Now, what becomes fundamental is to determine whether or not an individual is engaged in an interaction with an artificial agent. HRI has been investigated with various techniques, most of them quantitative and focusing on the robot. Although a number of instruments have been developed to gather feedback about dimensions such as usability or acceptability of robotic devices, less focus has been devoted to psychological and social aspects of the interaction. With this in mind, observational methods could provide valuable procedures to gather crucial information by analyzing the behavioral pattern of users, in terms of verbal and non-verbal features, while interacting with the robot.

In this paper, we present some efforts conducted to assess engagement during the interaction with an assistive robot. We introduce the Robot-Era project [18] during which a user-centered-design approach was applied via a multidisciplinary team [19], composed of technical and nontechnical researchers. The end-users collaborated together to identify the requirements of the older users and caregivers and to develop the most suitable technological solution for older people’s independent living at home.

The main objective of this work is to provide qualitative evidence of user experience with particular emphasis on individual psychological attitude toward the usage of robotic platforms as assistive companions, by measuring engagement during the interaction. Moreover, we aimed to conduct an explorative investigation with the attempt to find a relation between the system performance and the psychological attitude of participants. The underlying concept is to investigate the relation between the psychological status of the users, his/her perceptions during the interaction with the robot, and the performance of the robot itself. This further analysis was performed to provide a deep understanding of how the system performance could affect the user’s perception of the robotic platform.

Method

▪ Robot-Era architecture

The Robot-Era architecture (Figure 1) integrates a multi-robot system able to work in different environments such as outdoor, condominium, and indoor. It includes also a domestic wireless sensor network (WSN), constituting an ambient intelligence (AmI) infrastructure, which supervises the home and localizes the user. Other agents of the system include the elevator and the user interface subsystem (i.e., tablet and microphone). The system is composed of heterogeneous devices and includes three different robots acting in three environments [18].

Figure 1:The schematic representation of the Robot-Era system.

The domestic robot was designed upon the SCITOS G5 mobile platform (developed by Metralabs) and had to safely navigate in a domestic environment. It was equipped with a robotic arm for carrying small objects. Multicolor LEDs, mounted on the eyes, and speakers provided feedback to the user. The robot brought a removable tablet that the user could employ for service requests.

The condominium robot, also designed upon the SCITOS G5, had to navigate between floors through the elevator. Most of the hardware was shared with the domestic robot. The condominium robot did not have an arm, but was mounted on a roller mechanism to be able to exchange goods with the outdoor robot. The outdoor robot, designed on the DustCart platform, was an autonomous mobile robot for the transportation of objects in an urban environment. The robot consisted of a mobile base, a container for the objects, a robotic head and a touch screen used primarily for humanrobot interaction, and sensors for obstacle detection and localization.

Regarding the interaction modalities, the user could interact with the system using a web-based interface, run the robot from the domestic robot tablet or any smartphone, or employ a wearable microphone connected with a speech recognition software module.

▪ Participants

By fulfilling the Declaration of Helsinki, the approval of the study for experiments using human subjects was obtained from the local ethics committees on human experimentation. Written informed consent for research was obtained from each participant. The participants, consecutively recruited from August to November 2015 in the BioRobotics Institute (Scuola Superiore Sant’Anna, Pontedera, Italy), were screened for eligibility.

Inclusion criteria were: 1) age ≥ 65 years; 2) positive evaluation of mental status at Mini- Mental State Examination (MMSE) [20] (cut off to be enrolled= Score≥29); 3) autonomy in performing daily activities with domestic tools, evaluated with the instrumental activities of daily living (IADL) [21] (cut off to be enrolled= Score ≥2); 4) absence of psychiatric illness, substance abuse, and communication impairments. The following parameters were collected by a systematic interview with the patients: date of birth, gender, educational level, and social support network.

Experimental procedure

The experimentation was carried out at DomoCasa Lab, a domestic house in Peccioli (Italy). The participants were exposed to an informative video illustrating the Robot-Era system to familiarize them with its functionalities and services. Then, the participants were instructed with the experimental procedure, which envisaged a number of tasks related to each service of the Robot-Era system (Figure 2) shows examples of tasks performed by the users). Experimentation was set within a domestic house, with the participants placed in the sitting room of the house. Each task started with the participants calling the robot, which reached them in the sitting room. Specifically, the participants were asked to accomplish the following tasks:

Figure 2: Some examples of tasks performed by the participants.

• Indoor escort: The participants were invited to guide the robot around the house. The robot was let free to explore the environment.

• Grocery service: The participants were asked to provide the robot with a list of products for the robot to send to the grocery store.

• Object transportation: The participant asked the robot to bring them a bottle of water from the kitchen.

• Garbage service: The participants asked the robot to call the condominium to bring out the garbage.

• Surveillance: The robot warned the participant about a gas leaking and informed the participant how to ask for help.

• Reminder: The participant set an alarm through the robot. Subsequently, the robot alerted the participant when the alarm went off.

• Communication services: The participant was asked to make a video-call through the robot.

▪ Video recording analysis: behavioral data and system performance metrics

Video recording of users’ interaction with assistive robots was evaluated and coded using the behavioral categories proposed by Andrés and colleagues [22]. The researchers proposed a procedure to assess the complete humanrobot interactive activity. The evaluation system focuses on three key points: classifying scenario features, classifying (current/potential) social robot features, and assessing of behavioral units during (and after) interaction. With specific regard to user behavior, the following categories were proposed for investigation: emotions, proxemics, gaze, communication, facial expression, body gestures, interaction with robot, and interpersonal relationship (when allowed).

Specifically, the following general categories were considered:

ο Emotion: joy, sadness, fear, anger, disgust, neutral

ο Proxemics: intimate, personal and social, public

ο Gaze: directed gaze, mutual face gaze, none

ο Facial expressions: smile, laugh, raise eyebrows, frown, inexpressive

ο Body gestures: lifting shoulders, nodding head, shaking head, quiet

ο Interaction with the robot: exploration of the robot characteristics, simple manipulation, conceptual use, non-conceptual use, no interaction

Since the whole interaction was supposed to represent a conceptual use of the robot due to the specific experimental protocol, with regard to the category “Interaction with the robot,” we decided to introduce a particular sub-category within it, representing the instrumental use of the robot. More specifically, some tasks, such as the Skype call and indoor walking, required the participants to make use of the robot or the tablet. The duration of these tasks depended directly on the willingness of the user to interact with the system. The length of this specific characteristic could be interpreted as an indirect measure of engagement in the interaction, and for this reason we decided to introduce this further subcategory in the observational analysis.

On this basis, video data were analyzed and categorized according to the above-mentioned dimensions. More specifically, the percentage of time spent on each category was computed, and simple descriptive analysis was performed.

By taking the cue from the PARADISE framework [23], we considered those parameters which better fitted our experimental setup, and added some specific ones:

• Dialogue efficiency metrics

− elapsed time (length), system turns, user turns, total turns, response latency, latency due to reprompts

• Dialogue quality metrics

− errors, reprompt, rejects (raw)

− errors%, reprompt%, rejects% (normalized)

The dialogue efficiency metrics were calculated by hand from the dialogue recordings. The length of the recording was used to calculate the elapsed time in seconds (ET) from the beginning to the end of the interaction. Measures were calculated for the number of system turns the number of user turns, and the total amount of turns (total turns). Additionally, the agent mean response latency (latency) and the response latency to the user’s repeated requests, namely the time from the first attempt by the user and the answer from the robot (long_latency), were obtained.

The dialogue quality measures were derived from the recordings. A number of agent behaviors that affect the quality of the resulting dialogue were gathered. This included the number of times the user had to repeat a command to the system (reprompt), whenever it was due to an error of recognition by the system or to a timeout. The errors of speech recognition by the system were also computed, and the number of recognizer rejections (rejects) where the system’s confidence in its understanding was low and it said something like, “I have not understood. Can you repeat, please?” were detected. Finally, as in [24], we normalized the dialogue quality metrics by dividing the raw counts by the total number of utterances in the dialogue; this resulted in errors%, reprompt%, rejects% metrics. This solution was chosen because all the efficiency metrics appeared unlikely to generalize [3].

▪ Statistical analysis

As a first step, from data about behavioral observation, general dimensions of engagement and enjoyment were computed. Among the subcategories, the ones which positively correlated among each other were averaged to obtain a more general dimension of engagement and enjoyment.

Additional analysis envisaged the investigation of relation between psychological attitude and system performance metrics. Specifically, the statistical analysis entailed regression linear model analysis (performed through SAS 9.4) with the main aim to investigate the possible influence of system performance on the user’s psychological attitude. With this purpose, a number of stepwise regression analyses were performed by inserting the performance metrics as independent variables, while the behavioral dimensions were considered as dependent variables. More specifically, the following metrics were considered: engagement, enjoyment, no interaction, and direct gaze.

Results

Twenty-six older adults (13 males and 13 females) were recruited for this experiment. The mean age of the sample was 71.61 ± 5.67 years (range=65-85), and all participants were all retired. Regarding the educational level of the sample, only 42.00% of the participants attended high school and 11.54% had a master degree. Moreover, at the time of the experiment, 35.00% of them were in a stable relationship, and the same amount was widowers. The remaining persons were single or divorced. Additionally, 9 of them lived alone at home, 6 lived with a partner, 2 of them lived with a child, while 9 resided in a nursing home.

Participants were also asked about their familiarity with the usage of a number of technological devices using a five-point Likert scale. The telephone and TV remote controller were the devices they were more familiar with, while the instruments and systems that they stated they were less familiar with were smartphones, tablets, devices for telemedicine, and Skype.

▪ Psychological attitude

The average time spent in interacting with the robot was 25.63 minutes (SD= 6.16). In Table 1 the mean percentages of time spent by the users in certain behaviors are reported.

Table 1: Mean percentages of time spent by the users in certain behaviors.

Due to the experimental protocol, the interaction with the robot was exclusively conceptual. In fact, since the users were asked to accomplish specific tasks, their use of the system was subjected to some constrains. Nevertheless, the duration of certain tasks was mostly up to the user, and the duration of such interactions can be interpreted as the willingness of the user to be engaged in the interaction and willing to maintain a relation with the robot. That is why we decided to consider the specific sub-category “instrumental use of the robot.” From observational data, it emerged that the users spent most of their time interacting with the robot (96.00%) instead of ignoring it. Moreover, an interesting finding emerged regarding the user’s unexpected behavior. In fact, participants were supposed to interact with the robot in a pretty structured manner, and except for some freedom (i.e., they could spend all the time they wanted in using the robot for the indoor escort, or they could order the preferred number of items in the grocery service), they have prearranged tasks to accomplish. Nevertheless, it sometimes happened that participants started an explorative behavior and kept approaching the robot to obtain a better understanding of the robot’s features and functioning (exploration: 2.00%). This was interpreted as a particular sign of curiosity and interest toward the robotic platform. On the contrary, the time spent in none interacting with the system was really exiguous (2.00%) and was mostly due to the robot not always being responsive to the user’s requests, and consequently they distracted themselves while waiting for a feedback from the robot. In support of these results, the analysis of emotional attitude unveiled an overall positive attitude of the whole sample toward the robot, which can be inferred from non-verbal clues such facial expressions and gestures. Even though the participants seemed to appear mostly quiet while interacting (98.74%), in fact, they experienced only positive emotions. Their emotional status was never negative, and they sometimes enjoyed the interaction. Furthermore, it emerged that the facial expressions and the body gestures served also to increase the communicative strength of messages to the robot, suggesting a good degree of engagement during the interaction.

With regard to the investigation of possible relation between psychological attitude toward the robotic platform and some sociodemographic aspects, correlation analysis did not unveil specific associations.

As stated previously, an additional goal within our work was to explore the possible implication of system performance in determining the psychological attitude of participants. The next subsection presents the results obtained.

▪ System performance influence on user’s psychological attitude

Table 2 shows how the psychological attitude toward the interaction with the assistive robot can be affected by the robot’s performance itself. In the previously section, it clearly emerged how the interaction with the robot elicited mostly positive emotions in participants. From this further analysis, it emerged that the amount of utterances from the robotic platform seemed to be the main factor influencing the participant’s attitude. A lower frequency of robot utterances was related to higher degrees of engagement, enjoyment, instrumental use, and exploration behavior as well. Moreover, it emerged that a lack of interaction with the robotic platform could be ascribed to specific behaviors which, taken together, reflected a low responsiveness rate of the robot. In fact, repeated commands (Reprompt%), possibly related to the robot ignoring the participant (Timeout%), could increase the amount of time spent waiting for the robot to accomplish requests by the user (Tn1_Lat).

Table 2: Regression linear model analysis; psychological attitude of the users vs. interaction with the assistive robot and robot performance.

Finally, the time spent waiting for an answer from the robot after repeated requests from the participant seemed to affect the explorative behavior, as well.

Discussion

Participants interacted with a home assistive robot, which was designed to support frail older individuals with the ultimate goal of fostering their independence at home by preventing accidents and hospitalization. The main purpose of this work was to bring evidence fostering the effectiveness of information and communication technology (ICT) solutions in caring activities. With this in mind, robotic platforms are meant to be seen, not just as instruments, but more as agents able to interact with the users, where a good interaction is meant to represent a success in the purpose of assistive technology. To obtain a good interaction between agents and humans, a certain level of engagement is necessary via maintaining an exchange of information and compliance in the interaction.

Generally speaking, our findings provide promising information in this direction, and taken together, they suggest interest by the users, who demonstrated a definite positive attitude toward the robotic platform and its functions, supported by evidences of mostly positive or neutral feelings during the interaction. Nevertheless, some aspects should be considered while interpreting the results of our investigation. In fact, it is important to stress that the robot is not designed to be a social companion, and consequently it should not arouse amusement and enjoyment. Indeed, despite relatively low percentages in the sub-category “joy,” it seems worthy to highlight that the interaction, even when some technical issues occurred, was never perceived as annoying or frustrating. Conversely, the interaction was perceived as enjoyable by the users. Moreover, although the overall interaction was experienced with a neutral attitude by participants, it is important to highlight that they never perceived frustration or boredom, and that only positive emotions were elicited. This is a particularly valuable result, since it suggests a more likely positive long-term attitude toward robot usage. Taken together, our behavioral results suggest a good degree of engagement experienced by the users. This was the main point we were looking for, since a good engagement in interaction is more likely to turn into a higher degree of compliance in the usage of this technological solution as support both for older people and for caregivers.

An additional goal of this work was to explore which technical features of the robotic platform could be ascribable as aspects influencing the user’s perception of the interaction with the robot. This further effort was aimed to strengthen the exchange between the two different disciplines, life sciences and ICT to foster mutual support in developing solutions for older persons. Unexpectedly, those aspects such as system errors seemed not to affect in a significant way the interaction with the users. A possible reason could be that the users were aware that they were interacting with a prototype and consequently could have been more indulgent with the robot failures, or more simply, users tended to judge an agent who makes mistakes more “human-like.” With this regard, other recent studies showed that humans prefer to interact with a robot that makes mistakes and exhibits some uncertainty or delays [25- 28]. Nevertheless, it seems that the fluency of the dialogue could be a factor in the positive perception of the interaction. The balance between the numbers of the agent’s turns in the speech dialogue seems to be important in determining whether the user is engaged in the interaction. This suggests the importance to pay particular attention in developing capabilities in the robotic platforms by ensuring an adequate dialogue flow, possibly by modulating it through time according to the user’s preferences.

Finally, what importantly emerged from our analysis is an overall positive psychological attitude toward the robot interaction and a good level of trust and engagement, which eventually does not seem to be affected by environmental or methodological issues. These evidences provide valuable insight for research on assistive technology, since, from older individuals’ point of view, the possibility to rely on technological support could bring a higher level of independence, with benefits both in psychological and physical wellbeing, and the ultimate goal of extending their lives spent in their own homes. In addition, technology could provide relief to both informal caregivers, such as relatives, as well as formal caregivers by facilitating the management of their job.

Conclusions

Considering that technology will become even more pervasive in healthcare activities, it is important to understand to what extent it is considered as useful and acceptable by users, and to obtain a deeper understanding of how to improve it. Because home assistive technology is designed to accompany the majority of daily activities of older users, it is critical to investigate to find the best way to ensure engagement in the user’s behavior and attitude toward the technologic companion. The ultimate goal is the attempt to foster a healthy life-style and to prevent age-related issues. The present work revealed the importance of properly designing a robotic assistant in a tailored manner to improve the relationship with the user. Because our results offer an insight limited to the experimental session span, further research should focus on a longitudinal investigation with the aim of achieving better insight of the engagement over time.

Acknowledgments

This work was supported by the Robot-Era and ACCRA Project, founded by the European Commission respectively under the FP7 Programme (FP7-ICT-2011, grant agreement No. 288999 and Horizon 2020 Programme (H2020-SCIPM14- 2016, grant agreement No. 738251).

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this article.

References

1. World report on aging and health. Posted on 2015. OECD, Health at glance: OECD Indicators, Paris: OECD Publishing, (2015).
2. Colombo F, Llena-Nozal A, Mercier J, et al. Help Wanted? Providing and Paying for Long-Term Care, Paris: OECD Publishing, (2011).
3. Commission Staff Working Document, Long-term Care in Ageing Societies – Challenges and Policy Options, Brussels, (2013).
4. Turchetti G, Micera S, Cavallo F, et al. Technology and innovative services. IEEE. Pulse 2(2), 27-35 (2011).
5. D’Onofrio G, Sancarlo D, Ricciardi F, et al. Information and Communication Technologies for the Activities of Daily Living in Older Patients with Dementia: A Systematic Review. J. Alzheimer's Dis 57(3), 927-935 (2017).
6. Fasola J, Matarić MA socially assistive robot exercise coach for the elderly. Journal. Of. Human-Robot. Interaction 2(2), 3-32 (2013).
7. Cavallo F, Aquilano M, Bonaccorsi M, et al. Improving domiciliary robotic services by integrating the ASTRO robot in an AmI infrastructure. Springer. Tracts. In. Advanced. Robotics 94 (2014).
8. Marek KD, Rantz MJ. Aging in place: a new model for long-term care. Nursing. Administration. Quarterly 24(1), 1-11 (2000).
9. Robinson H, MacDonald B, Broadbent E. The role of healthcare robots for older people at home: A review. Int. J. Soc. Robot 6(1), 575-591 (2014).
10. White M, Vining Radomski M, Finkelestein M, et al. Assistive/Socially covery: Patient Perspectives. International. Journal. of. Telemedicine. And. Applications, 6 pages (2013).
11. Broadbent E, Tamagawa R, Patience A, et al. Attitudes towards health-care robots in a retirement village. Australas. J. Ageing 31(5), 115-120 (2012).
12. Heerink M, Kröse B, Evers V, et al. The influence of social presence on acceptance of a companion robot by older people. Journal. of. Physical. Agents 2(1), 33-40 (2012).
13. Sidner CL, Lee C, Kidd CD, et al. Explorations in engagement for humans and robots. Artificial. Intelligence 166(1-2), 140-164 (2005).
14. Anzalone SM, Boucenna S, Ivaldi S, et al. Evaluating the engagement with social robots. International. Journal. of. Social. Robotics 7(4), 465-478 (2015).
15. Breazeal CL. Designing sociable robots. MIT Press, (2004).
16. Anzalone SM, Ivaldi S, Sigaud O, et al. Multimodal people engagement with iCub. In Biologically Inspired Cognitive Architectures 2012: Proceedings of the Third Annual Meeting of the BICA Society, (2012).
17. Cavallo F, Limosani R, Manzi A, et al. Development of a Socially Believable Multi-Robot Solution from Town to Home. Cognitive. Computation 6(4), 954–967 (2014).
18. Di Nuovo A, Broz F, Wang N, et al. The multi-modal interface of Robot-Era multi-robot services tailored for the elderly. Intelligent. Service. Robotics 11(1), 109–126 (2017).
19. J Cockrell e M. Folstein. Mini-Mental State Examination (MMSE),» Psychopharmacology bulletin, vol. 4, n. 24, pp. 689-692, (1987).
20. Barberger‐Gateau P, Commenges D, Gagnon M, et al. Instrumental activities of daily living as a screening tool for cognitive impairment and dementia in elderly community dwellers. J. Am. Geriatr. Soc 11(40), 1129-1134 (1992).
21. Litman DJ, Walker MA, Kearns MJ. Automatic detection of poor speech recognition at dialogue level. In Proceedings of the 37th Annual Meeting of the Association of Computational Linguistics, (1999).
22. Andrès A, Pardo DE, Diaz M, et al. New Instrumentation for human robot interaction assessment based on observational methods. JAISE 7(1), 397-413 (2015).
23. Litman DJ, Pan S. Designing and evaluating an adaptive spoken dialogue system. User. Model. User-Adap 12(2-3), 111-137 (2002).
24. Bethel C, Murphy R. Review of human studies methods in HRI and recommendations. International Journal. of. Social. Robotics 2(1), 347-359 (2010).
25. Steinfeld A, Fong T, Kabe D, et al. Common metrics for human robot interaction. In 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, HRI ’06, New York, NY, USA, (2006).
26. Rich C, Ponsler B, Holroyd A, et al. Recognizing Engagement in Human-Robot Interaction. In 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Osaka, (2010).
27. Admoni H, Dragan A, Srinivasa SS, et al. Deliberate Delays During Robot-to-Human Handovers Improve Compliance With Gaze Communication. In ACM/IEEE International Conference on Human-robot Interaction, HRI '14, (2014).
28. Short E, Hart J, Vu M, et al. No fair!! An interaction whit a cheating robot. In 5th ACM/IEEE International Conference on Human-Robot Interaction, (2010).