Publications

As socially assistive robots (SARs) enter more diverse care settings, including users' homes, it is critical to identify the shifting privacy risks and dimensions of privacy this technology and its data collection capabilities may affect. We propose a new model of privacy to address the complex nature of SARs as a multimodal technology within the healthcare space. To construct this new model, we combine a previous three-dimensional model from the healthcare literature and synthesize it with a seven-dimension technology-related model. We then use this new combined model to analyze self-report data from several workshops with prospective users of the dog-like robot Therabot to map out the dimensions of privacy identified as future concerns for clinicians and those living with depression. Finally, we suggest this model can be used in future studies to support the in-depth exploration of privacy implications of SARs within healthcare through discussions about privacy among clinicians, those receiving care, and robot designers.

Imagining and creating a future with a robot tailored to an individual's needs and wants provides a unique challenge to both designers and perspective users. Using both an existing base zoomorphic robot, TherabotTM in a specially designed tabletop role-playing game, we have explored what it would look like for someone living with depression to go through a "day in the life" with their newly designed robot companion. Inspired by this collaborative co-design technique, eight participants explained their own relationship with depression, identified a character that they connected with, and chose the physical form, sensors, and abilities for a socially assistive robot. Using this character and robot, participants described how they might overcome depression symptoms and identified uses when managing depression symptoms for the robot.

Socially assistive robots can be used as therapeutic technologies to address depression symptoms. Through three sets of workshops with individuals living with depression and clinicians, we developed design guidelines for a personalized therapeutic robot for adults living with depression. Building on the design of Therabot, workshop participants discussed various aspects of the robot's design, sensors, behaviors, and a robot connected mobile phone app. Similarities among participants and workshops included a preference for a soft textured exterior and natural colors and sounds. There were also differences - clinicians wanted the robot to be able to call for aid, while participants with depression differed in their degree of comfort in sharing data collected by the robot with clinicians.

Effective emotional communication can have benefits in social interactions between a user and a robot. By developing a tool that can pivot easily between distinct emotional states in response to user presence or input, we have applied a multi-modal method for implementing affect communication in everyday interactions. Our interactive tool engages with users through three emotional avenues and has the potential for usage as an emotional or social support companion.

Deploying socially assistive robots (SARs) at home, such as robotic companion pets, can be useful for tracking behavioral and health-related changes in humans during lifestyle fluctuations over time, like those experienced during CoVID-19. However, a fundamental problem required when deploying autonomous agents such as SARs in people’s everyday living spaces is understanding how users interact with those robots when not observed by researchers. One way to address that is to utilize novel modeling methods based on the robot’s sensor data, combined with newer types of interaction evaluation such as ecological momentary assessment (EMA), to recognize behavior modalities. This paper presents such a study of human-specific behavior classification based on data collected through EMA and sensors attached onboard a SAR, which was deployed in user homes. Classification was conducted using generative replay models, which attempt to use encoding/decoding methods to emulate how human dreaming is thought to create perturbations of the same experience in order to learn more efficiently from less data. Both multi-class and binary classification were explored for comparison, using several types of generative replay (variational autoencoders, generative adversarial networks, semi-supervised GANs). The highest-performing binary model showed approximately 79% accuracy (AUC 0.83), though multi-class classification across all modalities only attained 33% accuracy (AUC 0.62, F1 0.25), despite various attempts to improve it. The paper here highlights the strengths and weaknesses of using generative replay for modeling during human–robot interaction in the real world and also suggests a number of research paths for future improvement.

Therabot is a socially therapeutic assistive robot (STAR) in the form of a stuffed dog designed for use in mental healthcare as a supplement to therapist directed interventions. The modular sensor collar is an adjunctive tool designed to facilitate the collection of contextual data in home environments across zoomorphic robotic platforms for the purpose of evaluating sensors and informing the design of machine learning models for context understanding. This demonstration will allow attendees to interact with the latest version of Therabot in both covered and uncovered (underlying robotic platform visible) forms. Additionally, attendees will be able to use and examine the open source modular sensor collar platform. The purpose of this demo is to invite conversation and spark ideas among the community about the design of social robots for use in mental healthcare applications.

Artificial Intelligence (AI)-driven Digital Health (DH) systems are poised to play a critical role in the future of healthcare. In 2021, $57.2 billion was invested in DH systems around the world, recognizing the promise this concept holds for aiding in delivery and care management. DH systems traditionally include a blend of various technologies, AI, and physiological biomarkers and have shown a potential to provide support for individuals with various health conditions. Digital therapeutics (DTx) is a more specific set of technology-enabled interventions within the broader DH sphere intended to produce a measurable therapeutic effect. DTx tools can empower both patients and healthcare providers, informing the course of treatment through data-driven interventions while collecting data in real-time and potentially reducing the number of patient office visits needed. In particular, socially assistive robots (SARs), as a DTx tool, can be a beneficial asset to DH systems since data gathered from sensors onboard the robot can help identify in-home behaviors, activity patterns, and health status of patients remotely. Furthermore, linking the robotic sensor data to other DH system components, and enabling SAR to function as part of an Internet of Things (IoT) ecosystem, can create a broader picture of patient health outcomes. The main challenge with DTx, and DH systems in general, is that the sheer volume and limited oversight of different DH systems and DTxs is hindering validation efforts (from technical, clinical, system, and privacy standpoints) and consequently slowing widespread adoption of these treatment tools.

Socially-assistive robots (SARs) have significant potential to help manage chronic diseases (e.g. dementia, depression, diabetes) in spaces where people live, averse to clinic-based care. However, the challenge is designing SARs so that they perform appropriate interactions with people who have different characteristics, such as age, gender, and cultural identity. Those characteristics impact how human behaviors are performed as well as user expectations of robot responses. Although cross-cultural studies with robots have been conducted to understand differing population characteristics, they have mainly focused on statistical comparisons of groups. In this study, we utilize deep learning (DL) and machine learning (ML) models to evaluate whether cultural differences show up in robotic sensor data during human-robot interaction (HRI). To do so, a SAR was distributed to user's homes for three weeks in the US and Korea (25 participants), while collecting data on the human activity and the surrounding environment through on-board sensor devices. DL models based on that data were able to predict the user’s cultural identity with roughly 95% accuracy. Such findings have potential implications for the design and development of culturally-adaptive SARs to provide services across diverse cultural locales and multi-cultural environments where users’ cultural background cannot be assumed a priori.

The use of socially assistive robots is able to alleviate some depression symptoms, according to existing research. However, due to comorbidities that often accompany depression and the unique experiences of each individual, it is necessary to get a better understanding of how SARs should be personalized. Through 10 hourlong workshops with 10 individuals living with depression, we explored the customization of a zoomorphic SAR for adults with depression. By using the SAR Therabot? as a base platform, participants designed their own unique covering for the robot, and discussed desired robot behaviors and privacy concerns around data collection. Though the physical designs of the robots varied greatly, participants expressed common themes regarding their preference for a soft touchable exterior, comfort with sharing data with their therapists, and interest in the robot producing more realistic sounds and movements, among other design features.

This chapter provides an overview on approaches for planning, designing, and executing human studies for Human-Robot Interactions (HRI). Recent literature is presented on approaches used for conducting studies in human-robot interactions. There is a detailed section on terminology commonly used in HRI studies, along with some statistical calculations that can be performed to evaluate the effect sizes of the data collected during HRI studies. Two improvements are described, using insights from the psychology and social science disciplines. First is to use appropriate sample sizes to better represent the populations being investigated to have a higher probability of obtaining statistically significant results. Second is the application of three or more methods of evaluation to have reliable and accurate results, and convergent validity. Five primary methods of evaluation exist: self-assessments, behavioral observations, psychophysiological measures, interviews, and task performance metrics. The chapter describes specific tools and procedures to operationalize these improvements, as well as suggestions for recruiting participants. A large-scale, complex, controlled human study in HRI using 128 participants and four methods of evaluation is presented to illustrate planning, design, and execution choices. The chapter concludes with ten recommendations and additional comments associated with the experimental design and execution of human studies for human-robot interactions.

The objective of this chapter is to provide an overview of the use of the forensic interview approach for conducting qualitative interviews especially with children in human-robot interaction studies. Presented is a discussion of related work for using qualitative interviews in human-robot interaction studies. A detailed approach on the phases of a forensic interview are presented, which includes introduction and guidelines, rapport building, narrative practice, substantive disclosure interview, and the cool-down and wrap-up. There is a discussion on the process of transcription and coding of the qualitative data from the forensic interview approach. A presentation is provided detailing an exemplar study including the analyses of the qualitative data. There is a brief discussion of the methods for reporting this type of data and results along with the conclusions from using this approach for human-robot interaction studies.

Therabot is a robotic therapy support system designed to supplement a therapist and to provide support to patients diagnosed with conditions associated with trauma and adverse events. The system takes on the form factor of a floppy-eared dog which fits in a person»s lap and is designed for patients to provide support and encouragement for home therapy exercises and in counseling.

The goal of this paper is to begin a discussion of the benefits, challenges, and ethical concerns related to the use of robots as intermediaries for obtaining sensitive information from children within the human-robot interaction (HRI), criminology, sociology, legal, and psychological communities. This work examines how robots may impede disclosures from children, encourage inaccurate disclosures, facilitate unintended disclosures, provide a more reliable interviewer, decrease the likelihood of misleading children, and enhance forensic interviews through high fidelity data logging. Open research questions, proposed research studies, and pathways toward deployment of robots as forensic interviewers are provided. As HRI researchers working in an interdisciplinary team, with members trained by the National Child Advocacy Center in Child Forensic Interview Protocols, we believe sustaining a dialogue concerning the design and appropriate use of robots in this area is essential for continued progress.

Therabot is an assistive-robotic therapy system designed to provide support during counseling sessions and home therapy practice to patients diagnosed with conditions associated with trauma. Studies were conducted to determine desired features of potential end-users of the system, such as clinicians, with feedback from past survivors of trauma to guide the participatory design process. The results from a survey of 1,045 respondents revealed a preferred form factor of a floppy-eared dog with coloring similar to that of a beagle. The most requested features were that the robot be of a size that would be comfortable to fit in a person's lap and a covering that was soft, durable, and had multiple textures.

Therabot™ is an assistive-robotic therapy system designed to provide support during counseling sessions and home therapy practice to patients diagnosed with conditions associated with trauma. It has the form factor of a floppy-eared dog with coloring similar to that of a beagle, and comfortably fits in a person's lap.

Sky Writer is a collaborative communication medium that augments the traditional display of a UAV pilot and allows other stakeholders to communicate their needs and intentions to the pilot. UAV pilots engaging in time-critical missions, such as urban disaster responses, often must allocate most of their cognitive capacity towards flight tasks, making communication and collaboration with other stakeholders difficult or dangerous. Sky Writer addresses the needs of stakeholders while requiring minimal cognitive effort from the UAV pilot. The application presents stakeholders with an interface that provides contextual flight information and a live video stream of the flight. Stakeholders are able to sketch directly on the video stream or use a spotlight indicator that is mirrored across all displays in the system, including the pilot's display. The application can be used in any modern web browser and works with traditional and touch devices. Concept experimentation performed at Disaster City with two pilots indicated that the spotlight feature was particularly useful while the UAV was in motion, and the sketching features were most useful while the UAV was stationary. The system will be tested with professional responders soon to determine its efficacy in a simulated response, and to inform the ongoing design process.

This paper describes a novel, low cost HRI testbed for the evaluation of robot movement, gaze, audio style, and media content as a function of proximity. Numerous human-robot interaction studies have established the importance of proxemics in establishing trust and social consonance, but each has used a robot capable of only some component, for example gaze but not audio style. The Survivor Buddy proxemics testbed is expected to serve as blueprint for duplication or inspire the creation of other robots, enabling researchers to rapidly develop and test new schemes of proxemic based control. It is a small, four-degree of freedom, multi-media "head" costing approximately $2,000 USD to build and can be mounted on other robots or used independently. To enable proxemics support, Survivor Buddy can be coupled with either a dedicated range sensor or distance can be extracted from the embedded camera using computer vision. The paper presents a sample demonstration of proxemic competence for Survivor Buddy mounted on a search and rescue robot following the victim management scenario developed by Bethel and Murphy.

Humans regulate their social behavior based on proximity to other social actors. Likewise, when a robot fulfills the role of a social actor it too should regulate its interaction based on proximity. This paper describes work in progress to establish methods for autonomous modification of social behavior based on proximity and to quantify human preferences between methods of scaling a robot's social behaviors based on distance from a human. The preliminary results of a 72 participant human study examine the reaction to scaling with linear methods and perception-based methods. Results indicate significantly higher ratings in multiple areas (comfort, natural movement, safety, self-control, intelligence, likability, submissiveness (p<.05) when using a perception-based scaling function, as opposed to a linear or no scaling function. Work in progress is analyzing the biometric measures collected.

This paper describes an open source speech translator toolkit created as part of the "Survivor Buddy" project which allows written or spoken word from multiple independent controllers to be translated into either a single synthetic voice, synthetic voices for each controller, or unchanged natural voice of each controller. The human controllers can work over the internet or be physically co-located with the Survivor Buddy. The toolkit is expected to be of use for exploring voice in general human-robot interaction.

This video describes the Survivor Buddy social medium robot.