CA paper No.4

1. Title & Reference:

Q. Vera Liao, Muhammed Mas-ud Hussain, Praveen Chandar, Matthew Davis, Yasaman Khazaeni, Marco Patricio Crasso, Dakuo Wang, Michael Muller, N. Sadat Shami, and Werner Geyer. 2018. All Work and No Play?. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI ‘18). ACM, New York, NY, USA, Paper 3, 13 pages. DOI: https://doi.org/10.1145/3173574.3173577

1. Memo:

Problem

대부분의 CA 연구가 Lab 환경에서 실험되고 있음. certain interactions with CAs may only be observed in the wild.

Objects of research

• Real-life 환경에서,
• What kinds of conversational interactions did users have with the QA agent in the wild? (어떤 유형의 대화 상호작용을 했는지)
• What kinds of conversational interactions can be used as signals for inferring user satisfaction with the agent’s functional performance, and playful interactions. (어떤종류의 대화가 에이전트에 대한 기능적 만족과, Playful한 interaction에 만족을 주는지 알아봄)
• our goal is to inform for whom to design for playfulness by identifying behavioral signals of users seeking satisfaction from playful interactions through a data-driven approach. (기존의 연구는 Why playful이고 How playful에 초점을 맞추어 왔음)

What they did

• We recruited 337 participants through HR contacts in three groups with different starting dates, each using Chip (Text-Based Agent. 회사에 대한 질문에 대해 답해주는 HR Agent)or 5-6 week
• We considered two types of features as independent variables (predictive features): conversational acts and lexical features representing conversational behaviors

Findings

• user satisfaction strongly impacted by social designs such as the agent’s representation [56] and personality (Satisfaction이랑은 관련있지만, Function에 대한 기대감을 높일 수 있음)
• CAs should target similar goals of human conversations, which are to fulfill propositional goals—conveying meaningful information, and interactional goals—ensuring the communication process to be enjoyable
• 하지만, it may differ from that of human. 사람마다 다르고, Agent에 대한 Relation building 행동이 나타나지 않음.
• 실제 Reception Agent에 대한 사람들의 사용을 살펴 보니, relational behaviors—politeness(e.g.,greeting),sociable behaviors(e.g., small talk), and negative behaviors (e,g., insult) 행동들이 나타났음. 질문의 1/3이 small talk이지만, Intelligent Test 목적이였다는 한계.
• found remarkable differences in their preference for interaction styles of CAs
• agent ability check and closing negatively predict the satisfaction on functionality
  • Agent ability-check can be considered signals of user struggling with the gulf of execution
  • users closing the conversation (“bye”) after errors, signaling frustration and refusal to continue using the system
• no signaling effect of chat opening (다른 연구에서는 인사를 하느냐가 tendency to anthropomorphize an embodied agent에 영향을 주었는데 여기서는 큰 영향 없었음)
• 그냥 chat 윈도우에 인사하는 것은 habitual utterances with the chat interface instead of consciously anthropomorphizing the agent
• An evident pattern in lexical features signaling playfulness is frequent occurrences of second-person pronouns (e.g., “how do you”, “how are you”) in positive features and first-person pronouns in negative features. (이인칭 대명사 => Signaling Playfulness/ 1인칭 대명사 => Negative Feature (★★★★★))
• “testing intelligence” to be a manifestation of playfulness. (=» Really?? )
  • words information and search to be strong signals for playfulness (=» Really?? )
• Reliable Signals: a) User들이 남긴 System에 대한 피드백이 기능에 관한 것만은 아니였다. 일부는 Playful intention을 가지고 있었음. b) chit-chat is indeed an indication of users seeking satisfaction from playful interactions. c) 사용자의 Agent ability checking behavior는 signals of users struggling with the system’s functional affordance. d) Communicative utterances such as opening and acknowledging (아는척하기) are more of habitual behaviors in using the chat interface
• a challenging task in the development of CAs is to anticipate chit-chat in a free-form input. (Free form 농담을 어떻게 대응할 것인가??? )
  • 대부분의 농담이 Agent의 traits and status of agents에 관련되어 나타나는 경향이 있으므로, Designing an agent with a comprehensive and consistent persona and elaborating on descriptions of such a persona (e.g., what does the agent like) may help prepare for addressing this type of chit-chat. - 포괄적이고 일관된 페르소나로 Agent를 설계하고, 페르소나에 대한 설명-무엇을 좋아하고, 싫어하고-을 상세히 설명하면, 이러한 유형의 채팅을 준비하는데 도움될 것

• “what can you do” although on the surface was asked in an anthropomorphic way, served more for the instrumental usage of the system (Web page에서 about, help section 찾아가듯이)

• understanding how people manifest internal status in communicative behaviors with human partners is important for building realistically social agents
• there are also rich signals in conversational behaviors to infer user satisfaction with the functional performance of CAs, especially for signaling frustration.
• Compared to query-based information systems, users are more likely to reveal their emotional status in conversational interactions to provide real-time feedback. (리얼타임으로 Feedback을 줘야 하기 때문에 query-based information system과 비교 했을때, 사용자는 emotional status를 더 reveal한다) (★★★★★)
• playfulness as a desirable characteristic in computer interactions because it promotes adoption, satisfaction and learning outcome
• playfulness was considered as a“point of entry”
• playful behaviors may decline in the long run (장기적인 사용에서는 감소)
• supporting playful interactions can enhance adoption
• promote and sustain playfulness in the long run 할수 있는 방법들
  • proactive social interaction / manifesting personality (명백한 personality)/ continuously revealing new features and responses / avoiding complexity / providing transparency and user control for system status
• monitoring the use of pronouns. By detecting these signals,a CA may adapt its interaction style.
• playful interactions may also serve as affordance cues, where over boasting of “social smarts” can belie (잘못전달함) the true system capabilities and raise incorrect user expectations
• it may be an opportunity to communicate about the system’s functional scope when users initiate chitchat about agent’s traits and status (Agent가 Functional Scope에 설명하는 것이 좋음. Transparency관련 )