Daily Note: Agent Personality and Contextual Triggers

These notes are a summary of concepts presented in “Incremental Acquisition and Reuse of Multimodal Affective Behaviors in a Conversational Agent.”

Maike Paetzel, James Kennedy, Ginevra Castellano, and Jill Fain Lehman. 2018. Incremental Acquisition and Reuse of Multimodal Affective Behaviors in a Conversational Agent. In 6th International Conference on Human-Agent Interaction (HAI ’18), December 15–18, 2018, Southampton, United Kingdom. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3284432.3284469

  1. Affect and Behavior
    • Availability of content and behaviors for various affect states
    • Predict perceived affective state based on behavior
  2. Russell Affect Model
    • Framework for understanding and categorizing affective states
  3. Challenges in Conversational Design
    • Corpus creation for multimodal behaviors
      • Need for diverse content supporting affect-driven narratives
      • Ensuring choice in conversational turns for any affective state
    • Inconsistencies in agent personality
      • Inter-dialogue inconsistencies due to lack of personality markers
      • Intra-dialogue inconsistencies from affect changes across turns
      • Limited history for crowd workers leading to inconsistent affect development
    • Data-driven approaches
      • Reliance on corpora (authored sentences or stochastic models)
      • Occasional inconsistencies in real interactions
    • Signaling Affect
      • Integration of conversational content, non-linguistic utterances, and full-body behaviors
      • Crowdsourcing affective behavior (video, written content, affective labeling)
  4. Strategies for Affect-Driven Narratives
    • Dialogue authoring and editing
      • Creating diverse affective states and intensities
    • Content reuse
      • Reuse lines from similar situations while adhering to affective constraints
    • Prediction and modeling
      • Reuse content across personalities while predicting perceived affect
  5. Multimodal Delivery
    • Broadening affect intensity via voice tone and gestures
    • Reducing corpus size by strategically enriching multimodal behaviors
    • Predicting perceived affect for re-situated content
  6. Crowdsourcing
    • Crowd workers struggle with unfamiliar contexts and target affect constraints
    • Affect-driven narratives produce broader affect spectrums
  7. Implementation
    • Reusing lines and multimodal enrichment reduces corpus size
    • Predicting infrequent “affective flips” is a challenge
      • Rare flips can disrupt perceived intelligence and agency
    • Class imbalance hinders precision in predicting flipped or rejected lines
  8. Semi-Situated vs. Fully-Situated Perception
    • Differences in online and face-to-face affect perception
    • Importance of social presence over physical presence