Daily Note: Mixed Reality, Empathic Computing, and Agent Technologies

These notes are a summary of concepts presented in “EMiRAs-Empathic Mixed Reality Agents.”

Zhuang Chang, Yun Suen Pai, Jiashuo Cao, Kunal Gupta, and Mark Billinghurst. 2024. EMiRAs-Empathic Mixed Reality Agents. In Proceedings of the 3rd Empathy-Centric Design Workshop: Scrutinizing Empathy Beyond the Individual (EmpathiCH ’24). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3661790.3661791

  1. . Empathic Mixed Reality Agents
    • Convergence of Empathic Agent, Mixed Reality Agent, and Empathic Mixed Reality
  2. Key Concepts
    • Corporeal Presence and Interactive Capacity (CPIC) Matrix
      • Tool to evaluate agent embodiment and interaction capabilities
    • Empathic communication: Agents sensing and acting on human users
    • Creating nuanced interactions in both virtual and physical domains
  3. Empathic Mixed Reality Systems
    • Purpose of fostering deeper empathy and understanding between people and between people and environments
    • Integration of physiological sensors (e.g., EMG, EEG, EOG, EDA, PPG) to detect user states
    • Role of haptic devices (e.g., controllers, gloves) in creating an empathic touch
  4. Agent Capabilities
    • Sensing
      • Virtual environment, physical environment, and mixed reality environment
    • Embodiment
      • Virtual, physical, and mixed reality
    • Perception-Action Model
      • Empathic appraisal – perceiving emotional cues
      • Empathic response – generating emotions in response to cues
  5. Classification of Empathic Mixed Reality Agents
    • Types based on environment interaction
      • Virtual embodiment, virtual interaction
      • Physical embodiment, physical interaction
      • Mixed reality embodiment
        • Virtual and physical environment interaction
    • Agent-Environment Interaction
      • Interaction with human users
      • Interaction with non-human objects
  6. Agent Design for Empathy
    • Importance of building sensing and action capabilities using advanced technologies
      • Eye tracking, motion tracking, position tracking
      • Physiological sensors for implicit understanding
    • Eliciting empathy from humans towards agents and vice versa
  7. CPIC Matrix and Interaction Levels
    • Definitions of corporeal presence and interactive capacity
    • Matrix classifications
      • Virtual embodiment vs. physical embodiment
      • Interaction in single or mixed environments
    • Examples of interaction levels
      • Virtual embodiment interacting with physical non-human objects
      • Mixed reality embodiment with a focus on human interaction