Daily Note: Affect-Aware Systems and Intelligent Emotion Recognition

These notes are a summary of concepts presented in “Affect-awareness framework for intelligent tutoring systems.”

Landowska, Agnieszka. (2013). Affect-awareness framework for intelligent tutoring systems. 2013 6th International Conference on Human System Interactions, HSI 2013. 540-547. 10.1109/HSI.2013.6577878.

  1. Definition and Purpose of Affect-Aware Systems
    • Software programs that recognize user emotions
    • Control mechanisms handle affect-related data to optimize interactions
  2. Handling Uncertainty in Emotional Recognition
    • Sources of uncertainty
      • Fuzzy nature of emotions
      • Insufficient algorithmic accuracy
      • Instability of emotions (frequent changes)
      • Limitations of representation models
    • Frameworks to manage uncertainty
      • Probability
      • Fuzzy set
      • Evidence
      • Possibilities
      • Interval analysis
      • Rough sets
    • Certainty factor quantifies algorithm confidence in results
  3. Components of Affect-Aware Systems
    • Affect Recognition
      • Video
      • Voice
      • Text
      • Physiological measurements
    • Multimodal input schemas ensure robustness
    • Algorithms
      • Gaussian Processes
      • Support Vector Machines (SVMs)
  4. Interpretation
    • Six basic emotions (e.g., joy, sadness, fear, etc.) as combinations (Ekman’s model)
  5. Discrete Models
    • Dimensional Models
      • Whissel Wheel
        • PAD model by Russel and Mehrabian
    • Trustworthiness of recognition and classification
    • Emotional stereotyping and user affect modeling
  6. Affect-Aware Reaction:
    • Affect-aware control mechanisms for interventions
    • Multimodal responses based on behavior libraries
    • Emotional states classified into action-triggering and non-triggering subsets
  7. Recognition and Representation Mechanisms
    • Emotion Recognition Algorithms
    • Multimodal approaches combining multiple data inputs
    • Conditions
      • Input collectability in the application environment
      • Output compatibility with chosen representation models
      • Certainty information
  8. Intervention Strategies and Mechanisms
    • Affective Intervention
      • Modifies standard control flow for counterproductive emotional states
      • Intervention scenarios include emotional state and system reaction pair
  9. Decision-Making:
    • Considers hypothesis certainty to minimize disruptive interventions.
    • Multimodal execution (text, audio, video)
  10. Patterns and Analysis
    • Emotional reaction patterns influenced by
      • Previous experiences
      • Task type
      • Neural system differences
    • Analysis opportunities:
      • Timeline trends (burn-down curves).
      • Pattern recognition in emotional responses to stimuli and tasks.
  11. Recognition Enhancements:
    • Lexical analysis: Keyword spotting, smiley detection, punctuation, capitalization heuristics.
    • Keystroke dynamics
    • Physiological response
  12. Decision Frameworks
    • Dempster-Shafer theory of evidence
    • Goal-Question-Metric method for system evaluation