Daily Note: Nonverbal Communication and Intelligent Systems

These notes are a summary of concepts presented in “Nonverbal Cues in Human–Robot Interaction: A Communication Studies Perspectives.”

Jacqueline Urakami and Katie Seaborn. 2023. Nonverbal Cues in Human–Robot Interaction: A Communication Studies Perspective. ACM Trans. Hum.-Robot Interact. 12, 2, Article 22 (March 2023), 21 pages. https://doi.org/10.1145/3570169

  1. Nonverbal Communication
    • Communication is an interactive process where receivers assign meaning to stimuli transmitted by senders
    • Nonverbal communication involves messages that do not include words, such as gestures, facial expressions, and tone of voice
    • Psychological model: sender transmits a message; receiver decodes it
  2. Nonverbal Communication in HRI
    • Nonverbal communication enhances human-robot interaction (HRI) by making robots appear more humanlike
    • Nonverbal cues include gestures, postures, eye movements, and vocal intonations
    • Redundancy in nonverbal cues helps ensure message clarity
  3. Cultural and Contextual Aspects
    • Nonverbal cues are often culturally and generationally specific
    • Robots must account for cultural expectations, particularly in body shape, clothing, and gestures
  4. Physical Appearance (Nonverbal Cues)
    • Body types (endomorph, ectomorph, mesomorph) influence perceptions of robot capabilities
    • Clothing shapes first impressions and can align with robot roles
    • Kinesics (Body Movement)
      • Gestures: iconic (literal meaning), metaphoric (abstract meaning), deictic (pointing)
      • Posture: communicates attitude and relational status
      • Proxemics: spatial distance and its impact on communication
    • Facial Expressions
      • Display emotions or social responses (e.g., anger after task failure)
    • Oculesics (Eye Behavior)
      • Eye contact, pupil dilation indicating interest or focus
  5. Auditory (Nonverbal Codes)
    • Paralanguage
      • Vocal qualities (pitch, tone), vocalizations (groans, giggles), and fillers (“uh,” “um”)
    • Chronemics
      • Use of time, such as pauses, impacts perception of communication
  6. Tactile and Other Sensory Nonverbal Codes
    • Haptics
      • Touch as a communication tool, balancing intimacy with cultural rules
    • Olfactics (Smell)
      • Scents may enhance robot roles but are underexplored
    • Gustatory Sense (Taste)
      • Relevant for robots assisting with feeding tasks
  7. Designing for nonverbal cues and interaction
    • Continuous nonverbal cues are hard to achieve due to their fluid nature
    • Misinterpretations of robotic nonverbal cues can result from technical limitations or cultural mismatch
    • Emotional cues, such as a robot’s voice tone, must align with user expectations to avoid confusion
    • Contradictory cues (e.g., gestures vs. speech) undermine communication impact
    • Robots should exhibit sensitivity, emotion, and intuition to succeed as social agents (kokoro function)
    • Physiological cues (e.g., pupil dilation, posture changes) can enhance humanlike interactions