These notes are a summary of concepts presented in “Large Language Objects: The Design of Physical AI and Generative Experiences.”
Marcelo Coelho and Jean-Baptiste Labrune. 2024. Large Language Objects: The Design of Physical AI and Generative Experiences. interactions 31, 4 (July – August 2024), 43–48. https://doi.org/10.1145/3672534
- Definition of Large Language Objects
- Physical interfaces extending large language models into the physical world
- Provide multimodal input, end effectors, user feedback, world knowledge, and real-time context
- Inspired by situated robotics with integrated natural language and generative feedback
- Role and Capabilities of Large Language Objects
- Act as tools for shaping functionality and behavior
- Integrate design and fabrication processes
- Merge human and artificial intelligence in creative workflows
- Neurometric Design
- Enhances computation with more expressive, contextual, and accessible interactions
- Bridges the gap between digital and physical realms using descriptive, grounded language
- Generative Experiences and Dynamic Mental Models
- Shift from static mental models to on-demand, real-time generation
- Adapt to unforeseen user behavior and environmental changes
- Offer fluid and tailored functionalities for diverse users and scenarios
- Unique Opportunities in Generative Experiences
- Progressive interactions for both beginners and advanced users
- Focus on robust human communication and collaboration
- Enables inference through physical and contextual constraints
- Impacts and Applications
- Support hyper-personalization and market segmentation
- Promote new materials and fabrication techniques
- Facilitate novel ways for collaboration, communication, and teamwork