Calm, Hands-Free Teaching: Torso–AR–Ear-EEG Fusion for Stress-Aware Robot Instruction

Song et al. (2023, RO-MAN) show that torso dynamics (via an instrumented seat and IMU) can precisely control robots hands-free. Jiang et al. (2024, HRI) find that AR-based kinesthetic teaching excels for collecting demonstrations from non-experts. Mai et al. (2024, Cyberworlds) demonstrate that wearable ear-EEG can monitor emotion and cognitive load in HRI. We combine these: users teach a robot via AR “virtual kinesthetic” guidance while seated; torso lean and rotation control coarse motion and selection, freeing the hands for task objects; ear-EEG tracks moment-to-moment load/anxiety, guiding interface adaptation (e.g., slowing the pace, simplifying prompts, or offering an LLM-generated micro-tutorial). Given that LLM-powered robots can raise expectations and anxiety (Kim et al., 2024, HRI), we design the conversational layer to be calm and grounding, with clear task scaffolds. We measure subjective quality using human-centered metrics identified by Mizuchi et al. (2023) and evaluate throughput/accuracy of demonstrations. This challenges the assumption that effective robot teaching requires hand-held or direct physical manipulation. It also synthesizes physiological sensing with AR and conversational guidance (cf. Secco, 2025) for a more accessible, inclusive interface (beneficial for mobility-limited users). Impact: a stress-aware teaching modality that broadens who can program robots, improves demo quality under pressure, and sets a template for affect-adaptive HRI authoring tools.

References:

1. Designing Evaluation Metrics for Quality of Human-Robot Interaction in Guiding Human Behavior. Y. Mizuchi, Yusuke Tanno, T. Inamura (2023). International Conference on Human-Agent Interaction.
2. Understanding Large-Language Model (LLM)-powered Human-Robot Interaction. Callie Y. Kim, Christine P. Lee, Bilge Mutlu (2024). IEEE/ACM International Conference on Human-Robot Interaction.
3. Design and Validation of a Torso-Dynamics Estimation System (TES) for Hands-Free Physical Human-Robot Interaction*. Seung Yun Song, Yixiang Guo, Chentai Yuan, Nadja Marin, Chenzhang Xiao, Adam W. Bleakney, Jeannette Elliott, João Ramos, E. Hsiao-Wecksler (2023). IEEE International Symposium on Robot and Human Interactive Communication.
4. A Comprehensive User Study on Augmented Reality-Based Data Collection Interfaces for Robot Learning. Xinkai Jiang, Paul Mattes, Xiaogang Jia, Nicolas Schreiber, Gerhard Neumann, Rudolf Lioutikov (2024). IEEE/ACM International Conference on Human-Robot Interaction.
5. Interactive Conversational AI with IoT Devices for Enhanced Human-Robot Interaction. E. Secco (2025). Journal of Intelligent Communication.
6. Wearable Ear EEG Device for Emotion Recognition in Human-Robot Interaction. Ngoc-Dau Mai, Kentaro Go, Xiaoyang Mao, Wan-Young Chung (2024). International Conference on Cyberworlds.