VLA-Based XAI HMI Design
- Year
- 2026
- Field
- Physical Computing, HMI, Research
- Context
- 10 weeks · 10-member team · Design Lead · Figma, Figma MCP, Git · ACM AutomotiveUI 2026 Student Research
A passenger-in-the-loop testbed for SOTIF error situations in L5 automated vehicles. The interface uses a VLA (Vision-Language-Action) model so the vehicle can diagnose functional insufficiency and explain the cause to passengers in natural language. As Design Lead, I established the UX guidelines and developed the real-time synchronized multimodal web application.
Even in SAE Level 5 driving, SOTIF errors can occur due to algorithmic limits or perception gaps, leaving passengers in extreme uncertainty if the system works as a black box. This project designs a Situation Awareness (SA)-based multimodal interface built on Explainable AI.
Two SOTIF scenarios without mechanical faults were defined to create cognitive uncertainty. Three HMI conditions were compared: No HMI, Voice Only, and Multimodal (visual + voice).
The research established a self-explaining interface through interdisciplinary collaboration and a dedicated Passenger-in-the-Loop testbed. The paper was submitted to the Student Research Track of ACM AutomotiveUI 2026.
