At XRCC 2026, I joined a global team to work on this AI-powered MR(Mixed Reality) project made for smart glasses. Running on Meta Quest series, it listens to customer conversations when turned on & given permissions, identifies the products being discussed, classifies customer intent in natural language, and overlays the right product specifications, comparisons, and upsell suggestions directly into the associate's field of view — in real time, eyes up, hands free. We choose Washing Machine as our product as the proof of concept.

The Problem

The cost of this knowledge gap is quantifiable and direct. Fisher, Krishnan, and Netessine (2006) at the Wharton School conducted one of the largest empirical retail operations studies, finding that trained sales associates were 46% more productive than untrained peers, generating significantly more sales per hour — with half of that gain attributable to product knowledge training specifically. Independent research from Experticity confirmed that sales associates with strong brand expertise sell 87% more than peers without it.

EchoRep is a system that delivers the knowledge advantage of a fully trained associate to every employee, on day one, without waiting for training to catch up.

Implementation

As the XR Specialist on this team, I’m responsible for designing and integrating interactions with the UI in the 3D space. The system uses multiple hand gestures and micro gestures to navigate and manipulate the UI. Hand gestures and micro gestures are a universal language shared among Meta Quest and Meta Ray Ban. It is scalable to use these features to allow us to migrate the systems to Meta smart glasses in the future.

A scalable and maintainable UI navigation system was built to collaborate with a comprehensive gesture input manager to navigate through interactables such as buttons, toggles and sliders. At the same time, a state-machine-based interface management system was engineered to maximize our ability to quickly scale up and integrate multiple interfaces for new features.

Considering the scenario where the sales representative needs to walk around and introduce machines to customers, the UI needs to automatically follow the salesman and as intuitive as possible to use. Therefore, I developed feature to automatically update the UI’s position and rotation based on user’s position and forward direction. It can smoothly moves and rotates when the user moves or rotates beyond a certain threshold.

Award

I’m thrilled that EchoRep was chosen as one of the finalists on XRCC 2026 and we aim to bring trophy from the final on June 30th.