EchoRep

An AI powered MR Copilot for Sales Representative to provide smarter services.

—-XR Creator Con 2026 Finalist

At XRCC 2026, I joined a global team to work on this AI-powered MR(Mixed Reality) project made for smart glasses. Targeting Meta Ray Ban & Spectacles, we prototyped with Meta Quest series due to the device limitation. It listens to customer conversations, identifies the products being discussed, classifies customer intent in natural language, and overlays the right product specifications, comparisons, and upsell suggestions directly into the associate's field of view — in real time, eyes up, hands free.

The Problem

The cost of this knowledge gap is quantifiable and direct. Fisher, Krishnan, and Netessine (2006) at the Wharton School conducted one of the largest empirical retail operations studies, finding that trained sales associates were 46% more productive than untrained peers, generating significantly more sales per hour — with half of that gain attributable to product knowledge training specifically. Independent research from Experticity confirmed that sales associates with strong brand expertise sell 87% more than peers without it.

EchoRep is a system that delivers the knowledge advantage of a fully trained associate to every employee, on day one, without waiting for training to catch up.

Features

  • Scan Product: Check product information with QR Code scanning

  • Comparison: Compare products when scanning two more products

  • Ask AI: Proper response proposed from LLM. Quick answers to any questions related to products.

  • Dashboard: Provide stock and stats information.

As the XR Specialist on this team, I designed and integrated interactions with the 3D space UI. The system uses hand gestures and microgestures to navigate and manipulate the UI. Hand gestures and microgestures are an universal language shared among Meta Quest, Meta Ray Ban and Spectables. It is scalable to follow these to allow us to migrate the systems to smart glasses.

During tests, we found some people were confused when they use the microgestures for the first time. Therefore, I designed and implemented a tutorial where the user uses the swipe and tap gestures to finish a progress bar. Tooltip images and gesture indicators follow the hand rig to help them to learn the gestures, which made it intuitive and easy to understand.

A scalable and maintainable UI navigation system was built to collaborate with a comprehensive gesture input manager to navigate through interactables such as buttons, toggles and sliders. At the same time, a state-machine-based interface management system was engineered to maximize our ability to quickly scale up and integrate multiple interfaces for new features.

LLM

We utilized Groq API to integrate LLM feature, providing smart communication suggestions between Sales representatives and customers. This is a great practical use case with LLM to solve a real world problem and improve customer experience. We tried different LLM such as Opus and Gemini. We finally chose Gemini 3.1 Flash lite to balance between the speed and quality. It was also a great practice to integrate LLM to a product where prompts are carefully designed and provide clear context for LLM to generate proper response.

Award and Potential

I’m thrilled that EchoRep was chosen as one of the finalists on XRCC 2026. We acknowledged that this project has a lot of potentials to integrate on actual smart glasses and provide smarter sales experience on different products. The key is to make the LLM feature reliable and trustworthy on a long standing smart glasses. Although we faced many challenges and limitations, I’m proud of being one of this global team and contributed to this project.