XR
Also written: XR Blocks, XR headset, Extended Reality headset
Extended Reality (XR) merges virtual, augmented, and mixed reality, creating immersive digital experiences that blend with the physical world.
Extended Reality, or XR, is a broad term for technologies that blend the real and virtual worlds. Think of it as an umbrella term covering virtual reality (VR), augmented reality (AR), and mixed reality (MR).
VR completely immerses you in a digital world, like playing a video game with a headset. AR overlays digital information onto the real world, like a navigation app showing directions on your phone's camera view. MR takes AR a step further, allowing digital objects to interact with your real surroundings. Google's AgentHands project, for example, uses XR to make AI agents interact more naturally with people in these blended environments.
Extended Reality (XR) is an umbrella term encompassing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). VR provides full immersion in a simulated environment, typically via a headset. AR overlays digital information onto the user's view of the real world, often through mobile devices or smart glasses. MR allows digital objects to interact with and be perceived within the physical environment, enabling a more seamless blend of digital and physical elements. The AgentHands project exemplifies XR's application in human-AI interaction, aiming for more intuitive and spatially aware communication.
XR is gaining traction as AI models become more capable of generating interactive and context-aware experiences. Projects like Google's AgentHands demonstrate how advanced AI can enhance the realism and utility of XR applications, moving beyond simple displays to dynamic interactions. This integration promises more natural and effective ways for humans to engage with both AI and digital information in shared spaces.
- AgentHands: Generating interactive hand gestures for spatially grounded agent conversations in XRGoogle · Google Research Blog ·
- Does Witholding Claude Mythos Even Matter?Anthropic · AI Inside ·
- Vibe Coding XR: Accelerating AI + XR prototyping with XR Blocks and GeminiGoogle · Google Research Blog ·
- Phantom DatacentersOpenAI · AI Inside ·
- Google XR Director Juston Payne: Project Aura, Android XR, Lessons from Google GlassGoogle · AI Inside ·
- Samsung Galaxy XR Gets RealAnthropic · AI Inside ·
- Google Gemini
- Google's AgentHands project explores generating interactive hand gestures for spatially grounded agent conversations within XR environments.
- Virtual Reality (VR)
- VR immerses users in a completely digital environment, whe
- Augmented Reality (AR)
- AR overlays digital elements onto the real world, while
- Mixed Reality (MR)
- MR allows digital objects to interact with the real world, a
The term Extended Reality (XR) was popularized by Jeremy Bailenson, a founding director of Stanford's Virtual Human Interaction Lab, around 2016. It serves as an inclusive term for all immersive technologies. Source ↗
While direct XR interaction with current frontier AI models is limited, you can explore AR features on your smartphone. Use an AR measurement app to see how digital objects are placed in your real space. Then, imagine an AI assistant providing real-time information or guidance overlaid on your view.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- WorkflowAgentHands workflow for generating embodied XR agent gesturesSteps to generate synchronized, spatially grounded hand gestures for XR agents based on LLM responses. · 15 min
- System promptSystem prompt for Gemini to teach XR Blocks architectureTeaches Gemini the XR Blocks architecture, samples, and best practices for XR development. · 2 min
- ChecklistXR Blocks Gem evaluation checklist for Vibe Coding XRChecklist for evaluating XR Blocks Gem v0.11.0 using the VCXR60 dataset. · 30 min
- ToolVibe Coding XR workflow for AI + XR prototypingAccelerates AI and XR prototyping by translating prompts into interactive WebXR apps. · 5 min
- WorkflowRapid XR prototyping with Vibe Coding XR and GeminiTranslate natural language prompts into interactive XR apps using Gemini and XR Blocks. · 10 min
Questions people ask
- What is the main difference between VR, AR, and MR?
- VR fully immerses you in a digital world. AR overlays digital information onto your view of the real world. MR allows digital objects to interact with your real surroundings.
- How does AI relate to XR?
- AI can power more intelligent and interactive XR experiences, such as virtual characters that respond realistically or AR systems that understand and augment your environment.
First seen on Subvolts Oct 22, 2025. Updated Oct 12, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.