NVIDIA opens public beta for XR AI, a developer toolkit aimed at agentic AI on smart glasses

NVIDIA is pushing agentic artificial intelligence out of the conversation layer and closer to real work. In a new post, the company announced that NVIDIA XR AI is now available in public beta, positioning the software framework as a way for developers to build multimodal AI agents for augmented reality (AR) glasses and other XR devices.

The core idea is to give agents more than the ability to answer questions. NVIDIA says these systems must perceive the physical world through video, audio, and sensors, understand fast-changing conditions and spatial context, retrieve relevant information from enterprise systems, and then reason about and execute the next best action. The company also emphasizes low latency and a design requirement to support users without causing distraction.

NVIDIA XR AI is described as a developer library that connects AR and XR inputs to AI models, enterprise data, tools, and accelerated computing. In practical terms, the toolkit aims to simplify how developers stitch together multimodal perception with enterprise retrieval, reasoning models, and “agent orchestration,” the logic that coordinates skills and tool use so the agent can complete tasks.

The platform brings together four core capabilities. NVIDIA points to NVIDIA NeMo Agent Toolkit as the component intended to enable tool use, reasoning workflows, and multi-agent coordination. It pairs that with NVIDIA’s accelerated computing options, including DGX Spark, DGX Station, and RTX PRO systems, which the company says are used to run inference across cloud, data center, and edge environments.

NVIDIA’s pitch is that the combination enables spatially aware agents that can access knowledge and act in real time, with the expectation that the computational load can be matched to where the device and application run. The company says developers and enterprises are already embedding XR AI based agents in manufacturing, science, healthcare, design, and immersive learning use cases.

Among the examples, Siemens is exploring XR AI and DGX Spark in a research setting to help factory engineers locate maintenance information, troubleshoot issues, verify work, and record outcomes on the shop floor. NVIDIA’s account describes a scenario where an engineer wearing lightweight glasses could ask about a programmable logic controller issue and receive real-time guidance, linking industrial systems, digital twins, and automation workflows.

In life sciences, NVIDIA cites AutoBio and a system called LabOS. The post says LabOS introduces “spatial intelligence” into scientific workflows using the XR AI architecture. According to NVIDIA, LabOS provides hands-free guidance for complex experimental steps, starting with areas such as stem cell therapy and gene-editing research. The company describes the system as a “co-scientist” that helps researchers identify the right sample and CRISPR gene editor, guide each step, and capture a structured, reproducible record while humans, robots, and AI collaborate at the bench.

The company also points to healthcare, including a demonstration by the Surreality Lab at the University of Pittsburgh Medical Center. NVIDIA says the pipeline, running on XR AI and DGX Station, is designed to help surgical teams find information and guide attention while minimizing visual clutter for the surgeon. The post adds that the system is intended to understand what not to occlude in the surgeon’s view, so useful context can be surfaced without pulling focus from the patient and procedure.

For caveats, NVIDIA does not lay out performance benchmarks, battery or hardware constraints for specific glasses models, or detailed availability terms for the public beta. The company also does not specify which enterprise connectors, model choices, or security and privacy controls are available by default, nor does it quantify latency targets or the reliability of multimodal agent actions in uncontrolled environments.

Looking ahead, developers will likely watch how NVIDIA XR AI’s beta matures around integration, device support, and real-world deployment. The post highlights compatibility with smart glasses from Meta, Rokid, and VITURE, and it lists multiple application partners across industry and research, but it leaves open how quickly developers can move from prototypes to production and what standards for safety, permissions, and evaluation NVIDIA will require as agentic capabilities expand.