Google Research and DeepMind unveil AMIE, a multi-agent medical AI for real-time video consultations

Google Research and DeepMind are taking a step toward “real-time” health AI with AMIE, a research medical system designed to handle the kind of back-and-forth clinicians experience in a video visit, not just text-based symptom summaries.

In a first-of-its-kind demonstration described in a Google Research blog post, AMIE is built on Gemini and Project Astra and uses a multi-agent architecture, a setup in which multiple AI “agents” coordinate to perceive inputs and carry out different subtasks. Google says AMIE can interpret both visual and auditory cues during a consultation, guide virtual physical exams, and reason diagnostically in real time.

The company framed the project around what it said is often missing from current digital triage tools: clinicians do not only ask questions, they also observe visible signs of illness and register nonverbal cues such as discomfort, movement, and speech patterns. Google said AMIE is meant to extend the consultation beyond words by incorporating those observations into its ongoing assessment.

To evaluate AMIE, Google described a randomized study using simulated consultations with patient actors, paired with a group of primary care physicians. Clinical evaluators then assessed AMIE favorably across several core competencies, including history-taking thoroughness, diagnostic accuracy, management appropriateness, and communication quality.

In addition to evaluator ratings, Google said the patient actors preferred the video experience over text chat. The preference result points to an acceptance challenge beyond medical accuracy, namely whether patients find an AI-mediated consultation usable and clear when the interaction is visual and auditory rather than purely conversational in text.

Google emphasized that AMIE remains a research system. It said more work is required before responsible real-world clinical deployment, underscoring that performance in simulation does not necessarily translate to operational safety, regulatory compliance, or robustness in everyday clinical settings where information quality can vary widely.

The announcement arrives as Alphabet continues to broaden Gemini and other AI capabilities into multimodal applications, where systems combine text, images, and audio to handle tasks that more closely match human perception. Project Astra, mentioned by Google in the AMIE description, is part of this broader push toward real-time, context-aware AI assistance, although Google did not provide further operational details about Astra’s role inside AMIE beyond the system’s stated foundation.

Within healthcare, real-time video consultation raises technical and ethical expectations at once. A system must integrate observations and questions as they unfold, maintain internal consistency during an interview, and present recommendations in a way that clinicians can interpret and patients can understand. Google’s decision to describe multiple evaluation categories, rather than only diagnostic accuracy, suggests it is also targeting the “workflow” aspects of clinical communication and management decisions, not just recognition of symptoms.

Even so, the company did not disclose several specifics that readers will likely want to see as AMIE moves toward any future product discussions, including the number of study participants, the size and characteristics of the simulated case set, how “real-time” was defined in experimental terms, and the exact scoring methodology used by clinical evaluators. Google also did not detail how the system would behave with missing, noisy, or adversarial inputs, which are common in real video visits.

The next test for AMIE, if Google pursues it further, is whether it can perform reliably across broader patient populations and clinically realistic conditions, while meeting safety constraints and regulatory requirements. For Alphabet, the near-term value of the work is likely less about immediate clinical use and more about demonstrating that multimodal, coordinated AI agents can handle the temporal structure of a consultation, which could influence how health AI features are designed going forward.