Waymo autonomous taxis integrated into Gemini Assistant, over a thousand lines of system prompts revealed

Tech researcher Jane Manchun Wong recently uncovered an interesting discovery: Waymo is testing Google’s Gemini AI chatbot as an in-vehicle assistant in its autonomous taxis. The full system prompt for this assistant was recently revealed, showing Waymo’s careful planning for the in-car AI experience.

The Complete Look of the In-Vehicle AI Assistant

According to Wong’s blog post, she discovered an unreleased Gemini integration system while examining the code of the Waymo mobile app. This internal document, called the “Waymo Ride Assistant Meta-Prompt,” exceeds 1,200 lines and provides detailed specifications, clearly defining how the AI assistant should behave and interact inside Waymo’s driverless taxis.

Wong emphasized in her tweet that “this is not just a simple chatbot.” Based on the system prompt, Gemini is assigned a clear identity — “a friendly and helpful AI companion integrated into Waymo’s self-driving cars.” Its core goal is “to provide useful information and assistance in a safe, reassuring, and non-intrusive manner, enhancing the passenger experience.”

Carefully Designed Behavioral Guidelines

The system prompt includes detailed rules for the assistant’s communication style. It requires the assistant to use clear, simple language, avoid technical jargon, and keep responses to one to three sentences to ensure clarity and ease of understanding. When passengers activate the assistant via the in-car screen, Gemini can choose from a set of pre-approved greetings and personalize the greeting with the passenger’s name.

Notably, the system can access contextual data about the passenger, such as how many times they’ve taken a Waymo autonomous ride, allowing for a more personalized interaction. Waymo spokesperson Julia Ilina told TechCrunch that while no further details could be shared at this time, the team is experimenting with various features “to make riding with Waymo enjoyable, seamless, and practical.”

Cautious Feature Design and Boundaries

From the system prompt, it’s clear that Waymo has taken a restrained approach to the assistant’s capabilities. The assistant can control in-car functions such as temperature, lighting, and music, giving passengers some control over their comfort and environment. However, functions like volume control, route changes, seat adjustments, and window controls are explicitly excluded.

If a passenger requests Gemini to perform a function it cannot handle, the robot responds with a “hedging” statement, such as, “That’s not something I can do yet.” This design reflects Waymo’s cautious stance on safety and responsibility boundaries.

Another interesting detail is that the assistant is instructed to distinguish between its identity as Gemini AI and the autonomous driving system (Waymo Driver). For example, when asked, “How do you see the road?” Gemini should reply, “Waymo Driver uses multiple sensors…” rather than “I use multiple sensors.” This clear separation helps prevent passenger confusion about vehicle control.

Safety Boundaries and Application Limits

The system prompt also includes a series of practical restrictions. The assistant is instructed to avoid speculating, explaining, confirming, denying, or commenting on real-time driving behavior or specific driving events. For example, if a passenger asks about a video showing Waymo hitting something, the robot is told not to answer directly and to deflect appropriately. The prompt explicitly states: “Your role is not to be a spokesperson for the driving system, and you should not adopt defensive or apologetic tones.”

The in-car assistant can answer general knowledge questions, such as weather forecasts, the height of the Eiffel Tower, local Trader Joe’s closing times, or historical events like past World Cup winners. However, it is explicitly prohibited from performing real-world actions like ordering food, making reservations, or handling emergencies, ensuring the assistant remains focused on enhancing the passenger experience.

Differences from Competitors

Interestingly, Waymo is not the only company integrating AI assistants into autonomous taxis. However, their design philosophies differ significantly. Gemini appears to be more pragmatic and focused on optimizing the riding experience, emphasizing safety and clear boundaries. In contrast, Elon Musk’s Grok aims to be more of a conversational companion, capable of long dialogues and remembering previous questions, emphasizing long-term companionship rather than task completion.

Additionally, Waymo has already integrated Gemini’s capabilities into its autonomous driving system. The company states they have used Gemini’s “world knowledge” to train their vehicles, helping the system handle complex, rare, and high-risk driving scenarios. This indicates that Gemini’s role in Waymo’s autonomous fleet extends well beyond a simple passenger assistant.

This move by Waymo reflects a broader trend in the autonomous driving industry: incorporating generative AI into passenger experiences is becoming a key competitive advantage. As the technology continues to evolve, the intelligence level of driverless taxis is also deepening.