Gemini 3 is built to accept multiple types of inputs—text, images, audio, video, and PDFs—in a single request. The API lets you send these inputs as separate “parts,” and the model processes them together as one structured context. In real applications, the model can look at a screenshot, read accompanying text, and interpret a user’s question all at once. This is not a bolt-on feature; it is part of the core architecture, allowing the model to reason across different content formats without needing separate preprocessing pipelines.
Developers can use this in practical ways. For example, you can send a product screenshot and ask for accessibility issues, attach a long PDF and ask for comparisons against a policy document, or submit a meeting video and request an action summary. The model handles alignment across text and visual elements, meaning it can refer to what it “sees” in an image while using the surrounding text as guidance. This reduces the need for external tools like OCR or image classifiers, because Gemini 3 directly understands the multimodal content.
When combined with retrieval, multimodal processing becomes even more powerful. Suppose you index PDF pages, slide images, or video stills in a vector database such asMilvus or Zilliz Cloud.. At query time, you retrieve the most relevant pages or key frames and feed them to Gemini 3 along with the user’s prompt. The model can reason across these inputs as a connected set of evidence. This makes workflows like document review, UI analysis, or multimedia summarization much easier to implement in a production environment.