
A shopper photographs a running shoe and asks which brand makes it and whether it suits flat feet. The answer engine has to read the photo, then match it to reviews and product data. That job needs multimodal AI. It also means your product photos and videos now feed answers alongside your text.
A modality is a type of data: text, images, audio, video or documents. A multimodal model handles more than one. Early systems chained separate tools. OpenAI’s first ChatGPT voice mode, for instance, ran speech-to-text, then a text model, then text-to-speech.
Newer models are trained across modalities at once. OpenAI described GPT-4o in May 2024 as one network for text, vision and audio. Google built Gemini as a multimodal model from the start. So one model can look at a photo and reason about it in words.
Each input type passes through an encoder that turns it into embeddings. Images and words land in the same vector space, so a photo of a trail shoe sits near the words that describe it. That’s how a model matches a picture to a product name or a review.
Multimodal models are a branch of foundation models, trained on paired data such as images with captions or videos with transcripts.
| Surface | Input | What it means for brands |
|---|---|---|
| Google Lens | Camera photo or screenshot | Products and places identified from images |
| Search Live in AI Mode | Live camera and voice | Questions about what the user is looking at |
| Google’s new Search box | Images, files, videos, tabs | Any of these can start a query |
| ChatGPT and Gemini apps | Uploaded images and voice | Comparisons built from a photo or a PDF |
| Shopping answers | Product images and feeds | Cards with image, price and merchant |
Volume is already large. In October 2024, Google said Lens handles nearly 20 billion visual searches every month. At I/O in May 2026, it began rolling out a Search box that takes text, images, files, videos or Chrome tabs as input.
Product answers are turning visual. Shopping results in AI answers show cards with an image, a price and a merchant. Those details come from product feeds and page markup, so a missing image or a stale price lands straight in the answer. Feeds managed in Google Merchant Center are one of the main sources for Google’s shopping results.
Video counts too. YouTube had an 11.6% share of third-party citations in AI answers. Source: Ansvisor tracking data, September 15 to October 14, 2026. Transcripts and chapter titles turn a video’s content into text that search systems can index.
Our playbook on AI search strategies by industry covers the retail side. It calls for product structured data and a Merchant Center feed that match the visible page.
Ansvisor’s AI Shopping Analytics tracks the product cards that appear in AI answers on ChatGPT, Copilot, Perplexity and Google. It shows which of your products surface, which merchants are attached and which prompts trigger cards, by platform and region. Competitor product counts sit next to yours, so you can see where their products appear and yours don’t.
Each prompt you win brings more mentions, more citations and more buyers. Ansvisor keeps finding the next one.
Multimodal AI is an AI system that can take in or produce several data types, such as images and text, within one model. GPT-4o and Gemini are examples. You can show them a photo and ask a question about it in words.
Google Lens is the best-known one: you point a camera at an object and get results. Google said in 2024 that Lens handles nearly 20 billion visual searches a month. AI Mode also accepts photos and files, and Search Live lets you talk with Search about what your camera sees.
Generative AI describes what a model does: it creates new content. Multimodal describes which data types it handles. Many current models are both, such as one that reads a product photo and writes a description of it.
Images, video and product data become inputs that AI answers read and sometimes cite. Descriptive alt text, accurate product feeds and video transcripts make visual content easier for these systems to interpret.
Multimodal models can read text in images and describe what they show. Search systems still lean on page text, alt text and structured data to index and rank pages. Facts such as prices or specs should also appear as plain text on the page.
See how AI answers describe your brand, find out why competitors get picked, and close the gaps with AI agents that work alongside your team.
14-day free trial or 50% off your first month with code SAVE50.
Continue exploring key AI visibility concepts.
Measure and improve how often your brand appears in AI-generated answers.
Learn more →Strategies for increasing visibility in answer engines and AI summaries.
Learn more →Optimizing content for AI-powered discovery experiences.
Learn more →Understand how OpenAI retrieves and synthesizes information.
Learn more →AI-generated summaries that appear directly in Google Search.
Learn more →Explore how Perplexity cites and presents sources.
Learn more →References and sources used by AI systems to support answers.
Learn more →Measure the quality and influence of cited sources.
Learn more →How easily AI systems can discover and reuse your content.
Learn more →New terms are added regularly.
Help us improve the page or suggest a new term →
Co-founder at Ansvisor
Cihan Geyik is the co-founder of Ansvisor, an open-source AI Visibility platform for AI Search. With more than 15 years of experience in digital marketing and growth, he writes about AI visibility, AI search, AEO, GEO, citations, and answer engines. He focuses on helping brands understand and improve their presence across ChatGPT, Gemini, Google AI Overviews, Bing AI Mode and other AI answer engines.
Add Ansvisor as a preferred source and Google will show you more of our AI search, AEO and GEO guides.
© 2026 Ansvisor. All rights reserved. Ansvisor is an open-source AI Search Execution Platform for AI Visibility. Ansvisor is a brand owned and operated by Empler AI Inc.
The open-source AI Search Execution Platform. Track how AI answers talk about your brand and act on it.
Open source · MIT license© 2026 Ansvisor, a brand of Empler AI Inc. · 1401 PE Ave. Unit 105, Wilmington, DE 19806, United States
Cookie preferences
We use cookies to keep the site working and, with your OK, to measure traffic and improve our marketing. Read our Cookies & Tracking policy.