Visual reasoning
Also written: Visual intelligence, AI vision, Visual patches, Visual thinking
AI's ability to understand and reason about visual information like images or screen content.
Visual reasoning is how artificial intelligence systems can understand and make sense of what they see. This goes beyond just recognizing objects; it involves understanding the relationships between them and the context of the scene.
For example, imagine showing an AI a picture of a table with a fork and a knife. Visual reasoning allows the AI to understand that the fork and knife are eating utensils and that they are placed on the table, ready for a meal.
Visual reasoning refers to an AI model's capacity to interpret, analyze, and draw logical conclusions from visual data, such as images, videos, or screen displays. This involves understanding not only the presence of objects but also their spatial relationships, actions, and the overall scene context.
Advanced models can perform tasks like visual question answering (VQA), image captioning, and scene understanding, often leveraging transformer architectures adapted for multimodal inputs. Tradeoffs include computational cost and the potential for biases inherited from training data.
Recent Google Gemini updates highlight its ability to directly interact with computer interfaces, suggesting a leap in visual reasoning. This allows AI to understand and control digital environments, moving beyond just interpreting static images to actively engaging with dynamic screen content for tasks like automation and testing.
- কম্পিউটার ব্যবহারGoogle · Gemini Models ·
- Gemini 2.5 Computer Use 模型 | Gemini APIGoogle · Gemini Models ·
- Computer UseGoogle · Gemini Models ·
- Использование компьютераGoogle · Gemini Models ·
- Përdorimi i kompjuteritGoogle · Gemini Models ·
- Korzystanie z komputeraGoogle · Gemini Models ·
- Sử dụng máy tínhGoogle · Gemini Models ·
- A state-of-the-art model that makes precise, immediately usable visualsOpenAI · OpenAI Research ·
- Nano Banana Pro: Hands-on with the World’s Most Powerful Image ModelGoogle · Google AI: Release Notes ·
- Sora System CardOpenAI · OpenAI News ·
- Fei-Fei Li: Human-Centered AIMicrosoft · Behind The Tech with Kevin Scott ·
- Google Gemini
- Google Gemini's recent advancements focus on its "Computer Use" capability, enabling the AI to directly interact with and control digital interfaces like software and browsers.
- OpenAI ChatGPT
- OpenAI's recent work on image generation, like ChatGPT Images 2.0, shows progress in creating precise and usable visuals, implying advancements in how AI understands and produces visual information.
- Image recognition
- This focuses on identifying objects within an image, but does
- Computer vision
- A broader field encompassing image recognition and visual reaso
- Natural language processing
- This deals with understanding and generating human
The concept of visual reasoning in AI emerged from the fields of computer vision and artificial intelligence, aiming to replicate human cognitive abilities to understand the visual world. Early work dates back decades, with significant progress in recent years driven by deep learning.
Use an AI chatbot that supports image input. Upload a picture and ask the AI to describe what is happening in the image, identify specific objects, or explain the relationships between elements. For example, ask it to describe a scene from a vacation photo.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- HeuristicVisual patches as scalable representation for AIUse visual patches, similar to text tokens, for scalable training of generative AI models on visual data. · 5 min
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
Questions people ask
- Can AI truly 'see' like humans?
- AI can process and interpret visual data to a high degree, but it doesn't 'see' or have subjective experiences in the way humans do. Its understanding is based on patterns learned from data.
- What's the difference between visual recognition and visual reasoning?
- Visual recognition is about identifying objects or features. Visual reasoning adds the ability to understand the context, relationships, and implications of what is seen.
First seen on Subvolts Jun 21, 2019. Updated Oct 17, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.