Image-to-video generation with Vercel AI SDK
Use the Vercel AI SDK to send an image and prompt for video generation.
prompt: {
image: "[[image_url_or_data]]",
text: "[[animation_description]]"
} json
The prompt parameter accepts an object with image and text fields for image-to-video generation. The image field can be a URL string, base64-encoded string, Uint8Array, ArrayBuffer, or Buffer.
This snippet shows how to format a prompt for image-to-video generation using the Vercel AI SDK. It specifies that the prompt object should contain an 'image' field for the source image and a 'text' field for the animation instructions.
The Vercel AI SDK is designed to abstract away the complexities of different AI model APIs. By providing a structured prompt object with specific fields like 'image' and 'text', it ensures the model receives the necessary inputs in the correct format for image-to-video transformation.
- Image-to-Video ↗xAI · xAI Docs: Guides · Jul 30, 2026 · quoted as shared
First seen on Subvolts Jul 30, 2026. Updated Sep 11, 2026. Spotted a problem? Tell us.
- Image-to-VideoxAI · xAI Docs: Guides · Jul 30, 2026
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- Prompt patternPlatform-specific video prompt adjustmentInstruct AI to adapt video content for specific platforms like TikTok by adjusting pacing, tone, and visuals. · 5 min
- Setting tipUse AI agents for distinct video creation tasksEmploy specialized AI models as agents, each handling a specific part of the video production workflow. · 5 min
- HeuristicAdd imperfections for human-like AI videoIncorporate flaws like off-level framing to make AI-generated scenes feel more natural. · 5 min
- Code snippetConfigure grok-imagine-video-1.5 for reference-to-video generationSet providerOptions.xai.mode to 'reference-to-video' and pass reference image URLs. · 5 min
- Anti-patternRelying on manual prompt iteration for video generationAvoid iterating through many prompts; use AI to internalize cinematic decision-making. · 5 min
- Model pickGPT-4.1 mini vs. GPT-5 for specific AI tasksUse GPT-4.1 mini for deterministic tasks, GPT-5 for ambiguous interpretation and reasoning. · 5 min
Questions people ask
- Which AI tools does this work with?
- Any current model. It was seen with in the sources.
- Where did this come from?
- xAI (xAI Docs: Guides). Subvolts collected, normalized and credited it; the link above opens the original.