Writing better prompts for image-to-video: camera moves, motion and start frames

Image-to-video models can take prompts that are too vague and interpret them in unexpected ways, like inventing motion that wasn't intended. However, by providing a clear start frame and a concise description of what should move, how the camera should move, and the pace of the action, the model can produce more predictable results. Here's a simple checklist for writing better prompts for image-to-video:

1. Let the start frame do the heavy lifting by showing the key composition elements. Avoid repeating these details in the prompt, as the model will likely re-imagine the scene.

2. Limit camera moves to one per clip, focusing on specific actions like push-in, pull-out, orbit, pan, or static shots. This will help maintain a coherent shot.

3. Describe motion using verbs and pace words, such as "turns her head," "smiles slightly," or "slowly." Avoid writing "The woman moves" as it provides little direction to the model.

4. Include small, subtle motions in the background to give the clip a sense of life, like hair in the breeze or steam rising from a cup. Phrase these requests as "subtle" or "light" to prevent them from competing with the main action.

5. Use a template to structure your prompts, separating the camera move, main subject action, secondary ambient motion, lighting changes, and style or mood constraints. For example: "[camera move], [main subject action with pace], [secondary ambient motion], [lighting change if any], [style/mood constraint]."

6. Test different models with the same start frame and prompt to find the best performer for your specific needs. Some models may excel at smooth camera work, while others are better at faces or physics.

7. If you encounter issues like subject morphing, camera drift, or flickering textures, adjust your prompt accordingly. Shorten the prompt, remove unnecessary adjectives, add "locked-off camera" or specify one camera move, slow down the action, or simplify the start frame. By separating the composition from the motion, you'll achieve more consistent and predictable results.