Video foundation models have achieved significant advancement with the help of network upgrade as well as model scale-up. However, they are still hard to meet requirements of applications due to unsatisfied generation quality. To solve this problem, we propose to align video foundation models with human preferences from the perspective of post-training in this paper. Consequently, we introduce an Iterative Preference Optimization strategy to enhance generated video quality by incorporating human feedback. Specifically, IPO exploits a critic model to justify video generations for pairwise ranking as in Direct Preference Optimization or point-wise scoring as in Kahneman-Tversky Optimization. Given this, IPO optimizes video foundation models with guidance of signals from preference feedback, which helps improve generated video quality in subject consistency, motion smoothness and aesthetic quality, etc. In addition, IPO incorporates the critic model with the multi-modality large language model, which enables it to automatically assign preference labels without need of retraining or relabeling. In this way, IPO can efficiently perform multi-round preference optimization in an iterative manner, without the need of tediously manual labeling. Comprehensive experiments demonstrate that the proposed IPO can effectively improve the video generation quality of a pretrained model and help a model with only 2B parameters surpass the one with 5B parameters. Besides, IPO achieves new state-of-the-art performance on VBench benchmark. We will release our source codes, models as well as dataset to advance future research and applications
An astronaut in a sandy-colored spacesuit is seated on a majestic lion with a golden mane in the middle of a vast desert. The lion's paws leave deep prints in the sand as it prowls forward. The astronaut holds a compass, looking for a way out of the endless expanse. The sun beats down mercilessly, and the heat shimmers in the air.
A man with a beard is playing the piano in a concert hall. He wears a black tuxedo and a bow tie. His fingers dance on the keys as he plays a beautiful melody.
A silver flying motorcycle with a sleek design and a powerful engine zooms through a city. The motorcycle's rider wears a helmet and leather jacket as they navigate through traffic.
A futuristic robot astronaut, clad in a sleek silver suit with glowing blue accents, runs across a barren, rocky landscape under a starlit sky. In its hands, it carries a small, high-tech musical instrument, emitting a soft, ethereal glow. The robot's movements are fluid and purposeful, blending athleticism with a sense of otherworldly grace. The terrain is rugged and desolate, with distant mountains and a faint horizon, creating a stark yet captivating backdrop for this unique scene.
A woman with flowing dark hair and a serene expression sits at a cozy caf\u00e9, sipping from a steaming ceramic mug. She wears a soft, cream-colored sweater and a light brown scarf, adding to the warm, inviting atmosphere. The caf\u00e9 is dimly lit with soft, ambient lighting, and a few potted plants add a touch of greenery. She looks content and relaxed, enjoying her drink in the peaceful environment.
A young girl in a futuristic spacesuit runs across a vast, icy landscape on a distant planet, with a towering mammoth-like creature beside her. The mammoth's massive, shaggy form and long tusks contrast with the stark, alien environment. The sky above is a deep, star-filled space, with distant planets and nebulae visible. The girl's determined expression and the mammoth's steady pace convey a sense of adventure and discovery in this otherworldly setting.
An actor dressed in rugged, casual attire, including a dark jacket, jeans, and hiking boots, walks alongside a dingo in a natural, wooded setting. The dingo, with its distinctive reddish-brown fur and alert eyes, moves gracefully beside the actor, who looks calm and engaged. The background features lush greenery and dappled sunlight filtering through the trees, creating a serene and harmonious atmosphere. The actor and the dingo walk in sync, their bond evident in their synchronized steps and mutual awareness of each other.
A massive woolly mammoth, covered in thick, shaggy fur, walks through a prehistoric landscape. Its large tusks curve outward, and its heavy footsteps create deep impressions in the snow-covered ground. The mammoth moves steadily, its trunk swaying gently as it navigates through a dense, icy forest. The sky above is overcast, casting a soft, diffused light that highlights the mammoth's imposing presence and the stark beauty of its environment.
