OpenAI has unveiled Sora, a text-to-video model designed to simulate the physical world in motion, marking a significant leap forward in AI’s ability to understand and interact with real-world scenarios.
Basically, you just write a text prompt and video is created using AI, similar to the company’s DALL-E for image generation, but this time for videos.
Sora can make videos up to a minute long, maintaining high visual quality and closely adhering to user prompts. For any artist, filmmaker or student that needs to create a video, this opens up so many possibilities.
OpenAI says Sora has been made available to red teamers for assessing potential harms or risks, as well as to a select group of creative professionals for feedback on its utility in professional settings. OpenAI plans to refine Sora based on this feedback, ensuring it meets the needs of its users effectively. The demo videos are pretty surreal.
Sora can create complex scenes with multiple people, specific types of motion, and detailed backgrounds. It can generate videos that accurately reflect the user’s prompts. For instance, Sora can create videos depicting a stylish woman walking down a neon-lit Tokyo street, giant wooly mammoths in a snowy meadow, and even a movie trailer featuring a space man’s adventures. The demo videos were pretty impressive.
There are limitations to Sora, however, including challenges in simulating the physics of complex scenes and understanding specific cause-and-effect scenarios. OpenAI says Sora may also confuse spatial details and struggle with precise event descriptions over time.
What about safety? OpenAI is working with domain experts to test the model for misinformation, hateful content, and bias, as well as developing tools to detect misleading content. The company also plans to incorporate C2PA metadata in future versions to ensure content authenticity.
Sora was created based on previous research in DALL-E and GPT models. The feature can also turn a picture and make a video from it. Existing videos can also be extended or have missing frames filled in. This just sounds so crazy that it’s possible with AI.
Check out some of the videos made by users requests and shared by OpenAI head Sam Altman, below:
1) What https://t.co/w6b9T1WWue
— Sam Altman (@sama) February 15, 2024
