OpenAI's Sora is an impressive video-generating tool, but it has its limitations, especially when it comes to depicting fast-moving people. Here's why capturing such dynamic scenes is a challenge.

One of the main reasons Sora struggles with fast-moving people is its current understanding of physics and motion. Most generative video models, including Sora, are trained to predict new pixels based on previous frames. This approach works well for static or slow-moving objects but falls short when objects move rapidly through space. Fast movements create complex changes in shape and position that the model finds difficult to accurately predict.
Another factor is the training data used to develop Sora. While the model has been trained on a vast amount of video data, it may not have enough examples of fast-moving people to learn from. This lack of specific training data means the model hasn't fully grasped the nuances of rapid human motion.
Sora uses the same transformer model architecture that large language models (LLMs) use. While this architecture is powerful for processing text, it's not inherently suited for understanding the complex physics of moving objects. Teaching a transformer model to comprehend the rules of physics in arbitrary situations remains a work in progress for AI researchers.
OpenAI is also cautious about the potential misuse of Sora, especially when it comes to generating videos of real people. Fast-moving videos can be particularly prone to creating unrealistic or misleading content, which could be misused for deepfakes or misinformation. To mitigate these risks, OpenAI has implemented strict safety measures and limited access to certain features.
Despite these challenges, OpenAI is continuously working to improve Sora's capabilities. By refining its understanding of physics and expanding its training data, the hope is that future versions of Sora will be better equipped to handle fast-moving people and other dynamic scenes.
While Sora is a powerful tool for generating videos, capturing fast-moving people remains a significant challenge due to the complexities of physics, training data limitations, and safety concerns. As AI technology continues to evolve, we can look forward to more advanced models that can handle these dynamic scenarios more effectively.