Last month, OpenAI unveiled its new text-to-video model Sora, causing a stir among professionals and casual users alike. Although it is yet to be released publicly, we decided to discuss whether it can be used in game development, as well as its potential pitfalls and limitations, with experts from ZiMAD, Skywaylab, Playkot, Lost Lore, and Charisma.
A screenshot of a video generated using Sora (Source: OpenAI)
OpenAI has only teased what Sora could be potentially capable of. What were your first impressions of the examples shown and the announcement in general?
Kirill Zhukovsky, CPO at ZiMAD
Sora is impressive with the level of quality of generated videos, but it’s not the only solution of its kind. Those who have seen that AI meme of Will Smith eating spaghetti, generated by one of the previous generation models, will understand how the quality has improved. The tool has a number of competitors, and they all are now revolutionizing video production. At this point, they can replace stock videos, devaluing to some extent the work of independent videographers who put short videos up for sale.
Alexey Emelianov, CBDO at Skywaylab
With the opportunities that OpenAI provides with its latest product, Sora, we are looking at a unique future for the video production industry. Sora not only demonstrates the ability to generate video content based on text descriptions, but also provides us with a perspective where each viewer will be able to influence the story development and details of the video, making them unique and personalized. This could mean revolutionary changes not only in the way entertainment content is consumed, but also in the advertising industry.
This is how I imagine the video production world in the future: each viewer can change the film themselves in real time, for example, make the film have a happy ending instead of a dramatic one, or change the storyline according to their preferences. This is likely to increase emotional engagement and open up new horizons of interactivity.
In the advertising space, the development Sora could lead to customized ads that are precisely targeted to the interests and preferences of a specific user. This will certainly reduce CPI and increase the effectiveness of advertising campaigns.
Taking an even broader view, Sora could have a strong impact on educational projects. In the future, we can expect educational videos that automatically adapt to the person’s knowledge level and interests, making the learning process more interesting and personalized.
I believe this is a time of great opportunity and we look forward to having access to this tool to explore everything in practice.
Evgeny Smirnov, Casual CPO at Playkot
What OpenAI has revealed about Sora’s capabilities seems to be a significant breakthrough for AI. Not just in terms of video generation. This technology shows significant practical potential not only for simulating real and virtual worlds, but also for modeling future scenarios, which is a crucial step in the development of AGI.
Eugene Kitkin, founder and CEO of Lost Lore
Every time I see new products from OpenAI, I think that I need to relearn again. Evolve again as a company and rebuild processes. The same is true with Sora. More flexible companies will be able to outperform larger ones in advertising, in prototyping, in attracting attention in all areas, not just in game development. The world will change again.
Guy Gadney, co-founder and CEO of Charisma
It looks like Sora has solved one of the most complex image generation problems: consistency between frames. So whereas most AI serves distort or hallucinate between video frames, Sora keeps a human running naturally, or buildings being the same shape. This is a key moment in AI generation and will have a profound image on short video generation. When it launches, expect a ton of “I inputted [xxx] into Sora and look what it created!!!”, but this will be shortlived as it was with ChatGPT as more creative people get to grips with it and produce more directed forms of entertainment. Regardless, it is as impactful as the home video camera.
How do you think Sora could be applied to game development?
Kirill Zhukovsky, CPO at ZiMAD
In gamedev, Sora could be used in creating ad creatives. It could also be a blessing for indie developers who don’t have enough funds or skills needed for video production. Theoretically speaking, it is possible to make cutscenes using Sora, but at the current stage it won’t be possible to put together a high-quality game. What seems possible is to use this tool to make a pitch deck for investors.
Alexey Emelianov, CBDO at Skywaylab
In gamedev, Sora will definitely have an impact on both development and marketing. Here are the areas that come to mind first:
- On-the-fly content creation: Sora can be used to generate dynamic scenes, characters, and even entire worlds based on text descriptions in real time. This will allow developers to quickly experiment with different storylines and visual styles, significantly reducing production time and costs;
- Personalized gameplay: games can become more flexible, offering each player a unique storyline, personalized quests, or even unique worlds to explore. There will be such a thing as a unique gaming experience that won’t be available to other people. In this regard, engagement should increase, and game streamers may become even more popular;
- Game marketing: this is our area and we are fully aware that the emergence and possibility of using such tools is a great gift to the entire gamedev. You just need to learn how to do it correctly and all game developers will get out of the crisis faster and significantly increase ROAS (return on advertising spend). This will especially give small game studios a good chance to grow due to greater flexibility and boldness in experimentation.
Evgeny Smirnov, Casual CPO at Playkot
With a view to the here and now, Sora can be useful in use cases such as the marketability assessment of new projects and the development of advertising creatives. Similarly, such AI tools can be a valuable asset for tasks involving automation and optimization. This is important as audiences still expect high production value games, but the mobile market is becoming more complex and companies are increasingly reluctant to take risks and invest in high-budget projects.
Looking into the future, we could see very different pipelines. For example, the move to AI rendering, where everything is built through simplified low-res graphics and then generated into more sophisticated, detailed and vibrant graphics, while also adding simulation of more complex processes within the game. Another example is that such technologies can significantly automate the work of artists: creating concept art, 3D models and animations based on their minimal designs. In addition, such AI can assist in the generation of game prototypes through a cascade of neural networks of various specializations.
Eugene Kitkin, founder and CEO of Lost Lore
Imagine that we could test gameplay without developing a prototype, using playable ads or video ads. Create more content, faster, cheaper. Choose-Your-Own-Adventure games will reach a whole new level of immersion by creating entire locations in real time. And who knows what will happen to game streaming services if such technology begins to generate gameplay or camera in real time, depending on the actions of the players!
Do you expect Sora to fit into your own pipeline?
Kirill Zhukovsky, CPO at ZiMAD
We personally don’t use Sora at ZiMAD, but we’re looking closely at it and other similar tools and solutions.
Alexey Emelianov, CBDO at Skywaylab
At Skywaylab, we test all the tools that appear on the market and despite the fact that we haven’t yet received access to Sora, the meetings we conducted with the team showed that the tool has good potential for speeding up and reducing the cost of testing marketing hypotheses, and this is key aspect in creating advertising campaigns.
However, our interest in Sora isn’t limited to creatives for buying traffic. We see great potential in the direction of viral content to drive traffic on social networks such as YouTube, TikTok, and Instagram. Despite the difficulties with controlling the quality of traffic, we think this channel will be extremely important for game developers, as it offers access to a large audience at a relatively low cost. Using Sora will allow us to significantly speed up and simplify the process of creating viral gaming content, making it more accessible and effective for our clients.
We see Sora as not just a tool for automating video creation, but also an opportunity to experiment with new formats and approaches to advertising. This may include creating personalized commercials that speak directly to the interests and preferences of the target audience, as well as developing dynamic content that can adapt to current trends and viewer interests in real time.
Ultimately, this technology opens the door to creating deeper, more engaging content that can attract huge numbers of players and maintain their interest over time, reducing acquisition costs and increasing the overall effectiveness of our campaigns.
Evgeny Smirnov, Casual CPO at Playkot
We keep a close eye on all innovations. And on the surface, we definitely see several potential use cases, such as developing gameplay trailers for creatives from fake shots, creating advertising creatives, optimizing cinematic development and automating animation development.
Eugene Kitkin, founder and CEO of Lost Lore
At Lost Lore, we have been using LLMs since 2022. In 2023, we have already developed three games using Midjourney, Stable Diffusion, Stable Video Diffusion (to create animations for characters and environments), several Audio AI, text-to-speech models, and ChatGPT4 for narrative and game design. We are one of the few AI studios that weekly explores the TheresAnAIForThat website and tries to apply new models to the daily tasks of the studio. As a result, we produce more content faster and get a higher quality final result.
There definitely be a place for Sora in our pipeline. We will try to become users at the closed test stage, if there is one.
What pitfalls and limitations do you see for Sora in terms of future use cases?
Kirill Zhukovsky, CPO at ZiMAD
The main problem is that the results are often unpredictable, plus there are not enough settings. Some scenes are difficult to put together due to strong differences in camera angles, dynamics, lighting, and character appearance. Over time, the settings will get more flexible, but due to its randomness, the tool won’t be suitable for full-fledged professional use anytime soon.
Alexey Emelianov, CBDO at Skywaylab
We don’t see many pitfalls so far, but they are quite significant. The implementation of innovative technologies such as OpenAI Sora always comes with certain challenges and limitations:
- Legal and ethical issues: one of the key pitfalls of using technology based on AI content generation is copyright and intellectual property issues. Determining who owns the rights to AI-generated content can be complex, and disputes may arise over the use of protected material without permission;
- Quality control: when generating video content using Sora, it can be difficult to ensure consistent and predictable quality. AI algorithms can interpret text descriptions differently, leading to mixed results. Monitoring and correction mechanisms must be developed to ensure that the generated content meets stated quality standards and expectations;
- Data dependency: the quality and variety of content generated is directly dependent on the training data on which the model was trained. There is a risk that the model may reproduce existing biases or not have enough diversity in the data to produce the content the creative producer intended.
In conclusion, I would like to add that despite all the potential pitfalls and difficulties in introducing new technologies into existing pipelines and processes, this must be done, otherwise there is a chance of being left behind the ship sailing to success.
Evgeny Smirnov, Casual CPO at Playkot
It is currently unclear how much control users will have over the results of Sora’s generation, and this determines how we might use the technology in our work. It’s also unclear at this stage how Sora will be retrained to adapt to the tasks of specific users and teams.
Eugene Kitkin, founder and CEO of Lost Lore
OpenAI servers will have a limit on the length of the generated video, and they will unlikely allow to install the model on third-party servers due to its proprietary architecture (most likely Sora has several layers of LLMs). The video creation time will be long, not suitable for regular use at first.
I would also express concerns about the safety of this technology. For this reason, the product is still not publicly available.