The AI revolution is in full swing, and modern artificial intelligence methods for learning, planning, and optimizing have begun changing the rules in many industries, from advertising and social media to mining or manufacturing. But not video games. Or at least, not yet. AI academic and modl.ai co-founder Julian Togelius has examined how these technologies will alter video games in the coming years.
Julian Togelius, Research Director and Co-Founder at modl.ai
People often closely associate AI with video games and in-game characters, but it’s not that straightforward. The kind of AI that controls non-player characters (NPCs) in most games is more like an intricate set of rules, manually written by the game designers. That said, in recent years, the videogame industry has woken up to the transformative potential of AI, with major studios and games companies, like Ubisoft or Electronic Arts, investing heavily in research & development.
EA devoted a lot of time and resources to make bots in, the poorly received, Battlefield 2042 as realistic as possible. But the AI enemies ended up switching between incompetent and unbeatable, leading to considerable backlash from fans of the series. But fine-tuning NPC difficulty and making them feel ‘right’ to play against isn’t an exact science. Aside from NPCs, AI has a myriad of beneficial applications in gaming. New breakthroughs in AI are bringing developers unprecedented new approaches to game design, altering the way games are made and played.
A tailored experience
Most games, or at least those with any significant budget, regularly collect data about their players. That could be as simple as when a session starts and ends or be much more detailed, like every action the player took. This data has many uses. Machine Learning can be used for player modelling, which in simplest terms is the practice of creating computational models based on players. Unsurprisingly, the most important question for many developers is when will the player stop or which players, and under which circumstances, will pay money for boosts and in-game items. However, data from players can be used for so much more.
In academia, player modeling has been studied for 15 years, and research suggests many more things can be modelled in this way. We can learn to predict what players will experience based on their playstyle, or if they think a specific kind of level is fun to play, or if a character will be intriguing to interact with. It is also possible to learn models of a player across multiple games, so platform holders and publishers with multiple games have a unique opportunity to learn a lot about their players and use the models to make predictions in games that have not been released yet. Such models can help improve games by adapting them to the particular player.
Battlefield 2042
Racing and other sports games in particular, already perform a simple form of adaptation to the player in the form of dynamic difficulty adjustment. With better player models, we could take this to the next level. For instance, developers could select content that matches not just a player’s skill level – and usually, skill isn’t one-dimensional, some players might be good at strategy while others may have quick reactions — but they could also select content that they know the players will like. So, tailoring their experience by offering them an enjoyable, but not overwhelming, challenge and keeping them playing for longer.
With enough data and Machine Learning methods we will be able to predict not only what a player will enjoy right now, but even what the player will enjoy in the future. So the game will be able to decide when to introduce new types of quests and dilemmas, or new visual environments, keeping players engaged with new compelling content.
Reigning in the randomness of PCG
These types of models could simply select which content to serve the player from among an arbitrary set of levels, characters and quests created by the game’s designers. But this kind of approach becomes exponentially more powerful if the game can automatically create new content as well – and that’s where Procedural Content Generation (PCG) comes in.
PCG is various techniques for automatically generating content. It’s been part of many games since the early ’80s, but its more prominent use is generating levels in roguelike games such as Diablo or Hades. One particularly high-profile use of PCG is in the space exploration game No Man’s Sky. The game’s creators, Hello Games, used PCG to generate everything from unique plants and animals to entire planets — around 18 quintillion of them! There are more planets than anyone could visit in their lifetime, and every single planet has a unique flora and fauna.
No Man’s Sky
The PCG algorithms that make these games possible rely heavily on randomness, so it is hard to control their output, which of course, is not a problem if it is a design feature — like in No Man’s Sky. But these shortcomings mean there are limits to what you can do with PCG, right now. However, a string of AI innovations has brought us new PCG methods that could bring radical new possibilities for automatically generating game content.
Combining PCG with models of what players do and what they enjoy, we could automatically generate completely new, tailored content for players. This means that not only will players have more tailored experiences in games, but the games will be ever-changing, never running out of new content. Imagine a future version of Grand Theft Auto where you could drive in any direction, for as long as you wanted, and the game would give you new cities to explore, with new people to meet, new quests fulfil, new architecture, new politics, and so on — and all of this created specifically for you!
Despite some developers’ concerns around the technology, AI won’t replace human game designers, at least not yet. Instead, it’ll give them superpowers. A designer will be able to draw up the general shape of a level and place key features, with the system automatically filling in the blanks. Or they will then be able to adjust the generated level using high-level instructions like ‘make it easier for players with slow reflexes’ or ‘make it less symmetric’.
Testing
In the last decade, enormous research efforts have gone into creating AI players that can play games. From board games like Chess or Go, classic Atari games to FPSs like DOOM and Counter-Strike, we’ve seen some very impressive work on building or training well-playing agents. Rather than being designed to assist game development, this work, often carried out by large and well-funded teams at major tech companies, mostly serves as a vehicle to develop and test new AI methods. However, these game-playing agents do have a wide range of uses in development. One of the most promising is game testing.
The scope and scale of modern games, combined with an array of gaming platforms they must be optimised for, means that testing has become an even more critical phase of development. Games that are shipped laden with bugs and glitches are reviled by fans, as exemplified by CD Projekt Red’s gross miscalculation with the highly anticipated yet vilified Cyberpunk 2077 — the fallout from the launch even led the company’s valuation to drop by $1 billion.
Cyberpunk 2077
Testing is very labour-intensive and consumes a sizable chunk of most game development budgets, which is perhaps why so many get it wrong. Methods that can automate game testing, partially or completely, have the ability to reshape the development process completely.
These kinds of agents can also help game balancing. By testing a game with many different AI-driven agents, we can see how it would be played by different types of players, and adjust the level’s parameters accordingly.
For example, we may want a role-playing game to be only moderately more difficult for players that rush through it than for completionists who meticulously scour every area, while also being more challenging for trigger-happy players. We can let an algorithm automatically play it through using different playing styles and gradually adjust the parameters of the game to achieve the right balance.
The advantages of this approach only become greater in multiplayer games. The challenge of balancing a Battle Royale game with a hundred people on the same map manually are immense, starting with that you need a hundred committed humans who understand the game every time you want to test a design change.
Levelling up NPCs
We’re beginning to see progress when it comes to computer-controlled players. There have been some really exceptional examples of NPC design in recent years, like The Last of Us, Bioshock Infinite or Halo 2, where developers really pushed the boundaries of game design. But generally speaking, bots really haven’t advanced that much, simply because many of the common approaches to NPC design haven’t really changed since they were first created. But that can all change with AI.
One way game-playing AI can be used to improve NPCs is by providing better sidekicks or team members. It’s a common gripe for many gamers as to just how useless NPCs that are supposed to help you can be. With better player modelling, we can create AI agents that can predict with some accuracy what the player will do, and plan around that to carry out their goals.
A major breakthrough in gaming will be the use of real-time generated natural language by in-game characters. Almost universally, game characters “speak” in pre-written sentences, and interaction usually follows a ‘dialogue tree’ where players choose from a limited set of different dialogue options.
This is a very stifling form of interaction, and it was never anyone’s idea about how best to interact with game characters. It’s more of a workaround, that became the default over the past four decades of video games, simply because we don’t know how to actually make game characters talk.
The Last of Us Part II
The massive progress in deep learning-based language models promises a way of ultimately getting around this. By giving language models the right prompts, it is now possible to generate written interactive dialogue on particular topics and styles. Open AI’s GPT-3 language generator has been a huge step forward in this area of research. The system has given us a glimpse into the future of AI – writing articles, chatting with humans online and also powering the game AI Dungeon. The text adventure game allows players to type in commands – anything from ‘make tea’ to ‘attack that dragon with a banana’ – and the system does its best to provide a realistic response and continue the story.
But it is still very hard to control the output of language models, and NPCs built this way may end up saying things that interfere with the game’s design or worse. In the case of AI Dungeon, the game’s USP was limitless possibilities, but the game’s designers had to change this to near-limitless, as many players abused the system to act out sexual encounters. While the system is far from perfect, the pace of progress in researching, refining and controlling language models, it’s likely that we will see deep learning-powered NPC dialogue in the not too distant future.
Graphical upgrades
Since games first came into existence, the most common benchmark of technical progress has been graphics. While AI is usually associated with behaviour, deep learning has made significant inroads along various parts of the graphics pipeline.
Researchers at Intel Labs have been using Machine Learning filters to complete graphical output in older games. They used deep learning, trained on real-world images, to make GTA V’s graphics photorealistic, with improved shadows, reflections and textures. But this kind of work isn’t just reserved for researchers. There’s a growing community of modders that are graphically upscaling old games using similar approaches. The results are far from perfect, and from being productised, but it’s easy to see where this kind of work will lead us in the future.
Aside from re-releasing games with huge graphical upgrades, with more sustained research in this area, in theory, this method could be used to take a lot of the rendering burden from a game’s engine, allowing designers to devote its processing power to other areas of the game.
An exciting future
Many of the benefits that these AI techniques will bring will streamline development, while greatly enhancing the experience for the players as well. But to see the real, transformative change artificial intelligence is capable of delivering, there needs to be further research across the entire stack.
It would be a mistake to only think of these methods in isolation. Most of them can augment and even support each other. Better PCG will make it possible to train more diverse and generally capable game-playing agents, and these agents will, in turn, make it easier to test games, and to develop better content generation methods.
The future of the industry is bright, and once these AI techniques are further refined and perfected it’ll likely change gaming forever. From shouldering the creative burden of designers, streamlining testing, while also bringing players entirely unique, tailored experiences with endless replayability.