Events in Pixonic’s War Robots: How studio developed and tested hypotheses

What follows is the edited transcript of the keynote Vadim presented at White Nights Moscow.

Vadim Charugin

Events: how we understand them, what are the pros and cons

What do we at Pixonic mean by events? These are in-game events that last for a limited time. They introduce additional content or abilities unavailable to players at other times.

What are the main advantages of events?

• They increase the player base;

During holidays, players have much more time to check out the game, play it, and participate in activities. They communicate more, watch more videos.

For example, on our YouTube channel, all videos related to the events get way more views than the regular ones (sometimes by millions).

• Events are perfect for retargeting;

Users respond best to campaigns related to events. They are especially effective in retargeting former users if you promise to award them with new content if they return to the game.

• Events increase purchase count;

During events, users tend to spend more that they usually do. This is because people make emotional purchases during the holidays. In addition, events typically coincide with sales and offers.

• Events are a great way to test your hypotheses and fine-tune the mechanics.

Through events, we add additional features or content, track players’ reaction, collect feedback. We don’t have to communicate to players that the change is only temporary, it’s assumed. However, if they like it very much, they can ask to keep it for good, which we typically do.

There are some disadvantages as well:

• Sales temporarily drop after events;

During the events users play more actively than usual. There is a risk that people get fed up with the game. It passes over time, but within this period of recovery from fatigue, the game metrics may be lower than the average.

• Getting used to the events;

If events take place in the game too often, players can get used to them. They will suspect that an event with special content and temporary sales can begin at any moment. And instead of spending money here and now, they will wait for an event. This will lead to a drop in revenue.

• Strict release schedules for seasonal and holiday events.

Many events are associated with specific dates. For example, you can’t release the New Year’s event in February. This keeps the team under constant pressure. To ship new content on time, sometimes you have to start preparing it six months before the release. Because they are impossible to delay, these events often come out with bugs.

Our first event

We only started doing events in War Robots in 2016. By then, we clearly understood their pros and cons. We also knew that they became a standard for the industry, so we absolutely had to have them.

However, there were two things to consider before we launched our first event that autumn.

First of all, by that time the game already had enough content for all types of users. And we were ready to make more.

Secondly, we wanted to try something new with the game.

Start time and duration

Since the launch of the first event, we have experimented a lot with the event start time and duration. Here are the conclusions we came to.

An event should take place simultaneously across all platforms that our game was released on. If it starts one day on one platform, and two days later on the other, it will be very inconvenient for the players and may confuse the developers.

An event shouldn’t be too long. Otherwise, the users will get tired.

For example, we had an event that began in early December and ended in mid-January. In the first weeks, players responded very positively. But when the event went on for the fourth week, the players were tired. As a result, by mid-January we saw player activity drop. It took the game the rest of January to recover to the previous level of player activity.

Events should not be too short either. It is important that as many users as possible take part in the event. We came to the conclusion that the optimal duration for an event is 17-18 days. It is better to start events on a Thursday, when the work week is nearing its end.

What do we do during an event

Since we started doing events, we’ve changed a lot of things about them.

Store icon

This is the change that any player sees as soon as the game is updated. People understand that the game has somehow changed, and this encourages them to return to it.

But don’t change icons too much or too often. Otherwise, players might even fail to recognize your game or they would be confused: is it a new icon? Or the old one?

New visual elements for maps

During events, it’s a good idea to add new elements that do not affect the gameplay but introduce visual diversity. Things like these show players that an event has begun.

New visuals usually provoke discussion in communities because some people love them, others don’t. Players go to the forums, share their opinions. That alone increases the engagement.

But don’t keep the new designs forever. People can get tired and react negatively. You don’t want to still have Christmas trees in the game in March.

Let me tell you about our first Halloween event. We came up with an idea to add ghost robots to maps. It made sense because Halloween is associated with ghosts. Our ghostly robots were supposed to appear here and there, run around the map, scare the players, wreak some extra havoc on the battlefield.

Before the launch of the event, we released a video about this temporary update. The video got a lot of views and was warmly received. The players were waiting for this feature.

However, when we rolled it out, the game started crashing due to large numbers of players. On that Halloween, not everyone was able to see ghost robots. So this event has become something of a legend that many people are still asking and waiting for.

New visual elements for the main screen and the UI

Today we try not to make major changes on the maps for the events. Instead, we focus primarily on the hangar and pop-up windows. The reason for this is simple: any player that enters our game sees the hangar first and gets a pop-up.

The main difficulty for us is to maintain the game’s performance at the same level. If you start the game and it lags, instead of positive emotions you will get negative ones. And the event will have the opposite effect.

Additional tasks

Now we always add quests as part of the events. It makes users spend more time in the game during the events.

Initially, we had quest chains. They worked as follows: as soon as the player completed the first quest, they could proceed to the next one. At the end of each sequence of these tasks, the player received an award. The more chains completed, the better the reward.

The system had a positive impact on user activity. The average number of fights per day per person during events increased significantly. Moreover, after the end of the event, the activity began to return to the previous level. There were no more declines.

We also found that people completed some quests more often than others. This made us realize that players saw the value of in-game objects differently than us. Subsequently, we improved the situation. To do this, it was enough to redistribute quests in such a way that the players first received weapons and only then robots.

We also noticed that the players that advance farthest into quest chains are also the ones who:

• pay most often;
• pay the most;
• play the most.

However, this system had one big flaw. If you entered the game at the end of the event, you couldn’t complete all the quests in the chain: you simply didn’t have enough time.

So we redesigned the system. We decided to let go of the quest chains, made the quests easier, limited their completion time, and started giving them out every day.

So it doesn’t matter if you’ve completed previous quests or not. If you sign in on a Tuesday, you will receive a Tuesday quest, and on Wednesday you will receive the next one.

We also reduced the total number of awards the players could receive in general. At the same time, we have increased the percentage of people who received at least one. Overall, we were able to increase the number of people who receive awards up to 20 times compared to previous events.

Testing during events

I would like to discuss something else now. Let me tell you what we tested and what we’re testing now as part of the events.

New paint job system for robots

We initially introduced the paint job system as a temporary feature. The new mechanic allowed players to purchase paint jobs and change how the robots looked.

We wanted to see how many players would be interested.

We didn’t like the result very much. Only a small fraction of our audience appreciated the new mechanic. It’s probably because players in War Robots don’t have time to study the skins of the allies, and you can’t really see the paint job on your enemies because of the predominantly long-range gameplay.

Increasing clan size

Another feature that we tested during the events was the option to increase the maximum size of clans.

Our clan system allowed players to accumulate activity points. The more players there were in a clan, the more activity points they had and the higher their clan ranked.

We wanted to test how many people would be interested. In the end, we saw that our clans did not interest a lot of players. Subsequently, we worked really hard to improve the entire system.

Game modes

The most interesting thing that we test as part of the events is the new game modes. Each of them significantly changes the gameplay, and offers a new experience to the players. If the mode becomes widely popular, it stays after the end of the event.

One of these was the Skirmish mode. This was our first unrated mode, in which players fought each other not with their robots, but with the pre-installed assemblies from developers. The assemblies themselves changed regularly.

This mode had three tasks:

• advertise our new content among high-level players;
• allow new players to evaluate the entire gaming arsenal;
• increase retention.

The feature only increased players’ interest slightly. However, we discovered that players liked this mode because the rules were constantly changing there (for example, some types of robots came equipped with weapons that normally were not fit on them).

Thanks to the mode, the DAU/ WAU ratio grew by 5%.

A/B testing during events

Events also allow us to regularly conduct A/B tests.

Pop-ups

For example, we compared the effectiveness of different pop-up windows that notify about the end of an event.

Group A was shown a pop-up with a plain text describing what awaits players during the event, as well as the “Go to” button that linked to the social network for more the details.

Group B saw a pop-up with stylized pictures, and Group C saw beautiful pictures, some of which were even animated.

We thought that group C would show the highest conversion. However, a picture with stylized pictures turned out to be more productive. The smallest conversion was seen in group A.

Notifications

We also did A/B testing of push notifications.

We divided all players into eight groups. To each group we sent three push notifications during the event:

• Notification that the event started;
• Notification that the event continued (sent during the second week);
• Notification before the last weekend that the event would end soon.

The push notifications were very different in terms of subject matter. In addition to the main information, some of them informed about gifts, others discussed quests or special offers.

The largest conversion was shown by the group in which every new notification was significantly different from the previous ones (in topic, in content and in proposal).

An interesting fact: all the notifications that used Emoji performed better than the others. Since then, we have been using Emoji in our notifications as often as possible.

Ongoing experiments

Another thing I would like to share with you is our current experiments.

For example, last fall we tried to add a regional event to the game which would be visible to all players but focused primarily on a specific country.

In October, we had an event dedicated to China National Day. This holiday falls on the beginning of October. Usually, no one works in China for the whole week. Obviously, users have more time to play.

What we wanted to see:

• How will Chinese users respond to the event?
• Will they really play more on the weekends?
• Will an event like this be interesting to users from other countries?

The answers to all these questions were positive.

***

In its sixth year, War Robots has crossed 120 million installs and shows no signs of slowing down.