In the fourth episode of our Growth Loop mini-series, Yevgeny Peres, VP Growth at ironSource, breaks down step 3 of the Growth Loop: optimizing monetization. Yevgeny explains which KPIs are important to track and when, best practices for optimizing engagement and revenue, a new way to look at ad monetization products (hint: it's not all about the money), and guidelines for setting up A/B tests.
Be sure to listen to episodes one, two, and three of the Growth Loop mini-series. Listen to episode four below or read on for the transcript.
“Step 3 of the Growth Loop deals with analyzing users who have been playing your game and then deriving the right insights, with the right optimization decisions and execution to follow. That is not possible without the right measurement in place about everything about how users behave within your game, whether it’s technical analytics around crashes or monitoring all of the in-app events setup within your analytics stack, or measuring monetization like how much revenue each user generates. Putting all that in place within the product is critical, before you even bring the first user. Without it, you’re bringing users without ever knowing how they're interacting with content.”
“Be sure to measure external sources that represent the product - like star ratings in the store or reviews where users may be complaining about something specific. It’s outside the game but should be implemented in the product, so users can be prompted to leave a review.”
When it comes to the metrics themselves, it depends on the strategy of that game’s business and to a certain extent, the stage. If you’re launching the game and it’s in soft launch, and you want to understand if the game is working, and will look at crash rates or churn rates or retention. As the game stabilizes into something that’s a healthier business at a higher scale, you’ll be looking at deeper metrics and predicting monetization - measuring ARPU, ARPDAU, retention and predicting your LTV curve. That’s up to you based on your strategy. Obviously, in the beginning you want to stabilize the product and after that’s ready, maximizing monetization is the strategy. This is where you're going to be looking at onboarding, the FTUE, churn rates, number of sessions, average session length, ARPDAU, ARPU curve within D3, D7, D14, engagement rate, and usage rate.”
“High-level monetization metrics would be how much revenue you generated on a certain day, and non-cohorted. If on May 1 you generated $10K and 10K DAU, then your ARPDAU is going to be $0.10. The next day, that can go up due to a change in product or monetization. When you look at the cohort that arrived May 1, and today is May 15, you have 14 mature days to know how much revenue those users generated each day. Now you have a measured curve of what that ARPU looks like - which is divided into in-app purchase and ad revenue. That will help you predict LTV and how much you can bid. And it can help you with modeling cash flow for user acquisition.
“In terms of KPIs for specific ad products - engagement rate and usage rate is important for rewarded video. If you’re monetizing with IAPs and you're selling a pack of 100 gems or 2000 gems - what is the engagement of each of those products? What are people after? How do you price? Based on that, you can decide what should be your next steps.”
“In step 2 of the Growth Loop, like we discussed last episode, you’re monetizing the product out of the gate. This is the first time people are engaging with the product. The most common best practice here is rewarded first, in terms of an ad monetization strategy. And also game design and designing the core loop of the game to include rewarded products - so, for example if a user fails in the middle of the level, you give him the option of reviving or continuing from the same spot. It’s best practice to reward users with virtual items or unlock specific features or boost specific phases of the game, not necessarily currency. Something like the offerwall, which is another user-initiated product, is more around the currency and pricing that is critical.”
“Since the ads are user-initiated, it’s very important to optimize the engagement of those products. If you want to optimize the revenue of those products, you need to first look at what drives them to engage with that product or not. Is it the visibility of the placement? Is it part of the loop? Is it the value?
The most common mistake is focusing on what’s happening inside - eCPM, fill rate, what happened yesterday with eCPM. And that’s not what’s really incremental, because a change of engagement of 30% to 35% is more significant than a 10% change in eCPM. Since these products are rewarding the users, they have incremental impact on IAP monetization, session length, and number of sessions. If you have an implementation of rewarded video that resets every 6 hours (every 6 hours users come in to collect a reward), and you maximize engagement, you’re actually calling users to come in more a day than without this product. In this way, rewarded videos are more engagement products than they are monetization products. Monetization is the outcome of the users engaging with those placements for the purpose of that reward.”
“The offerwall is usually a product that’s placed within the shop of the game. It's much less visible than rewarded video and only a certain percentage comes in and engages with that product. But what if you were actually able to promote that placement outside of the shop and call users to come in and engage with the offerwall? Usually that’s done for special double credit promotions where over the weekend you double the reward, prompt users and say ‘hey for the next 48 hours you’ll get 2x normal rewards.’ That placement now educates 100% of users that are playing the game on that day about the offerwall. A big part of those users who were never aware of the offerwall earlier are now being introduced to that feature. It’s a good example of something that really spikes engagement rates.”
For users who are not playing the game during that day and aren’t aware of the sale, by sending a push notification telling them there’s a sale happening, won’t only drive higher engagements with the product, but also engagement will spike because people will come into the game and spend more time and money. The outcome of this is that the morning after, the engagement doesn’t drop to the same point as before, it drops to a higher level - because more users are educated about this monetization product as being an alternative payment method that lets them progress further in the game.”
“One of the value points of user-initiated products is that it educates the users - what is currency and how do you use it? Over time, the users understand the value of a gem, a coin, or a star. Now they’re starting to be wiser with how they utilize their wallet. The reward for rewarded video as I mentioned should be around virtual items, or unlocking features. Coins are easy to A/B test at the end of the day - should I give 50 or 200?
“The offerwall is more critical and where I see more mistakes. Part of the configuration of the offerwall for the developer is deciding how much value do you want to exchange with the user. If that offerwall conversion generated $10, how many coins do you want to give the user? Usually what happens is that the value exchange set for offerwall is the same as your IAP value exchange. But here, instead of the user paying a dollar, it’s the user’s time that’s worth a dollar - but the user doesn’t know how much his time is worth. That’s why finding the sweet spot requires testing - what exactly is that value of that reward? Perhaps $1 should be 5,000 coins or 50 coins.
It’s important to mention that in many games, the economy inflates. In slot games, you start with 500 coins but as you progress and start winning, you become this bajillionaire - and being rewarded 50 coins doesn’t make sense anymore. The rewards need to be adaptive and dynamic to your state. If this isn’t implemented properly and the rewards are static while the economy inflates, engagements are going to drop to zero.”
“At the end of the day, the purpose of the product is to maximize monetization and LTV. That does not mean maximizing retention or maximizing ARPDAU. In order to maximize D7 ARPU, where most of the revenue is generated for ad-based games, you need to A/B test different strategies. It requires a lot of hypotheses and following best practices. It starts with following a certain monetization setup, like we talked about in step 2, and maybe that means only showing banners after 10 sessions or maybe that means lowering the frequency cap 5 interstitials to 2, or maybe it means increasing the frequency cap from 4 to 9. Whatever it is, you need to reach that point from A/B testing. That’ll be the sweet spot of ARPU reaching its maximum.
If you push as many ads as possible, ARPDAU will be very high but retention is going to tank. No one is going to come back tomorrow morning because they were blasted with too many ads and the experience was bad. They’ll leave 1 star ratings which will make your life more difficult to acquire users the next morning. That’s something to keep in mind that requires testing, a steady stream of users, monitoring their monetization per day, per impression, per engagement, and seeing what that looks like and what you should do about it. A good example would be segmenting users differently - frequency, placements - and then you go granular as you progress.
“In general, A/B testing requires a steady stream of data. Assuming you have that, you need to start with the right order of questions. Some things require A/B testing before others. Since we’re focusing on ad monetization in this podcast, something that’s critical is starting with the question - should you introduce a new ad unit or not? Should you place interstitials into your game or not? The frequency? What should be the rewards? Then down the road, you can A/B test certain segments like non-payers and payers. You can A/B test content of your ads - should you show competitor ads? The X button after 2 seconds? 3 seconds? You can also theoretically test the usage of a waterfall against different bidding strategies. But there’s a certain order you should follow.”
It depends on the type of platforms you’re using. Most analytics platforms support A/B testing and automate it. When it comes to setting up those tests, especially if you’re using ad monetization as a strategy, it’s important to invest the time in setting things correctly. By correctly I mean - setting the example up - what’s your hypothesis? If the variable is the frequency of interstitials, then you’re testing 3 per session or 1 per session. Your control group is everything that happened until now, so that one group doesn’t impact the other. The platform you use needs to support that. The platform also needs to be sticky. Users that come in through one door stay in that pattern so you can measure that group properly and don’t move between sessions and days. Once you have that established and let that cohort mature in terms of length of time and size, is when you start tweaking and testing increments. Then you go to your next question. That’s a constant path that never stops and a good example of the loop.”
“It’s better to not A/B test at all than to A/B test incorrectly. If you’re asking the wrong questions, testing more than one variable at a time, or not using a good platform, you’ll end up seeing interactions that don't represent the hypothesis you’re trying to address. There’s a chance you go the wrong way which can drop performance or keep it as is, meaning you lose a growth opportunity where you could have accelerated the growth loop of the game.”
