AI Assurance for Gaming Use Case

2

Changing the rules of the gameLeveraging over billions of data points to influence the decisions that shape and constantly improve cutting-edge games, applying machine learning to the gaming space is a complex play.

Between the real-time demands of the in-game experience, and the insights derived for marketing decisions, product planning and personalization, creating and managing machine learning models that can instantly detect drifts and feed better insights to optimize games, campaigns and revenue is no small task.

While the games are constantly evolving, ads and promotions are highly dynamic and player behaviours volatile, how can data science teams make sure their models are optimized and provide real value while delivering value to support product, marketing, and prevent fraud?

Enter superwise! Providing a unique solution to monitor, analyze and optimize your models in production, and give more visibility and control to the data science and operational teams. The solution creates one source of truth for all AI stakeholders to foster more trust and efficient use of AI at scale.

3

What’s keeping her up at night?

• What is happening with models in production, in real-time?

• How to optimize models and improve the retraining strategy?

• How to make the models more transparent?

Who is in charge of models in production?

Business AnalystGeorge

Marketing AnalystNelly

Data Science TeamErica

What’s keeping him up at night?

Are there changes in user’s behaviours, and if yes what caused the changes?

What’s keeping her up at night?

Is the model optimized for new populations?

What’s keeping her up at night?

Can I detect new types of fraud on time?

Fraud Analyst Jen

Welcome to Day 2! The day your models move from research and training with clear rules and owners to production. When the hyper dynamism of real-life data meets the hyper sensitivity of your models, it takes a village to make sure that your models behave the way they should.

4

Who is in charge of models in production?

Business Analyst George

Marketing Analyst Nelly

George and his team need to analyze large and complex data sets to understand user’s behaviours to shape features, product roadmap and business strategies.

To do so, they rely heavily on the insights gathered from the data science team to personalize the gaming experience, such as the ability to customize the difficulty according to the player’s behaviour and preferences.

They need the predictions promptly and need to understand any changes in the data: was it due to new features in the game? New customer segments? Exogenous factors?

Nelly is here to go through the marketing performance and user conversion funnel to identify opportunities to improve ROI. She needs to move as fast as possible to give guidance to her fellow marketers about campaigns to attract new users, and leverages the insights of the ML models to predict conversions and in-app offerings.

To excel at her job, she wants her knowledge to be razor-sharp on specific subpopulations. She wants to understand how these populations behave and affect the quality of the predictions, so as to enable her to make better recommendations on whether to target new users and locations. Today, she lacks the granularity she needs to make better decisions.

5

Who is in charge of models in production?

Fraud Analysts Jen

Jen and her team use multiple ML models to detect fraudulent and suspicious activity in the game such as bot detection, hacked accounts, abusive users,etc.

They are also responsible for supervising the models and validating their decisions. The ML models are a great tool to find suspicious activity and identify new fraud trends, but she doesn't have the necessary visibility to understand what they rely on.

Right now she is very dependent on the data science team to leverage the insights of the ML models or validate her hypotheses, and that stops her from reacting fast to newly detected fraud trends.

Meet Erica! She and her team are in charge of building, maintaining and continuously improving all machine learning models used in the company, such as churn prediction, LTV, toxic player detection, in-game recommendation and fraud detection, to name just a few. They also work with various operational teams that use the models to help them leverage business insights.

Erica wants to better understand what is happening with the models in real-time and to optimize them based on production-driven insights. For instance, she thinks their weekly retraining strategy is costly and inefficient, and while automatic, every upload needs to be monitored.

Between supporting the other teams and supervising the weekly retraining, she has very little time to work on optimizing current models or research for new ones. Ultimately, this creates friction in the organization and a lack of efficiency for the data science team overall.

Data Science Team Erica

6

For data science users• Catch drifts at the feature level before they impact your business.

• Gain a better understanding of the behaviour of your predictions in production.

• Benefit from production-driven insights to optimize models and perform smarter retraining.

For operational users• Gain more independence from the data science team to

understand what is happening with the models and predictions.

• Benefit from granular insights on sub-segments to be able to calibrate the best next move: a campaign, a new feature, a change in process,etc.

• Be more efficient and work as fast as you need to achieve your goals while unlocking the benefits of your AI program.

Empowering AI stakeholders with superwise

7

Monitor, analyze & optimize All your models

Know exactly when your models misbehave and why

With real-time alerts on anomalies: data and model drifts, performance issues, correlated events, weak spots, and automated root-cause analysis to severely reduce the time needed to detect and fix issues.

Achieve full observability of your models in production

With a full analytical suite to slice & dice your data in production and discover trends. Thanks to out of the box and customized model KPIs, you can have a clear view of what is happening, track and predict performance over time and versions. The solution also enables you to detect underperforming segments, and pinpoint changes at a granular level.

Optimize models with production-driven insight

Thanks to unique views of your data and models in production, and innovative capabilities to compare models’ versions, you can define smarter retraining strategies and benefit from better models.

8

As more businesses rely on AI models to boost their impact and their bottom-line, the need for managing, monitoring and optimizing the real-life behaviour of these models grows.

Superwise.ai is the company that monitors and assures the health of AI models in production. Already used by top-tier organizations, Superwise.ai monitors millions of predictions daily to eliminate the risks derived by these models’ black-box nature: bad decisions, unwanted bias, and compliance issues. Their AI assurance solution acts as the one source of truth for all the stakeholders, and empowers data science and operational teams with the right insights to scale their use of AI by becoming more independent and agile, and gain confidence in their models’ operations.

Implemented use cases include Customer Lifetime Value (CLV) predictions, fraud detection, lead scoring, underwriting, credit risk, and more.

Gartner recently named superwise as a 2020 Cool Vendor in Enterprise AI Governance, recognizing it for its innovative technology and approach

www.superwise.ai

About Superwise

Schedule a Demo