overcoming obstacles with bi and big data · overcoming obstacles with bi and big data tdwi e-book...

OvercOming Obstacles with bi and big data

TDWI e - book December 2012

1 Making a Case for Contextual Bi

3 Q&a: the Big Data golD rush

5 how to aCQuire, analyze, anD aCt on Big Data to PreDiCt CustoMer Behavior anD Drive revenue

8 aBout DoMo

Sponsored by

tdwi.org

1 TDWI e-book overcomIng obsTacles WITh bI anD bIg DaTa

making a case fOr cOntextual bi

Domo believes traditional business intelligence (bI) is broken. What’s broken isn’t something that can be fixed easily. It has to do with the way traditional bI is designed. Domo says its approach to bI delivers something fundamentally different: a contextual view of information, regardless of where it exists, be it in the cloud or in an on-premises system.

BI upstart Domo, Inc. believes traditional BI is broken.

What’s broken isn’t something that can easily be fixed, argues Todd Beauchene, one of the company’s senior solution consultants. It can’t be easily fixed because it has to do with fundamentals, with the way BI systems are designed and maintained. Domo’s approach to BI delivers something fundamentally different, he claims: a contextual view of information, anywhere and everywhere, whether it’s in the cloud or in an on-premises system.

In Domo’s view, traditional BI is too rigid: it’s wedded—even welded—to a client/server, data warehouse–centric model that’s neither especially agile nor particularly practical. In the client/server era, this might’ve been a pragmatic necessity.

In the era of the loosely coupled cloud, it’s just plain unacceptable, he argues.

“A user’s needs will change as he understands better what the data is, what’s available to him, and how business

conditions have changed. A good BI system should be able to adapt to that, and that’s part of what we’ve built into Domo,” Beauchene explains. “We know things are going to change over time. As users understand better what their data looks like, we want them to be able to leverage [Domo] to get at the data they really need the way they want it.”

The 64-zettabyte question, now as ever, involves just how to achieve this. In the late 1990s, for example, traditional BI vendors pushed OLAP-driven discovery as one response to this problem. However, the OLAP model entailed issues of its own, for both IT and business end users.

Nowadays, BI vendors trumpet self service as a solution. Instead of waiting for IT to do this or that for them, self service empowers users to do it for themselves. That’s the promise, at least.

Chris Wintermeyer, who oversees enterprise solutions with Domo, thinks this is like putting lipstick on the proverbial pig. It does nothing, he maintains, to address traditional BI’s biggest problem: its dependency on an inflexible data warehouse–centric model. With a conventional data warehouse, Wintermeyer argues, it takes too long to prepare and deliver access to data.

By the time a user gets the data they thought they wanted, their needs have changed.

About DomoExpert Q&A How to Act on Big Data Contextual BI


“Every BI project that I’ve been involved in—and this is over a long career in BI—after you’ve shown [a user] what you’ve built for them, after they use it for a while, they always ask, ‘But what if we did this? Couldn’t we do it this way?’” says Beauchene. “When they first see [the finished product], they’re happy, but after they start using it, they identify things they want to change.”

The data management (DM) team goes back to the drawing board, does its stuff, and gives a user what they say they wanted—only to discover that (once again) users’ needs have changed. The goalposts are always moving because that’s the way the world works.

Domo touts what it says is a better way: a cloud-based BI service that’s able to consume data from other cloud providers (such as Salesforce.com, Marketo, and Adobe Site Catalyst) and from on-premises sources such as DB2, Oracle, SQL Server, Sybase, and Teradata databases.

The overarching theme is user-specific context, says Wintermeyer: the context a C-level executive requires is going to be different from that required by a director of marketing.

Domo’s platform permits a DM group to build role-based views for these and other classes of users. “The challenges they all face, regardless of whether they’re executives or further on down the food chain, is that if they have access to a lot of the information [they want], it’s not all in one place,” he argues. “They have to go into their sales system to get sales data, or their financial system to get financial information. They aren’t getting information contextually. It’s not possible for them to get information contextually.

“If we were to ask very specific individual questions of all of those data sources, we’d be writing lots of individual queries constantly. Instead, we write queries that we consider wide and deep. So let’s get a bunch of information about sales out of Salesforce.com, or a bunch of information about hiring and recruitment out of Taleo,” Wintermeyer explains.

Beauchene says Domo focuses on delivering a “nimble”—as distinct to an “agile”—BI experience. Agile, he argues, is too associated with the rigor of a methodology. The Domo approach emphasizes empirical iteration: a DM group builds a view for a certain group of users; these users try it out, identify the things they want changed, request new features, and hand it back to DM. The process starts all over again.

That’s precisely the point: because business needs are always changing; because Domo continues to build new features or capabilities into its cloud platform; and because the services or systems Domo pulls data from are themselves changing, users are always going to ask for more.

“Our approach is to take a big problem, break it into small parts, and let the users consume the pieces as they come up,” he says.

There’s another wrinkle here, too. One problem with self service in a traditional BI environment is that DM teams have to be very careful about how much self service they give to users—especially if (as part of self service) they’re giving users query access to operational systems. Give a nonexpert user too much self-service rope and they could easily hang themselves with it, issuing toxic queries that bring operational systems to their knees.

Industry luminary Claudia Imhoff, a principal with consultancy Intelligent Solutions, Inc., warns against precisely this scenario. “If you are going against an operational source, one of the things that I worry about is what’s the impact on that operational source? [Does a DM group] have the ability to monitor what’s going on in that billing system that knows how to create bills [but] isn’t used to having queries run against it?” said Imhoff, at a recent industry event.

For this reason, Imhoff argues, DM groups have to be very careful about how they both support and police self-service.

“If you do see an impact, what can you do to shut it down quickly? What can you do to alleviate the impact on those operational sources?”

To avoid self-service issues, Domo is structured in such a way that users can “slice, dice, and drill as they see fit, without having to worry about negative ramifications on live systems,” says Wintermeyer.

The goalposts are always moving because that’s

the way the world works.



Is big data more than just the latest fad? What best practices can your enterprise employ to successfully mine big data for deeper insights? For answers, we contacted Tom mcconnon, product marketing manager for Domo, a company focused on driving value from bI investments.

TDWI: Is big data just another California Gold Rush, or is there real value beyond the hype?

Tom McConnon: Like the Gold Rush, the big data movement is producing real winners and losers. The hype translates to wealth when organizations understand how to extract real value from the ever-increasing amounts of data they collect. Businesses walk away empty-handed when they spend resources collecting and analyzing big data but never make the results available to the decision makers in a format that they can (1) consume and understand and (2) act on.

What are the benefits and drawbacks of big data? Is it really worth it? Do the benefits outweigh the drawbacks? If you’re smart, they most certainly do. Let’s explore both. First, there are several drawbacks. Big data requires significant infrastructure and resources to collect and mine, and it can be overwhelming and distracting to teams that don’t know what to do with it.

There are benefits. For example, big data insights improve decision making—this is the “gold” that comes from big data.

Q&a: the big data gOld rush



How do you define big data “gold”?

When initiating a big data project, we always start with one question: “What is the most important decision your CXO makes on a regular basis, and what information could be used to make that decision?” The answer outlines the information we seek to derive from data. Big data gold can be defined as the information your CXO needs to make better decisions every day.

What are realistic expectations for big data consumption?

The truth is that nobody really consumes big data. Big data is only valuable when it’s boiled down into actionable insights. Today’s expectation is that these big data insights should be highly consumable. We assert that big data insights are highly consumable when they meet four standards.

First, they’re lean, meaning there’s no extraneous information, no extra fat. We see time and time again where business users request information from IT, and the deliverable comes back as a spreadsheet that contains information to answer the question, as well as lots of other unnecessary information.

Second, they’re real time. Data is collected in real time, so it’s reasonable to expect that big data insights should be consumable in real time as well. More than ever before, businesses rely on timely information to get a competitive edge and increase performance. Real-time data is a key factor in increasing the speed of business.

Third, they’re visual. Business leaders don’t want to consume big data insights in spreadsheets. When insights are presented graphically (via dashboards, charts, graphs, etc.) they’re much easier (and faster) to consume.

Finally, they’re accessible on any device. There’s no justification for missing out on big data insights due to the device you’re using. Whether it’s a desktop, laptop, tablet,

or smartphone, big data insights should be right at your fingertips at any given moment.

Who in the organization should be benefiting from big data gold?

Big data gold is truly for the masses. When organizations empower their workers with better information, it helps everyone pull in the same direction. Although insights should be provided to as many people as possible, scale is always a factor. You have to start somewhere, so begin by providing big data insights to the individuals who regularly make the highest-impact decisions. Then, as resources allow, work down the chain to empower the rest of your workforce.

What are some of the best practices of organizations that have struck gold with big data?

I’d like to highlight two of them.

First, enable all relevant employees to collaborate around big data gold. When you disseminate valuable information within your organization, great things happen. First, you harness the wisdom of the crowds. Let’s say big data insights indicate when your next manufacturing project will be completed. When you share that information with the collective that’s involved with that initiative, their qualitative insights add value to the quantitative data. Second, you increase communication within the organization while reducing formal meetings.

My second best practice: manage by exception. Big data has the potential to overwhelm and overload. To get the most value out of the incoming data, effective organizations manage by exception. Instead of pouring over all the data that’s at your fingertips, pay attention to the outliers. Don’t worry about website leads every day—pay attention when they fall below a predefined threshold. Don’t look at deals in the sales pipeline every morning—tune in if it’s significantly below or above projections. Managing by exception allows effective organizations to handle higher quantities of information more efficiently.

Big data insights should be right at your

fingertips at any given moment.



hOw tO acQuire, analyze, and act On big data tO predict custOmer behaviOr and drive revenue

To find the most meaningful data, organizations must move beyond how to best acquire information and focus on how to analyze and act on it.

Today, with millions of websites on the Internet, creating a personalized, customer-focused experience is at the forefront of every organization’s business strategy. Consumers want individualized shopping and user experiences and companies are looking to optimize their big data initiatives to provide this personalized level of service.

However, big data is both a blessing and a curse. The ability to cheaply and quickly store large amounts of information is valuable; yet, analysis tools that can make sense of the data are lacking. Algorithms and techniques that used to work well with hundreds of thousands of transactions are unusable now that the problem involves tens of millions of transactions, and the size of the data is growing exponentially.

Big data analysis must respond to real-time shifts in customer data and perform analyses of the relationships

between consumers, products, pricing, promotions, and sales. For example, if sales decline, what can the company do to revitalize demand? How can the data analysis enhance promotional offers and increase profits?

To find the most meaningful data, organizations must move beyond how to best acquire information and also focus on how to analyze and act on it. If data is not actionable and cannot drive operational decisions, it is not very useful. If data can be used to assist in deciding how to best bundle services and solutions, and to determine pricing and packaging, it has value.

Increasingly, it is consumptive pricing models, not subscriptions, that will influence consumer behavior and service usage. Consumptive pricing will help drive customer and partner behavior, but to do this, companies must harness big data from many sources, including in the cloud as well as in mobile and virtualized environments. Turning data into information will ultimately help organizations increase revenues, enhance growth, and reduce churn.

By Scott Swartz



User data can typically be broken down into two distinct types: transactional and sub-transactional data. User purchases or transactional data isn’t typically thought of as big data, and although such data is valuable from a monetary standpoint, it only tells part of a story. However, when you add in sub-transactional data or clickstream-based data, you get a story about the user’s behavior and intentions that is far clearer and more predictive.

If you know that I bought a book from Amazon.com for $14.95, you only have limited view of the user. When you understand what other books I bought prior to this purchase; what reviews I read; what time of day, month, and year I make purchases; from which type of device I shop; and how often I look at other items prior to purchase, and you compare that information with other purchasers who have similar shopping paths, you can predict my future behavior. Amazon commercialized this concept of item-based collaborative filtering to generate suggestions for shoppers on its site. The company has also been able to predict purchasers’ behavior based on their visits from tablets or PCs.

As developers of applications today, you also need to consider how you architect your solution to track user behavior or engagement with your solution. A simpler way to start may even be to leverage the crop of new big-data customer engagement applications popping up to take advantage of insatiable demand to know more. By simply including JavaScript code snippets, you can create an application to track many details about the user and his or her clickstream behaviors. You can even configure these tools to provide real-time, in-app messaging or offers to customers based on their specific behavior patterns.

Having the ability to track a user’s behavior and present suggestions or offers is only part of the solution. Being able to drive dynamic offers and then monetize those offers immediately is where you can act on this rich data set to drive more revenue. To move this from the realm of science fiction to fact, think of airlines that change their prices based on the sales trajectory of a specific plane’s seats over a specific time period. This constant repricing based on purchase data is called yield management. With today’s off-the-shelf technology, you can track your users’ behavior on a

more granular basis than simple purchase and yield timing. Then, by adding in multiple offers and a flexible commerce engine, you can bring revenue to light that simply wasn’t available before.

Improve Customer Satisfaction

However, just as important as selling your customers more products and services may be, when you understand their actions and activities in-app, you can address an even bigger issue for recurring-revenue-based business: churn. Customers lost to churn need to be acquired, and in today’s online world, keeping customer acquisition costs (CAC) at less than their annual recurring revenue (ARR) is important.

Simply looking at the transactional data may show the value of a purchase or a customer, but by looking at the sub-transactional big data, you can learn when customers begin to use your solutions less frequently, when they are hitting specific computational or functional walls, or even when they are about to make specific mistakes using your application. By fine-tuning your user experience, offering functional alternatives or stepping in to prevent problems, you can drive a far more effective customer success model and prevent dissatisfaction before it arises. Although it might seem like a privacy issue or user stalking, real-time data analysis leads to far more engaged customers.

Subscriptions Are Dumb; Consumption Is Fat

Many cloud and SaaS-based business models focus on the beauty of subscriptions. This is often reinforced by venture capitalists who preach the mantra that recurring revenue is an annuity stream than can carry astronomical valuations for practitioners. However, relying on subscriptions can be dangerous. Subscriptions tell only part of the story. They represent dumb transactional data that is inherently limited in providing a holistic understanding of your customer. Understanding something about usage and even building

Although it might seem like a privacy issue or

user stalking, real-time data analysis leads to

far more engaged customers.



pricing models that take into account a user’s clicks, downloads, and views enables you to scale pricing in a far more flexible manner.

In our work at MetraTech, we see far more businesses moving beyond simple subscriptions to consumptive-based business models. Consumptive models are rich with fat, big user data that can be mined and leveraged for additional revenue.

The future belongs to those who know their customers not just on a superficial, persona-based level. Winning businesses will leverage how their users speak through their purchase and usage patterns, and those businesses will respond with pricing and offers based on real-time awareness.

What Future Commerce Requires

A3—acquiring, analyzing, and acting on big data—isn’t as complicated as it sounds. Begin by introducing big-data-activity-based capture into your online solutions. Track user behavior and provide offers and options based on those actions. Then introduce a modern commerce-and-compensation engine to act upon and monetize your insights. You’ll be able to model and test nearly any pricing model and quickly respond to your users, thereby driving revenue in ways that were never possible before.

Scott Swartz is the founder and CEO of MetraTech. You can contact the author at [email protected].



www.domo.com

Domo is a cloud-based executive management platform that is redefining the business intelligence market and transforming the way business is managed. Domo gives users direct, real-time access to all the business information they care about, all in one place. Domo solves universal pain points felt by CEOs, other managers, and IT organizations for whom traditional BI reporting is often too cumbersome, too complex, and too slow.

The company was started by cofounder and longtime CEO of Web analytics powerhouse, Josh James. With $63 million in funding, Domo is backed by an all-star list of angels and investors, including Benchmark Capital, IVP, Andreessen Horowitz, Ron Conway and David Lee of SV Angel, Hummer Winblad, plus the who’s who of SaaS and Internet technology.

Domo’s founding team consists of some of the most sought-after talent in the industry with experience that includes Amazon.com, AmericanExpress, Ancestry.com, eBay, Endeca, Facebook, Google, LinkedIn, MLB.com, Salesforce.com, Omniture, and SAP.

• www.domo.com

• www.domo.com/BI

tdwi.org

TDWI, a division of 1105 Media, Inc., is the premier provider of in-depth, high-quality education and research in the business intelligence and data warehousing industry. TDWI is dedicated to educating business and information technology professionals about the best practices, strategies, techniques, and tools required to successfully design, build, maintain, and enhance business intelligence and data warehousing solutions. TDWI also fosters the advancement of business intelligence and data warehousing research and contributes to knowledge transfer and the professional development of its members. TDWI offers a worldwide membership program, five major educational conferences, topical educational seminars, role-based training, on-site courses, certification, solution provider partnerships, an awards program for best practices, live Webinars, resourceful publications, an in-depth research program, and a comprehensive website, tdwi.org.

© 2012 by TDWI (The Data Warehousing InstituteTM), a division of 1105 Media, Inc. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. E-mail requests or feedback to [email protected].

Product and company names mentioned herein may be trademarks and/or registered trademarks of their respective companies.

.


overcoming obstacles with bi and big data · overcoming obstacles with bi and big data tdwi e-book...

Documents