building the next analytic app platform in the cloud
DESCRIPTION
Strata 2012 - Alteryx Presentation George Mathew, President & COO of Alteryx explains how Alteryx built a scalable, fault-tolerant Analytic Plaform in the Cloud. This deck was presented at Strata Conference in Oct 2012 and George addresses JSON vs XML, elastic framework, IAAS, Automated deployment and many more issues that went into the development of gallery.alteryx.comTRANSCRIPT
1© 2012 Alteryx, Inc. Confidential.
Building the Next Analytic App Platform in the Cloud
George K. MathewPresident & Chief Operating Officer,Alteryx
2© 2012 Alteryx, Inc. Confidential.
Conversation is shifting from Infrastructure to Outcomes…
3© 2012 Alteryx, Inc. Confidential.
Social Media
ClickStreams/ Log Files
Device Generated Data Cloud Applications
…And For Good Reason:New Sources of Relevant Information
Analytic Platform
4© 2012 Alteryx, Inc. Confidential.
77% agree more employees with big data insight = informed decisions
74% say more data shared = more effective decisions
More employees need Big Data Insight
5© 2012 Alteryx, Inc. Confidential.
6© 2012 Alteryx, Inc. Confidential.
7© 2012 Alteryx, Inc. Confidential.
Data Integration & Analytic Workflow
Integrate any data source
Inte
gra
te
Un-Structured Content Rapid design of
predictive analytics
Analyze
App &
Data
All Relevant Data
Enrich
Packaged Market &
Customer Data
Step 1:
8© 2012 Alteryx, Inc. Confidential.
Create & Share Analytic Apps in Cloud
Asse
mble
App
Private or Public Cloud
Publish
Run
Step 2:
9© 2012 Alteryx, Inc. Confidential.
Alteryx Strategic Analytics 8.0First Cloud for Strategic Analytics
10© 2012 Alteryx, Inc. Confidential.
1. Relevant and Useful2. Scalable and Elastic3. Public/Private Deployment4. Fault-Tolerant
Attributes of Next-Gen Analytic Cloud Service
11© 2012 Alteryx, Inc. Confidential.
Trends
Make it useful…
• JSON vs. XML (SOAP): JSON is the clear winner.• Web Tier: WCF• Focus on standards-based approach (i.e. HTTP for inter-
process communication vs. proprietary communication protocols)
12© 2012 Alteryx, Inc. Confidential.
JSON RESTful Service• A completely stateless web service
tier built on WCF.• No need for session stickiness –
initially.• Plan for security and versioning at the
service operation level.• Have no other hosting dependencies
(like IIS).
Implication: Web service tier is completely elastic. A simple start with plenty of options moving forward – including more advanced traffic routing and versioned APIs.
Yes, but how did we do it?
13© 2012 Alteryx, Inc. Confidential.
Don’t figure out the scaling alone. Great options are available:
• Persistence• Out of the box maintenance, monitoring, replication• Strong developer support, multiple language drivers
• Scalability/Elasticity Framework• Web-based• Automation API• Monitoring• Multiple IaaS provider support (private/hybrid options)
• IaaS• Global, API-driven, Windows & Linux support
Make it scalable…
14© 2012 Alteryx, Inc. Confidential.
Automate Deployment and Scalability
• Use automation and configuration APIs of RightScale and AWS to fully instrument deployment of nodes in the web, persistence and analytic processing tiers.
• Monitor and management tools for nodes via programmatic APIs.
Implication: 1 FTE to the ProdOps for the Alteryx Gallery to monitor and manage our entire cloud deployment.
Yes, but how did we do it?
15© 2012 Alteryx, Inc. Confidential.
Not all organizations want a public cloud service.
• Designed the Gallery architecture to be a hybrid
• Near seamless transition from public to private contexts
• Mixed-context execution: Cloud execution environment with private data (DRO)
Make it deployable…
16© 2012 Alteryx, Inc. Confidential.
Analytic Application Processing• Stateless controller/worker topology
allowing for rapid expansion/contraction of analytic processing capacity.
• Compressed and encrypted data streams over traditional HTTP.
• Generic persistence interfaces to allow both relational and non-relational data stores (SQL vs. No-SQL)
• Customizable throttles to limit app execution in a cloud environment.
Implication: Analytic processing in the cloud can scale to meet the needs of 10’s to 1000’s of users in a secure and flexible way with the appropriate limits to protect both the user and the execution environment.
Yes, but how did you do it?
17© 2012 Alteryx, Inc. Confidential.
18© 2012 Alteryx, Inc. Confidential.
• Implemented a completely stateless architecture for analytic processing.
• Use real-time compression and encryption to move data and analytic processes over HTTP.
• Design for future Data Residency Options (DRO).
• Create analytic processing arrays that are isolated processes and self-recoverable.
Make it fault-tolerant…
19© 2012 Alteryx, Inc. Confidential.
Initial testing indicates that even heavy-weight analytic processing (spatial, non-spatial, predictive and reporting) is uniformly distributed.
• Web Tier• Analytic Workers• Node MongoDB
replicated cluster
~= 50 OPS
Implication: Performance is just a “scale-out” operation, as opposed to “scale-up” - with tremendous cost savings benefits.
Plenty of room to grow…
20© 2012 Alteryx, Inc. Confidential.
Enhance Data,Add
Context
Analytic Apps, Data Loading
Access and Integrate Big Data
Humanizing Big Data: Single Platform to Deliver Big Data Insight & Foresight
21© 2012 Alteryx, Inc. Confidential.
Alteryx Strategic Analytics 8.0First Cloud for Strategic Analytics
http://gallery.alteryx.com
22© 2012 Alteryx, Inc. Confidential.
Key Terms
• Alteryx• Analytics Gallery• Analytic Apps• Analytics in Cloud• Cloud Apps• Big Data Analytics• Strategic Analytics• Predictive Analytics• Unstructured data• Humanizing Big Data• Analytics Platform• Data Analysts• Data Scientist
• RESTful• JSON services• Public/Private
Deployment• Fault-Tolerant• JSON vs. XML (SOAP)• Mongo DB• Amazon Web Services• IAAS• RightScale• George Mathew, Alteryx• Strata 2012• O'Reilly Strata
Conference• Strata Conf, New York• Strata, Oct 2012