performance monitoring and testing in the salesforce cloud
DESCRIPTION
TRANSCRIPT
Performance Monitoring and Testing
in the Salesforce Cloud
Your Success. Our Cloud
Raj Advani, salesforce.com, Sr. Technical Solution Architect
Randy Case, salesforce.com, Director, Customer Centric Engineering
Safe Harbor
Safe harbor statement under the Private Securities Litigation Reform Act of 1995:
This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties
materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results
expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be
deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other
financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any
statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.
The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new
functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our
operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of
intellectual property and other litigation, risks associated with possible mergers and acquisitions, the immature market in which we
operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new
releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization
and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of
salesforce.com, inc. is included in our annual report on Form 10-Q for the most recent fiscal quarter ended July 31, 2012. This
documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of
our Web site.
Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently
available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based
upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-
looking statements.
Agenda
What is performance testing
The scalable Salesforce architecture
Internal Salesforce performance testing
Why do you need to test?
Performance testing approach and testing tools
Questions and Answers
What is Performance Testing?
‘Performance testing is typically done to help identify bottlenecks in a
system, establish a baseline for future testing, support a performance
tuning effort, determine compliance with performance goals and
requirements, and/or collect other performance-related data to help
stakeholders make informed decisions related to the overall quality of the
application being tested. In addition, the results from performance testing
and analysis can help you to estimate the hardware configuration
required to support the application(s) when you “go live” to production
operation. ‘
6
WOW!
What is Performance Testing?
‘Performance testing is typically done to help identify bottlenecks in a
system, establish a baseline for future testing, support a performance
tuning effort, determine compliance with performance goals and
requirements, and/or collect other performance-related data to help
stakeholders make informed decisions related to the overall quality of the
application being tested.
7
WOW!
Agenda
What is performance Testing
The Scalable Salesforce Architecture
Internal Salesforce Performance Testing
Why do you need to test?
Performance Testing Approach and Testing Tools
Transactional Volumes
It’s in the numbers!
>800 million
operations/day
< 275 ms avg. response time
>40 million account records
>30 million opportunity records
100s millions custom object records
100s thousands users
100s gigabytes of data
>20 million contact records
>60 billion requests/quarter 104,000+ businesses
Multi-tenant, Cloud Computing Platform
Pod Architecture Enhances Availability, Scale, and Platform
for Future Growth
Network
Services
Storage
Services
Backup
Services
Monitoring
Services
NA0 Pod
NA1 Pod
NA2 Pod
NA3 Pod
NA4 Pod
NA5 Pod
EMEA Pod
APAC Pod
Sandbox Pod
EMEA2 Pod
NA6 Pod
NA7 Pod
“N” Pod
Security
Services
A Scalable, High Performing Architecture
Database cluster
8 Node RAC cluster
Scalability and fault tolerance
Application Servers
Web requests, API calls, Batch processes, Search requests
30 plus servers, load balanced for performance
Scale vertically and horizontally
Scalable, High Performing Architecture
Network
10 carriers for geographic
performance and redundancy.
Data centers located at core
Internet hubs.
Access to thousands of global
Internet peering points.
Redundant routers, switches,
firewalls and load balancers
Agenda
What is performance Testing
The Scalable Salesforce Architecture
Internal Salesforce Performance Testing
Why do you need to test?
Performance Testing Approach and Testing Tools
Performance Engineering
Primarily focuses on benchmarking, simulation, tuning, analysis, and monitoring/profiling of the applications.
Uses sophisticated simulation methodologies, automation, tools, and environments to discover the problems.
Performance Engineering
Performance testing typically falls into two major categories.
Playback-based:
• Plays back the Production traffic logs within a secure environment.
• Private information appropriately obfuscated.
• Capture real-world data skews, volumes, and transactions.
Synthetic:
• Custom tools to generate and profile workloads and data shapes
• Simulate current loads and beyond that are currently in use in the production
systems.
What’s Tested?
Full Environment
Database Performance
User Interface Search Programmatic
Features
Capacity Planning
Proactive Approach
Customer transaction monitoring
Capacity alerts
Reactive monitoring
Multi year growth plan
Scale vertically or horizontally
Excess capacity always exists!
Agenda
What is performance Testing
The Scalable Salesforce Architecture
Internal Salesforce Performance Testing
Why do you need to test?
Performance Testing Approach & Testing Tools
Candidates for Performance Testing
Heavy Custom Development
• Testing should be geared towards exercising the custom code of an application
High Activity
• Large Transaction based testing
• Large User based testing
• Browser Rendering
• Real-time monitoring
What Is Performance Testing
22
Response Time Based Testing
Confirm that a set of VF Pages and Apex controllers perform in such a manner as to give the end user a positive experience
Load Based Testing
Confirm that a set of Apex code, VF Pages will be able to support a particular load.
What is Performance Testing? Performance Testing categories
Load, Endurance, Spike, Configuration, Isolation.
Performance Testing Architecture
Metrics Collected
Page Response Times
Measurement of End User
perspective of tested pages
Server Response Times
Measurement of SFDC POD
Response Times
Transaction Throughput Rate
Measurement of number of completed
transactions in a time period
Agenda
What is performance Testing
The Scalable Salesforce Architecture
Internal Salesforce Performance Testing
Why do you need to test?
Performance Testing Approach & Tools
Recommended Performance Testing Approach
Build Test Plan
Run Baseline Test
Identify X
•Identify Key Business
Transactions
•Test Tool
•Environment for
testing
•Projected X Load for
each script
Run .5X, .75X, and X
Tests
•Low User Test
•Used to confirm script
•Get baseline numbers
•Use baseline numbers
•Data analysis from
existing sources
•Make sure number is
realistic
•Run your full scale
tests
•Gradually increasing
load
X = Projected realistic goal based on script and
wait time
Performance Testing Tools
Salesforce supports most all major performance testing tools
Tools capture all data and metrics for reporting
More popular tools:
Open Source
•Jmeter
•Fiddler
•OpenSTA
Commercial
•LoadRunner
•Silk Performer
•RadView
Engaging Salesforce for Performance Testing
Create tests that measure transaction and business process time.
Create a test plan
Open a Salesforce support case with the following:
Brief explanation of the test
When test will be conducted
Sandbox instance and organization ID where the test will be conducted.
Contact information during the test
Attach your test plan
If plan or timeframe changes, let us know.
Wrap Up
•Most clients will not need to conduct their own testing
• Average of 60 billion transactions a day with an average response time of 250ms
• Multi-Tenet architecture supports scalability and high performance
• Capacity planning and Performance engineer teams conduct performance test to
ensure application and platform performance
•Heavy use of custom code or high user load clients may conduct
performance tests
• Main collected metrics include, response times, server times, and transaction
throughput rate
• Identify realistic load, create test plan and run tests
• Engage Salesforce by creating a support case
Raj Advani Sr. Technical Architect,
Randy Case Director, Software
Engineering
