the best of both worlds - combining performance and functional mobile app testing
Post on 29-Jan-2018
378 Views
Preview:
TRANSCRIPT
Presenters
Speaker
Henrik Rexed
Performance Engineer
Neotys
Speaker
Lingkai Shao
Marketing Analyst
Bitbar
Speaker
Dave Horner
Director of Sales
Bitbar
Neotys Continuous Performance Testing Solution
System Testing
Integrated Component Testing/System-wide Testing
ReleaseComponent Testing
Continuous Integration & Component Testing
Proactive Production Monitoring End-User
Experience
Dev Ops
Continuous
Release &
Deployment
Collaborative
Development
Continuous
TestingContinuous
Monitoring
Continuous
Integration
Continuous
Feedback
DevOps is no longer a
unicorn’s discipline.TestDev Ops
Shift Left Shift Right
DevOps
According to Adobe 2016 Online Shopping Data, mobile drove 55% of visits to retail websites on Black Friday (45% coming from smartphones, 10% from tablets) and accounted for 36% of sales (25% from smartphones, 11% from tablets) *
Why Functional and Performance Testing for Holiday Season
*=http://news.adobe.com/press-release/marketing-cloud/media-alert-adobe-data-shows-black-friday-breaks-online-sales-record-3
Performance matters
53% of visits to mobile sites are
abandoned after 3 secondsaccording to research from Google
According to Google doubleClick faster websites has 70% longer average session lengths, 35% lower bounce rates and 25% higher ad
viewability than their slower counterparts.
Etam reduces its average page load time from 1.2s to 500msand increased conversions by 20%
Walmart saw up to a 2% increase in conversions for every 1 second of improvement in load time.
Every 100ms improvement also resulted in up to a 1% increase in revenue.
● Preparation: typically 10 weeks ahead of peak days● Functional test suite including
Stage One: Functional & Compatibility
Standard performance testing workflow
Build the performance strategy
Create the testing scenarios
Prepare
Execute
AnalyseTuning
Mobile Application Architecture Landscape
Native app
Hybrid app
Web app
Web appWeb Model
Mobile Model
Rest Interface
Business logic
Data Access logic
XML
Legacy
Relational
Presentation Business Data
Third party
Mobile Performance Testing Touchpoints
BACK-END
EXECUTION PLATFORM
RIA/FRAMEWORKS
DATABASE
NETWORK
3G, 4G / LTE, Wi-Fi
VIDEO
HTTP Adaptive StreamingThroughput
LatencyPacket loss
SERVERINFRASTRUCTURE
OS
CLIENTAPP
DEVICE
iPhone 4, 5, 6SamsungBlackberry…(7,000 different device types are used to access Facebook)
Screen SizeCPUMemory
Native Browser-BasedHybrid
The most important network constraints are:• Latency
• Bandwidth
At a certain point, increasing the bandwidth will only have a minor effect.
Latency & Bandwidth . . .
Packet Loss Consequences
• Packet Retransmission
• End user experience degraded because packet loss increases jitter
• Errors could occur if packet loss is in pure data transmission
Packet Loss is Inevitable with Mobile
• Mobility: Quickly switching to the next cell tower may result in temporary signal loss due to congestion on the current tower
Mobile Protocols Try to Reduce Packet Loss Rate
Packet Loss . . .
Lots of users
• Mobile applications often have large user bases
• High growth in popularity can create the need for highly scalable load generation infrastructure
Geographical distributed load
• Users are accessing apps from locations around the globe
• Distance and location can greatly affect performance
Mix of Web & Mobile Users
• Many mobile apps have a corresponding web app with a shared backend
• High load on the web app can affect performance on mobile
High volume, Geographically Distributed
… Applying the Bitbar Testing Integration…
00:00:00 00:00:30 00:01:00 00:01:30
Load
NeoLoadApplication
Server
App Error Rate
Virtual Users Load
UserExperience
Why Synthetic Monitoring?
● Alarm and Alert● Better MTTR! (Mean Time To Resolution)● Find and fix issues before impact to users● Baselining and Benchmarking (Pre-production Staging & Post Production)● Prepare for peak traffic and new markets● Monitor Complex Transactions● Measure and Adhere to SLAs● Hold 3rd Party Vendors Accountable● Test from end user perspective● Competitive Benchmarking
Stage Three: Proactive Production Monitoring
Understand Performance of Complex User Transactions and Hold 3rd Party Services Accountable (SLAs)
Benefits of Synthetic Monitoring
Understand Performance In New Markets, Optimize Content Delivery, Benchmark Pre & Post Change
Benefits of Synthetic Monitoring
top related