too slow?: use vs2010 profiling & load testing to manage performance issues benjamin day
TRANSCRIPT
Too Slow?: Use VS2010 Profiling & Load Testing to Manage Performance Issues
Benjamin Day
Benjamin Day
• Consultant, Coach, Trainer• Scrum.org Classes
Professional Scrum Developer (PSD) Professional Scrum Foundations (PSF)
• TechEd, VSLive, DevTeach, O’Reilly OSCON• Visual Studio Magazine, Redmond Developer News• Microsoft MVP for Visual Studio ALM• Team Foundation Server, TDD, Testing Best Practices,
Silverlight, Windows Azure• http://blog.benday.com• [email protected]
Thanks.
• To my content reviewers Jerri Chiu, Microsoft Dennis Bass, Microsoft
The Agenda
• Basics of Web Performance Tests Load Tests Load Test Rigs Performance Explorer
• Existing application Identify performance problems Go from bad performance to good ASP.NET & WCF
Disclaimer: I’m going to use the term “load testing” imprecisely…a lot.
Here we go.
The Purpose of Load Testing
• Identify the capabilities of your application• Possible goals
Explore Verify Find the limits Crush
Best Practice:The Obligatory “Eat Your Vegetables”
• Load test throughout the development cycle.
• Catch problems early• Establish performance baseline• Watch for performance trends
• Cheaper to fix early
• Plan for performance
Cheaper To Fix Early
Early Less Early Nearly Done Production0
10
20
30
40
50
60
Time
Co
st t
o f
ix
Dose Of Reality
• You know you should work that way.• You probably don’t.
• This talk is for you.
• You have problems.• How do you figure out what’s going wrong?
Inspiration for this talk
• Real customer engagement• Spent years developing a web application• Released it • Seriously grumpy customers• Huge hardware• ~20 simultaneous users• Fixed it
4000+ simultaneous users Reduced hardware
The Tools
Visual Studio
Ultimate
• Web Performance Tests• Load Tests• Load Test Rigs
Visual Studio
Premium
• Profiling Tools• Performance Wizard
TOUR OF THE APPLICATIONdemo
Disaster Strikes.
• The application is deployed to production.• Users are losing their minds.
Intermittent errors on the site. Site is slow.
• Your boss is seriously angry. “Fix it!”
The Problem.
• “No Repro” “Works on my box.” You don’t know what the errors are. The site seems to work fine.
• Wild guess: it dies under load.
• You need to create some load.
The Plan.
1. Script user activity with Web Performance Tests
2. Simulate a mix of users using Load Tests
3. Simulate lots of users with a Load Test Rig
4. Reproduce the errors...hopefully
5. Find code problems with the Performance Wizard
6. Fix the code
Web Performance Tests.
The Plan.
1. Script user activity with Web Performance Tests
2. Simulate a mix of users using Load Tests
3. Simulate lots of users with a Load Test Rig
4. Reproduce the errors...hopefully
5. Find code problems with the Performance Wizard
6. Fix the code
What is a Web Performance Test?
• Test type in Visual Studio 2010 Ultimate• Simulate a user using an ASP.NET application
HTTP traffic
• Building block of Load Tests
Tip: Web Tests & User Stories
• Web Tests should simulate a User Story• Example:
As an administrator, I need to search for a person by username so that I can deactivate that person’s account.
As a customer, I want to order a pizza and pay with a credit card.
• Helps to organize your suite
CREATE A BASIC WEB TESTdemo
Data-driven Web Tests
• Run web test once per row in data source• Simulate multiple users doing *similar* actions• Data source types
OLE DB, CSV, XML
• Parameterization
• Better tests, better coverage
WEB TESTS IN THE UNDERPERFORMING APPLICATION
demo
Load Tests.
The Plan.
1. Script user activity with Web Performance Tests
2. Simulate a mix of users using Load Tests
3. Simulate lots of users with a Load Test Rig
4. Reproduce the errors...hopefully
5. Find code problems with the Performance Wizard
6. Fix the code
What are Load Tests?
• Mix of Web Performance and Unit Tests
• Simulates a mix of users doing a mix of things
• Simulate Network speeds Different browsers Varying user activity loads
Why would you put a Unit Test in a Load Test?• Helpful for testing WCF• Cumbersome to simulate WCF via Web Test
(You’d have to re-invent the wheel.)
• Unit tests can re-use the WCF libraries (Done and done.)
• Important for Silverlight and SOA• http://wcfloadtest.codeplex.com/• http://sqlloadtest.codeplex.com/
HEY KIDS, LET’S CREATE SOME LOAD TESTS!
demo
(…and then you all say, “Hooray!”)
Running Load Tests.
The Plan.
1. Script user activity with Web Performance Tests
2. Simulate a mix of users using Load Tests
3. Simulate lots of users with a Load Test Rig
4. Reproduce the errors...hopefully
5. Find code problems with the Performance Wizard
6. Fix the code
Two ways to run load tests.
• Generate load from Visual Studio 2010 Uses only 1 core/processor Limited to 250 simulated users
• Generate load from a Test Rig Far better test Simulate lots more users
Load Test Agent Licensing• Microsoft Visual Studio Team System 2010
Test Load Virtual User Pack• Simulate up to 1000 virtual users• Used to cost $$$
Visual Studio 2010 Load Test Feature Pack• Used to cost ~$3500.00 per 1000 simulated users
• Free starting March 8, 2011• Visual Studio 2010 Ultimate with MSDN• Simulate unlimited amount of load
(Limited only by your hardware.)
• http://www.microsoft.com/visualstudio/en-us/products/2010-editions/load-test-virtual-user-pack
Load Test Rig & The Application
• Controller Orchestrates Licensing Perf Mon
Counters
• Agent (1..n) Simulates
users
The Plan.
1. Script user activity with Web Performance Tests
2. Simulate a mix of users using Load Tests
3. Simulate lots of users with a Load Test Rig
4. Reproduce the errors...hopefully
5. Find code problems with the Performance Wizard
6. Fix the code
RUN LOAD TESTS ON THE RIGdemo
The Core Problem.
‘sp_who2’ says lots of open connections.
Performance Profiling
The Plan.
1. Script user activity with Web Performance Tests
2. Simulate a mix of users using Load Tests
3. Simulate lots of users with a Load Test Rig
4. Reproduce the errors...hopefully
5. Find code problems with the Performance Wizard
6. Fix the code
Profiling Tools
• Load Tests help you find issues in a system “Surface” issues
• Profiling helps you find issues in the code
• Unit Tests, Web Tests, Load Tests Repeatable Good for targeted tuning
• System under “natural” load Not repeatable Looking for rough patterns
Profiling Methods
• Gathers information at intervals (clock cycles)• No code modifications• Use this for first pass explorations
Sampling
• Modifies your code• Gathers detailed timing and count informationInstrumentation
• Multi-threaded code profilingConcurrency
• Object creation• Garbage collection.NET Memory
• Interaction between your application and SQL Server via ADO.NETTier Interaction
Load Test Performance Sessions Via Wizard
• Must be only Web Performance Tests• Can’t profile via Load Test Rig• ASP.NET must be running in x86 mode
LET’S GO PROFILE SOME STUFF.
demo
The Plan.
1. Script user activity with Web Performance Tests
2. Simulate a mix of users using Load Tests
3. Simulate lots of users with a Load Test Rig
4. Reproduce the errors...hopefully
5. Find code problems with the Performance Wizard
6. Fix the code
Any last questions?
For More Information: Articles
http://tinyurl.com/3xjqgvz
http://tinyurl.com/2ulvvvr
For More Information: Microsoft Links
• Visual Studio Performance Testing Quick Reference Guidehttp://vsptqrg.codeplex.com/
• Content Index for Web Tests & Load Testshttp://tinyurl.com/ye97e52
For More Information: Microsoft Bloggers
• VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/
• Ed Glas - http://blogs.msdn.com/edglas/• Bill Barnett - http://blogs.msdn.com/billbar/• Sean Lumley - http://blogs.msdn.com/slumley/• Dennis Stone - http://blogs.msdn.com/densto/• Mike Taute - http://blogs.msdn.com/mtaute/• Neelesh Kamkolkar -
http://blogs.msdn.com/b/nkamkolkar/
Additional information that I practically never have time to cover.
Miscellaneous Tips.
Include Load Tests In Your Nightly Build
• Create a Test List (*.vsmdi)• Include the Test List in the build• Captures trend data at 24 hours intervals• Compare the load test data with Excel
Static Code Analysis / FxCop
• Help you to find issues in your code• Show you IDispose errors
• Rules are configurable• Use your judgment about which are relevant for your team
Avoid Static Methods & CA1822
• Static Code Analysis Rule CA1822 is evil.
• Sacrifices maintainability for perfomance.
• Blog post: “Static Methods Are A Code Smell”
• http://bit.ly/cnEsuV
Only optimize known performance problems.• You’d be surprised by what *ISN’T* a performance problem• Don’t spend a ton of time coding fancy solutions to
performance problems you *THINK* you might have• Find and fix *REAL* problems
Tip: WCF Message Size
• Data structure formatting can make a huge difference• Example: Banks and the States they operate in• Option 1: Dictionary<string, string>
Key = Bank Id Value = State Abbreviation
• Option 2: Dictionary<string, string[]> Key = State Abbreviation Value = Array of Bank Id’s
• Option 2 was approximately 90% smaller
Tip: Add Performance Counters
• Create Performance Counters (perfmon) in your app• Do this early in the development cycle
• More directed profiling of your app during Load Tests
• Good for managing your app when it’s in production (Operations people love this.)
Using Performance Sessions on x64
• You can’t profile x64 IIS processes from the IDE
• Option #1: Profile from the command line Start ASP.NET profiling using VSPerfAspNetCmd.exe Run the web test
• Option #2: Run the ASP.NET process in x86 mode
How To Run ASP.NET in x86 Mode
• Set “Enable 32-bit Applications” to true on your application’s AppPool