responsibly forecasting©2003 orapub, inc. responsibly forecasting oracle system performance craig...
TRANSCRIPT
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
ResponsiblyForecasting Oracle
System Performance
Craig A. ShallahamerOraPub, Inc.
Portland, Oregon [email protected]
OracleWorld 2003Session 36691
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Presentation Objectives
• What is Oracle performance management.
• Know an industry standard methodology.
• Know about many proven models.
• How to begin using forecasting in your daily routine.
• Where to go from here.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
What is performance management?
• What is management?
• For a DBA, PM is both reactive and proactive.• Reactive:
– It’s were most DBAs live.
– Is what we have been taught to do.
• Proactive:– It’s where DBAs must learn and live.
– Forecasting is a significant part of PPM.
– It’s not as difficult as you might think.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
We must forecast methodically.
• We desire reproducible consistency.– Without method there is chaos.
– With too much method, creativity is halted.
• Forecasting is a scientific thing, therefore we should be able to develop a method.
• There are many different methods.
• The project plan, if there is one, should embrace and enhance our method.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Methodically forecasting performance
• Determine the “study question.”
• Characterize the workload.
• Develop the appropriate model.
• Validate the forecast forecast.
• Forecast!
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Determine the “study question”
• Each project has a fundamental question that needs answering.
• For example:– Can the system handle the load six months from now?
– Will response time significantly change if we add five more CPUs?
– How many CPUs and IO devices do we need to keep people happy?
• Write it down.
• Get everyone to agree to it…over and over again.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Characterize the workload
• Objective: To capture the most relevant characteristics of the real or proposed workload.
• Workload characterisation is simply segmenting and linking processes by common characteristics. (e.g., time of operation, CPU requirements, calls, hits, etc.)
• It is important because before any analysis or forecasts can be made, the workload must be understood.
– You can’t ask about something if you don’t know anything about it.
• A “capacity planning” tool must allow for flexible characterisation of the workload.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Forecast model development.
• Every forecast uses a some type of model.• Models are a reality abstraction.
– Something simple representing something complex.
• We use models every day of our lives.• Models:
– Are not real
– May seem overly simplified
– May seem overly complex
– Can be tunable or optimized
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Some benefits of usingmathematical forecast models.
• Greater chance of dealing appropriately with reality• Reduced complexity• The same WC must be performed.• Lower cost (labor, equipment, traveling, etc.)• Unlimited scenario testing (“what if”)• Very flexible when slight changes are needed• Very portable from one project to the next• Non-technical people can many times use the
model.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
The costs of usingmathematical models.
• Dealing with unbelieving clients and colleagues
• Mathematical modeling skills
• Must have the ability to think both abstractly, high level, and very detailed.
• Modeling doesn’t deal with algorithmic, scaling, and code issues very well.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Forecast model characteristics.
• Precision possibility. Each model has a limit on it’s possible precision. You want to match required precision with precision possibility.
• Project duration. Some models typically are involved with projects of specific duration.
• System status. Some models can only be applied to an existing production system, while some can can work well with a proposed system.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Forecast model characteristics.
• Required Data. Some models require detailed transaction level detail while others only require general WC and O/S data.
• Scope. Some models only deal with one system component (e.g., CPU) and do not account for O/S subsystem interaction.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Your unique situation will lead you to the best model.
Forecast Model Scope Required DataSystem Status
Precision Possibility
Project Duration
Simple Math componentapplication
specificproduction proposed
low very short
Ratio Modeling componentdetail application
and broad system
production proposed
low short
Linear Regression componentdetail application
and broad system
production high medium
Simple Queuing componenttransaction
detailproduction proposed
high long
Simulationcomponent
system
detail application and broad system
production high medium
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Some models are extremely simple.
• Simple Math. 2MB * 100 users = 200MB– Fast and simple
– We use this all the time!
• Ratio Modeling relates a WC category to a specific system resource.– 150 OLTP transactions = 1 CPU
– We use it all the time
– Relatively fast
– Amazingly precise given its simplicity
– No associated statistics
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Linear Regression is simple.
• LR relates one or more WC category to a specific system resource.
– It’s relatively fast
– Very precise
– Easy to learn
– Statistically sound
– Dangerous
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Simple Queuing is very versatile.
• Simple Queuing deals with the reality of waiting and non-uniform stuff.
– Very precise
– Statistically sound.
– Very versatile
– low precision or
– high precision
Response Time vs Throughput
0.000
1.000
2.000
3.000
4.000
5.000
6.000
7.000
8.000
9.000
10.000
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
Throughput
Re
sp
on
se
Tim
e
Queue Time
Service Time
Response Time
Arrival Rate
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Simulation is common in the research community.
• Simulation places transactions into a controlled system and records what happens.
– Very powerful and very versatile
– Statistically sound
– Different types
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
How many cycles does it take?
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Comparing different forecast models.
Forecast Model Scope Required DataSystem Status
Precision Possibility
Project Duration
Simple Math componentapplication
specificproduction proposed
low very short
Ratio Modeling componentdetail application
and broad system
production proposed
low short
Linear Regression componentdetail application
and broad system
production high medium
Simple Queuing componenttransaction
detailproduction proposed
high long
Simulationcomponent
system
detail application and broad system
production high medium
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
What model validation is used for.
• Validating a model helps us understand its precision capabilities.
• If a model is not validated, we can not responsibly use it.
• Validation usually provides valuable statistics used to quantify error.
• Basic statistics is required to understand the model validation results.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
How a model is validated.
• Validation applies unseen historical data to a model, forecasts, and then compares the forecasted values to the actual historical values.
• The smaller the error, the more precise the forecast model.
• Validation can focus on different “forecast areas” of interest.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Forecasting…finally!
• Forecasting is very satisfying, especially after all you have gone through to get there.
• Forecasting takes “never before seen data” and applies it to the forecast model.
• When following our methodology, the forecast output combined with the statistics, enables for very responsible forecasts.
• Forecasting is very model specific.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
An example simulation forecast summary
Workload Increase
(%)
% Rt Change
CPUs CPU BusyIO
DevicesIO Busy Notes
0 4 11 16 30 Baseline0 1 3 13 16 430 1 2 16 16 43 0 - 1 - 16 - unstable system
100 4 4 15 16 75 current system in 6 months200 - 4 - 16 - unstable : current system in 12 months
100 1 3 18 20 61 Mod system in 6 months200 4 3 23 20 83 Mod system in 12 months200 1 3 22 25 68 Mod system in 12 months200 0 3 22 30 58 Mod system in 12 months200 1 3 22 25 68 Mod system in 12 months300 2 3 26 25 84 Mod system in 18 months300 0 3 26 30 71 Mod system in 18 months
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Thoughts about integrating forecasting into your daily routine.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
• Don’t wait and don’t wait for the perfect project.
• Look for natural forecasting opportunities in you existing work.
• Add additional value to your existing work.
• Don’t wait for the perfect opportunity.
• There is no perfect forecast project.
Start forecasting right away.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
• You may not notice this yet, but people frequently ask forecast type questions.
• They may not expect you to be able to answer them.
• Start listening to people around you.
• As you listen be thinking about how they could be helped using forecasting.
• Don’t wait for them, listen and ask them if they could benefit from a forecast.
Listen for opportunities
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
• Don’t wait for the big perfect project.• Start with small, very well defined, and
very relevant forecasts.• Don’t try to impress people, just get it
done.• If you do a good job, people will begin
asking for more forecasts.• Don’t be talked into doing something larger
than you, or for that matter anyone, could do.
Answer very simple questions.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
• Forecasts need lots of data.• Starting collecting lots of data now.• Don’t wait until you need the data…it will
be do late.• Gather Oracle, application, and O/S data.• Store it in a DB so you can get to it
whenever you need to.• The gathering frequency should be
between 30 minutes to 2 hours. One hour is a good place to start.
• The OSM tools are a good way to start.
Build a historical database.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
• Use the tools and services you received from this course.
• Don’t try and create your own mathematical tools right away.
• Use what you have and enhance as appropriate.
• If you have a general modification, please share.
Begin with existing tools and services.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
• Don’t let the math scare you away.
• Don’t let the math or the tool distract you from answering the study question.
• More complex math does mean more precise forecasts.
Don’t get caught up in the math.
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Final Exam!
• What are the five methodology steps?
• Name four forecasting models?
• What model makes quick low precision forecasts?
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Resources
• www.orapub.com
• Simulation with Arena
• Practical Performance Analyst
• Fast Track Demonstration
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Thank You!
Responsibly Forecasting©2003 OraPub, Inc.
www.orapub.com
Craig A. ShallahamerOraPub, Inc.
Portland, Oregon [email protected]
OracleWorld 2003
ResponsiblyForecasting Oracle
System Performance